book holster

A strap or container attached to a belt or shoulder strap, where people put a book to be carried around hands-free.

When there is only the strap around the book itself, not attached to anything else, use book strap.