Gnerally speaking any novel illustration that is greyscale will not have any text unless it is in a speech bubble or is a character profile, while any coloured illustration will have text, usually either lines from the scene being depicted or just the name of the character.
There are exceptions, such as post #1570122, but these are relatively unusual. I just checked 40 of the light novel series I own and only 2 of them (OreTwi and Inou Battle) do this, and the latter only did it a couple of times.
As for how to tell for sure... well, most light novels do have free samples available on the website or on nico seiga, but I accept that this is quite a bit of effort.
As to how useful it is... well, I guess it would tell people "if you buy the light novel then this picture will probably have text over it".