The Water Bolt looks the same in every world. Before you pick it up, this weapon looks like a book with blue covers and a yellow strap. In your inventory, it has a red color. If you don’t want to mine each bookshelf, you can try and search for the Water Bolt by its visual appearance, even though this method is more time-consuming and dangerous considering all the enemies in a Dungeon.