There are only 5 gems throughout the dungeons on BBI, not counting the gem you always get in one of the chests when you beat Daimon. That makes a total of 6. The first 5 gems found throughout The Island do not respawn, but the chest with the gem after killing Daimon respawns every time.