You can find topaz in Gerudo Desert. Link can mine it with his pickaxe.