Usually this happens when the gelatin doesn’t properly dissolve, or hasn’t properly mixed in with the milk mixture. It can also happen if the heavy cream and milk mixture isn’t emulsified and properly mixed before pouring into dishes. So to make sure this doesn’t happen, thoroughly mix the mixture after adding the gelatin AND after adding the cream. You could even wait for the mixture to cool down a little, and mix it again before pouring the panna cotta into individual dishes.