It's possible the gelatin wasn't fully dissolved before adding the other ingredients. Don't rush it. It takes about 10 minutes to fully softened and dissolve in the liquid. If it is not dissolved it will leave the panna cotta with a grainy texture.