Usually it's because it's because the sugar isn't fully dissolved and the fudge ends up being grainy. Be patient and give the sugar time to dissolve.