The short answer is because sugar crystals formed during the heating process. For example, if you moved the chocolate mixture at all during the cooling phase, sugar crystals can form and contribute to the grit. Keep in mind though that this is an old fashioned fudge recipe and since it contains no cream, it tends to have a finite amount of grit naturally.