This is probably because the mixture got too hot on the stovetop and the sugar re-crystallized making for a gritty, grainy texture. Make sure to keep an eye on that candy thermometer otherwise you might end up with crumbly fudge!