Yes, you can reduce the calorie count by using low-fat or non-fat alternatives for some of the ingredients. For example, you could use non-fat buttermilk instead of regular buttermilk and reduced-fat butter or margarine instead of regular butter. Additionally, you could reduce the amount of sugar used in this.