Fudge is mainly made with sugar, butter, and milk, while chocolate fudge includes cocoa or chocolate.