Getting the marshmallow and butter mixture too hot can lead to your peanut butter chocolate Rice Krispie treats getting too hard. Also, if you don’t use fresh marshmallows or butter, it can impact your treats.