Technically fudge does not need to be refrigerated to set. It just needs to be cool which can happen on the counter. I like to put it in the fridge to speed up the process. The kids are usually standing around waiting for it!