Yes, it's important. Watch the video below - I did not refrigerate the pie for long enough (I let it rest for one hour on the counter, then one more hour in the fridge), resulting in a very gooey center. Which I actually like, but it makes cutting the pie more challenging. So yes, ideally, you should let the pie rest for the full four hours before attempting to slice it.