Trust me, you do. Frozen hash browns take twice as long to cook, plus the consistency of the casserole is just not quite right. Take the time to let the frozen potatoes thaw. It’s worth it.