The main difference between sweet potato pie and sweet potato casserole is casseroles are baked with a crunchy nut topping and marshmallows on top. In contrast, sweet potato pies typically do not have any toppings.