Sweet potato casserole uses brown sugar with a topping of nuts or marshmallows but sweet potato pie uses white sugar and can be served with a whipped topping or no topping.