Sweet potatoes and yams are often confused, but they are different. Sweet potatoes have a smoother texture and sweeter flavor compared to yams. For traditional sweet potato casserole, it’s best to use sweet potatoes. Yams are starchier and can be drier, which might affect the consistency of your casserole.