If you’re new to using this ingredient in dessert recipes, you might be asking yourself the same question. Although peeling sweet potatoes is optional when baking them, you do need to peel sweet potatoes before making cupcakes with them.