You can absolutely bake your sweet potatoes, keep in mind that this will take a lot longer than microwaving. Simply preheat your oven to 400°F. Pierce potato skins all over with a fork and place them on a foil lined baking sheet. It will take about an hour for the potatoes to finish baking.