Yes if you want to have that gooey, caramelized texture that makes a crisp so delicious. You can also keep the skin on as it will sort of soften during baking.