It's highly recommended for two reasons - the crust can be a little dry, and it just looks prettier with them removed.