You’re not required to wear them throughout every aspect of your workout if you don’t want to. Doing workouts such as planks don’t require hand protection, but if it’s easier to leave them on, then that is perfectly fine. It’s more important to make sure you’re protecting your hands when doing bar work or lifting.