Yes, you can do that. The stress built up during the exercises in the gym session will wear off with some yoga practice. Gym workouts and yoga sessions compliment each other, so go ahead with both.