One purpose of punch mitts is to train a fighter to punch at different levels and different angles. You might hold them at chest height for straight jabs, at waist height for rising uppercuts, at head height for hooks. So holding the mitts in a fixed spot isn’t desirable. Instead, hold them at the height and angle needed for the fighter to strike with proper form and practice a variety of strikes.