Instead of throwing away velocity that isn't perpendicular to the swing every update just change the current velocities direction. Now velocity isn't dampened, but it does increase while swinging. This can be fixed by applying a damping I choose