Revert "Remove confusing optimization"

This reverts commit e05c80a08859f6c4f0f599ca0b262ced00ad66fe.

we gain a speed up of 1.5% under gcc !

No functional change.