Revert "sched: Micro-optimize the smart wake-affine logic"

This reverts commit b582d20650ae74fde6bb47c5d1305f346d630c3a.