Revert "Faster scaling through lower end clocks"

Part of commit found here:
https://github.com/Snuzzo/funky_kernel/commit/1b53f437ced0973ef3b8edbffe77620437f47232