Revert "cpufreq: Don't use cpu removed during cpufreq_driver_unregister"

This reverts commit bbcdb66be1f7c7d1a32e3d7fcb1800422615ba4a.