Change the way scaling was being performed

Refactor some scaling code to make it cleaner and remove unnecessary
code.

This should fix #5