Cleanup color classes

Cleaned up more of the classes that work with color while trying to get
it to work correctly.