The word color is very old in English. It dates back to when Middle English was spoken, sometime in the 14th century. But it wasn't until many years later that people started spelling it as colour. The spelling changed to colour along with other words that had the "or" sound at the end of them, such as honour and labour. Nobody really knows who came up with this rule or why it was decided upon but it has been followed by many people over a long period of time.