Optimize colorDictionary to only be built once