Don't reset colors if the string isn't colored