Remove custom "ls" colors if we're disabling customizations.