Just use one file for storing ssh-configs
symlink ssh-config to .ssh/config
Omit superfluous color plugins.