I think you're on to something here, but I think it's probably a little more complex than what you shared with us in this message. We tend to catch fevers because viruses inside our bodies are adapting the environment to themselves. Any hive or nest building species changes the environment to help themselves. Honeybees, by helping to spread the pollen of their favorite flowers, are changing their broader environment to help themselves. Coral reefs certainly change their environments dramatically, in ways that help themselves.
Parasites often change their environment to help themselves.
Perhaps it is the fact that we don't just change the environment where we live, but that we change the environment in places where we plan to live. Perhaps scale has something to do with it. Just something to chew on.