When having taken on too many stimuli from their surroundings and are in serious need of recharging, the Empath wants nothing more than to be invisible to others. 

And when heading towards a fatigue meltdown, the last thing an Empath can deal with is someone offloading their troubles, like so many like to do.  Even polite conversation is too much.  This is often interpreted as a rejection or an insult. 

Because others don’t feel what an Empath feels, it is difficult for them to understand why they have to withdraw the way they do. 

Sadly, the more insecure someone is, the more they are offended by your quiet ways.  If they see you as blowing hot and cold, they may consider this as disrespect and reject you as a way to hurt you. 

So, if someone develops an aversion towards you, for no good reason, remember not to take it personally.  It is just the Mirror of the Empath at work.