Passive aggression often stems from underlying anger, sadness, or insecurity, of which the person may or may not be consciously aware.  If this seems like a passive-aggressive response, you may have to fight fire with fire.