Quirrell is, in effect, turned into a temporary Horcrux by Voldemort. He is greatly depleted by the physical strain of fighting the far stronger, evil soul inside him.