Pump will shoot water or soap currently when I press down, but when the device goes back up the water or soap shoots out of the little holes on the sides of the valve. We have two, and both do this.