It's called “transient” hypertrophy for a reason. Muscle pumps disappear about as fast as they come on, because once you finish working out, your body has no reason to hold onto all that blood in non-working muscles. As for the lactic acid, your body wants to flush that out as fast as possible.