Fix EMFILE errors by killing keepalive

See detailed explanation in the diff.