Fix major issue with threaded worker keepalive

The keepalive queue management is fixed to not pop extra connections
incorrectly.

Close #816