Clean shutdown of ReliableProxySpec, see #2846

* Solved by adding missing Dequeue in throttler
* Some boy scouting