this is less fugly

Signed-off-by: Stephen Gran <steve@lobefin.net>
Fix infinite loop in listeners on server shutdown