Remove assert which prevented quiescing

Mike, this is the PJSIP fix you suggested I make to allow closing down multiple TCP factories. Quick review?