Issue when calling HttpListenerBase Start/Stop quickly

In a integration test setup where we Start on AppHostHttpListenerBase derived classes in the setup and Stop them sometimes again immediately we saw NullReference exceptions.
The exceptions happen on the ThreadPool worker and I didn't find a way to catch them.

The problem is, that the queued user work item on the thread pool doesn't make sure that the listener is still around between the while condition check and its usage.

I created a test that you can execute manually and have to check the output (I saw repeatedly NullReferenceExceptions in ReSharper's Test output) that shows that error.
Beside that, I added locks around the important pointswhere the Listener is accessed.

Cheers,
Dominik