Revert "Fixed potential SEGFAULT."

This reverts commit 79b81f48402c1276d4cfd3bf0cbfb4084952b36b.
Was causing:

Assertion failed: destroyed (socket_base.cpp:154)
/bin/bash: line 5: 31344 Aborted                 ${dir}$tst
FAIL: test_many_sockets

On TravisCI.
Merge pull request #923 from hintjens/master

Revert "Fixed potential SEGFAULT."