Revert "Temporary ignore socket.error issues"

This reverts commit 13ddeb8fccd87b56293912b9fadedd849cccbf08.

Theoretically, they fixed the network flow so we don't need this
anymore.