Add in error ignore for the socket exception in the packet bufferer.

I believe this is the implementation that both @k0rd and I agreed on.  If we agree, we can accept this and close the other pr/issue.