Return if the stream is nonblocking

If the stream is nonblocking we actually get this invalid packet and
should just stop.