matttproud/golang_protobuf_extensions — pull request #3 by u-c-l (2014-07-02 18:03:44+00:00)
Title: Bring decode.go Reader usage in line with the Reader contract.

Following items are important here:

- A Reader should never return 0, nil, but if it does, it should be
treated as a no-op.

- A Reader will read what's there and then return. Later calls will
then block until something can be read or an error occurs. (Or it
will return 0, nil, but see above.)

- If a Reader returns an error, the bytes already read MUST be
considered before considering the error. A common example is that a
Reader reads everything as expected, but already knows the next call
will result in 0, io.EOF. In that case, some Reader implementations
return n, io.EOF (with n being the expected number of read bytes) to
give the caller the opportunity for performance optimizations (like
dropping the Reader without trying another read).

Especially the last item was causing trouble if reading directly from
the http response body (as it is one of those that returns io.EOF
ASAP). The second item causes trouble in subtle ways (imagine a stop
in the middle of reading a varint - or you will try to unmarshal an
incomplete proto message).

BTW: The error "truncated message" would never trigger. Fixed
implicitly now.

The fix was easy for the part where the proto message is read because
we already know the length and can use io.ReadFull to avoid both, the
stopping problem and the early io.EOF problem. (Check
http://golang.org/src/pkg/io/io.go?s=10103:10169#L273 to see how it
does it.) THe varint reading is tricky. If we read less than
binary.MaxVarintLen32 bytes, we might actually be fine already. We
cannot simply wait until we have read binary.MaxVarintLen32 bytes
because the whole shebang (varint plus the whole proto message) might
fit into less. So we have to try if we can parse the varint from what
we have and then try again until we either hit an error or can parse a
varint or we have read the maximum and can still not parse a varint
(which is an error, too).