GHSA-v2ch-c8v8-fgr7: Versity panic induced by AWS chunked data sent to port
Sending AWS chunk data with no Content-Length HTTP header causes the panic, every time.ReproductionSetup versity server running on port 7071, no SSL (for ease of packet tracing with tshark). Problem can be reproduced with or without SSL on the versity end.Use nginx to reverse proxy on port 7070. This does have to be SSL enabled for the repro to occur. nginx config:Use aws s3 cp to copy a large file (one that will trigger multipart upload) into versity via the nginx proxy on port 7070. AWS CLI must be version 2 for the repro to occur. Connecting directly to versity on port 7070 does not trigger the repro.Initial crash analysisThe versity server enters a panic exception in function `HashReader` (csum-reader.go). The panic is due to a null value in the field r which should be a pointer to an io.Reader.The reason this field is blank goes back to code in the `fasthttp` library, `ContinueReadBodyStream`. This function exits prematurely if the incoming request `ContentLength` field is set to `-2`, and therefore does not set up an io.Reader in the request context structure.`-2` is not a valid content-length for an HTTP message, it is a special value used internally by the `fasthttp` module. (Unfortunately the author hard coded the value rather than using a meaningful macro, which would have aided the understanding here. )The reason for the `-2` can be found in function `parseHeaders`(headers.go in fasthttp). The header struct content length is set to `-2` at the start of the function, and will be overwritten if an HTTP content-length header is encountered during the parsing. If no such header is present in the request, it stays as `-2` with the later result of skipping the creation of an io.Reader. Seems reasonable - no content, no need for a reader? (Note that the presence of content-length triggers body reader creation even if the value is 0.) The fasthttp code replaces the illegal `-2` value with `0` before passing the request up to Versity.So the pathway of this crash is receiving a request with no content-length, which means no body reader is set in the request context structure, but Versity tries to read anyway in the hash routines.This pathology was confirmed by artificially creating a content-length on all incoming packets that didn’t have one, the outcome being that this panic did not occur, although the overall upload still reported failure (I didn’t go into exactly why as the point had been proven).Further AnalysisThe question becomes why there is no HTTP content-length header in this request. The answer to this is that the large transfer is being done using AWS chunking. Wireshark shows the packet structure. There is an initial POST request with content-length zero:
Hypertext Transfer Protocol.. followed soon after by a PUT request with ‘content-encdoding: aws-chunked’. According to the AWS documentation, this encoding is used when the size of the upload is not known in advance, and a content-length header is expressly forbidden. AWS supplies other headers with size information, but these are not recognised by fasthttp.
Hypertext Transfer ProtocolThe initial response to the first POST request appears to be identical in both the direct and via nginx traces:
DIRECT:
Hypertext Transfer ProtocolVIA NGINX PROXY:
Hypertext Transfer ProtocolWhat is yet to knowWireshark traces show that the exact same AWS command results in a different upload pattern when connecting directly to Versity (with or without SSL) rather than going through nginx. There is a similar initial POST but followed by a stream of raw TCP with ‘100 - Continue’ responses from the server. There is no PUT request with content-encoding aws-chunked:
Hypertext Transfer ProtocolIt is unclear why AWS switches to the chunked mode when connecting through nginx. It is possible that changes to the nginx config could work around this, however the Versity behaviour remains a problem. At the very least this is a potential DDOS vulnerability. There is no structured exception handling in Versity at all and no defensive coding such as verifying callbacks are non-null before attempting to use them.
CVSS v4.0: 7.7