Project: mesos
Issue MESOS-2438: Improve support for streaming HTTP Responses in libprocess.
Currently libprocess' HTTP::Response supports a PIPE construct for doing streaming responses: {code} struct Response { ... // Either provide a "body", an absolute "path" to a file, or a // "pipe" for streaming a response. Distinguish between the cases // using 'type' below. // // BODY: Uses 'body' as the body of the response. These may be // encoded using gzip for efficiency, if 'Content-Encoding' is not // already specified. // // PATH: Attempts to perform a 'sendfile' operation on the file // found at 'path'. // // PIPE: Splices data from 'pipe' using 'Transfer-Encoding=chunked'. // Note that the read end of the pipe will be closed by libprocess // either after the write end has been closed or if the socket the // data is being spliced to has been closed (i.e., nobody is // listening any longer). This can cause writes to the pipe to // generate a SIGPIPE (which will terminate your program unless you // explicitly ignore them or handle them). // // In all cases (BODY, PATH, PIPE), you are expected to properly // specify the 'Content-Type' header, but the 'Content-Length' and // or 'Transfer-Encoding' headers will be filled in for you. enum { NONE, BODY, PATH, PIPE } type; ... }; {code} This interface is too low level and difficult to program against: * Connection closure is signaled with SIGPIPE, which is difficult for callers to deal with (must suppress SIGPIPE locally or globally in order to get EPIPE instead). * Pipes are generally for inter-process communication, and the pipe has finite size. With a blocking pipe the caller must deal with blocking when the pipe's buffer limit is exceeded. With a non-blocking pipe, the caller must deal with retrying the write. We'll want to consider a few use cases: # Sending an HTTP::Response with streaming data. # Making a request with http::get and http::post in which the data is returned in a streaming manner. # Making a request in which the request content is streaming. This ticket will focus on 1 as it is required for the HTTP API.
Story points: 8