eczarny/xmlrpc — Issue #11: Support for data streaming
Author: koke
Created: 2011-12-14 02:50:02+00:00
State: open

# The problem

If you're trying to use binary data through XML-RPC, the current implementation does it all in memory.
Trying to upload a 100MB video from an iPhone can be a big problem because of this:

* The current implementation expects a NSArray with the parameters, including data (100MB+ of data)
* It does the Base64 conversion and stores the result in memory (another 100MB * 4/3)

The same goes for parsing XML-RPC responses with big data.

# The proposed solution

I've found that a good solution would be accepting NSInputStream objects inside the parameters, and encode those on the fly.
Here's a working proof-of-concept, which is not ready at all for merge, but it works.
https://gist.github.com/1474978

Since many users won't probably upload big data, maybe it's best to create a XMLRPCStreamedEncoder or similar?

The main change is that XMLRPCEncoder is now a subclass of NSInputStream. Then instead of setting `request.HTTPBody = [encoder encode]` you can set `request.HTTPBodyStream = encoder`.

The network libraries take care of opening the stream and asking for more data, usually in 32KB chunks from what I've seen on iOS.

I still haven't played much with responses since I don't really need them, but it'd go something like:

* We have `[NSURLConnection connection:didReceiveData:]` and `NSXMLParser` doesn't need the full document, so we start parsing the moment we got data
* When a base64 section is found, instead of decoding the whole thing to memory and returning `NSData`, it's stored in a cache/temporal file
* The `XMLRPCResponse` object contains a `NSFileHandle` or some other file representation where the `NSData` would go