26684 multipart related

This PR addresses the following:

- Push large attachments as MIME Multipart/Related parts
- Pull attachments as binary HTTP streams separately from document bodies
- Fix some of the interactions between the AttachmentManager and replications, ensuring correct handling of stub attachments revpos such that already known attachments are not needlessly transferred.

Extra PushConfiguration/PullConfiguration properties are introduced:

`PushConfiguration.pushAttachmentsInline`: (default small)
* false = always push multipart
* small = push small attachments inline
* true = always push inline

Currently "small" attachments are defined as being < 65536 bytes.

`PullConfiguration.pullAttachmentsInline`: (default false)
* false = always pull attachments as separate HTTP streams
* true = always pull attachments inline, as base64 data inlined in document body

It may be more efficient to pull attachments inline if it is known in advance that they will be small.

Actually it wasn't as difficult as I thought. Just pushed a change which
should address this.


On Thu, Jun 19, 2014 at 5:20 PM, Tom Blench <tom.blench@cloudant.com> wrote:

> Downloading to a temp file might require a little refactoring, so maybe
> postpone that just to get this merged.
>
>
> On Thu, Jun 19, 2014 at 3:30 PM, Mike Rhodes <notifications@github.com>
> wrote:
>
>> In src/main/java/com/cloudant/sync/replication/BasicPullStrategy.java:
>>
>> > +                    if (!Boolean.parseBoolean(System.getProperty("pull_attachments_inline", "false"))) {
>> > +                        for (DocumentRevs documentRevs : result) {
>> > +                            Map<String, Object> attachments = documentRevs.getAttachments();
>> > +                            for (String a : attachments.keySet()) {
>> > +                                Boolean stub = ((Map<String, Boolean>) attachments.get(a)).get("stub");
>> > +                                if (stub != null && stub.booleanValue()) {
>> > +                                    continue;
>> > +                                }
>> > +                                String contentType = ((Map<String, String>) attachments.get(a)).get("content_type");
>> > +                                String encoding = (String) ((Map<String, Object>) attachments.get(a)).get("encoding");
>> > +                                UnsavedStreamAttachment usa = this.sourceDb.getAttachmentStream(documentRevs.getId(), documentRevs.getRev(), a, contentType);
>> > +                                DocumentRevision doc = this.targetDb.getDbCore().getDocument(documentRevs.getId());
>> > +                                this.targetDb.safeAddAttachment(usa, doc, encoding);
>> > +                            }
>> > +                        }
>> > +                    }
>>
>> Did you ever fix this?
>>
>> You could start a transaction before bulkInsert, and the nesting should
>> work fine. However, you will end up holding open the transaction for an
>> indeterminate length of time. I still think this might have to happen in a
>> different order, where the attachment is downloaded and added to the
>> attachments store before the document referencing it is inserted.
>>
>> The long-transaction could be a stop-gap to get this merged, if we make
>> sure we add a task to get the better solution done.
>>
>> —
>> Reply to this email directly or view it on GitHub
>> <https://github.com/cloudant/sync-android/pull/10/files#r13970419>.
>>
>
>