twitter/bijection — pull request #174 by miguno (2014-08-11 09:53:59+00:00)
Title: Add compression support to bijection-avro

This API enhancement is backwards compatible.

We add `withCompression()` factory methods to `SpecificAvroCodecs` and `GenericAvroCodecs`, and also adds the following three convenience methods:

* `withBzip2Compression`
* `withDeflateCompression`
* `withSnappyCompression`

(I did not add a `withXzCompression` method as this codec was apparently introduced in Avro 1.7.6, and Bijection is currently still using the older 1.7.5 version.)

**Usage examples**

```scala
// Encode data, no compression (already supported in bijection-avro)
implicit val specificInjection = SpecificAvroCodecs[FiscalRecord]

// Encode data, with Snappy compression (new feature)
implicit val specificInjection = SpecificAvroCodecs.withSnappyCompression[FiscalRecord]
```

Compression in Avro is transparent to readers of the data, which means that there is no change needed on the decoding side of things.

**No compression support added to `toBinary` methods**

Please note that I did not add corresponding compression support to `GenericAvroCodecs.toBinary` and `SpecificAvroCodecs.toBinary` (i.e. the Injection variants that _do not_ embed the Avro schema into the encoded binary data). This is because Avro's API provides [compression only at the file container level (i.e. block compression)](http://grokbase.com/t/avro/user/1349vbdew2/enabling-compression). In other words, without using Avro's `DataFileWriter` class -- which is what Bijection does for `apply` but not for `toBinary` -- we cannot set a compression codec. We can try to work around that limitation, but this would turn `toBinary` into a renamed `apply` method, and it would make the code inconsistent because suddenly `toBinary` would embed the Avro schema into each encoded record (which it does not do in the current code, and which is IMHO the core semantic difference between `toBinary` and `apply`).