eregs/regulations-core — pull request #17 by cmc333333 (2013-08-30 23:52:47+00:00)
Title: Compress encoded fields if they are large enough

The definitions layer was tripping the maximum packet size on mysql. Instead of decreasing the batch size (to ~20 from 100), I figured a better route would be to compress the embedded json. I believe this is required as many regulations are significantly longer than E and will surely trip up the maximum packet size sooner.

Compression only kicks in if the content is > 1000 characters long and if the compression actually benefits us. In my testing, decompression adds roughly 10ms on the largest blob size, which is quickly drown out by HTTP overhead.

Unfortunately, django 1.5 doesn't handle binary fields well, so everything gets base64 encoded.