saltstack/salt — pull request #2492 by ryanschneider (2012-11-09 00:24:31+00:00)
Title: Add gzip_compression support to cp.get_file

Another smaller pull request.

We are using the new templated file name feature to push large JSON documents to each host. The files are roughly 1 megabyte each, but gzip down to 32kb (they are pretty printed to be human readable).

I started to manage the gzipping on my end, but then realized it could be useful in salt in general, so here you go. :)

Usage:

```bash
salt "*" cp.get_file salt://vimrc /etc/vimrc gzip_compression=5
```

Added an integration test (with a "big" file larger than 8k, also set the master file_buffer_size to 8192, as far as I could tell the other integration tests weren't pushing anything larger than that) and a section in the docs.

One thing I don't do is cache the gzipped data. So, if you have a large number of minions requesting the same file, there's an unnecessarily high amount of CPU overhead. I might add that down the road if we run into a scenario where we need to use this to distribute a single file to a large number of hosts. Basically, what I would do is keep .gzipped copies of the files in a new subdir of cachedir (and keep the mtime in sync with the src to know when the cache is invalid).

Thanks,
Ryan