docker/docker — pull request #7542 by unclejack (2014-08-12 16:40:04+00:00)
Title: Refactor archive buffering

This PR refactors archive to add buffering to help fix #7291. This PR is the first one of the two PRs required to fix #7291.

The first commit (move some io related utils to pkg/ioutils) moves some Readers and Writers to pkg/ioutils. I've moved these first so that I would avoid adding WriteCloseWrapper to utils/utils.go. I've moved only the io utils which had the least dependencies on other things (e.g. no HTTP related utils were moved and so on).

This PR adds archive/pool to help reuse buffers for reading and for writing. This allows us to use larger buffers by reusing them.

The PR also adds buffering to archive writing, thus helping speed up archive writing and a few other operations.

Open questions:
- [ ] Should we make the pool global and share it across components of Docker? We could reuse it in other places as well.
- [ ] Should we increase the 512K buffers to 1024K to help make downloads even smoother?