Run the remote image presence checks in parallel (speeds up pushes)

This PR makes all of the requests to check for the image already existing in the registry in parallel, then pushes only those that it needs to one by one. This can cut down times where you are pushing a small number of new layers. For an image with 15 layers this can cut down push times from ~20s to ~6s.

This currently doesn't provide a huge speedup for the docker hub, but will for anything using docker-registry. This is due to the hub responding with a 401 for every check (meaning a push will always be attempted). This does still cut down on the overall time however (~20s to ~17s).

Possible considerations:

* Is this too many requests at once? 
* Do we want to also do the pushes themselves in parallel?
* Are there more requests we can bundle together / streamline?
* I'm pretty new to go, is this a poor way of doing this?

# Examples of output

Current master:

    $ sudo time docker push tutum.co/iancal/touched
    The push refers to a repository [tutum.co/iancal/touched] (len: 1)
    Sending image list
    Pushing repository tutum.co/iancal/touched (1 tags)
    Image 511136ea3c5a already pushed, skipping
    Image df7546f9f060 already pushed, skipping
    Image e433a6c5b276 already pushed, skipping
    Image e72ac664f4f0 already pushed, skipping
    Image 4b7b4bdd15f5 already pushed, skipping
    Image dfa042d4c557 already pushed, skipping
    Image 5ca9e32c43a4 already pushed, skipping
    Image 530a1e076a6f already pushed, skipping
    Image 9b5ea9aa53b9 already pushed, skipping
    Image 8179f0d15d9b already pushed, skipping
    Image c591bbba5ed4 already pushed, skipping
    Image a6ee666e4fff already pushed, skipping
    Image 0f2344772f88 already pushed, skipping
    Image 0881e8acb442 already pushed, skipping
    Image 346a283c1060 already pushed, skipping
    Image 3857b7bcd7e9 already pushed, skipping
    Image feede2781b3d already pushed, skipping
    Image 8f9f59ed1c63 already pushed, skipping
    Image bd59c53042c8 already pushed, skipping
    Pushing tag for rev [bd59c53042c8] on {https://tutum.co/v1/repositories/iancal/touched/tags/latest}
    0.00user 0.01system 0:23.02elapsed 0%CPU (0avgtext+0avgdata 6356maxresident)k
    0inputs+0outputs (0major+2025minor)pagefaults 0swaps

New build:

    $ sudo time docker push tutum.co/iancal/touched
    The push refers to a repository [tutum.co/iancal/touched] (len: 1)
    Sending image list
    Pushing repository tutum.co/iancal/touched (1 tags)
    Image 0f2344772f88 already pushed, skipping
    Image dfa042d4c557 already pushed, skipping
    Image 4b7b4bdd15f5 already pushed, skipping
    Image 346a283c1060 already pushed, skipping
    Image c591bbba5ed4 already pushed, skipping
    Image 5ca9e32c43a4 already pushed, skipping
    Image e433a6c5b276 already pushed, skipping
    Image df7546f9f060 already pushed, skipping
    Image a6ee666e4fff already pushed, skipping
    Image 530a1e076a6f already pushed, skipping
    Image 9b5ea9aa53b9 already pushed, skipping
    Image e72ac664f4f0 already pushed, skipping
    Image 0881e8acb442 already pushed, skipping
    Image 8179f0d15d9b already pushed, skipping
    Image 511136ea3c5a already pushed, skipping
    Pushing tag for rev [346a283c1060] on {https://tutum.co/v1/repositories/iancal/touched/tags/latest}
    0.01user 0.00system 0:05.19elapsed 0%CPU (0avgtext+0avgdata 8488maxresident)k

Checking new layers can still be pushed:
    
    $ sudo time docker push tutum.co/iancal/touched
    The push refers to a repository [tutum.co/iancal/touched] (len: 1)
    Sending image list
    Pushing repository tutum.co/iancal/touched (1 tags)
    Image a6ee666e4fff already pushed, skipping
    Image 8179f0d15d9b already pushed, skipping
    Image 530a1e076a6f already pushed, skipping
    Image 346a283c1060 already pushed, skipping
    Image e433a6c5b276 already pushed, skipping
    Image dfa042d4c557 already pushed, skipping
    Image 0881e8acb442 already pushed, skipping
    Image 0f2344772f88 already pushed, skipping
    Image df7546f9f060 already pushed, skipping
    Image e72ac664f4f0 already pushed, skipping
    Image 9b5ea9aa53b9 already pushed, skipping
    Image 5ca9e32c43a4 already pushed, skipping
    Image c591bbba5ed4 already pushed, skipping
    Image 4b7b4bdd15f5 already pushed, skipping
    Image 511136ea3c5a already pushed, skipping
    Image 3857b7bcd7e9 not pushed, adding to queue
    Image bd59c53042c8 not pushed, adding to queue
    Image 8f9f59ed1c63 not pushed, adding to queue
    Image feede2781b3d not pushed, adding to queue
    3857b7bcd7e9: Image successfully pushed
    feede2781b3d: Image successfully pushed
    8f9f59ed1c63: Image successfully pushed
    bd59c53042c8: Image successfully pushed
    Pushing tag for rev [bd59c53042c8] on {https://tutum.co/v1/repositories/iancal/touched/tags/latest}
    0.00user 0.00system 0:21.15elapsed 0%CPU (0avgtext+0avgdata 8448maxresident)k
    0inputs+0outputs (0major+2043minor)pagefaults 0swaps

OK, sounds good  I'll try moving the push inside the goroutine when I'm
back after Christmas. It should also speed up pushes to the hub :)

On Tuesday, 23 December 2014, Alexander Morozov <notifications@github.com>
wrote:

> In graph/push.go
> <https://github.com/docker/docker/pull/9714#discussion-diff-22227815>:
>
> >  	for _, ep := range repoData.Endpoints {
> >  		out.Write(sf.FormatStatus("", "Pushing repository %s (%d tags)", localName, nTag))
> > -		for _, imgId := range imgList {
> > -			if err := r.LookupRemoteImage(imgId, ep, repoData.Tokens); err != nil {
> > -				log.Errorf("Error in LookupRemoteImage: %s", err)
> > +
> > +		for i, imgId := range imgList {
> > +			go func(i int, imgId string) {
> > +				if err := r.LookupRemoteImage(imgId, ep, repoData.Tokens); err == nil {
> > +					out.Write(sf.FormatStatus("", "Image %s already pushed, skipping", utils.TruncateID(imgId)))
> > +					needsPush[i] = false
> > +				} else {
> > +					log.Errorf("Error in LookupRemoteImage: %s", err)
> > +					out.Write(sf.FormatStatus("", "Image %s not pushed, adding to queue", utils.TruncateID(imgId)))
> > +					needsPush[i] = true
>
> Current implementation with needPush is racy at least. I think if you push
> them in goroutines - code will be much simplier.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/docker/docker/pull/9714/files#r22227815>.
>

-- 
******************************************************************************** 
  
DISCLAIMER: This e-mail is confidential and should not be used by anyone 
who is
not the original intended recipient. If you have received this e-mail in 
error
please inform the sender and delete it from your mailbox or any other 
storage
mechanism. Neither Macmillan Publishers Limited nor any of its agents accept
liability for any statements made which are clearly the sender's own and not
expressly made on behalf of Macmillan Publishers Limited or one of its 
agents.
Please note that neither Macmillan Publishers Limited nor any of its agents
accept any responsibility for viruses that may be contained in this e-mail 
or
its attachments and it is your responsibility to scan the e-mail and 
attachments (if any). No contracts may be concluded on behalf of Macmillan 
Publishers Limited or its agents by means of e-mail communication. 
Macmillan 
Publishers Limited Registered in England and Wales with registered number 
785998 
Registered Office Brunel Road, Houndmills, Basingstoke RG21 6XS   
********************************************************************************