tags: Factor get_tags out of the _get_tags endpoint

Gettings a nicely-parsed list of tags is useful internally, so pull
the tag-listing logic out into it's own function.  The decorated
endpoint behind GET /v1/repositories/<path:repository>/tags is *not*
useful internally, so prefix that function with an underscore.

To convert the get_tags iterator to a dictionary, I've used a dict
comprehension [1], which landed in Python 2.7 and 3.0.  I couldn't
find any docs on docker-registry's minimal supported Python version,
so I imagine compatibility with ancient Pythons is not high on the
agenda.

[1]: http://www.python.org/dev/peps/pep-0274/
tags: Factor delete_tag out of the _delete_tag endpoint

Deleting a tag (and raising the associated signal) is useful
internally, so pull the tag-deleting logic out into it's own function.
The decorated endpoint behind
DELETE /v1/repositories/<path:repository>/tags/<tag> is *not* useful
internally, so prefix that function with an underscore.
Workaround headers too large problem with Glance v1 api
Switch order of add and pip install

This just updates the Dockerfile to take advantage of the ADD caching
added in 0.7.3 to speed up docker builds of the registry.

Docker-DCO-1.1-Signed-off-by: Phil Kates <me@philkates.com> (github: philk)
Merge pull request #258 from ewindisch/glance-headers-too-large

Workaround headers too large problem with Glance v1 api
added ignore files
Merge branch 'master' of github.com:dotcloud/docker-registry
Merge pull request #257 from philk/use_add_caching

Switch order of add and pip install
The secret_key is not generated automatically.

This "feature" introduced too many configuration bugs. Gunicorn now
set the pass the key automatically in the environment. If the key is not
set, the registry will not start.

Fixes #261

Docker-DCO-1.1-Signed-off-by: Sam Alba <sam.alba@gmail.com> (github: samalba)
tags: Extend delete_tag to remove additional metadata files

Storage.repository_tag_json_path and (if the tag == 'latest')
Storage.repository_json_path are both created by put_tag, so we should
remove them in delete_tag.
storage: Add Storage.repository_path

Besides factoring out a bunch of repetitive

  {self.repositories}/{namespace}/{repository}

construction, this gives us a path to recursively remove when we want
to delete a repository.
signals: Add repository-* signals

These aren't wired up yet; I'll do that in subsequent commits.
index: Trigger repository-created and repository-updated signals

Passing in the parsed JSON image data (ids and checksums).
Use exec to eliminate extra shell processes
Merge pull request #265 from rocketraman/shell-exec

Use exec to eliminate extra shell processes
index: Whitelist 'id' and 'checksum' keys for image data

The image data behind the "repository images" endpoints is documented
as only having these two keys [1], but on my existing docker-registry
instance, I see additional keys:

  # cat /var/lib/docker-registry/repositories/wking/gentoo/_index_images  | jq .
  [
    {
      "id": "9917ba282bd23e78d1caa25aac74fb9d5d305aee10db0ef84f3230ef7fc93c59",
      "Tag": "latest"
    }
  ]

Instead of storing any image-specific data that the user sends up,
this commit restricts the set of allowed keys.  This avoids
interfering with the Server.pullRepository code in the Docker source,
which uses the presence of img.Tag to decide whether or not to
download the image [2].  It also avoids trouble with the Docker server
trying to store auxiliary tags in the hard-coded ImgData struct [3].
The drawback to this extra protection against over-eager clients
pushing undocumented keys is that we'll have to update the whitelist
if new keys are added to ImgData.  This commit also doesn't cull
undocumented keys from existing image index files.  Folks with
existing registries can use jq [4] to remove them by hand.  In POSIX
shell:

  JQ='[.[] | {"id": .id, "checksum": .checksum}] | del(.[][] | select(. == null))'
  for FILE in $(find /var/lib/docker-registry/repositories/ -name _index_images)
  do
    FILTERED=$(jq "${JQ}" < "${FILE}")
    echo "${FILTERED}" > "${FILE}"
  done

[1]: http://docs.docker.io/en/latest/reference/api/index_api/#repository-images
     PUT /v1/repositories/(namespace)/(repo_name)/images
     GET /v1/repositories/(namespace)/(repo_name)/images
     PUT /v1/repositories/(repo_name)/images
     GET /v1/repositories/(repo_name)/images
[2]: https://github.com/dotcloud/docker/blob/v0.8.1/server.go#L1179
[3]: https://github.com/dotcloud/docker/blob/v0.8.1/registry/registry.go#L673
[4]: http://stedolan.github.io/jq/
Merge pull request #262 from dotcloud/fix-secret-key-config

Fix secret key config
Merge pull request #266 from wking/whitelist-image-index-keys

index: Whitelist 'id' and 'checksum' keys for image data
tags: Restore delete_repository URL to the spec value

The rename from

  DELETE /v1/repositories/<namespace>/<path:repository>/

(which matches the spec [1,2]) to

  DELETE /v1/repositories/<namespace>/<path:repository>/tags

(not in the spec) happened in 1026a87 (renaming DELETE call on a
repos, 2013-06-26).  The idea was that if the registry is *not* in
standalone mode and the index-side handling was disabled [2], we'd
still want an endpoint that implemented the registry-side cleanup
[1,3].

This commit restores the original URL (but leaves the function to the
tags module to work regardless of standalone mode), triggers
tag-deleted signals for each tag, removes the remaining repository
data en masse (which handles the index-side removal as well), and
triggers the repository-deleted signal.  It doesn't remove referenced
images, because those images might be required by other repositories.
We'd want some sort of refcounting (number of repositories/tags
requiring this image) to decide when image-removal was allowed.

[1]: http://docs.docker.io/en/latest/reference/api/registry_api/#delete--v1-repositories-%28namespace%29-%28repository%29-
[2]: http://docs.docker.io/en/latest/reference/api/index_api/#delete--v1-repositories-%28namespace%29-%28repo_name%29-
[3]: https://github.com/dotcloud/docker-registry/pull/268#issuecomment-36470080
index: Add '# nopep8' tags to long URL lines

This makes the lines even longer, but avoids tox failures:

  /.../registry/index.py:110:80: E501 line too long (116 > 79 characters)
  /.../registry/index.py:111:80: E501 line too long (112 > 79 characters)

Alternatively, we could ignore E501 entirely (with an entry in
tox.ini's flake8.ignore), but Sam Alba prefers ignoring just those
lines (and not all long lines) [1].

[1]: https://github.com/dotcloud/docker-registry/pull/247#issuecomment-36075734