Ignore EPERM when scanning for packages

We won't make much progress reporting the error back up the chain, so we may as
well carry on!
Two typos
auto merge of #761 : kali/cargo/patch-1, r=alexcrichton
auto merge of #718 : alexcrichton/cargo/issue-715, r=brson

We won't make much progress reporting the error back up the chain, so we may as
well carry on!

Closes #715
Update how cargo talks to the registry

This commit includes a laundry list of updates and tweaks to reflect the current
API of the registry:

* `registry.host` has been renamed to `registry.index`
* New top-level manifest keys are now accepted:
  * `homepage` - url
  * `documentation` - url
  * `repository` - url
  * `description` - a markdown-less blurb
  * `license` - string (verified by the registry on upload)
  * `keywords` - string array
  * `readme` - string pointing at a file
* Authors are now uploaded to the registry
* The upload format to the registry has changed to a body json payload
* Unpacking tarballs respects the executable bit for scripts and such.
* Downloading now follows redirects to go to S3.
* The download URL for a package has changed slightly.
* Verify path dependencies have a version listed when being uploaded
* Rename `upload` to `publish`
* Rename `ops::cargo_upload` to `ops::registry`
* Add a new `registry` package for interoperating with the registry
* Add the ability to modify owners via `cargo owner`
* Add a `readme` key to the manifest, and upload its contents to the registry.
* Add the ability to yank crates and their versions
* When packaging a library, verify that it builds from the packaged source by
  unpacking the tarball and simulate running `cargo build` inside of it.
Remove the hokey add_lockfile_sources

This method shouldn't be necessary as it should be possible to simply add all
sources known in the lockfile to a package registry, and
core::{resolve, registry} should take care of weeding them out if necessary.

In the process of doing so, this actually ends up fixing a problem with
rewriting a dependency to a new one which shares transitive deps. Previously all
the transitive deps were updated, but now the transitive deps remain locked at
where they were previously locked.
Don't seed Registry with known source ids

In the normal case this isn't necessary due to the Registry dynamically
discovering sources when dependencies are queried for. Consequently there's no
need to register all these sources ahead-of-time, and it reduces the cognitive
load when thinking about sources and updates.
Don't de-dup added sources

We have a hash map on the side and a method that already aborts re-adding a
source if it was already added.
Rewrite registry tests for easier mocking and such
Make it easier to map dependencies in a summary

This will hopefully become useful in the upcoming changes to resolve for
lockfiles with the registry.
Restructure resolve/lockfile ops

Move functionality from cargo_fetch and cargo_generate_lockfile into dedicated
lockfile/resolve modules. The plan is to expand the resolve module significantly
to deal with the lockfile that's loaded.
Integrate the lockfile and registry-based deps

This commit radically changes the approach of how lockfiles are handled in the
package registry and how resolve interacts with it. Previously "using a
lockfile" entailed just adding all of the precise sources to the registry and
relying on them not being updated to remain locked. This strategy does not work
out well for the registry, however, as one source can provide many many packages
and it may need to be updated for other reasons.

This new strategy is to rewrite instances of `Summary` in an on-demand fashion
to mention locked versions and sources wherever possible. This will ensure that
any relevant `Dependency` will end up having an exact version requirement as
well as a precise source to originate from (if possible). This rewriting is
performed in a few locations:

1. The top-level package has its dependencies rewritten to their precise
   variants if the dependency still matches the precise variant. This covers the
   case where a dependency was updated the the lockfile now needs to be updated.
2. Any `Summary` returned from the package registry which matches a locked
   `PackageId` will unconditionally have all of its dependencies rewritten to
   their precise variants. This is done because any previously locked package
   must remain locked during resolution.
3. Any `Summary` which points at a package which was not previously locked still
   has its dependencies modified to point at any matching locked package. This
   is done to ensure that updates are as conservative as possible.

There are still two outstanding problems with lockfiles and the registry which
this commit does not attempt to solve:

* Yanked versions are not respected
* The registry is still unconditionally updated on all compiles.
Make SourceKind private

This is a bit of an implementation detail of the `SourceId` and we should be
able to get away without exposing it.
Fix a flaky test relying on nondeterminsm

When updating a source with multiple packages, the registry will lazily discover
both the precise and imprecise versions of the source at some point. Previously
the source would be updated depending on which was discovered first, but this
commit adds logic to understand that if an imprecise version is discovered after
a precise version then the imprecise version should be favored (because it will
always trigger an update).

This needs to understand, however, that if `cargo update --precise` is used that
those sources should not be updated again, so the sources treated in
`add_sources` are considered "special" in the sense that they're locked and will
not be updated again.
Respect yanks in the registry

In general the semantics of a yank are that the code itself is not removed from
the registry, but rather packages are no longer allowed to depend on the
version. A yank is normally done to remove broken code or perhaps secrets, but
actually deleting code means that all packages depending on the yanked version
all of a sudden break. For these reasons a yank does not actually delete code,
but only flags the version as yanked in the index.

Yanked packages are therefore able to be depended upon if a lockfile points at a
yanked version, but are not allowed to become new dependencies of packages.

Implementation-wise, the following changes were made:

* SourceIds originating from a lockfile for registries will have a precise
  version listed (just a generic string "locked")
* Dependencies which use precise source ids are allowed to read yanked versions
* Dependencies without a precise source id are not allowed to use yanked
  versions

When using a lockfile (or a previous instance of resolve), all operations will
rewrite dependencies to have the precise source ids where applicable, meaning
the locked versions have access to yanked versions, but the unlocked versions do
not.
Add a cache to the registry source

This should help avoid hitting disk wherever possible and will also be leveraged
in just a second!
Remove some unused imports
Avoid updating the registry too frequently

This logic is based off the precision of the registry source's id as well as the
dependencies being passed in. More info can be found in the comments.
Document the new manifest keys