merge: split out mostly-non-interactive working dir updates

In a later patch, we'll run these updates in parallel.

--HG--
extra : rebase_source : 1117cd1c2e37d0617dc95c214fd1700cff542c0f
tests: update test output (will be folded into parent)

--HG--
extra : rebase_source : 3d86457c7e34ae54957bd326b55b567253b98a5d
merge: handle subrepo merges and .hgsubstate specially

In an upcoming patch, we will update .hgsubstate in a non-interactive
worker process. Merges of subrepo contents will still need to occur in the
master process (since they may be interactive), so we move that code into
a place where it will always run in what will become the master process.

--HG--
extra : rebase_source : ac573b1d115a3bef3d136a39ecf54d1c0450bc5a
merge: report non-interactive progress in chunks

Instead of a monotonic count, getupdates yields the number of files it
has updated since it last reported, and its caller sums the numbers when
updating progress.

Once we run these updates in parallel, this will allow worker processes
to report progress less often, reducing overhead.

--HG--
extra : rebase_source : 8bf56428051169819a7d408bfa4c74451e20f309
tests: getremove test output changes (fold into previous patch)

--HG--
extra : rebase_source : 038d6ed6a3e535c0ff4f1215bf2272f036bbc4dd
worker: count the number of CPUs

This works on the major platforms, and falls back to a safe guess of
1 elsewhere.

--HG--
extra : rebase_source : 8fa5cfa4e2331b711d2ff05ee0ba2b2b05c68e93
worker: estimate whether it's worth running a task in parallel

Not implemented for Windows yet.

--HG--
extra : rebase_source : 2766d0e8965b8db1c113960585f4d807307c0cc6
worker: partition a list (of tasks) into equal-sized chunks

--HG--
extra : rebase_source : 8a22e212b610e3844a634d77799fa55382510b1b
worker: allow a function to be run in multiple worker processes

If we estimate that it will be worth the cost, we run the function in
multiple processes. Otherwise, we run it in-process.

Children report progress to the parent through a pipe.

Not yet implemented on Windows.

--HG--
extra : rebase_source : 638b187aa4bbbd0ba5817b7bc84c633369fc5e54
merge: apply non-interactive working dir updates in parallel

This has a big effect on the performance of working dir updates.

Here are the results of update from null to the given rev in several
repos, on a Linux 3.2 system with 32 cores running ext4, with the progress
extension enabled.

repo             rev           plain  parallel   speedup
hg               7068089c95a2    0.9       0.3       3
mozilla-central  fe1600b22c77   42.8       7.7       5.5
linux-2.6        9ef4b770e069   31.4       4.9       6.4

--HG--
extra : rebase_source : 3148be691c75fea342483e732b773a6ff76c5040
merge: don't fiddle with name lookups or i18n in hot loops

We perform attribute dereferences and i18n lookups before looping.

--HG--
extra : rebase_source : ea1045ca352f45d155c4388eff731dd1db3e0bdf
Merge crew and main.
profiling: replace '+' markup of nested lines with indentation

The display of nested lines for hg --profile was very non-obvious and made it
look like sort didn't work.

The '+' immediately before CallCount was not related to the CallCount and did
not mean plus in any integer sense.

The '+' before module looked like a part of the module name and not like ascii
art.

Instead we now indent the subordinate module names to clearly show the
structure.
bundlerepo: replace basemap with the base field in the index

Bundle revisions had some info in their fake revlog intries and some info in a
dict with all the bundle revisions. This dict was used to get the stored data
and to distinguish repo revisions from bundle revisions.

Real repo revisions and bundle revisions will now be distinguished by comparing
with the tip revision of the original repo. This reintroduces something similar
to disktiprev which was unused and removed in a928865b4a4f and let that replace
the O(reposize) dict.
spelling: fix some minor issues found by spell checker
hgweb: simplify internal staticfile return codes
hgweb: make the test suite use hgweb in a more WSGI compliant way

- as checked by wsgiref.validate.

This makes sure that we don't optimize hgweb for invalid use cases.
localrepo: create context used for actual commit earlier

localrepo.commit creates a workingctx, calls self.status, does some
munging on the changes status returns, does some validation on those
changes, and then creates a new workingctx from the changes.  This
moves the creation of the new workginctx ahead of some validation,
with the intention of refactoring some of that validation logic into
the workingctx, so that it can be reused elsewhere.
localrepo: use workingctx for validation in commit

This changes localrepo.commit to use the workingctx it creates form
the munged output of localrepo.status while running some precommit
validation.  Specifically, it uses functions that were already present
on the workingctx.  I believe this is a net readabilty improvement,
and that this makes these lines consistent with the refactoring in a
subsequent patch that pulls some of the validation logic into
workingctx so that it can be reused elsewhere.
commit: factor out post-commit cleanup into workingctx

This pulls some of the logic for the cleanup that needs to happen
after a commit has been made otu of localrepo.commit and into
workingctx.  This is part of a larger refactoring effort that will
eventually allow us to perform some types of merges in-memory.