transaction: drop the third item in `tr.backupentries`

This third item is always None and never used.
transaction: document the contents of `tr.backupentries`

Now that all items are known we can document it.
transaction: mark backup-related attributes private

As the transaction is gaining more functions and attributes, it is important to clarify
what is part of the public API.
transaction: document startgroup and endgroup

These enigmatic methods are only used in repair. We document them to clarify
there purpose and user.
transaction: drop backupentries logic from startgroup and endgroup

The `startgroup` and `endgroup` methods are used in a very specific
context to wrap a very specific operation (revlog truncation). It does
not make sense to perform any other operations during such a "group"
(eg:file backup). There is currently no user of backupfile during a
"group" so we drop the group-specific code and restrict authorized
operations during "group".
transaction: document `tr.add`
transaction: factorise append-only file registration

The addition is done in two different places but differs slightly. We factorise
this addition to ensure it is consistent in all places.

--HG--
extra : amend_source : 7c8f562f9fa0cbd0f0722404d0a0e86e503b8e40
debugrevlog: fix computing chain length in debugrevlog -d

The chain length was computed correctly only when generaldelta
feature was enabled. Now it's fixed.

When generaldelta is disabled the base revision in revlog index is not
the revision we have delta against - it's always previous revision.

Instead of incorrect chainbaseandlen in command.py we are now using two
single-responsibility functions in revlog.py:
 - chainbase(rev)
 - chainlen(rev)
Only chainlen(rev) was missing so it was written to mimic the way the
chain of deltas is actually found during file reconstruction.
revlog: add config variable for limiting delta-chain length

The current heuristic for deciding between storing delta and full texts
is based on ratio of (sizeofdeltas)/(sizeoffulltext).

In some cases (for example a manifest for ahuge repo) this approach
can result in extremely long delta chains (~30,000) which are very slow to
read. (In the case of a manifest ~500ms are added to every hg command because of that).

This commit introduces "revlog.maxchainlength" configuration variable that will
limit delta chain length.
localrepo: rename revlog.maxchainlen to format.maxchainlen

This is more consistent with other option names, as spotted by Pierre-Yves. Thanks!

--HG--
extra : rebase_source : 8d1c508181e6efedd6d1618895b0c0e2c9032f66
context.status: explain "caching reasons" more fully

Where we "load earliest manifest first for caching reasons", elaborate
on what "caching reasons" refers to. Text provided by Matt in
http://thread.gmane.org/gmane.comp.version-control.mercurial.devel/73235/focus=73578.

--HG--
extra : rebase_source : 363abc026681e3ed6721d285b8439c9d31c98395
add: add back forgotten files even when not matching exactly (BC)

I accidentally did 'hg forget .' and tried to undo the operation with
'hg add .'. I expected the files to be reported as either modified or
clean, but they were still reported as removed. It turns out that
forgotten files are only added back if they are listed explicitly, as
shown by the following two invocations. This makes it hard to recover
from the mistake of forgetting a lot of files.

  $ hg forget README && hg add README && hg status -A README
  C README
  $ hg forget README && hg add . && hg status -A README
  R README

The problem lies in cmdutil.add(). That method checks that the file
isn't already tracked before adding it, but it does so by checking the
dirstate, which does have an entry for forgotten files (state 'r'). We
should instead be checking whether the file exists in the
workingctx. The workingctx is also what we later call add() on, and
that method takes care of transforming the add() into a normallookup()
on the dirstate.

Since we're changing repo.dirstate into wctx, let's also change
repo.walk into wctx.walk for consistency (repo.walk calls wctx.walk,
so we're simply inlining the call).

--HG--
extra : rebase_source : 8327c911a25181ec7fc9cdd761d286e3ae794fc4
addremove: add back forgotten files (BC)

After running "hg forget README && hg addremove", README will still be
reported as removed, while "hg forget README && hg add README" adds it
back so it gets reported as clean. It seems like they should behave
the same. Furthermore, it seems like no files should remain untracked
after 'hg addremove && hg commit' (or 'hg commit -A'). For these
reasons, change the behavior of addremove so it does add forgotten
files back.

The problem is with scmutil._interestingfiles(), which reports the
file as removed, so scmutil.addremove() does not add it. Fix by
teaching _interestingfiles() to report forgotten files separately from
removed files and make addremove() add forgotten files back. However,
do not treat forgotten files as sources for rename detection. Note
that since removed and forgotten files are treated the same before
this change, forgotten files were considered sources for rename
detection.

Also update the other caller, marktouched(), in the same way as
addremove().

--HG--
extra : rebase_source : bddc9621ddaa8632691c4eade01a0da79add04d1
sortdict: add iteritems method

Future patches will start using sortdict for log operations where order is
important. Adding iteritems removes the headache of having to remember to use
items() if the object is a sortdict.

--HG--
extra : rebase_source : 75fa9ea4514b8d93777bf68cc748fdab7aad3a2c
sortdict: add insert method

Future patches will allow extensions to choose which order a namespace should
output in the log, so we add a way for sortdict to insert to a specific
location.

--HG--
extra : rebase_source : 057291297378b249f16242301216c845bd47d997
hghave: use a less brittle have-json check

--HG--
extra : rebase_source : 469b3d5706a7a04dee2d82456958a9fee2d7a3a3
run-tests: use a try/except ladder instead of looking for a specific version

This ensures we get json instead of simplejson in as many places as possible.

--HG--
extra : rebase_source : 5442a5a7204ee82dacdaeff6654d15e8ad5884ea