largefiles: avoid redundant "updatelfiles" invocation in "overridetransplant"

At "hg transplant --merge REV", largefiles newly coming from the 2nd
parent (= REV) are marked as "a"(dded) by "patch.patch()", and have to
be marked as "n"(ormal) after commit.

But until changeset 3100d1cbce32, such largefiles were still marked as
"a" unexpectedly even after commit, because no additional entry is
added to filelog of such largefiles and they aren't listed in
"repo[newnode].files()" in this case: "newnode" is one of newly
committed changeset (= result of "repo.commit()").

"updatelfiles" invocation in "overridetransplant" shadows this problem
by forcibly synchronizing lfdirstate to dirstate.

Now, "updatelfiles" invocation in "overridetransplant" is redundant,
because changeset 3100d1cbce32 made "markcommitted" use "ctx.files()"
to get targets of "synclfdirstate" instead of "repo[newnode].files()".
largefiles: update standins only at the 1st commit of "transplant --continue"

Before this patch, "hg transplant --continue" may record incorrect
standins, because largefiles extension always avoid updating standins
while transplanting, even though largefiles in the working directory
may be modified manually at the 1st commit of "hg transplant --continue".

But, on the other hand, updating standins should be avoided at
subsequent commits for efficiency reason.

To update standins only at the 1st commit of "hg transplant
--continue", this patch uses "automatedcommithook", which updates
standins by "lfutil.updatestandinsbymatch()" only at the 1st commit of
resuming.

Even after this patch, "repo._istransplanting = True" is still needed
to avoid some status report while updating largefiles in
"lfcommands.updatelfiles()".

This is reason why this patch omits not "repo._istransplanting = True"
in "overriderebase" but examination of "getattr(repo,
"_istransplanting", False)" in "updatestandinsbymatch".
largefiles: avoid printing messages while transplanting by "_lfstatuswriters"

Putting "lambda *msg, **opts: None" (= avoid printing messages always)
into "_lfstatuswriters" while transplanting makes explicit passing
"printmessage = False" for "updatelfiles()" useless.

This patch also removes setting/unsetting "repo._istransplanting" in
"overridetransplant", because there is no code path referring it.
largefiles: move "copyalltostore" invocation into "markcommitted"

Before this patch, while "hg convert", largefiles avoids copying
largefiles in the working directory into the store area by combination
of setting "repo._isconverting" in "mercurialsink{before|after}" and
checking it in "copytostoreabsolute".

This avoiding is needed while "hg convert", because converting doesn't
update largefiles in the working directory.

But this implementation is not efficient, because:

  - invocation in "markcommitted" can easily ensure updating
    largefiles in the working directory

    "markcommitted" is invoked only when new revision is committed via
    "commit" of "localrepository" (= with files in the working
    directory). On the other hand, "commitctx" may be invoked directly
    for in-memory committing.

  - committing without updating the working directory (e.g. "import
    --bypass") also needs this kind of avoiding

For efficiency of this kind of avoiding, this patch does:

  - move "copyalltostore" invocation into "markcommitted"
  - remove meaningless procedures below:
    - hooking "mercurialsink{before|after}" to (un)set "repo._isconverting"
    - checking "repo._isconverting" in "copytostoreabsolute"

This patch invokes "copyalltostore" also in "_commitcontext", because
"_commitcontext" expects that largefiles in the working directory are
copied into store area after "commitctx". In this case, the working
directory is used as a kind of temporary area to write largefiles out,
even though converted revisions are committed via "commitctx" (without
updating normal files).
test-hup: use ls instead of echo

Having one file entry per line makes it easier to analyse diffs.
transaction: handle missing file in backupentries (instead of using entries)

The case where a backup of a missing file was requested was previously
handled by the 'entries' list. As the 'backupentries' is about to gain
ability to backup files outside of '.hg/store', we want it to be able
to handle the missing file too.

Reminder: using 'addbackup' on a missing file means that such file needs to be
deleted if we rollback the transaction.
transaction: gather backupjournal logic together in the __init__

The initialisation of file-backup related variable were a bit scattered, we
gather them together.
transaction: pass the transaction to 'pending' callback

The callback will likely need to perform some operation related to the
transaction (eg: backing files up). So we better pass the current transaction as
the callback argument. Otherwise callback that needs it has to rely on horrible
weak reference trick.

The first foreseen user of this is changelog._writepending. We would like it to
register the temporary file it create for cleanup purpose.
transaction: pass the transaction to 'finalize' callback

The callback will likely need to perform some operation related to the
transaction (eg: registering file update). So we better pass the current
transaction as the callback argument. Otherwise callback that needs it has to
rely on horrible weak reference trick.

This allow already allow us to slay a wild weak reference usage.
transaction: pass the transaction to 'postclose' callback

This mirrors the API for 'pending' and 'finalize' callbacks. I do not have
immediate usage planned for it, but I'm sure some callback will be happy to
access transaction related data.
transaction: extract backupentry registration in a dedicated function

We are about to use the 'backupentry' mechanism to allow cleaning up
transaction-related temporary files (such as 'changelog.i.a'). We start
by extracting the entry registration into its own method for easy reuse.

At that point, I would like to rename the backup-file related variable to
something generic but I'm a bit short of ideas.
revlog: make a predicate clearer with parens
revlog: store fulltext when compressed delta is bigger than it

This is a very silly case and not particularly likely to happen in the wild,
but it turns out we can hit it in a couple of places. As we tune the storage
parameters we're likely to hit more such cases.

The affected test cases all have smaller revlogs now.
revlog: compute length of compressed deltas along with chain length

In upcoming patches to the revlog, we're going to split up the notions of
bounding I/O and bounding CPU.
revlog: bound based on the length of the compressed deltas

This is only relevant for generaldelta clones.
revlog: increase I/O bound to 4x the amount of data consumed

This doesn't affect normal clones since they'd be bound by the CPU bound below
anyway -- it does, however, improve generaldelta clones significantly.

This also results in better deltaing for generaldelta clones -- in generaldelta
clones, we calculate deltas with respect to the closest base if it has a higher
revision number than either parent. If the base is on a significantly different
branch, this can result in pointlessly massive deltas. This reduces the number
of bases and hence the number of bad deltas.

Empirically, for a highly branchy repository, this resulted in an improvement
of around 15% to manifest size.

--HG--
extra : amend_source : fc7c093f6e7f0670ed1be6bc408869ddfd12642d
remove: move most of the implementation into cmdutils.remove()

This will allow access to the reusable parts from subrepos, similar to add(),
forget(), etc.

--HG--
extra : rebase_source : df7dd3008e664c558395f58eb42c4070f1347d24