scmutil.addremove: factor out dirstate walk into another function

Upcoming patches will reuse and expand on this function for other purposes.
scmutil.addremove: rename local 'copies' to 'renames'

An upcoming patch will refactor some code out into a method called
_findrenames. Having a line saying "copies = _findrenames..." is confusing.
Besides, 'renames' is a more precise name for this local anyway.
scmutil.addremove: factor out code to find renames

This code will be used in a different context in upcoming patches.
scmutil.addremove: factor out code to mark added/removed/renames

An upcoming patch will reuse this code in another function.
scmutil: add a function to mark that files have been operated on

Several places use scmutil.addremove as a means to declare that certain files
have been operated on. This is ugly because:
- addremove takes patterns relative to the cwd, not paths relative to the root,
  which means extra contortions for callers.
- addremove doesn't make clear what happens to files whose status hasn't
  changed.

This new method accepts filenames relative to the repo root, and has a much
clearer contract. It also allows future modifications that do more with files
whose status hasn't changed.
patch: use scmutil.marktouched instead of scmutil.addremove

addremove required paths relative to the cwd, which meant a lot of extra code
that transformed paths into relative ones. That code is now gone as well.
windows: check target type before actual unlinking to follow POSIX semantics

Creation and writing into target file via vfs (a.k.a opener) is done
after "unlink()" target file, if it exists.

For example, it is assumed that the revision X consists of file 'A',
and the revision Y consists of file 'A/B'. Merging revision X into Y
tries to "unlink()" on directory 'A' of 'A/B', before creation of file
'A'.

On POSIX environment, directories should be removed by "rmdir(2)", and
"unlink(2)" on directories fails. "unlink()" of Mercurial (and Python)
uses "unlink(2)" directly, so unlinking in the merge case above would
fail.

In the other hand, on Windows environment, "unlink()" of Mercurial
tries to rename before actual unlinking, to follow POSIX semantics:
already opened file can be unlinked safely.

This causes unexpected success in unlinking in the merge case above,
even though directory 'A' is renamed to another. This confuses users.

This patch checks whether target is directory or not before renaming,
and raises IOError(errno.EPERM) if so, to follow POSIX semantics.

--HG--
branch : stable
extra : rebase_source : 127d7233b406699617f6dbcbc34e5ea00c2aa223
largefiles: check existence of the file with case awareness of the filesystem

Before this patch, largefiles extension always unlinks largefiles
untracked on the target context in merging/updating after updating
working directory.

For example, it is assumed that the revision X consists of ".hglf/A"
(and "A" implicitly) and revision Y consists of "a" (not ".hglf/A").

In the case of updating from X to Y, largefiles extension tries to
unlink "A" after updating "a" in working directory. This causes
unexpected unlinking "a" on the case insensitive filesystem.

This patch checks existence of the file in the working context with
case awareness of the filesystem to prevent from such unexpected
unlinking.

"lfcommands._updatelfile()" also unlinks target file in the case
"largefile is tracked in the target context, but fails to be fetched".

This patch doesn't apply "repo.dirstate.normalize()" in this case,
because it should be already ensured in the manifest merging that
there is no normal file colliding against any largefiles.

--HG--
branch : stable
extra : rebase_source : c8ded337fceca38128c6c6dc05ae70759346abe5
largefiles: check unknown files with case awareness of the filesystem

Before this patch, largefiles extension checks unknown files in the
working directory always case sensitively.

This causes failure in updating from the revision X consisting of
'.hglf/A' (and "A" implicitly) to the revision Y consisting of 'a'
(not ".hglf/A") on case insensitive filesystem, because "A" in the
working directory is treated as colliding against and different from
'a' on the revision Y.

This patch uses "repo.dirstate.normalize()" to check unknown files
with case awareness of the filesystem.

--HG--
branch : stable
extra : rebase_source : 1b781ab0e2805127289ab40467b0166e0de60afc
manifestmerge: local unknown, remote created: don't traverse symlinks

To figure out what to do with locally unknown files, Mercurial attempts to read
them if they exist. When an attempt is made to read a file that exists but
traverses a symlink, Mercurial aborts.

With this patch, we first ensure that the file doesn't traverse a symlink
before opening it. This is fine because a file being "remote created" means the
symlink doesn't exist remotely, which means it will be deleted in the apply
phase.
blackbox: fix literal block syntax

--HG--
branch : stable
extra : rebase_source : 5cc43d6341fafea05189b31ff58110f410e55981
subrepo: open files in 'rb' mode to read exact data in (issue3926)

Before this patch, "subrepo._calcfilehash()" opens files by "open()"
without any mode specification. This implies "text mode" on Windows.

When target file contains '\x00' byte, "read()" in "text mode" reads
file contents in without data after '\x00'.

This causes invalid SHA1 hash calculation in "subrepo._calcfilehash()".

This patch opens files in 'rb' mode to read exact data in.

--HG--
branch : stable
Merge with stable.
merge with stable