Fix "arc diff" under Mercurial with a dirty working copy

Summary:
In D962, I switched us from using "hg summary" to "hg id" to find the working
copy revision. However, "hg id" reports the revision with a trailing "+" if the
working copy is dirty, so we fail before hitting the explicit check for this
later.

Trim off the trailing '+' if it is present.

Test Plan:
Ran "arc diff" in a dirty Mercurial working copy and got a good error message
about uncommitted changes.

  ~/repos/hg-working-copy $ arc diff
  WARNING: Mercurial support is largely imaginary right now.
  Usage Exception: You have uncommitted changes in this branch. Commit (or
revert) them before proceeding.

    Working copy: /INSECURE/repos/hg-working-copy/

    Uncommitted changes in working copy
      hello.c

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, Makinde

Differential Revision: 1070
Allow running `arc patch` without authentication.

Summary:
Allow `arc patch` without authentication if Phabricator instance has
'differential.anonymous-access' set to true.

Test Plan:
Set 'differential.anonymous-access' in Phabricator to true and run `arc patch`
without installing a certificate.  `arc patch` should work as expected.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1069
Don't flag "Untracked" files in Mercurial as "Uncommitted"

Summary: We incorrectly add the "Uncommitted" flag to untracked files, which
causes them to raise various prompts and not respect "--allow-untracked".

Test Plan: Ran "arc diff --allow-untracked" in a clean working copy with
untracked files, was not fatally error'd.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, Makinde, epriestley

Differential Revision: 1073
Correctly implement getFileDataAtRevision()

Summary: Apparently I just never tested this or something. Make it work
correctly.

Test Plan: Ran "arc diff" in a Mercurial working copy with a binary file in
outgoing.

Reviewers: Makinde

Reviewed By: Makinde

CC: aran, Makinde

Differential Revision: 1074
Fix "arc lint" exploding on new directories

Summary:
D1061 introduced a 'text file' check, but it fails under SVN for new
directories.

  - Revert D1061 (This reverts commit b2cd18252701be2093b52652fb3d1d94c5df571e.)
  - Make getChangedLines() return null to indicate that the operation doesn't
make sense. I think this was the intent of the code in the lint engine.
  - Fix a bug where running "arc lint" on a change in an SVN working copy from a
subdirectory would fail.
  - Fix a bug where warnings with no line information were incorrectly
discarded.

Test Plan:
  - Ran "arc lint" in an SVN working copy with a new directory (no failure).
  - Forced FilenameLinter to always raise a warning. Added a binary file and ran
"arc lint". The warning was reported for the new binary file, a new text file,
and a new directory.

Reviewers: jungejason, andrewjcg, nh, tuomaspelkonen, aran

Reviewed By: andrewjcg

CC: aran, andrewjcg, epriestley

Differential Revision: 1076
Save committed revision id in workflow.

Summary:
Save commited revision id in workflow, so it can be used in Arcanist
didRunWorkflow hook.

Test Plan:
Running getRevisionID on the workflow should return id of the committed
revision.

Reviewers: epriestley, jungejason

Reviewed By: epriestley

CC: aran, epriestley, mareksapota

Differential Revision: 1078
Add very hacky encoding transformation support for arc

Summary:
Adds a secret, undoucmented "encoding" key to ".arcconfig" which makes a very
half-hearted effort to convert encodings. This is probably good enough that
Differential can be used for code review, but there will be issues with 'arc
patch', 'arc export', paste, maybe conduit stuff, Diffusion, and whatever else I
haven't thought of.

This also doesn't store the original encoding so anything converted like this
won't reasonably be able to be made to work with all that stuff in the future.

See T452 for a broader discussion of the issues involved.

Test Plan:
Short circuited the UTF-8 detection to always fail, had my files "converted"
from ISO-8859-1 to UTF-8.

@davidreuss: you can test this by applying this patch to arcanist/, adding
'"encoding" : "ISO-8859-1"' to your .arcconfig, touching some non-ASCII file,
and then running "arc diff".

Reviewers: davidreuss, jungejason, tuomaspelkonen, aran

Reviewed By: davidreuss

CC: aran, davidreuss, epriestley, nshamg123

Differential Revision: 812
Disable color extensions in Mercurial

Summary: Mercurial has a --color flag which disables the use of ANSI colors. Use
it to prevent "hg diff" from giving us colorized diff output.

Test Plan: Ran "hg diff --color never", verified it disabled ANSI color in diff
output.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran

Differential Revision: 1079
Prevent ArcanistBundle from generating patches with overlapping hunks

Summary: We currently may generate patches which have hunks that include overlapping context. Git was okay with this until at least 1.7.3.4, but started rejecting these patches some time later.

We already attempt to detect and prevent this condition, we just don't do a very good job of it. Fix the check so that we avoid generating overlapping hunks.

Test Plan:

  - Cleanly applied troublesome patches under Git 1.7.7.2.
  - Used "arc export --git" to verify patches generate without overlapping sections.
  - Locally changed $context to 2 and 1, verified patch behavior was reasonable.

Reviewers: kdeggelman, bizrad6, jonathanhester, jungejason, nh, tuomaspelkonen, aran

CC:

Differential Revision: 1084
Remove "mercurial support is imaginary" warning from Arcanist

Summary: We ostensibly now support Mercurial at least mostly, so don't
underpromise quite so much.

Test Plan: Ran "arc diff" in a Mercurial working copy without being warned.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, Makinde

Differential Revision: 1085
Update Arcanist README text

Summary: Make this text a little more informative, this stuff is fairly stable
now and has actual documentation available.

Test Plan: Read the document.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, Makinde

Differential Revision: 1086
There's an off-by-one derp issue with this.
Disable "color" extension in Mercurial in an extension-agnostic way

Summary:
In D1079, I added "--color never", but this flag is provided by the "color"
extension, which is why I missed it originally, because it doesn't show up until
you enable that extension. Providing it causes installs which don't have it
enabled (disabled is the default) to fail.

Use "--config" to disable color instead. This sets a configuration setting and
works regardless of whether the color extension is present.

Test Plan: Ran "arc diff" in a mercurial working copy with the color extension
enabled and disabled.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: nh

CC: aran, nh

Differential Revision: 1092
bash completion: work around PHP CLI readline in command substitution

Summary: (At least my build of) PHP tries to do readline magic on the terminal
when invoked from the shell, even inside bash's command substitution operator.
Work around it by piping the output of ##echo## to ##php##, so the input isn't a
terminal.

Test Plan: ##bash$ arc li<TAB>## now gives me possible completions rather than
breaking and inserting a literal tab

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1096
Fix ArcanistBundle to generate disjoint, minimal hunks

Summary: 'patch' chokes on hunks with too much trailing stuff. 'git apply'
chokes on overlapping hunks. Make them both happy. Write some test cases so I
stop breaking this stuff.

Test Plan:
  - Applied a previously-failing patch via SVN.
  - Applied a previously-failing-then-succeeding patch via Git.
  - Ran unit tests.

Reviewers: jungejason, btrahan, nh, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 1099
Check that rev is accepted in arc mark-committed --finalize

Summary:
If arc amend is run on a rev that isn't accepted, it runs arc mark-committed
--finalize. The rev shouldn't be marked as committed if it hasn't been accepted,
so this diff adds in that check.

Test Plan:
Ran arc amend on a rev that hasn't been accepted, checked that it didn't get
marked as completed.

Reviewers: jungejason, epriestley

Reviewed By: epriestley

CC: aran, epriestley, nh

Differential Revision: 1104
make arc issue a warning if file upload fails during diff creation

Summary:
just a little try / catch action in ArcanistDiffWorkflow.

"Ideally, it would be good to flag these changes somehow so that "arc patch" can
issue the inverse warning (e.g., "This patch could not be completely applied
because some binary data was not uploaded.") but that hasn't come up in a real
use case yet."

I think a change w/ filetype of FILE_IMAGE || FILE_BINARY *and* no phid means
that the upload failed so there's no additional flag needed.  (True?)  however,
it would be easy enough to store metadata that explicilty stated whether or not
the file upload succeeded.

(also / related - i looked through the arc patch workflow a bit and i don't
understand how the svn codepath loads up the actual binary files...  for git,
toGitPatch => buildBinaryChange => getBlob is the right path )

Test Plan:
 - set 'storage.mysql-engine.max-size' to 0 in my conf, uploaded diffs with
files
 - noted in differential that it correctly detected images versus binary despite
file upload failing
 - noted in differential that images had some empty UI
 - reverted conf change, uploaded diffs with files
 - noted in differential file showed up
 - ran arc patch with DX, where DX had broken files
 - noted "Downloading binary data...

Exception:
ERR-BAD-PHID: No such file exists."

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1100
Use linear distance instead of square distance in levenshtein

Summary:
We used square distance that optimized for the wrong thing. Making the same
markers spread out instead of being together

Also added a very little cost for switching type. That will make diff types
stick together a bit more

Task ID: #623

Blame Rev:

Test Plan:
Ran my new unit test. And tested a few diff in phabricator

Revert Plan:

Tags:

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1112
Add events framework to Arcanist

Test Plan:
Added a test event listener, added an event dispatch in diff workflow, run the
workflow, the listener was called.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, jungejason, epriestley

Differential Revision: 1102
Allow modification of the svn commit message via an event listener

Test Plan:
Wrote an event listener modifying the commit message and the message was
successfully changed.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, jungejason, epriestley

Differential Revision: 1103