Add PytestTestEngine
Make arc patch slightly better about submodules

Summary:
Ref T3776, Ref T479. Say you have some DN, with a submodule X@Y. Later, X@Z in your working copy / repo. If you run arc patch DN, you'd end up with a dirty working copy claiming that X@Z was wrong and it should be X@Y.

To fix, basically run 'submodule init' and 'submodule update'. This makes it so after "arc patch" if you run "git status" it looks clean.

Gross part though now is if you then "git checkout master" you'll have a dirty checkout the other way. I think this is better though.

Test Plan: made a new repository where I added libphutil @ X, did some work (DX), then made libphutil @ y. When I arc patch'd DX, things looked good!

Reviewers: epriestley

Reviewed By: epriestley

CC: csilvers, Korvin, aran

Maniphest Tasks: T479, T3776

Differential Revision: https://secure.phabricator.com/D6837
Support PHPCS as a `.arclint` linter

Summary:
Ref T3186. Ref T2039. Ref T3771. A few effects here:

  # Expose PHPCS as a `.arclint` linter.
  # Turn PHPCS into an ArcanistExternalLinter linter.
  # Add test coverage for PHPCS.
  # Add a `severity.rules` option to `.arclint`. Some linters have very explicit builtin severities ("error", "warning") but their meanings are different from how arc interprets these terms. For example, PHPCS raises "wrong indentation level" as an "error". You can already use the "severity" map to adjust individual rules, but if you want to adjust an entire linter it's currently difficult. This rule map makes it easy. There's substantial precedent for this in other linters, notably all the Python linters.

For `severity.rules`, for example, this will turn all PHPCS "errors" into warnings, and all of its warnings into advice:

      "severity.rules" : {
        "(^PHPCS\\.E\\.)" : "warning",
        "(^PHPCS\\.W\\.)" : "advice"
      }

The user can use `severity` (or more rules) to get additional granularity adjustments if they desire.

Test Plan: https://github.com/epriestley/arclint-examples/commit/5bb919bc3a684d6654dcc28622de9a29ec08307d

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, ajtrichards

Maniphest Tasks: T2039, T3186, T3771

Differential Revision: https://secure.phabricator.com/D6830
Fixup lint testing for changes in D6798

Summary:
We have some linters that trigger based on the path name
in the tree (some rules apply in some dirs and not others).
The changes in D6798 caused all the paths to appear to be outside
the tree, so allow for passing a fake through from those test cases
that are sensitive to this.

We also have a test for the copyright linter, and that needs to read
settings from the .arcconfig file.  The change to faking a working
copy meant that this config option was effectively unset, so add a way
to pass the entire arcconfig through from the tests that need it.

Lastly, the logic to skip deleted files needs to be special cased
when we're faking paths like this: if we've added data for a file
in the testable engine, we should also consider that file as existing.

Test Plan:
`arc unit --everything` here, and passing our tests in
our repo over there.

Reviewers: epriestley, mareksapota

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6841
Don't error on first run of arc diff

Summary:
When running arc diff in a repository that supports commit ranges, it is
possible that the setting for the default relative commit hasn't been set.
If this is the case, the user will be prompted. This change makes sure that
the prompt happens (and thus the setting is set) before we run the
background lint and unit runs.

Test Plan:
```
rm .git/arc/default-relative-commit
arc diff
```

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T2351

Differential Revision: https://secure.phabricator.com/D6854
Tighten up "arc land"

Summary:
Make sure on failure (restoreBranch()) we call `git submodule update --init --recursive` to handle all those purdy submodules. For the pushing step, wrap the push commands in the try / catch block so everything gets cleaned up nice if there's failure. BONUS - add --recursive to arc patch workflow to so nested submodules work correctly. (Crazy git users)

Fixes T3407, T2945.

Test Plan: I wasn't sure how to simulate a good "push" failure but I think this should work.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T2945, T3407

Differential Revision: https://secure.phabricator.com/D6885
Fix arc + svn + delted binary file with properties + image heuristic

Summary:
See IRC. This fixes an issue when deleting an SVN file that ends with one of the extensions in the regexp, which may only affect newer versions of SVN.

Possibly we shouldn't have this heuristic, or should move it elsewhere or make it more explicit, but at least stop it from being broken for now.

Test Plan: Ran `arc diff --only` in a working copy with a deleted binary file ending in ".jpg".

Reviewers: btrahan, nh

Reviewed By: nh

CC: aran, mbishopim3

Differential Revision: https://secure.phabricator.com/D6893
List all DOM extension classes as extension classes for static analysis purposes

Summary: See D6895. We currenty whitelist one of the DOM classes, but should whitelist them all. Classes not in this whitelist raise warnings when users who don't have the corresponding extensions installed lint code. We don't want to require you to install every extension just to send diffs, so we whitelist stuff that we know would exist if you did install everything.

Test Plan: bwahaha

Reviewers: btrahan, Firehed

Reviewed By: Firehed

CC: aran

Differential Revision: https://secure.phabricator.com/D6896
Merge pull request #90 from dcramer/pytest

Add PytestTestEngine
[lint][spelling] delimeter -> delimiter

Summary:
I got bit by this one earlier (I swear I thought it was delimeter),
so I figured I would add it to the spelling data lint rule.

Test Plan:
changed an instance of delimiter to delimeter, ran `arc lint` fixed the
typo.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6936
Fix an issue with PHPCS output formatting

Summary:
See <https://github.com/facebook/arcanist/issues/102>. PHPCS changed its output format sometime between 1.4.6 (stable) and 1.5.0RC3.

Add a "no errors" test and make the linter work on both versions.

Test Plan: Ran `arc unit` on PHPCS 1.5.0RC3.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6926
parse ls-tree with git-submodule correctly

Summary:
git ls-tree gives type 'commit' for submodules.

This code-path is only used when a binary file is present in the diff.

Test Plan: arc diff with a binary file

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6943
Ignore errors from "svn upgrade" in unit tests

Summary: See discussion in D6893. Different versions of `svn` do different stuff, just ignore any possible error here.

Test Plan: Ran unit tests.

Reviewers: andrewjcg, btrahan

Reviewed By: andrewjcg

CC: aran

Differential Revision: https://secure.phabricator.com/D6946
Correct capitalization of "new" in lint

Summary: Automatically correct `New` to `new` in lint.

Test Plan: Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6948
Pluralize add files questions

Test Plan: Saw 'Do you want to add these files to the commit?'

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D6991
Perform case-insensitive check for PHPUnit test finder to not return original file

Summary: PhpunitTestEngine incorrectly included some source files as tests when run on a case-insensive fiesystem. This fixes that behavior. It could introduce problems for case-sensive users, but it's extremely unlikely to be an issue in practice (who would put a file in both Tests/ and tests/ and expect different results for the two?)

Test Plan: arc unit before and after change on OS X. Stubs and autoloaders in .../tests/... directories are no longer picked up.

Reviewers: epriestley

Reviewed By: epriestley

CC: aurelijus, Korvin, aran

Differential Revision: https://secure.phabricator.com/D7048
Prevent detailed code coverage from accessing missing report.

Summary:
Fixes T3856.

Detailed code coverage iterates over all of the files with changes in the current repository, however the code coverage tool might not generate a report for all changed files in the repository, and this would result in an undefined index error.

As an additional bonus, since changes to binary files won't be reported by code coverage tools, this also prevents binary data from being outputted to the console.

Test Plan:
Change a binary file in a repository and run a code coverage tool.  The binary file should be reported as 0% code coverage, but the file contents should not be rendered to the console.

Change a code file that is not covered by a code coverage tool and run code coverage.  The file should be reported as 0% code coverage, and the file contents should not be displayed.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T3856

Differential Revision: https://secure.phabricator.com/D7051
Add C# tools and xUnit unit test engines.

Summary:
This implements Arcanist support for the xUnit testing framework.  It also supports code coverage by way of `cstools` (which is something I've written).

The unit test support works under both Linux and Windows, while the code coverage support has only been tested under Linux (one would assume it would also work under Windows given that Windows has a super-set of functionality in the C# world).

The Arcanist support assumes that the directory layout will be something like:

  * MyProject
  * MyProject.Tests

When files are changed in either MyProject or MyProject.Tests, it causes MyProject.Tests to be built and the xUnit runner to be executed on the resulting binary.

Test Plan: I guess if really wanted to, you could create a C# project in MonoDevelop, set up `.arcconfig` to point to this unit test engine, and download and build xUnit and cstools.  Run `arc unit --coverage` to see the results of your unit test coverage.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T3859

Differential Revision: https://secure.phabricator.com/D7058
Fix an issue where method prototypes have different signatures

Summary: PHP whines about this:

> [2013-09-23 08:09:22] ERROR 2048: Declaration of CSharpToolsTestEngine::loadEnvironment() should be compatible with XUnitTestEngine::loadEnvironment($config_item = 'unit.xunit...') at [/INSECURE/devtools/arcanist/src/unit/engine/CSharpToolsTestEngine.php:13]

Auditors: jamesr, btrahan
Add `https.blindly-trust-domains` to Arcanist

Summary:
This allows users to specify that they want to implicitly trust their weird self-signed certificate without verification.

This can either be specified per user (which will make it apply every time the user runs `arc`, in any project):

  arc set-config https.blindly-trust-domains '["example.mycompany.com"]'

...or added to a `.arcconfig` file (which will make it apply to every user who runs `arc` in that project):

  "https.blindly-trust-domains" : ["example.mycompany.com"]

Depends on D7130.

Test Plan: Tweaked config and verified this setting sends HTTPSFuture down the right branch.

Reviewers: btrahan

Reviewed By: btrahan

CC: hlau, aran

Differential Revision: https://secure.phabricator.com/D7131