OAuth -- add support for Disqus

Summary:
also fix some bugs where we weren't properly capturing the expiry value or scope of access tokens.

This code isn't the cleanest as some providers don't confirm what scope you've been granted. In that case, assume the access token is of the minimum scope Phabricator requires. This seems more useful to me as only Phabricator at the moment really easily / consistently lets the user increase / decrease the granted scope so its basically always the correct assumption at the time we make it.

Test Plan: linked and unlinked Phabricator, Github, Disqus and Facebook accounts from Phabricator instaneces

Reviewers: epriestley

Reviewed By: epriestley

CC: zeeg, aran, Koolvin

Maniphest Tasks: T1110

Differential Revision: https://secure.phabricator.com/D2431
Use one daemon to discover commits in all repositories, not one per repository

Summary:
See D2418. This merges the commit discovery daemon into the same single daemon, and applies all the same rules to it.

There are relatively few implementation changes, but a few things did change:

  - I simplified/improved Mercurial importing, by finding full branch tip hashes with "--debug branches" and using "parents --template {node}" so we don't need to do separate "--debug id" calls.
  - Added a new "--not" flag to exclude repositories, since I switched to real arg parsing anyway.
  - I removed a web UI notification that you need to restart the daemons, this is no longer true.
  - I added a web UI notification that no pull daemon is running on the machine.

NOTE: @makinde, this doesn't change anything from your perspective, but it something breaks this is the likely cause.

This implicitly resolves T792, because discovery no longer runs before pulling.

Test Plan:

  - Swapped databases to a fresh install.
  - Ran "pulllocal" in debug mode. Verified it correctly does nothing (fixed a minor issue with min() on empty array).
  - Added an SVN repository. Verified it cloned and discovered correctly.
  - Added a Mercurial repository. Verified it cloned and discovered correctly.
  - Added a Git repository. Verified it cloned and discovered correctly.
  - Ran with arguments to verify behaviors: "--not MTEST --not STEST", "P --no-discovery", "P".

Reviewers: btrahan, csilvers, Makinde

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T792

Differential Revision: https://secure.phabricator.com/D2430
Improve branch table view with history link & other features

Summary: Add history column & history link per branch on branch table view, also later add some more features like last commit date, etc.

Test Plan: Checked if History column is in browser table view & history links are properly linked.

Reviewers: epriestley

Reviewed By: epriestley

CC: davidreuss, aran, Koolvin

Maniphest Tasks: T1201, T1202, T1200

Differential Revision: https://secure.phabricator.com/D2432
"arc upgrade" (server changes)

Summary: When the server version is ahead of the client version, send a more exciting error!!!

Test Plan: omg~~~

Reviewers: nh, Makinde, btrahan, jungejason

Reviewed By: nh

CC: aran

Differential Revision: https://secure.phabricator.com/D2436
Raise a "you should upgrade your storage" note for a missing database

Summary: We raise an improved exception for missing tables/columns, but not databases.

Test Plan: Hit a "no such database error", got a better error message pointing me at storage upgrades.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2429
Simplify daemon management: "phd start"

Summary:
  - Merge CommitTask daemon into PullLocal daemon. This is another artifact of past instability (and order-dependent parsers). We still publish to the timeline, although this was the last consumer. Long term we'll probably delete timeline and move to webhooks, since everyone who has asked about this stuff has been eager to trade away the durability and ordering of the timeline for the ease of use of webhooks. There's also no reason to timeline this anymore since parsing is no longer order-dependent.
  - Add `phd start` to start all the daemons you need. Add `phd restart` to restart all the daemons you need. So cool~
  - Simplify and improve phd and Diffusion daemon documentation.

Test Plan:
  - Ran `phd start`.
  - Ran `phd restart`.
  - Generated/read documentation.
  - Imported some stuff, got clean parses.

Reviewers: btrahan, csilvers

Reviewed By: csilvers

CC: aran, jungejason, nh

Differential Revision: https://secure.phabricator.com/D2433
Minor, fix OAuth registration, see D2431.

Auditors: btrahan
Minor, instructions suggest "phd list" but really mean "phd status".
"phd list" lists available (not running) daemons.

Auditors: csilvers
Minor, fix obvious typo that I missed in testing somehow.

Auditors: csilvers
Show "show more..." link in "Local Commits" view in Differential

Summary: Javascript! Depends on D2437 to do anything useful.

Test Plan: Clicked "show more", saw more.

Reviewers: csilvers, btrahan

Reviewed By: csilvers

CC: aran

Maniphest Tasks: T1189

Differential Revision: https://secure.phabricator.com/D2438
Improve Diffusion error messages and UI for partially imported repositories

Summary:
  - When you have an un-cloned repository, we currently throw random-looking Git/Hg exception. Instead, throw a useful error.
  - When you have a cloned but undiscovered repository, we show no commits. This is crazy confusing. Instead, show commits as "importing...".
  - Fix some warnings and errors for empty path table cases, etc.

Test Plan:
  - Wiped database.
  - Added Mercurial repo without running daemons. Viewed in Diffusion, got a good exception.
  - Pulled Mercurial repo without discovering it. Got "Importing...".
  - Discovered Mercurial repo without parsing it. Got "Importing..." plus date information.
  - Parsed Mercurial repo, got everything working properly.
  - Added Git repo without running daemons, did all the stuff above, same results.
  - This doesn't improve SVN much but that's a trickier case since we don't actually make SVN calls and rely only on the parse state.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T776

Differential Revision: https://secure.phabricator.com/D2439
Use subtler highlighting for copied and moved code

Summary:
The highlighting is distracting according to Nick Shrock and others.
Real designer, Lee Byron, helped me with this.

It also gives us unagressive target for jumping to the source line in future.

Another feature I will probably implement is highlighting also the source of copies/moves.
I will use the right side of the left column for it.

Test Plan:
Hover copied notifier.
Hover coverage notifier.

I've also checked that this doesn't break our super-flaky old/new code JavaScript detector.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin, leebyron, schrockn

Differential Revision: https://secure.phabricator.com/D2443
Fix warnings for undefined $next_step and $verb

Summary: Revisions don't necessarily have a backing repository, as in "--raw".

Test Plan: Looked at a "--raw" revision in accepted and closed states.

Reviewers: nh, btrahan

Reviewed By: nh

CC: aran

Differential Revision: https://secure.phabricator.com/D2444
Branch view improvements.

Summary:
Sorting by last commit date
Branch view limit to 25 branches
All branches table page with pagination on Git

Test Plan:
* Check repository view for expected behavior on branch table view
* Check all branches page & test pagination

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1200

Differential Revision: https://secure.phabricator.com/D2442
Use transactions in user status Conduit methods

Test Plan:
Add status, verify DB.
Remove status, verify DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2397
Support branch offset/limit in Mercurial

Summary:
  - Support offset/limit as added by D2442, for Mercurial.
  - We just list everything and slice, no clear way to do better and this isn't a major perf issue.
  - No clear way to easily get by-update sorting, we can implement this by looking up revs if someone asks.
  - Bury some of the Git implementation details inside the Git query.

Test Plan:
  - Looked at a Git repo, clicked "View All Branches".
  - Looked at a Hg repo, clicked "View All Branches".
  - Limited page size to 1, made sure it limited properly.

Reviewers: aurelijus, btrahan, vrana

Reviewed By: aurelijus

CC: aran

Differential Revision: https://secure.phabricator.com/D2453
Add a setting to blanket-disable autoclosing revisions in a repository

Summary: For git workflows where developers push personal feature branches to the origin, this is a quick workaround until T1210 is implemented properly. I'll also mention this in D2446.

Test Plan:
  - Committed a revision in an "autoclose" repository; it was autoclosed. Committed a revision in a no-autoclose repository, it was not autoclosed.
  - Edited repositories, saving the "autoclose" setting as enabled/disabled.

Reviewers: aurelijus, btrahan

Reviewed By: aurelijus

CC: aran

Maniphest Tasks: T1210

Differential Revision: https://secure.phabricator.com/D2448
Improve documentation on end-of-life for changes

Summary: This could use some additional explanation.

Test Plan: Read text.

Reviewers: btrahan, aurelijus

Reviewed By: aurelijus

CC: aran

Maniphest Tasks: T1204

Differential Revision: https://secure.phabricator.com/D2446
Fix "phd repository-launch-readonly"

Summary:
  - Delete an unreachable block of code.
  - Pass "--" to terminate overseer args.

Test Plan: Ran "phd repository-launch-readonly", didn't get argument errors out of the daemon.

Reviewers: jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2456
Document transactions in docs

Summary:
Also add some formatting and links.

Also fix test broken by D2393.

Test Plan:
  diviner .

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2461