Make ApplicationTransactions preview -> submit flow less janky

Summary: ...also make it so in Pholio when you add an inline comment the preview refreshes. Fixes T2649.

Test Plan: played around in pholio leaving commentary. noted that a new inline comment would refresh the preview.

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2649

Differential Revision: https://secure.phabricator.com/D6267
Add DoorkeeperExternalObject

Summary:
Ref T2852. This table holds data about external objects and allows us to write edges to them.

Objects are identified with an `<applicationType, applicationDomain, objectType, objectID>` tuple. For example, Asana tasks will be, e.g., `<asana, asana.com, asana:task, 93829279873>` or similar.

Test Plan: Ran storage upgrade.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6271
Add DoorkeeperObjectRef, DoorkeeperBridge, DoorkeeperBridgeAsana

Summary:
  - `DoorkeeperObjectRef` is a convenience object to keep track of `<applicationType, applicationDomain, objectType, objectID>` tuples.
  - `DoorkeeperBridge` provides pull/push between Phabricator and external systems.
  - `DoorkeeperBridgeAsana` is a bridge to Asana.

Test Plan:
Ran this snippet and got a task from Asana:

{P871}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6273
Add remarkup support for Asana URIs

Summary:
Ref T2852. Primarily, this expands API access to Asana. As a user-visible effect, it links Asana tasks in Remarkup.

When a user enters an Asana URI, we register an onload behavior to make an Ajax call for the lookup. This respects privacy imposed by the API without creating a significant performance impact.

Test Plan: {F47183}

Reviewers: btrahan

Reviewed By: btrahan

CC: chad, aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6274
Add `bin/auth refresh` for debugging OAuth token refresh issues

Summary: Ref T2852. Provide a script for inspecting/debugging OAuth token refresh.

Test Plan: Ran `bin/auth refresh` with various arguments, saw token refreshes.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6276
Provide an auto-refresh mechanism for OAuth providers to deliver fresh tokens

Summary:
Ref T2852. Give OAuth providers a formal method so you can ask them for tokens; they issue a refresh request if necessary.

We could automatically refresh these tokens in daemons as they near expiry to improve performance; refreshes are blocking in-process round trip requests. If we do this for all tokens, it's a lot of requests (say, 20k users * 2 auth mechanisms * 1-hour tokens ~= a million requests a day). We could do it selectively for tokens that are actually in use (i.e., if we refresh a token in response to a user request, we keep refreshing it for 24 hours automatically). For now, I'm not pursuing any of this.

If we fail to refresh a token, we don't have a great way to communicate it to the user right now. The remedy is "log out and log in again", but there's no way for them to figure this out. The major issue is that a lot of OAuth integrations should not throw if they fail, or can't reasonably be rasied to the user (e.g., activity in daemons, loading profile pictures, enriching links, etc). For now, this shouldn't really happen. In future diffs, I plan to make the "External Accounts" settings page provide some information about tokens again, and possibly push some flag to accounts like "you should refresh your X link", but we'll see if issues crop up.

Test Plan: Used `bin/auth refresh` to verify refreshes. I'll wait an hour and reload a page with an Asana link to verify the auto-refresh part.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6280
Show more information about OAuth tokens in the Account Settings -> External Accounts screen

Summary:
Ref T1536.

  - Allow providers to customize the look of external accounts.
  - For username/password auth, don't show the account view (it's confusing and not useful).
  - For OAuth accounts, show token status.

Test Plan:
{F47374}

{F47375}

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6289
Add a "refresh" action for external accounts

Summary:
Ref T1536. This is equivalent to logging out and logging back in again, but a bit less disruptive for users. For some providers (like Google), this may eventually do something different (Google has a "force" parameter which forces re-auth and is ostensibly required to refresh long-lived tokens).

Broadly, this process fixes OAuth accounts with busted access tokens so we can do API stuff. For other accounts, it mostly just syncs profile pictures.

Test Plan:
Refreshed LDAP and Oauth accounts, linked OAuth accounts, hit error conditions.

{F47390}
{F47391}
{F47392}
{F47393}
{F47394}
{F47395}

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6290
Add "invisible" styling/config to Phabricator

Summary: Ref T3322. Depends on D6297. Here are some Phabricator tweaks to complment D6297.

Test Plan: {F47522}

Reviewers: garoevans

Reviewed By: garoevans

CC: aran, chad

Maniphest Tasks: T3322

Differential Revision: https://secure.phabricator.com/D6298
Make ignored setup issues grey

Summary: When I ignore setup issues, I want them to look dealt with, and keep yellow for new ones. Also updated callout colors.

Test Plan: Ignored a number of issues.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6300
Fix issue where "phabricator" disqus account is used by default and not overridable

Summary: Currently setting "disqus.shortname" via config isn't actually used in the instantiation of disqus. This fix uses the shortname configured.

Test Plan: Open a phame post with disqus enabled and a shortname other than "phabricator" specified.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6310
Push feed publishing deeper into the task queue

Summary:
Ref T2852. I want to model Asana integration as a response to feed events. Currently, we queue one feed event for each HTTP hook.

Instead, always queue one feed event and then have it queue any necessary followup events (now, http hooks; soon, asana).

Add a script to make it easy to reproducibly fire feed event publishing.

Test Plan:
Republished a feed event and verified it hit configured HTTP hooks correctly.

  $ ./bin/feed republish 5765774156541908292 --trace
  >>> [2] <connect> phabricator2_feed
  <<< [2] <connect> 1,660 us
  >>> [3] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [3] <query> 595 us
  >>> [4] <connect> phabricator2_differential
  <<< [4] <connect> 760 us
  >>> [5] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [5] <query> 478 us
  >>> [6] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [6] <query> 449 us
  >>> [7] <connect> phabricator2_user
  <<< [7] <connect> 1,062 us
  >>> [8] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [8] <query> 540 us
  >>> [9] <connect> phabricator2_file
  <<< [9] <connect> 951 us
  >>> [10] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [10] <query> 498 us
  >>> [11] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [11] <query> 507 us
  Republishing story...
  >>> [12] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [12] <query> 685 us
  >>> [13] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [13] <query> 489 us
  >>> [14] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [14] <query> 512 us
  >>> [15] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [15] <query> 601 us
  >>> [16] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [16] <query> 405 us
  >>> [17] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [17] <query> 551 us
  >>> [18] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [18] <query> 507 us
  >>> [19] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [19] <query> 428 us
  >>> [20] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [20] <query> 419 us
  >>> [21] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [21] <query> 591 us
  >>> [22] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [22] <query> 406 us
  >>> [23] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [23] <query> 593 us
  >>> [24] <http> http://127.0.0.1/derp/
  <<< [24] <http> 746,157 us
  [2013-06-24 20:23:26] EXCEPTION: (HTTPFutureResponseStatusHTTP) [HTTP/500] Internal Server Error

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6291
Make EdgeQuery return `array()` for empty data instead of `null`

Summary:
Ref T2852. When we pull edge data and an edge has none, we currently populate `null` in the results. This is inconvenient and makes `idx()`'ing it clumsy. Instead, populate `array()` for empty.

(We've barely used edge data anywhere so far, which is why this hasn't come up before, but I have some use cases for it now.)

Test Plan:
  - Trivial / used in future diff.
  - Verified existing edge data callsites don't care about this API change (there are only 3).

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6293
Add DoorkeeperExternalObjectQuery, expose more ref/import APIs

Summary:
Ref T2852.

  - Broadly, we support "I have a Ref, I need a PHID" well but not "I have a PHID, I need a Ref".
  - Add DoorkeeperExternalObjectQuery, and use it to query ExternalObjects.
  - Allow external objects to be imported by their internal PHIDs. Basically, if we have an edge pointing at an ExternalObject, we can say "load all the data about this" from just the PHID and have it hit all the same code.
  - Allow construction of Refs from ExternalObjects. This makes the "I have a PHID, I need a Ref" easier.

Test Plan:
  - Verified Asana links still enrich properly at display time.
  - Used in future revision.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6294
Fix OAuth token refresh return value

Summary:
Ref T1536. Ref T2852. Currently, after refreshing the token we don't actually return it. This means that code relying on token refresh fails once per hour (for Asana) in a sort of subtle way. Derp.

Update `bin/auth refresh` to make this failure more clear.

Test Plan: Set `force refresh` flag and verified a return value.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1536, T2852

Differential Revision: https://secure.phabricator.com/D6295
Add PhabricatorWorker->log()

Summary:
Ref T2852. Add a `log()` method to `PhabricatorWorker` to make debugging easier.

I renamed the similar Drydock-specific method.

Test Plan:
Used logging in a future revision:

  ...
  <<< [36] <http> 211,704 us
  Updating main task.
  >>> [37] <http> https://app.asana.com/api/1.0/tasks/6153776820388
  ...

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6296
Pull some constants out of the Asana bridge

Summary: Ref T2852. Reduce the number of magical strings in use, and prepare the Asana bridge for eventual workspace/project support (a little bit).

Test Plan: Verified enriched links still work properly.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6301
Fix issue where https is not honoured when loading disqus api

Summary: Currently if disqus is used and a phame post is loaded over ssl, the disqus api is not loaded over https. This fixes that by honouring the protocol being used by the html document.

Test Plan: Open a phame post over https

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6311
Initial Asana sync for Differential

Summary:
Ref T2852. This is highly incomplete but seems structurally sound. Some additional context is available in the Google doc.

  - Add a workspace ID configuration. Without it, nothing else activates.
  - Add a worker which reacts to feed stories.
  - Feed stories about things which aren't Differential objects are ignored.
  - We load the revision, or fail permanently if we can't.
  - We get all the related user PHIDs (author, reviewers, CCs).
  - We check if any of them have linked Asana accounts, or fail permanently if they don't.
  - We check for an "ASANATASK" edge from the revision.
    - If we do not find one, we create a new task.
    - If we do find one, we load the task.
      - If we succeed, we check the chronological key of the most recent synchronized feed story ("cursor").
        - If this story is the same or newer, we update the task to synchronize it to the current state of the revision.
      - If we fail to load the task, we fail permanently ("asana task has been deleted").
  - We then publish the actual story text to the task.

Not in yet:

  - Updating followers requires separate API calls which we don't do yet.
  - No subtasks yet.
  - No sync of open/closed state.

Test Plan: {F47546}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6302
Synchronize review request state to Asana

Summary:
Ref T2852. Depends on D6302. This now creates, destroys, and synchronizes subtasks.

  - After finishing the parent task stuff, we pull a list of all known subtasks.
  - We load all those subtasks.
  - If we fail to load any, we delete their objects and edges on the Phabricator side.
  - Of the remaining subtasks, we find subtasks for users who aren't related to the object any more and delete them in Asana and locally (for example, if alincoln is removed as a reviewer, we delete his subtask).
  - For all the related users, we either synchronize their existing task or create a new one for them.
  - Then we write edges for any new tasks we added.

This doesn't handle a few weird edge cases in any specific way:

  - If a subtask is moved under a different parent, we ignore it.
  - If a new subtask is created that we don't know about, we ignore it.
  - If a subtask we know about is deleted, we just respawn it. This is consistent with "DON'T EDIT THESE". You can force sync to stop by deleting the parent.

Addititionally:

  - Make the "don't edit" warning more compelling and visceral.

Test Plan:
  - Kind of ran it a bit.
  - There are like 3,000 edge cases here so this is hard to test exhaustively.
  - Forced a few of the edge cases to happen.
  - Nothing seems immediately broken in an obvious way?

{F47551}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6303