Add more application query capabilities

Summary: Make the application query a little more flexible, and formalize the PHID type.

Test Plan: See next diffs.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7201
Add a config setting for storing application settings

Summary:
I'm just going to store application policy settings (like view/edit policy, and default policies for content) in config, because:

  1) We'll need access to it on every page, and Config is "free" since we already pull it.
  2) Building separate storage and transactions seems like overkill, we get less-nice but pretty-reasonable transactions for free with config.
  3) We could easily move it later if this is a bad call.

Also fix some formatting.

Test Plan: See future revisions.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7202
If a user can't see an application, prevent them from using its controllers

Summary:
Ref T603. Broadly, this allows you to implement a policy like "Only users in Engineering can use Differential."

This isn't complete, and there will be a long tail of special cases to deal with. Some examples:

  - If you can't use Differential, should you still be able to attach/detach revisions from tasks?
    - You currently will be able to.
    - This actually seems pretty reasonable.
    - But in other cases it might not be: the "send user a message" action should probably require access to Conpherence.
  - If you can't use Differential, should you still be able to see feed stories about it?
    - You currently will be able to, if you can see the revisions.
    - This seems not-so-reasonable and we should probably lock it down.
  - If you can't use Differential, can users CC you on revisions?
    - Currently, they can, and you can't do anything about it.
    - Probably they shouldn't be able to? This seems challenging to explain in the UI.
  - If you can't use Differential, can you write a Herald rule against it?
    - You currently will be able to.
    - Seems like you obviously shouldn't be able to.
    - I think this is a general issue right now (you can still write Differential herald rules even if you uninstall the application, I believe).

There are probably a few more things I haven't thought of. However, there are a finite number of these things and I suspect there aren't //too/ many more than this -- I can't come up with like 100 of them, and half of the ones above have easy fixes.

Despite the rough edges, I think this accomplishes 95% of what installs expect from it.

Test Plan: Restricted Differential and saw it vanish from the home page.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7203
Add some more policy strings

Summary: Provides more helpful text, notabl on `bin/policy show`.

Test Plan: Ran `bin/policy show`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7204
Allow application policies to be edited

Summary:
Ref T603. Enables:

  - Application policies can be edited.
  - Applications can define custom policies (this will be used for setting defaults, like "what is the default visibiltiy of new tasks", and meta-policies, like "who can create a task?").

Test Plan: Edited application policies. A future diff does more with custom policies.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7205
Support unlocking applications with bin/policy

Summary: Ref T603. If you get in trouble, `bin/policy unlock PHID-APPS-PhabricatorApplicationDifferential` and such can get you out now.

Test Plan: Unlocked an application.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7206
Fix two excessively aggressive policy checks

Summary:

  - Some applications need public access regardless of policy configuration.
  - The file data endpoint should ignore policies.
Add missing trailing slashes to crumb links in Diffusion

Summary:
In most cases this just makes the URIs more consistent, but it's funky/breakish for SVN repositories which are only partially tracked.

See also T3915, and IRC.

Test Plan:
  - Browsed some repositories, verified URIs generated as expected, with trailing slashes for directories.
  - Verified nothing goofy happened in the extremes (like double slashes on the first crumb).

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7209
Herald - add field + condition for Diffusion Commits for "On autoclose branch"

Summary:
Fixes T1461.

Adds

- FIELD_ALWAYS - now you could add this to a content type to always get notified
- FIELD_REPOSITORY_AUTOCLOSE_BRANCH - solves T1461
- CONDITION_UNCONDITIONALLY - used by these two fields to not show any value for the user to select

Test Plan: made a herald rule where diffs on autoclose branches would get flagged blue. made a diff on an autoclose branch and committed it. commit was flagged!

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T1461

Differential Revision: https://secure.phabricator.com/D7210
Fix controller-level access rules

Summary:
Ref T603. I had to partially revert this earlier because it accidentally blocked access to Conduit and File data for installs without "policy.allow-public", since the applications are available to "all users" but some endpoints actually need to be available even when not logged in.

This readjusts the gating in the controller to properly apply application visibility restrictions, and then adds a giant pile of unit test coverage to make sure it sticks and all the weird cases are covered.

Test Plan:
  - Added and executed unit tests.
  - Executed most of the tests manually, by using logged in / admin / public / disabled users.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7211
Adding support for 'adds' and 'removes' in diff content.

Summary:
Does what it says on the label. We already had 'Any changed file content', now we have 'Any added file content' and 'Any removed file content'.
- There is a bit of copied/pasted code here: I'm open to suggestions on how to refactor it so it's less redundant.
- The wording seems a little awkward, and as @epriestley mentioned in T3829, moved code will be detected less than ideally.

Test Plan: Created Herald Rules, verified via dry run that they were triggered in appropriate situations.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T3829

Differential Revision: https://secure.phabricator.com/D7214
Move "unlisted" apps to Query, use Query for app preferences

Summary:
  - Demagic "unlisted" apps.
  - Respect policies in application visibility settings.

Test Plan: Viewed applications, settings.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7212
Add support for bookmarks in Phabricator emails.

Summary: Right now emails don't include bookmark info (wasn't added in D2897). Lets include it so it's consistent with the web UI.

Test Plan: Inspected code, made sure it matched web UI code. Verified that web UI with these changes was consistent with rendering before refactoring.

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D7215
Prevent administrators from locking themselves out of applications

Summary: Ref T603. This could be a nicer UX, but limit the amount of foot-shooting that users can possibly do. You can still manage if you're really tricky ("Members of project X", then leave the project) but this should make it hard to make a mistake. It seems very unlikely any user ever intends to lock themselves out of an application.

Test Plan: Set an application's view policy to permissive ("Administrators") and nonpermissive ("No One") values. The former were accepted, the latter rejected.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7218
Tighten up some policy interactions in Herald

Summary:
Ref T603. Herald is a bit of a policy minefield right now, although I think pretty much everything has straightforward solutions. This change:

  - Introduces "create" and "create global" permisions for Herald.
    - Maybe "create" is sort of redundant since there's no reason to have access to the application if not creating rules, but I think this won't be the case for most applications, so having an explicit "create" permission is more consistent.
  - Add some application policy helper functions.
  - Improve rendering a bit -- I think we probably need to build some `PolicyType` class, similar to `PHIDType`, to really get this right.
  - Don't let users who can't use application X create Herald rules for application X.
  - Remove Maniphest/Pholio rules when those applications are not installed.

Test Plan:
  - Restricted access to Maniphest and uninstalled Pholio.
  - Verified Pholio rules no longer appear for anyone.
  - Verified Maniphest ruls no longer appear for restricted users.
  - Verified users without CREATE_GLOBAL can not create global ruls.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7219
Add Header to Registration

Summary: Adds an ObjectBox to Phabricator Registration

Test Plan: check logged out page for new header.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: Korvin, aran

Differential Revision: https://secure.phabricator.com/D7223
Modernize "Test Console" and fix a minor display bug with "Always"

Summary:
  - Use the box view in the test console.
  - Let the test console load tasks and mocks. We should move this to the adapters (`canAdaptObject($object)` or something).
  - Fix a minor issue with "Always": hiding the whole cell could make the table layout weird in Safari, at least. Just hide the select instead.

Test Plan:
  - Used test console on task.
  - Used test console on mock.
  - Created (silly) rule with "Always" and also some other conditions.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D7220
Make Herald transcripts policy-aware

Summary:
Ref T603. Herald transcripts potentially leak a bunch of content (task text, revision/commit content). Don't let users see them if they can't see the actual objects.

This is a little messy but ends up mostly reasonable-ish.

Test Plan:
  - Verified that transcripts for objects I couldn't see no longer appear in the list, and reject access.
  - Verified that transcripts for objects in applications I can't see reject access, albeit less gracefully.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D7221
lipsum - tighten up some test data generation

Summary:
maniphest tasks were fataling with priority 0 before making sure to add the return null if new object trick to the maniphest pro editor.

pholio had a problem where if you had no jpegs you were walking off array_rand. tighten the math and then just return a built-in if no uploaded user images could be found. Fixes T3889.

Test Plan: bin/lipsum generate for a few minutes and no errors

Reviewers: epriestley

Reviewed By: epriestley

CC: Korvin, aran

Maniphest Tasks: T3889

Differential Revision: https://secure.phabricator.com/D7222
Stop using loadRelationships in differential related herald adapters

Summary:
Used `DifferentialRevisionQuery` with the relevant `need*()` calls in the test controller.
And started assuming the revision has reviewers and CC phids in `HeraldDifferentialRevisionAdapter`.

Test Plan:
Added herald rules that use revisions (one for revisions another for commit) and reviewers.
Created, accepted and landed a revision that matched the rules and checked all rules were applied.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1279

Differential Revision: https://secure.phabricator.com/D6468

Conflicts:
	src/applications/herald/adapter/HeraldCommitAdapter.php
	src/applications/herald/adapter/HeraldDifferentialRevisionAdapter.php
	src/applications/herald/controller/HeraldTestConsoleController.php