Fix an XSS hole in Remarkup link handling, see T1601

Auditors: nh
Document %Ls in `csprintf()`

Test Plan: Read.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3156
Fix an issue where messages may not be processed by PhutilConsoleServer after the client exits

Summary:
When we call update(), it may move messages into the read() buffer but we won't process them right now. Process messages after update().

Also improve the correctness of the flush() implementation.

See D2841.

Test Plan: Ran test script with 4-second linter child, got 4 messages on stdout.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3152
Don't use stream_socket_shutdown() in socket channel

Summary: I'm not sure if `fclose()` does everything we need in Zend PHP but this solves all my troubles.

Test Plan:
  $ ./lint_mock.php # with 4 both with and without questions

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3157
Fix an issue where PHP_EOL_COMMENT tokens were discarded on EOF

Summary:
There was no token production rule from state <PHP_EOL_COMMENT> on <<EOF>>.

NOTE: I didn't bump the xhpast version since this is such a minor edge case that it didn't seem worth the trouble.

Test Plan: Unit test now passes with a clean xhpast build.

Reviewers: nh

Reviewed By: nh

CC: aran

Maniphest Tasks: T1599

Differential Revision: https://secure.phabricator.com/D3154
Provide a function for validating translations

Summary:
Based on discussion at T1139#32, we will not provide context to translators.
Thus we should limit the translations to have the same special characters.

This disallows some originals: e.g. 'print <filename>' in text context - one should use 'print <%s>' with `pht('filename')`.
And some translations: e.g. 'me & you' translated as 'me and you'.

We should call this from linter but I don't know how.
We can easily call it from `addTranslations()` in runtime but I'm not sure it's worth the overhead.

Test Plan: New test.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1139

Differential Revision: https://secure.phabricator.com/D3164
Provide prompt for console server

Test Plan: Used it in `arc diff`.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3167
Split Remarkup rule for document link from hyperlink

Summary:
We have other rules dealing with hyperlinks (`PhabricatorRemarkupRuleProxyImage`) which collides with one rule for hyperlink:

- If we apply proxy image first then it creates `[[ storedText ]]` which builds an invalid link.
- If we apply hyperlink first then there's nothing left for proxy image.

Test Plan: `[[ http://example.com/a.png | a ]]`

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3191
Add API for getting ANSI status
Allow redirecting output in console

Summary: This will allow us using plain `echo`, `passthru()` and such with console clients.

Test Plan: Called it from console client together with echo.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3208
Add context support to PHP highlighter

Summary:
 - Detect x::y, and mark y as an attribute with context x.
 - Export found contexts as a data attribute.

Test Plan: Load Diffusion. Examine DOM and click on things.

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3214
Make XHPASTSyntaxHighlighter more robust

Summary:
 - Also store symbol name as a DOM attribute, to allow cases where the
   symbol name isn't simply the text content of the `<span>`.
 - Check for interesting symbols first, to allow cases where the symbol
   doesn't have type T_STRING.

These changes support new features internal to Facebook, though they
also seem worthwhile in their own right.

Test Plan: Browse Diffusion. Examine DOM.

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3226
Support self::$foo case

Summary: See T1602#15.

Test Plan: Browse Diffusion. Click on things.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3237
Make error message from STDERR in exec channel

Summary:
If client prints to stderr then we fail horribly.
Our tests print to stderr quite often and there's not much I can do about it.
One example is when we throw in destructor (which is not supported by HPHP).

This is not ideal because message comes out of order.
I'll probably reorder STDERR before any input.

Test Plan:
  fprintf(STDERR, 'Test.'); // From client.

Reviewers: vrana

Reviewed By: vrana

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3213
Enable trace discard mode for overseers

Summary: We enable this for daemons but not for overseers. Overseers should also discard events; this prevents them from buffering a log of indefinite size.

Test Plan:
Ran:

  ./scripts/daemon/launch_daemon.php --trace --verbose --trace-memory PhutilNiceDaemon --conduit-uri=http://local.aphront.com/api/

Observed steady memory usage.

Reviewers: nh, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1636

Differential Revision: https://secure.phabricator.com/D3238
Fix doc links
Provide a wrapper around character conversions

Summary:
  - Centralize the check for the "mbstring" extension.
  - Centralize the check for "user explicitly specified encoding as utf-8 so just no-op".

Test Plan: Test coverage.

Reviewers: davidreuss, vrana, btrahan

Reviewed By: davidreuss

CC: aran

Maniphest Tasks: T452

Differential Revision: https://secure.phabricator.com/D3252
Fix typo in comment
Define and use a const instead of 77

Summary: Instead of randomly calling `exit(77)`, we should use a constant instead.

Test Plan: lint, unit, visual inspection

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3266
fix issue 179 of phabricator; ^ lookbehind is not really required