Remove deprecated autoload module from libphutil

Summary:
This has been marked deprecated for like a week so I'm guessing either someone
fixed the use in trunk, it no longer exists, or it'll get fixed once it starts
fataling. :)

Test Plan:
Grepped for uses, none for many weeks.

Reviewed By: jungejason
Reviewers: tuomaspelkonen, jungejason
Commenters: aran
CC: aran, jungejason, epriestley
Differential Revision: 330
Commit flex/bison scanner/parser to the repository

Summary:
Marcel suggested this as a good solution to the tricky install
process here: commit the generated scanner and parser to the repository so
users do not need to install flex and bison in order to build it. This is
explicitly permitted by the flex/bison licenses:

http://www.gnu.org/software/bison/manual/html_node/Conditions.html

Also detect and error/warn about bad versions of flex rather than exploding.

Test Plan:
Ran "make", "make parser scanner", "make cleanall", etc.

Reviewed By: aran
Reviewers: toulouse, jungejason, tuomaspelkonen, aran
CC: aran
Differential Revision: 444
Test Conduit connection before daemonizing if --daemonize and --conduit-uri are
both specified

Summary:
It's fairly common to have setup problems here but they currently occur in the
child and "phd launch xxxx" runs without error.

"phd debug" (D448) helps with this but we can be explicit about testing for this
even in the "phd launch" case.

(I don't reuse $this->conduit because I'm worried about how pcntl_fork() and
sockets may interact given some of my experiences in this area.)

Test Plan:
Took down Phabricator and ran "phd launch nice", got an error message instead of
a clean but doomed-to-failure daemon launch.

Reviewed By: jungejason
Reviewers: aran, jungejason, tuomaspelkonen
CC: aran, jungejason
Differential Revision: 449
Script to detect and show invalid UTF-8 sequences in documents.

Summary:
'arc' is now less forgiving about playing it fast-and-loose with encodings, so
make it easier to find and fix these problems in a codebase with preexisting
issues. See D431.

This script takes a list of files and either does a simple binary test for utf8
validity with the "-t" flag, or shows you exactly what problems exist without
it. The idea is that if you have an existing codebase with a bunch of code and
you want to fix all the UTF-8 problems, you can do something like this:

  find . -type f -name '*.php' -print0 | xargs -0 -n256 ./utf8.php -t

That will give you a list of all the files with problems. Now you can inspect
them individually or in groups and fix the issues:

  ./utf8.php file1 file2 ...

You can also find problems in a diff, or some other piece of command output:

  git diff -U99999 HEAD | ./utf8.php -

Test Plan:
Ran this script on various valid and invalid files.

Reviewed By: jungejason
Reviewers: edward, aran, jungejason, tuomaspelkonen
CC: aran, jungejason
Differential Revision: 433
Revert "Test Conduit connection before daemonizing if --daemonize and --conduit-uri..."

This reverts commit 4aec97acc9c7e4404f223770414e401dc64c5151.
Allow daemon logfiles to be set with --log.
Drop --log argument after evaluating it.
harden_directory.php: reduce footprint of directories by hardlinking them

Summary:
This is a small step toward a world of watirworld / sandcastle / integration
tests / async unit tests / web bisect. Do Sandcastle-style disk-size reduction
via hardlinks.

I've incorporated a couple of tricks and such:

  - Detect the hardlink limit thing and deal with it, I don't remember what
Sandcastle's solution was (presumably something similar?) but this should dodge
that.
  - If possible, use 'git show-tree' instead of hashing files ourselves.
  - If not, use git hashes.
  - Deal with executable bits correctly.
  - Deal correctly with git submodules.
  - Layout the workflow so that the "soft" directory can be used by other
processes and silently (and atomically) becomes the hard directory later.
Ideally this should let us use the "soft" directory immediately, but still
benefit from hardening it.

@slawekbiel, am I missing anything obvious here?

Test Plan:
Created a sandpit with every revision of Phabricator in it. It consumes 37MB of
disk + 3.0MB of hardlinks (for ~650 copies), versus 63MB for a single git
repository.
Made an attempt to verify all the special behaviors (no git, executable,
symlinks, hardlink limit).
I haven't played with the perf on this. It's pretty fast (<<1 second) for
incremental sandpits, although the initial copy of 12,000 files took a little
while.

Reviewed By: slawekbiel
Reviewers: slawekbiel, aran, jungejason, tuomaspelkonen
CC: aran, slawekbiel, epriestley
Differential Revision: 464
Use full "(?P<name>" syntax for named captures in HTTPFuture

Summary:
Some versions of PCRE reject the short "(?<" form and require "(?P<".

Also support use of non-default ports for Conduit.

Test Plan:
Will make cllorca apply this patch and tell me it fixes his problem.

Reviewers: llorca, toulouse
Commenters: llorca
CC: aran, epriestley, llorca
Differential Revision: 495
Add some utf-8 building blocks to libphutil

Summary:
The performance of these is absolutely terrible but I think they work as
advertised, at least. The biggest problem with utf-8 is that PHP support for it
is terrible.

If we hit perf problems with this stuff I'll almost certainly need to bundle a
PHP extension with libphutil. This will open up some opportunities for us but
make HPHP integration and general configuration more difficult. :/

Test Plan:
Ran unit tests.

Reviewed By: jungejason
Reviewers: jungejason, aran, tuomaspelkonen
CC: aran, epriestley, jungejason
Differential Revision: 505
Allow metadata to be exported out of PhutilMarkupEngine

Summary: For mentions, we need to get stuff back out of the engine. Allow rules to set "metadata" which the caller can read after marking up the text.

Test Plan: Built mentions on top of this.

Reviewers: tomo, mroch, jsp, jungejason, aran, tuomaspelkonen

CC:

Differential Revision: 518
Allow CodeBlock in Remarkup to accept some non-PHP default language

Summary:
Turns out not everyone uses PHP?!

Test Plan:
Generated Javelin documentation and got JS highlighting.

Reviewed By: cpojer
Reviewers: cpojer, tomo, mroch, jungejason, aran, tuomaspelkonen
CC: aran, cpojer
Differential Revision: 523
Make libphutil scripts exit with an error code when terminated by an uncaught
exception

Summary:
We install an exception handler but PHP gets all awkward when we do. See
comment. This restores sensible behavior and sanity to the world.

Test Plan:
Ran "phd debug IRC derpderp" and had the bot exit with an error instead of
exiting "successfully".

Reviewed By: codeblock
Reviewers: codeblock
CC: aran, codeblock
Differential Revision: 535
Fix incorrect call to json_encode()

Summary: I confused this with the signature of json_decode() when I wrote this.
PHP 5.2 json_encode() takes only one parameter, and PHP 5.3 json_encode() takes
a flags bitmask, so this call is strictly incorrect on all versions of PHP.
Test Plan: scrutinized http://us.php.net/manual/en/function.json-encode.php
Reviewed By: jungejason
Reviewers: davidreuss, jungejason, tuomaspelkonen, aran
CC: aran, jungejason
Differential Revision: 573
Provide a (mostly) utf-8 aware, smart shorten function

Summary: See T234. In D477 we introduced a textWrap() function that isn't UTF-8
aware; provide one that is.
Test Plan: Ran unit tests.
Reviewed By: cadamo
Reviewers: cadamo, jungejason, tuomaspelkonen, aran
CC: aran, cadamo
Differential Revision: 559
Split filename-based and language-based syntax highlighting APIs

Summary:
The major change here is the introduction of getLanguageFromFilename(). Then I
reorganized some things.

This needs some Phabricator-side changes although it doesn't break anything, it
just disrupts some highlighting.

Test Plan: Highlighted PHP, python, invalid-syntax PHP, and invalid-syntax
python in Paste.
Reviewed By: tuomaspelkonen
Reviewers: tuomaspelkonen, codeblock, jungejason, aran
CC: aran, epriestley, tuomaspelkonen
Differential Revision: 599
Guarded definition for 'idx'.

Summary: HPHP has it builtin.
Test Plan: Tested that the site was still working in my sandbox.
Reviewed By: epriestley
Reviewers: epriestley, jungejason
CC: aran, epriestley
Differential Revision: 640
Correctly language-detect files like "x.y.php"

Summary: The regexp was too greedy.
Test Plan: Unit tests.
Reviewed By: tuomaspelkonen
Reviewers: codeblock, tuomaspelkonen, aran, jungejason
CC: aran, tuomaspelkonen
Differential Revision: 642
Revert the function_exists from 'idx'.

Summary: We won!
Test Plan: sandbox
Reviewed By: epriestley
Reviewers: epriestley, jungejason
Commenters: jungejason
CC: aran, epriestley, jungejason
Differential Revision: 647
Added logging when 'pygmentize' fails.

Summary:
We are having some problems on 3/4 of our phabricator machines. The
pygmentize doesn't seem to work there for some reason. Let's log the
error and figure out what's going on.

Test Plan:
Renamed pygmentize binary on my sandbox and made sure the exception was logged.
Renamed it back and made sure a diff was syntax highlighted correctly.

Reviewed By: jungejason
Reviewers: jungejason, epriestley
Commenters: epriestley
CC: aran, jungejason, epriestley
Differential Revision: 663