Add an integration test for exopackage JAR structure

Summary: A separate diff broke this behavior in a rebase.  Add a test for it.

Test Plan: Buildbot.
Generalize DexStore a little

Summary:
Previously, each dex store was assumed to have a unique extension.
Generalize this a bit to support dex stores with more flexible naming.
An upcoming diff will use this for raw dex files.

Test Plan: Buildbot.
Make SplitZipStep remember what files it created

Summary:
Previously, it listed the output directory to provide the list of files
to be dexed.  Now, we remember the result from the zip splitter so we
can generate them directly.

The main motivation for this is that we can use an ordered list and
generate the file names with DexStore.fileNameForSecondary based on the
index, instead of relying on specific filenames and extensions.  This is
necessary to support raw dex files, which have a different naming scheme
from our existing secondary dex files.

Test Plan: Buildbot.
Use a real Enum arg for dexCompression

Summary:
With the new enum coercer, we can use less code and make this more
extensible at the same time!

Test Plan: Buildbot.
Remove Froyo support for secondary dex files

Summary:
Facebook apps no longer ship to Froyo, and secondary dex has never been
a supported feature in open-source Buck, so we can just drop this.  Now
we just pass the secondary dex directories as asset directories to
ApkBuilder instead of messing around with zip files.

Test Plan: Built and launched fb4a-noexo.  Buildbot should also be sufficient.
Add support for raw secondary dex files

Summary:
The next version of Android will probably have native support for
secondary dex files.  See details at
https://android-review.googlesource.com/#/c/95521/
Buck should be able to generate APKs that conform to this format.

Instead of secondary dex jars numbered from 1 under the assets path,
this code expects raw secondary dex files numbered from 2 directly under
the APK.

I made this format the default because most open-source projects are
likely to want this style.  Our own builds all pass this argument
explicitly, so the default doesn't affect us.

Note that even for non-pre-dexed builds, we still use our own zip
splitter rather than DX's new built-in one.

Test Plan:
- Integration tests.
- Build FB4A with this and installed it on a Nexus 5 with native multidex
  support.  (It was awesome, BTW.)
Unify some precondition styles

Test Plan: Buildbot.
New class GlobalState to hold process-wide logger data

Summary:
Currently, the LogFormatter class holds on to a process-wide
map of <thread ID: command ID> pairs.

Now that we can have multiple concurrent clients connected, the
previous assumptions that ConsoleHandler had about one console at a
time existing no longer hold, which caused #4985049.

I need the <thread ID: command ID> map in ConsoleHandler, so this
creates a new package-private class GlobalState which holds this
process-wide map (which will be shared between LogFormatter and
ConsoleHandler).

I made the new map a ConcurrentHashMap instead of a SynchronizedMap.
Let me know if this bothers anyone.

I plan on adding a few more maps to this class in a subsequent diff
to complete fixing #4985049.

Test Plan: Tested in subsequent diff.
New class CommandThreadAssociation factored out of LogFormatter

Summary:
Currently, the LogFormatter class contains a helper
class used to manipulate the <thread ID: command ID> map.

Since I need to factor that map out to fix #4985049, it made
sense to factor out the static classes as well; they really
don't belong there any more.

This factors out the CommandThreadAssociation class from LogFormatter
into a new class. A subsequent diff will make use of it.

I removed the Closeable implementation as well, since @natthu and
I discovered that using a try-with-resources block for this is
inappropriate, as we need to log inside a catch block, which depends
on the <thread ID: commmand ID> map.

Depends On D1519662

Test Plan: Tested in subsequent diff.
New class CommandThreadFactory factored out of LogFormatter

Summary:
Currently, the LogFormatter class contains a helper
class used to construct threads and update the global
<thread ID: command ID> map.

Since I need to factor that map out to fix #4985049, it made
sense to factor out the static classes as well; they really
don't belong there any more.

This factors out the CommandThreadFactory class from LogFormatter
into a new class. A subsequent diff will make use of it.

I filed a TODO (#4993059) when I realized CommandThreadFactory
never cleans up after itself.

Depends On D1519662

Test Plan: Tested in subsequent diff.
Refactor LogFormatter, use newly split-out classes, add test

Summary:
This makes use of the refactoring done in the previous
diffs to split out LogFormatter's static inner classes. I
also added a test for LogFormatter.

Depends On D1519662
Depends On D1519666
Depends On D1519668

Test Plan: Unit test included.
Add test utility method FakeOutputStream.getLastFlushSize()

Summary:
I need to write a test which checks for flushing.
This updates FakeOutputStream to add a method which tells us
the size of its buffer at the last time flush() was called.

Test Plan: Used in subsequent diff's test.
Properly support multiple OutputStreams in ConsoleHandler

Summary:
ConsoleHandler had a few nasty bugs that were revealed
once @jimp enabled multiple concurrent "read-only" commands like
`buck audit classpath`.

1) The most recently registered buck client was the only one which
received console output. Any earlier clients stopped receiving crucial
output (like exceptions raised inside Buck).

2) When I fixed this, I realized we had to filter log messages so only
ones created on threads for the command ID created for the client
should by emitted.

3) While testing multiple concurrent consoles, I realized whenever
we reloaded LogManager's configuration, it tore down our ConsoleHandler
and all its state, so I factored out its state, including the log
level override for --verbose=10.

4) I also found that we never flushed the console when error messages
were sent, so they were batched up and flushed at the end.

5) I also found that Java 7 try-with-resources blocks clean up the
resources *before* any catch or finally blocks are invoked, which
means the top-level block in Main to run the command, catch and log
exceptions, and exit prematurely unregistered the <thread ID: command
ID> map, so the log was missing the command ID.

To fix all these, I added two more maps to GlobalState to hold
<command ID: output stream> pairs and <command ID: log level override>
pairs, used them in ConsoleHandler, and added lots of tests.

Depends On D1519662
Depends On D1519666
Depends On D1519668
Depends On D1519676
Depends On D1519677

Test Plan:
Unit tests included. Manually tested by running:

  buck audit classpath //lib:ddmlib
  buck audit classpath //lib:ddmlib --verbose=10

from two concurrent consoles, confirming the verbose output
only went to one.

Manually induced an exception to be thrown in `AuditCommandRunner`.
Repeated manual test. Confirmed the exception was printed separately
to each console.
fix confusion between instance of AppleBundle and AbstractAppleNativeTargetBuildRule

Summary: fix confusion between instance of AppleBundle and AbstractAppleNativeTargetBuildRule

Test Plan: buildbot
Allow numbers in flavors

Summary:
This is useful when using flavors to namespace build rules (e.g.
when using the input source name as a flavor, which may contain
numbers).  However, this will likely also be useful if/when flavors
are used more prevalently (e.g. x86_64).

Test Plan: included test
Add `extendFlavoredBuildTarget` method

Summary:
This is similar to `createFlavoredBuildTarget` but accepts build
targets which are already flavored, in which case it creates and
uses a new flavor formed by concatenating the original flavor and
the new flavor.  This is useful for flavored target nodes (e.g.
C++ thrift support, C++ debug/optimized build mode support) which
uses flavors to name the potentially many build rules it creates
(e.g //foo:bar#cpp-libbar.a).

Test Plan:
- included tests
- used in and thrift C++ support to name build rules
Provide a way to explicitly set the `BuildRuleSourcePath` path

Summary:
This diff adds a new constructor to ##BuildRuleSourcePath## so that
the resolved path can be explicitly set, rather than using the output
path of the build rule that is wrapped.  This is useful for build
rules which generate multiple output sources (e.g. thrift, lex, yacc).

Test Plan:
- included test
- Used with C++ lex, yacc, and thrift support, as each of these produce
combinations of C++ sources and headers.
Fix naming of C/C++ archives

Summary: Don't include the flavor in the basename of the archive.

Test Plan: C/C++ tests
Improve native linkable support

Summary:
Adds support for setting up a standalone build rule for linking
native linkable inputs using C/C++ wrapping the linker.  The main
entry point is the `createNativeLinkableBuildRule` function, which
takes the original build params coming from the target graph and
sets up a native linkable which trims dependencies to only those
either contributing to the link via the top-level object inputs or
specified via `NativeLinkableInput` from `NativeLinkable` deps (e.g.
C++ library rules producing archives).

Test Plan: Added tests
Evolve C/C++ preprocessor support

Summary:
Support:
- Add the concept of a `CxxPreprocessorDep`, which models depdencies
which contribute to preprocessing of dependent rules (e.g. the C/C++
source form one C/C++ library depending on the headers from other
C/C++ libraries).
- Add a helper method to walk dependencies and collect all
`CxxPreprocessInput` classes from `CxxPreprocessorDep` dependencies.
- Add helper methods to setup build rules to manage headers from
C/C++ libraries, via setting up a symlink tree to set the headers
up in the correct namespace, and another rule to track changes to
header file locations and contents.

Test Plan: included tests