Initial commit of twitter-commons.

(sapling split of 8ea556778b879a611a03364d303c6723d254e704)
Many big changes in pants python support along with handy new python libraries this time around!

pants:
  + Fix bang meta-target ~duplicate dep handling
  + Add the ability to override javac compilerargs for compile.
  + Add support for egg dependencies
  + Add pex self-contained binary format generated for python_binary targets
  + Add support for python thrift stub generation
  + Add pylint support
  + Introduce test size layering by convention in BUILD files

jvm:
  + Fix a ZooKeeper client leak when using the get(timeout) API and a timeout occurs
  + Add CompatibilityCodec and support json/thrift binary serialization for ServerSets
  + StateMachine improvements
  + Generalized ObjectSizeCalculator so it correctly works for both 32-bit and 64-bit HotSpot JVM
  + Various checkstyle fixes
  + Add an installable EnumParser for @Cmdline Args
  + Dependency upgrades: slf4j 1.6.1 and guice 3.0

python:
  + Restructure twitter python under (src|tests)/python/twitter
  + Add deprecation decorators
  + Add recordio library for structured log reading and writing
  + Add an options library that supports declaring options at point of use instead of in ~__main__
  + Add glog style logging library

(sapling split of 89ae581d728aecc39bde1e8843209a0998cd2b96)
Replaced inaccurate Apache header with the correct notice reflecting Twitter's original intent to license the BUILD file under the Apache License, version 2.0.

(sapling split of 86ef2948ad2f3c92825f531f84cc92972e665157)
This month brings the introduction of the Twitter Text Processing Library:
  + lightweight annotation framework for text processing based on Lucene's TokenStream API
  + reference implementation of a basic tokenizer for Latin alphabet languages
  + example annotator program illustrating usage

Other changes include:

build:
  + java and python test targets have been normalized into :small, :medium, :large and :all suites

pants:
  + BUILD support for custom artifact urls via jar(..., url = 'url')
  + doc support for javadoc -linkoffline via jar(..., apidocs = 'url')
  + support for OS X Lion in protobuf and thrift generation
  + support for running individual junit test methods via ./pants [target] -Dtest=foo -Dmethods=bar,...,baz
  + support for custom annotation processors via the new build target annotation_processor
  + support for generating python parsers via antlr using the  new build target py_antlr_library
  + (idea|eclipse) properly handle source excludes

jvm:
  args:
    + replace reflections runtime @CmdLine scanning with annotation processor compile time scanning
    + install EnumParser as a default parser for enum types

  inject:
    + unsuppress checkstyle

  thrift:
    + change ThriftFactory default loadbalancing strategy to use MarkDeadStrategyWithHostCheck

  zookeeper:
    + implement basic support for ZooKeeperClient authentication with 1 set of credentials
    + add ACL support for SingletonService and DistributedLock
    + upgrade zookeeper dependency from 3.3.1 to 3.3.3 pick up bugfixes in the client
    + retire com.twitter.common.service.registration in favor of the zookeeper package's ServerSet

python:
  + initial cut at class file parser
  + a python JSON-to-Thrift decoder useful primarily for unit tests

(sapling split of f10bf41c85fda15eab658e9df6296589ddc5284d)
Fix pants doc artifact linking to work with new fully qualified publish.properties format.
Basic Python application framework for Twitter.

(sapling split of f8a404c7d64ba2e352d6f5710fa3411f19beae3c)
This push brings some improvements to the stats package with more flexibility in Percentil and a new MovingWindowDelta Stat.

Also in this push:

build:
  + fixup forked junit vm to inherit the ANT_OPTS jvm flags

pants:
  + fix ide commands to work with scala leaf targets
  + allow java_library and scala_library to float between src and tests
  + allow python_antlr_library to float anywhere in the source tree

(sapling split of b40990c4a113579b9a3d5eab6e78b4eca52f945e)
This month brings the release candidate for a new pants build system and many additions and improvements to the python application framework.

pants:
  + Add support for excluding java or scala from ide configurations
  + Fix idea checkstyle integration to include all suppression files
  + Support forward references in BUILD files
  + Fixup eclipse support so that the generated project recognizes any annotation_processors
  + Add a PANTS_DEV option to run pants straight from the repo sources to ease the pants dev cycle
  + Cache git build info to avoid running expensive git commands on every build
  + Add marker files for checkstyle runs to eliminate unnecessary checkstyle invocations
  + Add a doc target that supports conversion of markdown to html and nesting other doc dependencies

python:
  + Added support for subcommands and ini file based flag defaulting for subcommands
  + Added support for installable modules and provide logging and an http server in this form
  + Added jvm Amount analog along with a metrics module and /vars HTTP handler
  + Add glog style log file rolling
  + Added an Atlassian Confluence wiki posting component
  + Upgrade pytest to 2.6
  + Resource pool implementation
  + Added caching support to pants python builds
  + Basic ServerSet client
  + Improvements to java classfile parser
  + Add a file tailer
  + Support PEX_INHERIT_PATH to allow a pex to inherit the PYTHONPATH

jvm:
  Upgrade to guava 11.0.1, zookeeper 3.3.4 and finaglized thrift compiler

  app:
    + Implement a local service registry for startup actions, use this to boot the HTTP server
    + Several fixes and improvements to ShutdownRegistry ands its integration with AppLauncher
    + ExecutorServiceShutdown to encapsulate an orderly shutdown of an ExecutorService

  args:
    + @Positional Arg<List<T>> is now supported for gathering positional args
    + Added a twitter util Duration parser that uses Amount<Long, Time> syntax
    + Added a RangeParser to parse guava Ranges
    + Added support for registering custom Parsers and Verifiers globally via annotations+apt
    + Add CanExecute Verifier for files

  base:
    + Introduce a CachingSupplier

  objectsize:
    + Now has a public artifact: com.twitter.common#objectsize
    + Refactor to be parameterizable to arbitrary JVM memory layouts by the user

  reflect:
    + Added simple TypeToken based on guice's TypeLiteral but without the dependency burden

  stats:
    + Added ReservoirSampler - an implementation of http://en.wikipedia.org/wiki/Reservoir_sampling
    + Added Elapsed counting stat that reports the time since it was reset.
    + Added a /vars handler that exports data in JSON format and sort /vars text handler output
    + Added per pool gc cycle count and gc time, fd stats and more thread stats to JvmStats
    + Percentile improvements including support for multiple sampling windows

  testing:
    + TearDownRegistry to adapt a TearDownTestCase to an ShutdownRegistry used for shutdowns

  text:
    + Add a TokenStream(De)Serializer for serialization of TokenStreams and arbitrary attributes
    + Better definitions of punctuation
    + Rename PossetiveTokenCombiner to PossessiveContractionTokenCombiner and modify it to combine tokens in contraction form.
    + Implement JapaneseGomokuTokenizer, and remove JapaneseMeCabTokenizer

  util:
    + Utilities for working with temporary files and directories

  zookeeper:
    + The com.twitter.common#zookeeper jar has been broken up into one jar per major component

(sapling split of 9fc05c8cc927ade33cb14ee21d433669af218c91)
Fix add_target to walk the added target and thus find all its internal and external dependencies lke we do for the root set.

(sapling split of 709391881b82b98329aa5a0e00519c0aa57a0ab1)
Fix ivy resolve to handle those cases where no dependencies are resolved.

(sapling split of d5bde6e683cc705f77018d8bcbf05c576bbe4202)
A task to run JVM binaries in pants.

(sapling split of 14f075aa7d86e30be10c86b2f2c74c7b1d568584)
Code review comments addressed.

(sapling split of c72f0acd16be95c7def1704c88408a54c940ad95)
Fixup scala_compile to inject the selected scala runtime into all scala target dependencies

(sapling split of c1e3f65da5690c9a265bae96c070395618e94959)
A pants task to run a scala repl.

(sapling split of 085ce6f855c18f85f9e02d26d2565ad49925702c)
Kill commented out code.

(sapling split of 014b7c577a268fe9066e51c4938477bd7fc0878a)
Split user-supplied jvm-args and app args with shlex to get a more usable command line.

(sapling split of 29cce2ea296bd5fa9bd210894055f2a3e541c2c3)
make scala_library's java_sources field a pants dep instead of globs

(sapling split of cc1f28572035a59793d56c7c591bb467b7d18d6e)
Fix copypasta - pull debug args for jvm_run.py from the jcm config section.

(sapling split of dbcf6cf8cd4575f9bae5820b9236f342874af7c8)
This push brings several improvements to the release candidate for the new pants build system as well as continued improvements in the python application framework and pex support.

pants:
  + Add a dependency tracking java compiler for faster mapping of classfiles to sources.
  + Improve the junit runner to handle scala tests, abstract classes and classes with no tests.
  + Improve scala_library and scala_tests targets to accept java_sources for circular dependencies via pants() target pointers to java_libraries instead of using globs on a rigid parallel filesystem layout.
  + Fixup pants list to work against jar_library and other aggregator/aliasing targets.
  + Add support for generating an ivy resolve report.
  + Add --pex option to pants py and add PEX_MODULE as an invocation option during bootstrapping.
  + Fix pex generation for python library targets that only have resources.

python:
  + Add CherryPy and tornado eggs and upgrade pystachio egg.
  + Implement a TFinagleProtocol.
  + Fixup touch to use safe_open and factor this up.
  + Add parse_date/parse_time to twitter.common.quantity.
  + Add support for quantity __add__/__mul__.

jvm:
  Upgrade to guava 11.0.1, zookeeper 3.3.4 and finaglized thrift compiler

  app:
    + Fix and simplify LocalServiceRegistry and service launching.

  text:
    + Implement CapitalLetterDotTokenCombiner.
    + Implement Traditional2SimplifiedHanTransformer and LuceneSmartChineseTokenizer.

  zookeeper:
    + Handle interrupted exception on blocking get() in ZooKeeperClient.
    + Expose number of partitions in zookeeper/Partition.

(sapling split of 49d8f99f554457cedb9002ba79c51ea3d11c4850)
Fixup pants.ini

(sapling split of 2968de76d5c66d5c086bbc3670c00f289ab2210e)
Nice TTY behavior in pants repl mode.

(sapling split of ce3962d52c9f4ba5dffd1620704501bc40fcf12c)