[https://github.com/voldemort/voldemort/issues/108] allow getting byte and object arrays using voldemort-shell.sh
Merge pull request #109 from brentdmiller/master

allow voldemort-shell.sh to dump byte & object arrays
Avro build and push support removes azkaban dependancies and refactors
out all classes from batch commons
Refactoring reducer logic
Added support for specifying the key and value field in the avro record
some clean up
Rfactored partitioner and mapper
Rfactoring mapper logic out
Fixed bug in partitioner
Fixed error message in case fetcher raises an exception
Added retry logic to fetcher
Added destination name to the status message in the fetcher
Added copyright and fixed code formatting
Minor cleanup in preparation of making socket checkouts along non-blocking code paths asynchronous.

src/java/voldemort/store/nonblockingstore/ThreadPoolBasedNonblockingStoreImpl.java
- fix missing ns to ms conversion

src/java/voldemort/store/socket/clientrequest/ClientRequestExecutor.java
src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorFactory.java
- remove unnecessary selectionKey from checkTimeout interface

src/java/voldemort/utils/pool/KeyedResourcePool.java
- removed extraneous maxCreateAttempts
- attemptGrow returns true if pool grew
- refactored checkoutOrCreateResource
  - renamed to attemptCheckout
  - attemptCheckout is non blocking
  - attemptCheckout does not check timeouts
- refactored checkout
  - removed unreachable code (only one attempt could ever be made so attempts and maxCreateAttempts are extraneous.
  - made it clearer that exceptions control flow through try block
* even though code looks a lot different, I believe exact
  functionality is preserved except for one thing: the
  possibility of a single additional non-blocking get on the
  pool (resources) when attemptGrow returns true.
Adding a SlowStorageEngine to permit end-to-end testing with slow
servers in a cluster.

src/java/voldemort/store/memory/SlowStorageConfiguration.java
src/java/voldemort/store/memory/SlowStorageEngine.java
- The SlowStorageEngine/Configuration is inspired by the
  SlowStore used in other unit tests. The SlowStorageEngine
  produces delays on a per-operation-type basis. Delays can
  either be concurrent or queued: concurrent delays can overlap
  in time, queued delays occur in serial.

src/java/voldemort/server/VoldemortConfig.java
- Added config options for SlowStorageEngine

test/unit/voldemort/store/memory/SlowStorageEngineTest.java
- Unit test to confirm queued/concurrent delay behavior of a single SlowStorageEngine
he current nonblocking API can actually block if a server slows
down. The server slowdown can result in a socket checkout
blocking because all sockets are exhausted and there are a queue
of threads doing a blockingGet to get the socket. Adding an
integration test, E2ENonblockingCheckoutTest, that demonstrates
this undesirable behavior. Once a fix is in place, this
integration test may become a unit test for the correct behavior.

test/integration/voldemort/nonblocking/E2ENonblockingCheckoutTest.java
- Demonstrates the blocking behavior of the current socket
  checkout code. The integration test sets up a local voldemort
  instance with three servers, one of which is slow. Puts are
  done by two threads to a specific key that uses a fast node as
  the master, then the slow node, followed by the other fast node
  for the parallel puts. The second thread to attempt parallel
  operations blocks upon the first returning the checkout
  socket. This is observed in the console output.

test/common/voldemort/ServerTestUtils.java
- Added another startVoldemortServer method that takes a VoldemortConfig

src/java/voldemort/store/socket/SocketStore.java
- minor fix to logger.debug print out so .toString is not called on null object
Code clean up and refactoring in preparation of adding truly nonblocking requests.

src/java/voldemort/store/socket/SocketStore.java
- Moved logic from request async into ClientRequestExecutorPool::submitAsync
- Also moved private class NonblockingStoreCallbackClientRequest into ClientRequestExecutorPool

src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorPool.java
- added private class AsyncRequestContext to start parceling up request context
- added submitAsync and submitAsyncRequest methods. These currently do exactly what was done by SocketStore:requestAsync
- I.e., I do not believe I have changed the behavior of any of this code. I just moved it to where I need to change it.

src/java/voldemort/utils/pool/KeyedResourcePool.java
- made some private methods protected with the expectation of
  extending this class with a subclass that can queue up async
  requests for resources
- changed many variable names to be more consistent in naming/usage
- refactored attemptCheckout to be simpler; added attemptCheckoutGrowCheckout to parcel up old behavior of attemptCheckout
- Do not believe I changed any behavior with these code changes.
- added getResourcePoolForExistingKey method
  - Throws IllegalArgumentException whenever a non existing pool is requested
  - changed all bare resourcePoolMap.get(key) accesses in other methods to use this method
  - May have changed behavior in that NPEs due to pool==null will now be IllegalArgumentExceptions in some methods
- Documented that some getFooCount methods are approximate in face of concurrency via comments in the code
Additional refactoring and code prep before making connection checkout async.

src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorPool.java

Substantial refactoring and preparation of
ClientRequestExecutorPool to actually do an asynchronous
checkout. I believe I have essentially preserved prior
behavior. I.e., checkout of destination still blocks. But, the
pattern now looks a lot more like what is needed to do the
checkout asyncrhnously and then callback with the checkedout
resource. The exact exception strings likely changed during this
refactoring (but control flow should be the same).

src/java/voldemort/utils/pool/QueuedKeyedResourcePool.java

Skeleton of new class QueuedKeyedResourcePool that extends
KeyedResourcePool.
Initial commit of asynchronous checkouts.

src/java/log4j.properties
- print out more concise timestamp (don't print the date)
- print thread at end of message

src/java/voldemort/utils/pool/KeyedResourcePool.java
- reverted all protected data members back to private
- refactored attemptGrow to do the size check to determine if it
  is worth trying to attempt to grow. This made the method more
  useful to subclasses and cleaned up the local member that
  called it.
- added an internalClose method that returns whether or not the
  caller is "the one thread" responsible for closing everything
  down

src/java/voldemort/utils/pool/QueuedKeyedResourcePool.java
- first complete implementation of this class
- Four TODOs left in the code for the sake of interim code review

test/integration/voldemort/nonblocking/E2ENonblockingCheckoutTest.java
- minor tweaks to the test
Clean up of KeyedResourcePool and significant hardening of the unit test.

src/java/voldemort/utils/pool/KeyedResourcePool.java
- Documented the invariants (or lack thereof) guaranteed by this class.
- Documented this classes expectations of its users.
- Moved attemptGrow into the inner Pool class.
- Got rid of the attemptCheckoutGrowCheckout method. ;)

src/java/voldemort/utils/pool/QueuedKeyedResourcePool.java
- tweaked to match revised attemptGrow interface

test/unit/voldemort/utils/pool/KeyedResourcePoolTest.java
- Added a bunch of 'negative' tests. I.e., they demonstrate
  non-desirable behavior of current KeyedResourcePool.
- Added a contention test that has many threads checkout,
  possibly invalidate, and then checkin resources for some key.