Mlhr 1339

AbstractFSDirectoryInputOperator

  * Fixed timing issue with scanning that causes Unit Test to fail
  * Added files to failure list for retry if an IOException is thrown
  * Fixed bug where if two currentFiles got mapped to the same partition during repartition one of them would be dropped.
  * Fixed bug with directory scanner where each partition would accept a file if no regex was specified.
  * Scanning happens periodically to keep pending file lists up to date.

AbstractThroughputHashFSDirectoryInputOperator

  * Added this class. Does dynamic partitioning based on the number of files left in a directory.

Added FSInputOperator to Malhar and added unit tests

Tested as thoroughly as I could, but ran into SPOI-3053 and SPOI-3054.
Please check your IDE settings so no trailing whitespace is introduced.