HDInsight support and new file system layout.

This pull request addresses two separate issues:

  1. HDInsight support: This adds support for job submission to YARN's REST API as used by pre-releases of HDInsight.
  2. New file system layout: This redefines the file system structure of REEF's Driver and Evaluator working directories.

Details as follow below:

**HDInsight support**

YARN will get a REST API for job submission that expands on the current API for job management. The newly added module `reef-runtime-hdinsight` adds support for the current version of that under the assumption that data is also on Azure.  Future work will move this to the final YARN REST API and expand this in scope beyond HDInsight.

The module `reef-examples-hdinsight` contains the example code for this new runtime. For now, that is to run `HelloREEF`. The rationale for yet-another examples submodule is that in the case of HDInsight, we can't actually exclude the Hadoop JARs from the client, as it isn't executed on the cluster (but on the submitting machine).

**New filesystem layout**

REEF used to put all files for the current Driver or Evaluator into its working directory. This is unfortunate, as it precluded us from packaging things together and have YARN expand it for us. With this PR, we instead move to a different file system structure. The authorative answer is in the code in `com.microsoft.reef.runtime.common.files.REEFFileNames`, but here is the high level view:

In each working directory (Driver or Evaluator) we create the following folders:

 * `reef` serves as the root for all things REEF such that we don't clash with other libraries applications might be using.
 * `reef/local` contains all files local to this container (e.g. the configuration, local libraries, ...)
 * `reef/global` contains all files global to the application (e.g. libraries)

When we submit a job, we submit a JAR that contains all three folders above for the Driver. When we submit an Evaluator, we submit two JARs: One with the global stuff (that is also only copied to HDFS only once) and one with the local files for that Evaluators.

The upside of this approach is that the number of files an application consists of is immaterial for the number of filesystem operations we perform. Hence, it is now very much feasible to submit REEF jobs from an IDE without requiring the build of shaded jars or such upfront.

All three runtimes (YARN, HDInsight, Local) have been updated to adhere to this new layout. Also, the unit tests have been updated accordingly.
cosmetic changes to the weimer_hdinsight branch (not finished yet)
finished refactoring of the HDInsight code
Merge pull request #878 from Microsoft-CISL/weimer_hdinsight

HDInsight support and new file system layout.
Make TaskRuntime runnable instead of inheriting from Thread; other
cosmetic fixes
Improved the generation of FailedEvaluator

This closes #880:
  * Changed `EvaluatorRuntime` such that unexpected calls to the `RuntimeStopHandler` result in a `FailedEvaluator`
  * Added a test that proofs that it works

Also, I made unrelated updates to the code:
  * Renamed methods in `HeartBeatManager` from `onNext(...)` to more speaking names like `sendTaskStatus()`
minor cosmetic changes for better readability
Merge branch 'master' into sergiym_task_runtime_thread
cosmetic changes to TaskRuntime and ContextRuntime - before separating
Thread and Runnable in TaskRuntime
Merge branch 'master' of github.com:Microsoft-CISL/REEF into sergiym_task_runtime_thread
Provide ordered events to the Driver

This closes #882.

*Major changes*
  * `DispatchingEStage` now has a second constructor, which creates a new one that shares the thread pool and exception handler with the given one. That constructor is used in `EvaluatorMessageDispatcher`
  * Changed the order in `EvaluatorDispatcherThreads.dispatch()` to first send the event to services, then to the application
  * Changed the default on `EvaluatorDispatcherThreads`  to 1

*Minor code cleanups*
  * `DispatchingEStage` now uses the queueLength of the underlying `ThreadPoolStage`
  * Renamed `DispatchingEStage.dispatcher` to `DispatchingEStage.applicationDispatcher`
cosmetic fixes before merging into master
cosmetic fixes: remove redundant generic specializations in EvaluatorContext
Updated Evaluator Parameters

This PR is part of the ongoing work to move our named parameters from inner classes to top-level classes, which allows us to use the same parameters in the JVM and .NET implementation of REEF:

  * Moved all inner classes in `EvaluatorConfiguration`  to the package `com.microsoft.reef.runtime.common.evaluator.parameters`
  * Renamed `EvaluatorConfigurationModule` to `EvaluatorConfiguration` to bring it in line with the other uses of ConfigurationModules in REEF.
Merge pull request #884 from Microsoft-CISL/weimer_evaluator_parameters

Updated Evaluator Parameters
Introduction of TaskRepresenter

This closes #885

This PR is part of the longer term goal of rewriting the Driver code into smaller units. It introduces a new class `TaskRepresenter` which represents a running Task on the Driver. It is is responsible for taking action on Task status messages sent by an evaluator. It is in that capacity that #885 is addressed by ignoring repeated INIT messages from Tasks.

Also, this adds `TaskCountingTest`, which checks that we never hand a second `RunningTask` event to the user. As this test needs to run for a long time to trigger bad behavior, it is not part of the `AllTests` suite. Instead, I added a new test suite called `ProbabilisticTests` where we can collect those tests.
better logging when submitting a driver job to YARN; add common Hadoop
environment variables to classpath; other minor fixes
cosmetic fixes
Merge branch 'weimer_885'
Merge pull request #888 from Microsoft-CISL/sergiym_log_launch

better logging when submitting a driver job to YARN