[SPARK-1522]: YARN ClientBase throws a NPE if there is no YARN Application specific CP

The current implementation of ClientBase.getDefaultYarnApplicationClasspath inspects
the MRJobConfig class for the field DEFAULT_YARN_APPLICATION_CLASSPATH when it should
be really looking into YarnConfiguration. If the Application Configuration has no
yarn.application.classpath defined a NPE exception will be thrown.

Public API Changes
===========================

* YARN ClientBase getDefault*ApplicationClasspath returns Try[Array[String]]

This commit depicts how the `ClientBase` API could change the
`getDefaultYarnApplicationClasspath` and `getDefaultMRApplicationClasspath`
to return a `Try[Array[String]]` while recovering from
`NoSuchFieldError` and `NoSuchFieldException`.

Both methods that return the default application's *classpath*, for both *YARN*
as well as *Map Reduce (MR)*, use reflection and per the Java API
documentation they can throw the following exceptions:

* Class:getField(String name):
    * NoSuchFieldException - if a field with the specified name is not found.
    * NullPointerException - if name is null
    * SecurityException - If a security manager, s, is present and any of
      the following conditions is met:
        1. Invocation of s.checkMemberAccess(this, Member.PUBLIC) denies
        access to the field.
        2. The caller's class loader is not the same as or an ancestor of
        the class loader for the current class and invocation of
        `s.checkPackageAccess()` denies access to the package of this class.

* Field:Object get(Object obj):
    * IllegalAccessException - if this Field object is enforcing Java language access
      control and the underlying field is inaccessible.
    * IllegalArgumentException - if the specified object is not an instance of the class
      or interface declaring the underlying field (or a subclass or implementor thereof).
    * NullPointerException - if the specified object is null and the field
      is an instance field.
    * ExceptionInInitializerError - if the initialization provoked
      by this method fails.

**NOTE**: The above is based on the *Java API for JDK 1.7*

An interesting thing to notice is that the official JDK doesn't mention
the occurrence of the `NoSuchFieldError`. This is completely acceptable
per the JDK spec. The reason is that it is an *Error* and as
described by the Java Language Specification and depicted in
the *Error Class* documentation.

    An `Error` "indicates serious problems that a reasonable
    application should not try to catch."

While

    An `Exception` "indicates conditions that a reasonable
    application might want to catch."

If we actually dig deeper according to the *JVM SE7 Specification*

    "While Loading, Linking, and Initializing, if an error occurs during resolution
    of a symbolic reference, then an instance of
    IncompatibleClassChangeError (or a subclass) must be thrown..."

    "If an attempt by the Java Virtual Machine to resolve a symbolic reference fails
    because an error is thrown that is an instance of LinkageError (or a subclass),
    then subsequent attempts to resolve the reference always fail with the same error
    that was thrown as a result of the initial resolution attempt."

Now `NoSuchFieldError` extends `LinkageError` which in turn is a `IncompatibleClassChangeError`
and according to its documentation, the *LinkageError Class*,

    "indicates that a class has some dependency on another class;
    however, the latter class has incompatibly changed after the
    compilation of the former class."

Why all these is important and how it relates with a couple of lines of
code?

Well, the original approach catches the two most probable problems
you might encounter if you access, using reflection, a field that you
are almost sure that if it exist it will be of _public_ access but you
are not sure it will always be there. Interesting enough the original
implementation addresses one of the _exceptions_ as well as a potential
_linkage error_ but as mentioned neglects a documented _security exception_,
probably due its unlikeliness to occur.

Fact is that if an error _bubbles_ up the Spark YARN Client doesn't handle it
will terminate, in a probably obscure fashion. The current call stack is
as follows.

    Client >> run >> runApp >> ClientBase.setupLaunchEnv >> populateClasspath

In my opinion it is questionable to let an exception escape of this context, the _ClientBase Object_.
In my opinion such _ClientBase Object_ should fail gracefully by handling the potential
_exceptions_ and _linkage error_ while providing enough logging
to let a user know and identify what happened. Yet again, in my opinion
the implementation in this commit handles it in a better, more
resilient, manner than the previous implementation while adding logging
that will help clarify the issues in case of an _exception_.

Additional Changes include:
===========================

* Test Suite for ClientBase added
* Coding Style:
    * [Spark Style Guidelines](https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide)
    * [Scala Official Style Guidelines](http://docs.scala-lang.org/style/)
    * [Scalariform](https://github.com/mdr/scalariform)
* Code refactoring and cleanup per review by andrewor14

Ref.
    "JVM SE7 Specification" http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-5.html#jvms-5.4
    "Java API for JDK 1.7" http://docs.oracle.com/javase/7/docs/api/

[ticket: SPARK-1522] : https://issues.apache.org/jira/browse/SPARK-1522

Author      : berngp
Reviewer    : andrewor14, tgravescs
Testing     : ?