[SPARK-2848] Shade Guava in uber-jars.

For further discussion, please check the JIRA entry.

This change moves Guava classes to a different package so that they don't conflict with the user-provided Guava (or the Hadoop-provided one). Since one class (Optional) was exposed through Spark's public API, that class was forked from Guava at the current dependency version (14.0.1) so that it can be kept going forward (until the API is cleaned).

Since sbt doesn't provide the relocation feature maven-shade-plugin does, I implemented the functionality (based on the souce for maven's plugin).

Note this change has a few implications:
- *all* classes in the final jars will reference the relocated classes. If Hadoop classes are included (i.e. "-Phadoop-provided" is not activated), those will also reference the Guava 14 classes (instead of the Guava 11 classes from the Hadoop classpath).
- if the Guava version in Spark is ever changed, the new Guava will still reference the forked Optional class; this may or may not be a problem, but in the long term it's better to think about removing Optional from the public API.

For the end user, there are two visible implications:

- Guava is not provided as a transitive dependency anymore (since it's "provided" in Spark)
- At runtime, unless they provide their own, they'll either have no Guava or Hadoop's version of Guava (11), depending on how they set up their classpath.
Not according to my reading of
http://www.apache.org/dev/licensing-howto.html , in the part covering
bundled AL2 dependencies, but this stuff is always less than intuitive to
me.

I suppose the reasoning is that Spark's AL2 license already exactly
describes the terms of the licensing for that file. That's not quite true
for BSD or MIT licenses.

The copyright owner doesn't matter; there are hundreds of them. It matters
just how whoever they are license their IP to be used.
On Aug 11, 2014 12:22 AM, "Matei Zaharia" <notifications@github.com> wrote:

> In core/src/main/java/com/google/common/base/Optional.java:
>
> > @@ -0,0 +1,243 @@
> > +/*
> > + * Copyright (C) 2011 The Guava Authors
>
> Shouldn't we still list it in our LICENSE? We've listed all source files
> that are not copyright Apache, which would include this one.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/1813/files#r16034129>.
>