apache/spark — pull request #3658 by vanzin (2014-12-10 02:03:57+00:00)
Title: [SPARK-4809] Rework Guava library shading.

The current way of shading Guava is a little problematic. Code that
depends on "spark-core" does not see the transitive dependency, yet
classes in "spark-core" actually depend on Guava. So it's a little
tricky to run unit tests that use spark-core classes, since you need
a compatible version of Guava in your dependencies when running the
tests. This can become a little tricky, and is kind of a bad user
experience.

This change modifies the way Guava is shaded so that it's applied
uniformly across the Spark build. This means Guava is shaded inside
spark-core itself, so that the dependency issues above are solved.
Aside from that, all Spark sub-modules have their Guava references
relocated, so that they refer to the relocated classes now packaged
inside spark-core. Before, this was only done by the time the assembly
was built, so project that did not end up inside the assembly (such
as streaming backends) could still reference the original location
of Guava classes).

This relocation does not apply to the sub-modules under network/,
though. For those cases, we want to keep the Guava dependency alive,
since we want to use the same Guava as the rest of the Yarn NM
when deploying the auxiliary shuffle service. For this reason, also,
the network/ dependencies are shaded into the spark-core artifact
too, so that the raw Guava dependency doesn't leak.