Expose aplication ID in ApplicationStart event, use it in history server.

This change exposes the application ID generated by the Spark Master, Mesos or Yarn
via the SparkListenerApplicationStart event. It then uses that information to expose the
application via its ID in the history server, instead of using the internal directory name
generated by the event logger as an application id. This allows someone who knows
the application ID to easily figure out the URL for the application's entry in the HS, aside
from looking better.

In Yarn mode, this is used to generate a direct link from the RM application list to the
Spark history server entry (thus providing a fix for SPARK-2150).

Note this sort of assumes that the different managers will generate app ids that are
sufficiently different from each other that clashes will not occur.
The reflection code exists since the class can't be loaded in all env .
Tom, has this changed recently?
On 05-Aug-2014 10:27 pm, "Marcelo Vanzin" <notifications@github.com> wrote:

> In core/src/main/scala/org/apache/spark/SparkContext.scala:
>
> > @@ -1531,18 +1532,6 @@ object SparkContext extends Logging {
> >              throw new SparkException("YARN mode not available ?", e)
> >            }
> >          }
> > -        val backend = try {
> > -          val clazz =
> > -            Class.forName("org.apache.spark.scheduler.cluster.YarnClusterSchedulerBackend")
> > -          val cons = clazz.getConstructor(classOf[TaskSchedulerImpl], classOf[SparkContext])
> > -          cons.newInstance(scheduler, sc).asInstanceOf[CoarseGrainedSchedulerBackend]
> > -        } catch {
> > -          case e: Exception => {
> > -            throw new SparkException("YARN mode not available ?", e)
> > -          }
> > -        }
> > -        scheduler.initialize(backend)
> > -        scheduler
>
> Maybe I'm missing something, but this is just removing a bunch of
> reflection code and replacing it with a single line later on (in
> YarnClusterScheduler):
>
> initialize(new YarnClusterSchedulerBackend(this, sc))
>
> This looks much, much easier to read and cleaner to me, but if you guys
> somehow feel so strongly about it, I can revert the change.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/1218/files#r15825749>.
>