On Sat, Mar 15, 2014 at 9:55 PM, gzm55 <notifications@github.com> wrote:

> In
> core/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandlerMacro.scala:
>
> > +   * so we don't need to call getAMResponse(), just use AllocateResponse directly.
> > +   * This macro will test the existence of AMResponse,
> > +   * and generate diffenert expressions.
> > +   *
> > +   * This macro now is only used in spark's alpha version of yarn api.
> > +   * It stays in the core project, for the two-stage compiling of
> > +   * the scala macro system.
> > +   */
> > +  def getAMRespImpl(c: Context)(resp: c.Expr[Any]) = {
> > +    try {
> > +      import c.universe._
> > +      c.mirror.staticClass("org.apache.hadoop.yarn.api.records.AMResponse")
> > +      c.Expr[Any](Apply(Select(resp.tree, newTermName("getAMResponse")), List()))
> > +    } catch {
> > +      case _: Throwable => resp
> > +    }
>
> run time reflection also need hard code method name that does't exist in
> beta api, and have to pay the run time cost of reflect calls.
>

Cache the resolved method - it is fairly cheap once that happens.
There is a reason java does not support macro's.



> even if using runtime reflection, IMO, we would rarely deploy
> spark-assembly-xxx-hadoop2.0.0-cdh4.6.0.jar on a cluster running hadoop
> cdh4.2.0.
>


That is indeed a fair point.



> --
> Reply to this email directly or view it on GitHub<https://github.com/apache/spark/pull/151/files#r10638000>
> .
>