Right, the format of the config file should be the same (or similar). I
currently only parse the weight and pool name info because the current
implementation only uses weight for it's fair comparison (it doesn't have
the other bells and whistles such as min/max slots, etc). However, I think
the Hadoop fair allocation config file distinguished between map slots and
reduce slots, I think in our case we won't have that but instead maybe just
have minSlots, maxSlots, etc.






On Mon, Mar 11, 2013 at 11:52 PM, jason-dai <notifications@github.com>wrote:

> In
> core/src/main/scala/spark/scheduler/cluster/FairTaskSetQueuesManager.scala:
>
> > +  var pools = new ArrayBuffer[Pool]
> > +
> > +  loadPoolProperties()
> > +
> > +  def loadPoolProperties() {
> > +    //first check if the file exists
> > +    val file = new File(schedulerAllocFile)
> > +    if(!file.exists()) {
> > +    //if file does not exist, we just create 1 pool, default
> > +      val pool = new Pool("default",100)
> > +      pools += pool
> > +      poolNameToPool("default") = pool
> > +      logInfo("Created a default pool with weight = 100")
> > +    }
> > +    else {
> > +      val xml = XML.loadFile(file)
>
> An objective we have is to have the FairScheduler in Spark to reuse the
> FairScheduler config files in Hadoop, which will make it easy for people to
> deploy Spark alongside Hadoop. Any comments?
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/mesos/spark/pull/509/files#r3329480>
> .
>