If a user specifies a git tag, then both Spark and Shark have to be
compatible and we assume SparkBuild.scala is correctly specified in the git
repo. That use case is for advanced users (developers) who want to test
specific version cuts. If you look at the Shark codebase, the version
specified is typically 0.X.X-SNAPSHOT, so it will work out of the box.

First Spark will be compiled and published (because it is always listed
first in the module list), then Shark will be compiled.


On Thu, Jun 13, 2013 at 5:56 PM, shivaram <notifications@github.com> wrote:

> In shark/init.sh:
>
> > +SHARK_VERSION=`cat /root/shark/shark.version`
> > +HADOOP_MAJOR_VERSION=`cat /root/spark/hadoop.version`
> > +
> > +pushd /root
> > +
> > +# Github tag:
> > +if [[ "$SHARK_VERSION" == *\|* ]]
> > +then
> > +  pushd shark
> > +  git init
> > +  repo=`python -c "print '$SHARK_VERSION'.split('|')[0]"`
> > +  git_hash=`python -c "print '$SHARK_VERSION'.split('|')[1]"`
> > +  git remote add origin $repo
> > +  git fetch origin
> > +  git checkout $git_hash
> > +  sbt/sbt clean products
>
> Also in this case wouldn't shark need to compile against 0.8.0-SNAPSHOT or
> whatever the spark version published from the git hash was ? If that is the
> case we'll need edit SharkBuild.scala etc. for this and make sure spark
> gets published before shark etc.
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/shivaram/spark-ec2/pull/1/files#r4693200>
> .
>