caskdata/cdap — pull request #836 by rohitsinha54 (2014-12-16 23:34:28+00:00)
Title: [2.7.0] Added Spark in Workflow: CDAP-465

Jira Issue : CDAP-465

1. This PR adds Spark in our Workflow in a similar way as we run MapReduce. This does add some code duplication as our Workflow is not generic to ProgramTypes at this point. This issue required a rework on Workflow which will be done by @sagarkapare.

2. For the unit test I have added Spark in WorkflowTest this make sure that we can run Spark from a Workflow and also that different program (in this case MapReduce) can also be a part of the same workflow. I removed the CustomAction and moved its responsibility to Spark as it appeared that it was not doing anything there other than adding one more thing in Workflow and testing MapReduce run success.

3. Now SparkPageRank example runs the spark program as a part of Workflow.

Build is running here: https://builds.cask.co/browse/CDAP-DUT338-3