twitter/scalding — Issue #2: Error compiling source with scald.rb with 2.9.1
Author: schmmd
Created: 2012-01-12 22:30:36+00:00
State: open

Hi, near the top of scald.rb you have the code

COMPILE_CMD="java -cp project/boot/scala-2.8.1/lib/scala-library.jar:project/boot/scala-2.8.1/lib/scala-compiler.jar -Dscala.home=project/boot/scala-2.8.1/lib/ scala.tools.nsc.Main"

But this does not work, at least when the project was built with 2.9.1.

$ scripts/scald.rb --local tutorial/Tutorial1.scala
compiling tutorial/Tutorial1.scala
Exception in thread "main" java.lang.NoClassDefFoundError: scala/tools/nsc/Main
Caused by: java.lang.ClassNotFoundException: scala.tools.nsc.Main
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: scala.tools.nsc.Main. Program will exit.

I have scala 2.9.1 installed so I changed it to

COMPILE_CMD="scalac"

Voila!

$ scripts/scald.rb --local tutorial/Tutorial1.scala
compiling tutorial/Tutorial1.scala
12/01/12 14:29:49 INFO util.Version: Concurrent, Inc - Cascading 2.0.0 [hadoop-0.20.2+]
12/01/12 14:29:49 INFO flow.Flow: [] starting
12/01/12 14:29:49 INFO flow.Flow: [] source: FileTap["TextLine[['num', 'line']->[ALL]]"]["tutorial/data/hello.txt"]"]
12/01/12 14:29:49 INFO flow.Flow: [] sink: FileTap["TextLine[['num', 'line']->[ALL]]"]["tutorial/data/output1.txt"]"]
12/01/12 14:29:49 INFO flow.Flow: [] parallel execution is enabled: true
12/01/12 14:29:49 INFO flow.Flow: [] starting jobs: 1
12/01/12 14:29:49 INFO flow.Flow: [] allocating threads: 1
12/01/12 14:29:49 INFO planner.FlowStep: [] starting step: (1/1) local