kevinweil/elephant-bird — Issue #107: How to run example
Author: hassanrom
Created: 2011-11-24 23:43:31+00:00
State: open

Hello,

I'm running into trouble while trying to run one of the examples. Here's what I did:

$ ~/hadoop-0.20.2/bin/hadoop jar ~/elephant-bird/examples/dist/elephant-bird-examples-1.0.jar com.twitter.elephantbird.examples.ProtobufMRExample s3n://hassanrom-sna/name_age_sample.txt s3n://hassanrom-sna/name_age_sample.lzo11/11/25 00:42:57 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/protobuf/GeneratedMessage
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at com.twitter.elephantbird.examples.ProtobufMRExample.runLzoToText(Unknown Source)
at com.twitter.elephantbird.examples.ProtobufMRExample.main(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException: com.google.protobuf.GeneratedMessage
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 18 more

Any ideas what am I doing wrong? I've tried setting HADOOP_CLASSPATH to point to where the libraries are and that didn't work. I've also tried bundling all the jars together and that didn't work either. I also tried passing the jars via -libjars and that didn't work either. Some documentation on how to get the examples running would definitely be nice especially for folks who are new to hadoop.

Thanks in advance!
- Hassan