klbostee/dumbo — pull request #64 by d2207197 (2012-12-15 16:43:44+00:00)
Title: Fix typedbytes.pyc file path should start with file://

dumbo generate wrong typedbytes argument: `-jobconf 'tmpfiles=/usr/local/lib/python2.7/dist-packages/typedbytes.pyc' `
the correct one should be `-file '/usr/local/lib/python2.7/dist-packages/typedbytes.pyc'`

dumbo error messages:
EXEC: HADOOP_CLASSPATH=":$HADOOP_CLASSPATH" /usr/bin/hadoop jar /usr/lib/hadoop-mapreduce/hadoop-streaming.jar -outputformat 'org.apache.hadoop.mapred.SequenceFileOutputFormat' -inputformat 'org.apache.hadoop.streaming.AutoInputFormat' -reducer 'python -m tf-idf-dumbo red 0 262144000' -mapper 'python -m tf-idf-dumbo map 0 262144000' -file '/home/external/joe/nlp/lab11/vocab.mail.txt' -file '/home/external/joe/nlp/lab11/tf-idf-dumbo.py' -file '/usr/local/lib/python2.7/dist-packages/dumbo/backends/common.pyc' -output 'enron-tf-idf_pre1' -jobconf 'stream.map.input=typedbytes' -jobconf 'stream.reduce.input=typedbytes' -jobconf 'stream.map.output=typedbytes' -jobconf 'stream.reduce.output=typedbytes' -jobconf 'mapred.job.name=tf-idf-dumbo.py (1/2)' -jobconf 'tmpfiles=/usr/local/lib/python2.7/dist-packages/typedbytes.pyc' -input '/tmp/enronsample' -cmdenv 'dumbo_mrbase_class=dumbo.backends.common.MapRedBase' -cmdenv 'dumbo_jk_class=dumbo.backends.common.JoinKey' -cmdenv 'dumbo_runinfo_class=dumbo.backends.streaming.StreamingRunInfo' -cmdenv ''PYTHONPATH=common.pyc'
...
12/12/15 23:47:15 ERROR security.UserGroupInformation: PriviledgedActionException as:joe (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not exist: /usr/local/lib/python2.7/dist-packages/typedbytes.pyc
12/12/15 23:47:15 ERROR security.UserGroupInformation: PriviledgedActionException as:joe (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not exist: /usr/local/lib/python2.7/dist-packages/typedbytes.pyc
12/12/15 23:47:15 ERROR streaming.StreamJob: Error launching job , bad input path : File does notexist: /usr/local/lib/python2.7/dist-packages/typedbytes.pycStreaming Command Failed!