NICTA/scoobi — Issue #5: Unable to run examples/jobs due to NullPointerException during shuffleInMemory
Author: agemooij
Created: 2011-11-12 12:58:36+00:00
State: open

Hi

I've tried to run both my own sample code and the WordCount example but I keep running into a NullPointerException during the shuffle phase.

Setup:
Cloudera CDH3u2 (hadoop-0.20.2-cdh3u2) running in local pseudo-cluster mode on OSX 10.7.2 using jsk 1.6.0_29

I ran into some issues with the default arguments to WordCount being interpreted as hdfs paths during persist so I uploaded the generated all-words.txt to hdfs and ran it with both arguments as hdfs paths

Output:

$>: hadoop com.nicta.scoobi.examples.WordCount /in/all-words.txt /tmp/out
2011-11-12 13:46:59.740 java[15347:1903] Unable to load realm info from SCDynamicStore
11/11/12 13:47:04 INFO mapred.FileInputFormat: Total input paths to process : 1
11/11/12 13:47:05 INFO mapred.JobClient: Running job: job_201111121331_0001
11/11/12 13:47:06 INFO mapred.JobClient: map 0% reduce 0%
11/11/12 13:47:13 INFO mapred.JobClient: map 100% reduce 0%
11/11/12 13:47:21 INFO mapred.JobClient: Task Id : attempt_201111121331_0001_r_000000_0, Status : FAILED
Error: java.lang.NullPointerException
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1603)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1468)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1317)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1249)

attempt_201111121331_0001_r_000000_0: 2011-11-12 13:47:12.549 java[15432:1903] Unable to load realm info from SCDynamicStore
11/11/12 13:47:29 INFO mapred.JobClient: Task Id : attempt_201111121331_0001_r_000000_1, Status : FAILED
Error: java.lang.NullPointerException
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1603)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1468)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1317)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1249)

attempt_201111121331_0001_r_000000_1: 2011-11-12 13:47:21.072 java[15460:1903] Unable to load realm info from SCDynamicStore
11/11/12 13:47:36 INFO mapred.JobClient: Task Id : attempt_201111121331_0001_r_000000_2, Status : FAILED
Error: java.lang.NullPointerException
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1603)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1468)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1317)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1249)

attempt_201111121331_0001_r_000000_2: 2011-11-12 13:47:29.041 java[15489:1903] Unable to load realm info from SCDynamicStore
11/11/12 13:47:46 INFO mapred.JobClient: Job complete: job_201111121331_0001
11/11/12 13:47:46 INFO mapred.JobClient: Counters: 18
11/11/12 13:47:46 INFO mapred.JobClient: Job Counters
11/11/12 13:47:46 INFO mapred.JobClient: Launched reduce tasks=4
11/11/12 13:47:46 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=9815
11/11/12 13:47:46 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
11/11/12 13:47:46 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
11/11/12 13:47:46 INFO mapred.JobClient: Launched map tasks=2
11/11/12 13:47:46 INFO mapred.JobClient: Data-local map tasks=2
11/11/12 13:47:46 INFO mapred.JobClient: Failed reduce tasks=1
11/11/12 13:47:46 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=4946
11/11/12 13:47:46 INFO mapred.JobClient: FileSystemCounters
11/11/12 13:47:46 INFO mapred.JobClient: HDFS_BYTES_READ=58793
11/11/12 13:47:46 INFO mapred.JobClient: FILE_BYTES_WRITTEN=125291
11/11/12 13:47:46 INFO mapred.JobClient: Map-Reduce Framework
11/11/12 13:47:46 INFO mapred.JobClient: Combine output records=995
11/11/12 13:47:46 INFO mapred.JobClient: Map input records=5000
11/11/12 13:47:46 INFO mapred.JobClient: Spilled Records=995
11/11/12 13:47:46 INFO mapred.JobClient: Map output bytes=95000
11/11/12 13:47:46 INFO mapred.JobClient: Map input bytes=30000
11/11/12 13:47:46 INFO mapred.JobClient: Combine input records=5000
11/11/12 13:47:46 INFO mapred.JobClient: Map output records=5000
11/11/12 13:47:46 INFO mapred.JobClient: SPLIT_RAW_BYTES=394
11/11/12 13:47:46 INFO mapred.JobClient: Job Failed: NA
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1246)
at com.nicta.scoobi.impl.exec.MapReduceJob.run(MapReduceJob.scala:191)
at com.nicta.scoobi.impl.exec.Executor$.com$nicta$scoobi$impl$exec$Executor$$executeMSCR(Executor.scala:117)
at com.nicta.scoobi.impl.exec.Executor$$anonfun$executePlan$10.apply(Executor.scala:101)
at com.nicta.scoobi.impl.exec.Executor$$anonfun$executePlan$10.apply(Executor.scala:99)
at scala.collection.immutable.Set$Set1.foreach(Set.scala:86)
at com.nicta.scoobi.impl.exec.Executor$.executePlan(Executor.scala:99)
at com.nicta.scoobi.DList$.persist(DList.scala:133)
at com.nicta.scoobi.examples.WordCount$$anonfun$main$1.apply(WordCount.scala:62)
at com.nicta.scoobi.examples.WordCount$$anonfun$main$1.apply(WordCount.scala:26)
at com.nicta.scoobi.Scoobi$.withHadoopArgs(Scoobi.scala:44)
at com.nicta.scoobi.examples.WordCount$.main(WordCount.scala:26)
at com.nicta.scoobi.examples.WordCount.main(WordCount.scala)