nathanmarz/cascalog-contrib — Issue #3: simultaneous checkpoint step runs crashing
Author: Quantisan
Created: 2012-02-24 17:57:03+00:00
State: open

When I have ...

(workflow [tmp-path]
a-step ([] (...))
b-step ([:deps nil] ...)
c-step ([:deps [a-step b-step]] ...))

I get the exception below. I changed nil to [a-step] and it works fine.

java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 17 more
Caused by: cascading.flow.FlowException: internal error during mapper configuration
at cascading.flow.FlowMapper.configure(FlowMapper.java:65)
... 22 more
Caused by: com.esotericsoftware.kryo.SerializationException: Invalid class name reference: 16383
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:441)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:561)
at com.esotericsoftware.kryo.ObjectBuffer.readClassAndObject(ObjectBuffer.java:206)
at cascalog.KryoService.deserialize(KryoService.java:28)
at cascalog.ClojureCascadingBase.prepare(ClojureCascadingBase.java:51)
at cascading.pipe.Each$EachHandler.prepare(Each.java:499)
at cascading.flow.stack.EachMapperStackElement.prepare(EachMapperStackElement.java:67)
at cascading.flow.stack.StackElement.open(StackElement.java:178)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.StackElement.open(StackElement.java:182)
at cascading.flow.stack.FlowMapperStack.(FlowMapperStack.java:88)
at cascading.flow.FlowMapper.configure(FlowMapper.java:58)
... 22 more