kevinweil/elephant-bird — Issue #112: Writable loading seems broken in local mode
Author: tdunning
Created: 2012-01-04 04:32:11+00:00
State: open

With either pig 0.8.1 or pig 0.9.1 and elephant-bird 53a814f

I try this:

../pig/bin/pig -x local
register 'target/pig-vector-1.0-jar-with-dependencies.jar';
x = load 'x' using PigStorage('\t') as (a:int, b:chararray, c:chararray);
y = foreach x generate a,b;
store y into 'y.w' using com.twitter.elephantbird.pig.store.SequenceFileStorage (
'-c com.twitter.elephantbird.pig.util.IntWritableConverter',
'-c com.twitter.elephantbird.pig.util.TextConverter' ) ;
z = load 'y.w' using com.twitter.elephantbird.pig.load.SequenceFileLoader (
'-c com.twitter.elephantbird.pig.util.IntWritableConverter',
'-c com.twitter.elephantbird.pig.util.TextConverter' ) as (a:int, b:chararray);
illustrate z;

and this happens:

java.lang.NullPointerException: Signature is null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at com.twitter.elephantbird.pig.load.SequenceFileLoader.getContextProperties(SequenceFileLoader.java:296)
at com.twitter.elephantbird.pig.load.SequenceFileLoader.getContextProperty(SequenceFileLoader.java:306)
at com.twitter.elephantbird.pig.load.SequenceFileLoader.setLocation(SequenceFileLoader.java:411)
at org.apache.pig.impl.io.ReadToEndLoader.init(ReadToEndLoader.java:147)
at org.apache.pig.impl.io.ReadToEndLoader.(ReadToEndLoader.java:116)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad.setUp(POLoad.java:91)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad.getNext(POLoad.java:119)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:267)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:194)
at org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:257)
at org.apache.pig.pen.ExampleGenerator.readBaseData(ExampleGenerator.java:222)
at org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:154)
at org.apache.pig.PigServer.getExamples(PigServer.java:1245)
at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:698)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:591)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:306)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67)
at org.apache.pig.Main.run(Main.java:487)
at org.apache.pig.Main.main(Main.java:108)
2012-01-03 20:09:49,315 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Encountered IOException. Exception : Signature is null