kevinweil/elephant-bird — pull request #391 by saurfang (2014-05-13 19:28:14+00:00)
Title: SequenceFileLoader/Writer Unable to Recognize -c Parameter in commons-cli 1.0

While the elephant-bird specifies commons-cli 1.2 in the pom.xml, currently Pig compiles with commons-cli 1.0 by default. In my case, 1.0 was unluckily picked up while running the following script and caused the error message.

The reason was that in commons-cli 1.0, longOpt is assumed to contain the leading"--" while it's not so in 1.2. (The way values are retrieved are slightly different) Fortunately, both versions agree the shortOpt (the option name) do contain any hyphens. So the code can be made backward compatible by using shortOpt 'c' and 't' to getOptionValue in SequenceFileConfig and Writer.

To (hopefully) replicate, have common-clis 1.0 on your classpath, and run
```
raw_data = load 'INPUT' using com.twitter.elephantbird.pig.load.SequenceFileLoader (
'-c com.twitter.elephantbird.pig.util.NullWritableConverter',
'-c com.twitter.elephantbird.pig.util.ProtobufWritableConverter com.twitter.elephantbird.examples.proto.AddressBookProtos.Person'
);

dump raw_data;
```
The script fails with
> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'com.twitter.elephantbird.pig.load.SequenceFileLoader' with arguments '[-c com.twitter.elephantbird.pig.util.NullWritableConverter, -c com.twitter.elephantbird.pig.util.ProtobufWritableConverter com.twitter.elephantbird.examples.proto.AddressBookProtos.Person]'
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1722)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1430)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:381)
at org.apache.pig.PigServer.executeBatch(PigServer.java:406)
at org.apache.pig.PigServer.executeBatch(PigServer.java:392)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:170)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:747)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:228)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:203)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:608)
at org.apache.pig.Main.main(Main.java:156)
Caused by:
pig script failed to validate: java.lang.RuntimeException: could not instantiate 'com.twitter.elephantbird.pig.load.SequenceFileLoader' with arguments '[-c com.twitter.elephantbird.pig.util.NullWritableConverter, -c com.twitter.elephantbird.pig.util.ProtobufWritableConverter com.twitter.elephantbird.examples.proto.AddressBookProtos.Person]'
at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:881)
at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
... 13 more
Caused by: java.lang.RuntimeException: could not instantiate 'com.twitter.elephantbird.pig.load.SequenceFileLoader' with arguments '[-c com.twitter.elephantbird.pig.util.NullWritableConverter, -c com.twitter.elephantbird.pig.util.ProtobufWritableConverter com.twitter.elephantbird.examples.proto.AddressBookProtos.Person]'
at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:774)
at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:869)
... 19 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:742)
... 20 more
Caused by: java.lang.RuntimeException: Failed to create WritableConverter instance
at com.twitter.elephantbird.pig.util.SequenceFileConfig.getWritableConverter(SequenceFileConfig.java:225)
at com.twitter.elephantbird.pig.util.SequenceFileConfig.(SequenceFileConfig.java:102)
at com.twitter.elephantbird.pig.load.SequenceFileLoader$Config.(SequenceFileLoader.java:74)
at com.twitter.elephantbird.pig.load.SequenceFileLoader.(SequenceFileLoader.java:118)
at com.twitter.elephantbird.pig.load.SequenceFileLoader.(SequenceFileLoader.java:128)
... 25 more
Caused by: java.lang.NoSuchMethodException: com.twitter.elephantbird.pig.util.TextConverter.(java.lang.String)
at java.lang.Class.getConstructor0(Unknown Source)
at java.lang.Class.getConstructor(Unknown Source)
at com.twitter.elephantbird.pig.util.SequenceFileConfig.getWritableConverter(SequenceFileConfig.java:213)
... 29 more

The last inner error suggests the second argument was not correctly parsed and picked up the default `TextConverter` instead of the `ProtobufWritableConverter`.