cloudera/crunch — Issue #1: Using TupleN with less than 5 values results in ClassCastException
Author: eldenbishop
Created: 2011-10-14 01:03:52+00:00
State: open

Any read of serialized Tuple data instantiates a type based on the number of values in that Tuple. If you are using TupleN however and there are less than five values, you will get back Pair, Tuple3 or Tuple4 and the callbacks will throw a class cast exception. Basically, the type safety is broken in this bit of code.

This can be avoided because we already know the value count of the Tuple from the PType parameter, there is no reason to "guess" the type during the load (See usages of Tuple.tuplify() for the source of the bug).

...
...
PTable mapped = ...; // TupleN w/only 2 values
PCollection mapped = mapped.groupByKey().parallelDo(
new DoFn>, TupleN>() {
public void process(Pair> input, Emitter emitter) {
for (TupleN tuple : input.second()) {} // !!!! ClassCastException: Pair -> TupleN !!!!
},
ptf.tableOf(ptf.strings(), ptf.tuples(ptf.longs(), ptf.strings())
);

There is a working fix in my fork which I am using for some analysis at work.