twitter/scalding — pull request #233 by johnynek (2012-11-29 22:55:47+00:00)
Title: Removed dangers implicit conversions from TypedPipe <=> Grouped

The sum method (and aggregate would as well) expose a very dangerous condition: the implicits from TypedPipe[(K,V)] <=> Grouped[K,V].

The problem is that this exposed that some tests used .sum on a TypedPipe, which triggered an implicit before. When the patch adding sum to TypedPipe was added, this elimited the implicit conversion, which changed the result of the code.

I also had a problem of triggering extra map-reduce steps by putting the .withReducers(n) in the wrong place and actually doing a no-op map-reduce step due to the implicit conversion.

I think it is not worth it to have those conversions given the number of bugs we've already seen. To convert from pipe => grouped use .group, the reverse is done with .toTypedPipe (this are not new methods, but they must be called explicitly now).