MAHOUT-1615: drmFromHDFS returning the same Key for all Key,Vec Pairs for Text-Keyed SequenceFiles

SparkContext.sequenceFile(...) will yield the same key per partition for Text-Keyed Sequence files if the key a new copy of the key is not created when mapping to an RDD.  This patch checks for Text Keys and creates a copy of each Key if necessary.    
perhaps naming is to blame. key2val was meant to be the transformation from
file key (i.e. writable) to actual non-reused type such as Int etc. val2key
was meant to be inverse (and in the non-edited code it is), but it is not
used in context of this method and therefore should be omited. i.e it
should be simply

       val (key2val, exactTag) = .... match ...




On Wed, Sep 24, 2014 at 2:23 PM, Andrew Palumbo <notifications@github.com>
wrote:

> In spark/src/main/scala/org/apache/mahout/sparkbindings/SparkEngine.scala:
>
> >            (v: AnyRef) => v.asInstanceOf[LongWritable].get,
> > -          (x: Any) => new LongWritable(x.asInstanceOf[Int]),
> > +          (x: Any) => new LongWritable(x.asInstanceOf[LongWritable].get),
>
> I'm not sure here- we need to remove one of the functions them but I think
> we should be using val2key here not key2val correct?
>
>      key2val(v: AnyRef) => v.asInstanceOf[IntWritable].get
>      val2key(x: Any) => new Integer(x.asInstanceOf[IntWritable].get)
>
> so later when we map to the RDD:
>
> val drmRdd = rdd.map { t => val2keyFunc(t._1) -> t._2.get()}
>
> they will be of form [Integer][Vector]rather than [IntWritable][Vector]
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/mahout/pull/52/files#r18001536>.
>