The weight is not unique to a label.  Different trainers could want different different weights as input.  As a side issue, Seq[Double] is full of wrapped Doubles.  At the very least it should be a DoubleSeq.  Although I'm not advocating getting rid of this seq in the backend, I think it is error-prone to line up sequences of labels and weights in the midst of shuffling, splitting, re-splitting, etc.