thehyve/Rmodules — pull request #4 by cataphract (2014-01-16 10:02:47+00:00)
Title: Implement Fisher support for multi-series vars

This implements support for multiple rows (e.g. probes) returned from
high dimensional variables. The table with Fisher test expects that
whenever such variables exist, a cartesian product should be made,
like this:

Let's say we have two variables, a simple continuous one (C1) and a
multi-row high dimensional variable (H1) that for each patient gives
two values, one for probe p1 and another for probe p2. The values of
the variables for patients P1 and P2 would be:

PATIENT C1 H1
P1 1 [p1: 2, p2: 3]
P2 4 [p1: 5]

Then the result should be transformed like this:

PATIENT C1 H1 GROUP
P1 1 2 p1
P1 1 3 p2
P2 4 5 p1

If we have p patients and h high dimensional variables, each one with
exactly r_i rows (where i = 1,..., h), the number of rows in the final
result will be: p * r_1 * ... r_h

Implementing this feature required some significant changes. The table
(or more exactly its backing map) no longer stores the strings that
will be written directly in the cells of the output file. Instead, it
stores an intermediate result where the cell values are arbitrary
objects, not just string. In particular, for multi-row high
dimensional results, the backing map will store immutable maps where
the key is the label of the high dimensional row (e.g. the probe name)
and the value is the value for that patient, column number and high
dimensional row. Column::consumeResultingTableRows() was changed to
return Map instead of Map.

The column decorators NumericManualBinningColumnDecorator,
EvenSpacedBinningColumnDecorator and
EvenDistributionBinningColumnDecorator were changed in order to
support these map values. The changes in
NumericManualBinningColumnDecorator are modest, but the other two
decorators have to run the binning algorithm independently for each
high dimensional row and effectively transpose transpose the high
dimensional TabularResult at the end.

The cross product and effective creation of the final written result
is deferred into a new step class,
MultiRowAsGroupDumpTableResultsStep. The expanding itself is made in
the helper class ExpandingMapIterator.