Standard "Total Variance" Scaler

### Desired feature

A preprocessor that removes the mean for each feature, and then scales the total variance of the dataset, rather than the variance of each feature, to 1.

### Proposed Solution

A new preprocessor that operates like StandardScaler but automatically scales total-variance instead of the variance of individual feature

### Possible Alternatives

A new input parameter for StandardScaler that allows the user to set the variance of each feature, or which allows the user to identify groups of features to be considered individual "macro" features

### Additional context

Intended use case for situations where more than one feature (column) is associated with the same data-concept (like when including multiple points in space for sea surface temperature in the Pacific and also multiple points in space for sea level pressure in the Atlantic)

COMMENT:
@jnothman @fkdosilovic Thank you for thinking about this!

> But I do think it introduces complexity that would need to be justified not just through a theoretical use case, but through empirical demonstration that this actually makes a difference for learning from a dataset of more than a few points.

The use case is not so theoretical! In fact, it was inspired by my current project, and I believe I have such empirical documentation already. I am in the field of causal learning, and am working with [tigramite](https://github.com/jakobrunge/tigramite). I am working on methods for reducing information-losing dimension reduction prior to time-series causal learning, which typically takes only a single timeseries per conceptual node. Predictably, I found that after applying the StandardScaler to my complete dataset, the relative dimension of the different nodes dramatically affects learned relationships between them. 

I phrased the use case in a general way because I believe the problem is not specific to my current project -- I believe that this must also be a problem for all methods of computer learning with multi-dimensional variables, and, as a climate scientist, I assert that this is a problem for all analysis in my field.

>From an implementation angle we are just reshaping the data to one column before fitting the scalar etc and this might make sense as a parameter (single_distribution?) rather than a new estimator.

That makes sense @jnothman. I happened to do something slightly different after posting in order to see the empirical difference my requested estimator would make in my current analysis. Here's what I did: 

```python
class StandardTotalVarianceScaler(StandardScaler):
    '''As in sklearn's StandardScaler, but total variance is scaled rather  
    than feature variance. Total variance of an array A_ij with I samples  
    and J features is defined to be sum_ij(a_ij)/I.'''
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
    
    def fit(self, X, y=None, sample_weight=None):
        T = super().fit(X, sample_weight=sample_weight)
        T.scale_ = np.sqrt(np.sum(T.var_))*np.ones(T.scale_.shape)
        return T
```

Tigramite already has code that divides datasets into query-related groups and fits a deepcopy of the chosen sklearn transformation to each part, because we regularly later want to transform only a part of the data. I've been working on a [modification](https://github.com/rebeccaherman1/tigramite/tree/macro_info) that would divide the dataset into the conceptual-variables instead of query-related groups, and then apply deepcopy's of the chosen sklearn transformation to each part. This is the context in which I find the total-variance scaler useful, whether it is implemented as I have done above or as @jnothman suggests. 

But I do imagine that this new scaler would be more widely useful if there was an sklearn-supported way of dividing datasets into groups. To this end, I had originally imagined an alternate implementation employing an additional input parameter:

>> A new input parameter for StandardScaler that allows the user to set the variance of each feature, or which allows the user to identify groups of features to be considered individual "macro" features

I suppose I misspoke above; I was actually imagining an additional input parameter to the `fit` function, that would be somewhat parallel to `sample_weight`.

>I'm not a fan of this alternative approach since it seems as a rare use case. Additionally, this would require adding the same parameter to other preprocessing functions and transformers, e.g. MinMaxScaler, MaxAbsScaler, Normalizer, etc., to keep the API consistent.

I actually strongly disagree with your perception @fkdosilovic that the use case would be rare, and would love to see dataset-dividing logic for these other Scalers as well. But perhaps, as you say, this additional input parameter is not the best approach... 

It might be nice instead to have a `DatasetDivider` object which would be sortof like a pipeline, but instead of iteratively applying a list of transformations, it would divide the data into user-designated groups and then apply different instances of the chosen transformation (which could be a pipeline) separately to each group. In this scenario, this functionality would be available for all transformations without further modifications to the other Scalers. For some preprocessing transformations (like StandardScaler), using a dataset-divider would make no difference. However it would make a difference for any implementation of the total-variance scaler, and for all sklearn decompositions. If this new object could accept data from a subset of data groups for `transform` and `inverse_transform`, then we wouldn't need the logic I've been implementing in tigramite or the logic that's already there to deal with multiple parallel sklearn transformations, or any other similar and redundant code implemented by other sklearn users.

Labels: New Feature