Update the Ledoit-Wolf covariance shrinkage methodology to include modern methods

### Describe the workflow you want to enable

I've been working on implementing partial correlation with basis shrinkage in Python.  in particular, I've been porting R code to Python. 

The relevant partial correlation publications are the following:
* [Erb et al. 2020](https://www.sciencedirect.com/science/article/pii/S2590197420300082)
* [Jin et al. 2022](https://arxiv.org/pdf/2212.00496.pdf)

Which implemented in the [Propr](https://github.com/tpq/propr) compositional data analysis R package in the function called [bShrink](https://github.com/tpq/propr/blob/12553b3bcd159649f25d9a0e480250c1eee1d965/R/1-propr.R#L326).

This function uses the [cov.shrink function](https://rdrr.io/cran/corpcor/man/cov.shrink.html) from the [corpcor package](https://rdrr.io/cran/corpcor/).

The implementation they use builds on the original Ledoit-Wolf algorithm in the following ways:

> var.shrink computes the empirical variance of each considered random variable, and shrinks them towards their median. The shrinkage intensity is estimated using estimate.lambda.var (Opgen-Rhein and Strimmer 2007).
> 
> Similarly cor.shrink computes a shrinkage estimate of the correlation matrix by shrinking the empirical correlations towards the identity matrix. In this case the shrinkage intensity is computed using estimate.lambda (Sch\"afer and Strimmer 2005).


**Additional References:**

Opgen-Rhein, R., and K. Strimmer. 2007. Accurate ranking of differentially expressed genes by a distribution-free shrinkage approach. Statist. Appl. Genet. Mol. Biol. 6:9. <DOI:10.2202/1544-6115.1252>

Schafer, J., and K. Strimmer. 2005. A shrinkage approach to large-scale covariance estimation and implications for functional genomics. Statist. Appl. Genet. Mol. Biol. 4:32. <DOI:10.2202/1544-6115.1175>

### Describe your proposed solution

Rpy2 wrapper which makes the compute environment very bulky and dependency heavy

### Describe alternatives you've considered, if relevant

Reimplementing but outside of my expertise for this function.  I'm using the outdated solution currently but would be helpful to update to the more modern (though, 2007 isn't too new) solution. 

### Additional context

https://scikit-learn.org/stable/modules/generated/sklearn.covariance.ledoit_wolf.html#sklearn.covariance.ledoit_wolf

https://rdrr.io/cran/corpcor/man/cov.shrink.html




> In comparison with the standard empirical estimates (var, cov, and cor) the shrinkage estimates exhibit a number of favorable properties. For instance,
> 
> they are typically much more efficient, i.e. they show (sometimes dramatically) better mean squared error,
> 
> the estimated covariance and correlation matrices are always positive definite and well conditioned (so that there are no numerical problems when computing their inverse),
> 
> they are inexpensive to compute, and
> 
> they are fully automatic and do not require any tuning parameters (as the shrinkage intensity is analytically estimated from the data), and
> 
> they assume nothing about the underlying distributions, except for the existence of the first two moments.
> 
> These properties also carry over to derived quantities, such as partial variances and partial correlations (pvar.shrink and pcor.shrink).
> 
> As an extra benefit, the shrinkage estimators have a form that can be very efficiently inverted, especially if the number of variables is large and the sample size is small. Thus, instead of inverting the matrix output by cov.shrink and cor.shrink please use the functions invcov.shrink and invcor.shrink, respectively.

COMMENT:
> Does that mean the Ledoit algorithm will be removed from sklearn instead of updated?

I don't believe we intend to remove anything, rather @lorentzenchr and @glemaitre are discussing whether we intend to no longer accept further updates to the existing work. You can read a bit about the "soft deprecation" term [here](https://github.com/scikit-learn/scikit-learn/pull/27003).

I personally am not familiar with this area and would rather defer to those core-developers who are more up to date.

Labels: New Feature, Needs Decision