Module Expr Str Print NameLoad Import alias identifier Import alias identifier Import alias ImportFrom alias alias Assign NameStore Num Assign NameStore Num Assign NameStore Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad Call AttributeLoad NameLoad attr Num BinOpDiv NameLoad Num Num SubscriptLoad Call AttributeLoad NameLoad attr BinOpDiv NameLoad Num BinOpDiv NameLoad Num Num Slice Num Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad AttributeLoad NameLoad attr NameLoad For TupleStore NameStore NameStore Call NameLoad NameLoad body For NameStore Call NameLoad NameLoad body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMult NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr NameLoad Slice NameLoad Assign NameStore BinOpMult Num BinOpSub Call AttributeLoad AttributeLoad NameLoad attr attr Num keyword TupleLoad NameLoad NameLoad Num AugAssignAdd SubscriptStore NameLoad Index NameLoad NameLoad Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr NameLoad attr NameLoad Assign SubscriptStore NameLoad Index NameLoad NameLoad Assign NameStore Call AttributeLoad Call NameLoad attr NameLoad Assign SubscriptStore NameLoad Index TupleLoad NameLoad NameLoad Call AttributeLoad NameLoad attr BinOpPow AttributeLoad NameLoad attr Num Assign SubscriptStore NameLoad Index TupleLoad NameLoad NameLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign SubscriptStore NameLoad Index TupleLoad NameLoad NameLoad Call AttributeLoad NameLoad attr BinOpPow Call AttributeLoad NameLoad attr Num Num Assign SubscriptStore NameLoad Index TupleLoad NameLoad NameLoad Call AttributeLoad Call AttributeLoad Call NameLoad attr NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign NameStore SubscriptLoad NameLoad Index NameLoad Assign NameStore Call AttributeLoad NameLoad attr Num Assign NameStore Call AttributeLoad Call NameLoad attr NameLoad Assign SubscriptStore NameLoad Index TupleLoad NameLoad NameLoad Call AttributeLoad NameLoad attr BinOpPow NameLoad Num Assign SubscriptStore NameLoad Index TupleLoad NameLoad NameLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword Num Expr Call AttributeLoad NameLoad attr Num Num Num Assign NameStore Num Expr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr Num keyword BinOpDiv Call AttributeLoad NameLoad attr Num Call AttributeLoad NameLoad attr NameLoad keyword Str keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr Num keyword BinOpDiv Call AttributeLoad NameLoad attr Num Call AttributeLoad NameLoad attr NameLoad keyword Str keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr Num keyword BinOpDiv Call AttributeLoad NameLoad attr Num Call AttributeLoad NameLoad attr NameLoad keyword Str keyword NameLoad keyword Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr keyword Str keyword NameLoad Expr Call AttributeLoad NameLoad attr Num Num Num Assign NameStore AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr Num keyword Call AttributeLoad NameLoad attr Num keyword Str keyword Str Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice BinOpAdd BinOpDiv NameLoad Num Num SubscriptLoad Call AttributeLoad NameLoad attr Num Slice BinOpAdd BinOpDiv NameLoad Num Num keyword SubscriptLoad Call AttributeLoad NameLoad attr Num Slice BinOpAdd BinOpDiv NameLoad Num Num keyword Str keyword Str Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice BinOpDiv NameLoad Num BinOpSub BinOpDiv NameLoad Num Num SubscriptLoad Call AttributeLoad NameLoad attr Num Slice BinOpDiv NameLoad Num BinOpSub BinOpDiv NameLoad Num Num keyword Str keyword Str Expr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr Num keyword Call AttributeLoad NameLoad attr Num keyword Str keyword Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr keyword Str keyword NameLoad Expr Call AttributeLoad NameLoad attr
N/A N/A 
=======================================
Robust vs Empirical covariance estimate
=======================================

The usual covariance maximum likelihood estimate is very sensitive to the
presence of outliers in the data set. In such a case, it would be better to
use a robust estimator of covariance to guarantee that the estimation is
resistant to "erroneous" observations in the data set.

Minimum Covariance Determinant Estimator
----------------------------------------
The Minimum Covariance Determinant estimator is a robust, high-breakdown point
(i.e. it can be used to estimate the covariance matrix of highly contaminated
datasets, up to
:math:`\frac{n_\text{samples} - n_\text{features}-1}{2}` outliers) estimator of
covariance. The idea is to find
:math:`\frac{n_\text{samples} + n_\text{features}+1}{2}`
observations whose empirical covariance has the smallest determinant, yielding
a "pure" subset of observations from which to compute standards estimates of
location and covariance. After a correction step aiming at compensating the
fact that the estimates were learned from only a portion of the initial data,
we end up with robust estimates of the data set location and covariance.

The Minimum Covariance Determinant estimator (MCD) has been introduced by
P.J.Rousseuw in [1]_.

Evaluation
----------
In this example, we compare the estimation errors that are made when using
various types of location and covariance estimates on contaminated Gaussian
distributed data sets:

- The mean and the empirical covariance of the full dataset, which break
  down as soon as there are outliers in the data set
- The robust MCD, that has a low error provided
  :math:`n_\text{samples} > 5n_\text{features}`
- The mean and the empirical covariance of the observations that are known
  to be good ones. This can be considered as a "perfect" MCD estimation,
  so one can trust our implementation by comparing to this case.


References
----------
.. [1] P. J. Rousseeuw. Least median of squares regression. Journal of American
    Statistical Ass., 79:871, 1984.
.. [2] Johanna Hardin, David M Rocke. The distribution of robust distances.
    Journal of Computational and Graphical Statistics. December 1, 2005,
    14(4): 928-946.
.. [3] Zoubir A., Koivunen V., Chakhchoukh Y. and Muma M. (2012). Robust
    estimation in signal processing: A tutorial-style treatment of
    fundamental concepts. IEEE Signal Processing Magazine 29(4), 61-80.

 N/A __doc__ N/A numpy np N/A matplotlib.pyplot plt N/A matplotlib.font_manager sklearn.covariance EmpiricalCovariance MinCovDet N/A n_samples 80 N/A n_features 5 N/A repeat 10 N/A range_n_outliers N/A N/A np concatenate N/A N/A N/A np linspace 0 N/A n_samples 8 5 N/A N/A N/A np linspace N/A n_samples 8 N/A n_samples 2 5 N/A 1 -1 N/A err_loc_mcd N/A N/A np zeros N/A N/A range_n_outliers size repeat N/A err_cov_mcd N/A N/A np zeros N/A N/A range_n_outliers size repeat N/A err_loc_emp_full N/A N/A np zeros N/A N/A range_n_outliers size repeat N/A err_cov_emp_full N/A N/A np zeros N/A N/A range_n_outliers size repeat N/A err_loc_emp_pure N/A N/A np zeros N/A N/A range_n_outliers size repeat N/A err_cov_emp_pure N/A N/A np zeros N/A N/A range_n_outliers size repeat N/A N/A i n_outliers N/A enumerate range_n_outliers N/A N/A j N/A range repeat N/A N/A rng N/A N/A N/A np random RandomState N/A i j N/A X N/A N/A rng randn n_samples n_features N/A outliers_index N/A N/A N/A rng permutation n_samples N/A n_outliers N/A outliers_offset N/A 10.0 N/A N/A N/A N/A np random randint 2 size N/A n_outliers n_features 0.5 N/A N/A X N/A outliers_index outliers_offset N/A inliers_mask N/A N/A N/A N/A np ones n_samples astype bool N/A N/A inliers_mask N/A outliers_index False N/A mcd N/A N/A N/A MinCovDet fit X N/A N/A err_loc_mcd N/A N/A i j N/A N/A np sum N/A N/A mcd location_ 2 N/A N/A err_cov_mcd N/A N/A i j N/A N/A mcd error_norm N/A N/A np eye n_features N/A N/A err_loc_emp_full N/A N/A i j N/A N/A np sum N/A N/A N/A X mean 0 2 N/A N/A err_cov_emp_full N/A N/A i j N/A N/A N/A N/A N/A EmpiricalCovariance fit X error_norm N/A N/A np eye n_features N/A pure_X N/A X N/A inliers_mask N/A pure_location N/A N/A pure_X mean 0 N/A pure_emp_cov N/A N/A N/A EmpiricalCovariance fit pure_X N/A N/A err_loc_emp_pure N/A N/A i j N/A N/A np sum N/A pure_location 2 N/A N/A err_cov_emp_pure N/A N/A i j N/A N/A pure_emp_cov error_norm N/A N/A np eye n_features N/A font_prop N/A N/A N/A matplotlib font_manager FontProperties size 11 N/A N/A N/A plt subplot 2 1 1 N/A lw 2 N/A N/A N/A plt errorbar range_n_outliers N/A N/A err_loc_mcd mean 1 yerr N/A N/A N/A err_loc_mcd std 1 N/A N/A np sqrt repeat label Robust location lw lw color m N/A N/A N/A plt errorbar range_n_outliers N/A N/A err_loc_emp_full mean 1 yerr N/A N/A N/A err_loc_emp_full std 1 N/A N/A np sqrt repeat label Full data set mean lw lw color green N/A N/A N/A plt errorbar range_n_outliers N/A N/A err_loc_emp_pure mean 1 yerr N/A N/A N/A err_loc_emp_pure std 1 N/A N/A np sqrt repeat label Pure data set mean lw lw color black N/A N/A N/A plt title Influence of outliers on the location estimation N/A N/A N/A plt ylabel Error ($||\mu - \hat{\mu}||_2^2$) N/A N/A N/A plt legend loc upper left prop font_prop N/A N/A N/A plt subplot 2 1 2 N/A x_size N/A range_n_outliers size N/A N/A N/A plt errorbar range_n_outliers N/A N/A err_cov_mcd mean 1 yerr N/A N/A err_cov_mcd std 1 label Robust covariance (mcd) color m N/A N/A N/A plt errorbar N/A range_n_outliers N/A N/A N/A x_size 5 1 N/A N/A N/A err_cov_emp_full mean 1 N/A N/A N/A x_size 5 1 yerr N/A N/A N/A err_cov_emp_full std 1 N/A N/A N/A x_size 5 1 label Full data set empirical covariance color green N/A N/A N/A plt plot N/A range_n_outliers N/A N/A x_size 5 N/A N/A x_size 2 1 N/A N/A N/A err_cov_emp_full mean 1 N/A N/A x_size 5 N/A N/A x_size 2 1 color green ls -- N/A N/A N/A plt errorbar range_n_outliers N/A N/A err_cov_emp_pure mean 1 yerr N/A N/A err_cov_emp_pure std 1 label Pure data set empirical covariance color black N/A N/A N/A plt title Influence of outliers on the covariance estimation N/A N/A N/A plt xlabel Amount of contamination (%) N/A N/A N/A plt ylabel RMSE N/A N/A N/A plt legend loc upper center prop font_prop N/A N/A N/A plt show