An estimator of the mutual information based on a criterion for conditional independence
The mutual information is a measure of stochastic dependence. Here, we present a data-dependent nonparametric estimator of the mutual information. The algorithm, upon which this estimator rests, is based on Dobrushin's information theorem. The key idea is to build a succession of finer and finer partitions made of nested hyperrectangles, and stop the refinement process on any hyperrectangle as soon as local independence has been achieved. The bias and variance of this estimator are studied through simulations. This includes a comparison to maximum likelihood estimators.
mutual information nonparametric estimation statistical dependence data-dependent partitions