In statistics it is necessary to study the relation among many probability distributions.
Information geometry elucidates the geometric structure on the space of all distributions.
They uniquely exist in the space of finite measures, and are generally sufficient statistic. The optimal estimate on any model is given by projecting the ideal estimate onto that model. An error decomposition theorem splits the error of an estimate into the sum of statistical error and approximation error. They can be expanded to yield higher order asymptotics.