Talk:Fisher information metric

message: I suspect the appropriate fate of this article is to redirect it to Fisher information, incorporate everything here into that article, and greatly expand it so it's no longer a stub, putting in the usual topics on Fisher information, which at this point are missing. Michael Hardy 01:14, 7 Apr 2004 (UTC)
thread_title: Redirect

message: Anyone know what a "statistical manifold" is? Or a "statistical differential manifold"? Melcombe (talk) 14:15, 18 September 2009 (UTC)
replies: text: Statistical manifold = smooth parametric model = a finite-dimensional family of probability distributions that can be viewed as a smooth manifold = it can be smoothly parametrized by finite-dimensional θ, at least locally.  … stpasha »  17:40, 9 February 2010 (UTC)
replies: text: And the article doesn't say anything like that, at present. Melcombe (talk) 10:54, 10 February 2010 (UTC)
thread_title:  Question 

message: Why doesn't this article mention the Cramér-Rao bound? Seems to this reader that belongs in the first paragraph. It is clearly the most important application, and the link will lead readers to a more accessible treatment of the concept in a simpler situation.  — Preceding unsigned comment added by Robroot (talk • contribs) 23:20, 1 March 2014 (UTC)
thread_title: Another Question

message: In this edit, User:Melcombe asked for clarification. 
I'll clarify here, rather then mess up the article: yes, the  expressions are equivalent; plug one into the other and turn the crank. If you get stuck, recall that 
:\int p(x,\theta) dx = 1
and that the second derivative of 1 is zero. (and that the derivatives commute with the integral). linas (talk) 02:29, 12 July 2012 (UTC)
thread_title: Clarification

message: The section "Formal definition" is not a formal mathematical definition at all, it is incoherent. Here is the section in question: 
:Let X be an orientable manifold, and let (X,\Sigma,\mu) be a measure on X. Equivalently, let (\Omega, \mathcal{F},P) be a probability space on \Omega=X, with sigma algebra \mathcal{F}=\Sigma and probability P=\mu.
:The statistical manifold S(X) of X is defined as the space of all measures \mu on X (with the sigma-algebra \Sigma held fixed).  Note that this space is infinite-dimensional, and is commonly taken to be a Fréchet space.  The points of S(X) are measures.
:Pick a point \mu\in S(X) and consider the tangent space T_\mu S. The Fisher information metric is then an inner product on the tangent space.  With some abuse of notation, one may write this as
::g(\sigma_1,\sigma_2)=\int_X \frac{d\sigma_1}{d\mu}\frac{d\sigma_2}{d\mu}d\mu
:Here, \sigma_1 and \sigma_2 are vectors in the tangent space; that is,  \sigma_1,\sigma_2\in T_\mu S.  The abuse of notation is to write the tangent vectors as if they are derivatives, and to insert the extraneous d in writing the integral: the integration is meant to be carried out using the measure \mu over the whole space X.
The tangent space to a space of measures S at a measure μ is the linear space S itself. So in the displayed formula, each tangent vector σi is a measure. Then dσi/dμ must be the (absolutely continuous part of the) Radon-Nikodym derivative of σi with respect to μ. This is not an abuse of notation; it is standard notation and has a completely rigorous definition. There are not too many d's, that is fine, no need to apologize. And the expressions σi and dσi/dμ are not both tangent vectors. The first is a tangent vector, and the second is a linear image of it that is used for defining the inner product.
That is where the problems begin. In the formula, Wikipedia is attempting to multiply two such functions and integrate them against the underlying measure. There are two problems with this:
(1) The functions may not lie in L2, so that the definition makes no sense. (The integral cannot be carried out.)
(2) If σi is singular with respect to μ, then the given expression (the absolutely continuous part of the Radon-Nykodym derivative) is too small to capture the size of the vector σi relative to μ. In some cases  (for example when σ1=σ2), it could be useful to conventionally assume the integral to be infinity, and work on interpreting that. In other cases, it's like the objection in (1) but worse.
So I would not characterize this as a "formal" definition, but rather as a blackboard proposal of a definition that is interesting, but doesn't work yet.
There is a whole literature on putting both distance metrics and Riemannian metrics on spaces of measures and on Banach manifolds. The problems are also well known. For L^2 type Riemannian metrics, similar to what is being attempted here, there are some papers of Marsden and other nonlinear functional analysts starting in the 1970s. And so forth. 178.38.76.15 (talk) 09:52, 22 November 2014 (UTC)
replies: text: P.S. As an example of how to do this kind of thing correctly, see Kullback–Leibler_divergence#Definition, where all the caveats are correctly explained. 178.38.76.15 (talk) 10:08, 22 November 2014 (UTC)

text: P.P.S. The substantive problems go away if we restrict the scope of the definition to finite probability spaces, though some problems of notation and expression remain. 178.38.157.66 (talk) 15:42, 23 November 2014 (UTC)
replies: text: The category of probability spaces are precisely those spaces that have the Nikodym-Radon property; the Hilbert spaces are square-integrable, and so there is no question that the integral is convergent. I believe that the confusion about the derivative can be ascribed to the fact that it is an abuse of notation: you can get away with thinking of these as being actual "derivatives", for a little while, but then run into trouble. Instead, you have to pick a weak topology, and perform integration on that. When you do so, the use of the "d" symbol becomes non-sensical; its not terribly well-defined. That's why its an abuse of notation. The L^2 property that you are looking for is possessed by the Hilbert spaces, and the topology that you must endow these spaces with is the weak topology; the condition that something be square integrable is the same thing as saying that a Cauchy sequence converges to a finite value in the weak topology.   67.198.37.16 (talk) 18:23, 30 September 2016 (UTC)
thread_title:  Definition is mathematically incoherent 

message: There are dozens of places in this article, in most of the sections, where the mathematics is buggy or even incoherent.
Foe example:
:Definition The "equivalent formulation" misses the mark. It's the wrong equivalence for the derivation given.
:Relation to the Kullback–Leibler divergence Not explicit enough. Does it only work at extreme points?
:Change in free entropy Criticized as incoherent on the talk page
:Relation to the Jensen–Shannon divergence Notation is not understandable
:As Euclidean metric Criticized as incoherent on the talk page
:As Fubini–Study metric Fails to specify variables. A lot of incoherence.
:Continuously-valued probabilities More incoherence. Objects not defined correctly.
The article needs a serious warning: "Needs attention from an expert." I don't know how to add it.
2A02:1210:2642:4A00:9010:30F5:E080:70AE (talk) 18:40, 7 February 2024 (UTC)
thread_title:  The article is largely incoherent