## 6 Entropy Based on a Wavelet Transform and a Noise Model  
Image filtering allows, as a special case, thresholding and reading off segmented regions. Such approaches have been used for very fast - indeed one could say with justice, turbo-charged-clustering. See [18; 19]. Noise models are particularly important in the physical sciences (cf. CCD, charge-coupled device, detectors) and the following approach was developed in [20]. Observed data \(f\) in the physical sciences are generally corrupted by noise, which is often additive and which follows in many cases a Gaussian distribution, a Poisson distribution, or a combination of both. Other noise models may also be considered. Using [PERSON] theorem to evaluate the probability distribution of the realization of the original signal \(g\), knowing the data \(f\), we have:  
\[\mathrm{p}(g|f)=\frac{\mathrm{p}(f|g).\mathrm{p}(g)}{\mathrm{p}(f)} \tag{3}\]  
\(\mathrm{p}(f|g)\) is the conditional probability distribution of getting the data \(f\) given an original signal \(g\), i.e., it represents the distribution of the noise. It is given, in the case of uncorrelated Gaussian noise with variance \(\sigma^{2}\), by:  
\[\mathrm{p}(f|g)=\exp\left(-\sum_{pixels}\frac{(f-g)^{2}}{2\sigma^{2}}\right) \tag{4}\]  
The denominator in Equation (3) is independent of \(g\) and is considered as a constant (stationary noise). \(\mathrm{p}(g)\) is the a priori distribution of the solution \(g\). In the absence of any information on the solution \(g\) except its positivity, a possible course of action is to derive the probability of \(g\) from its entropy, which is defined from information theory. If we know the entropy \(H\) of the solution (we describe below different ways to calculate it), we derive its probability by:  
\[\mathrm{p}(g)=\exp(-\alpha H(g)) \tag{5}\]  
Given the data, the most probable image is obtained by maximizing \(\mathrm{p}(g|f)\). This leads to algorithms for noise filtering and to deconvolution [16]. We need a probability density \(p(g)\) of the data. The Shannon entropy, \(H_{S}\)[21], is the summing of the following for each pixel,  
\[H_{S}(g)=-\sum_{k=1}^{N_{b}}p_{k}\log p_{k} \tag{6}\]where \(X=\{g_{1},..g_{n}\}\) is an image containing integer values, \(N_{b}\) is the number of possible values of a given pixel \(g_{k}\) (256 for an 8-bit image), and the \(p_{k}\) values are derived from the histogram of \(g\) as \(p_{k}=\frac{m_{k}}{n}\), where \(m_{k}\) is the number of occurrences in the histogram's \(k\)th bin. The trouble with this approach is that, because the number of occurrences is finite, the estimate \(p_{k}\) will be in error by an amount proportional to \(m_{k}^{-\frac{1}{2}}\)[22]. The error becomes significant when \(m_{k}\) is small. Furthermore this kind of entropy definition is not easy to use for signal restoration, because its gradient is not easy to compute. For these reasons, other entropy functions are generally used, including:  
* Burg [23]: \[H_{B}(g)=-\sum_{k=1}^{n}\ln(g_{k})\] (7)
* Frieden [24]: \[H_{F}(g)=-\sum_{k=1}^{n}g_{k}\ln(g_{k})\] (8)
* Gull and Skilling [25]: \[H_{G}(g)=\sum_{k=1}^{n}g_{k}-M_{k}-g_{k}\ln\left(\frac{g_{k}}{M_{k}}\right)\] (9) where \(M\) is a given model, usually taken as a flat image  
In all definitions \(n\) is the number of pixels, and \(k\) represents an index pixel. For the three entropies above, unlike [PERSON]'s entropy, a signal has maximum information value when it is flat. The sign has been inverted (see Equation (5)), to arrange for the best solution to be the smoothest. Now consider the entropy of a signal as the sum of the information at each scale of its wavelet transform, and the information of a wavelet coefficient is related to the probability of it being due to noise. Let us look at how this definition holds up in practice.