# A Probabilistic Hyperspectral Imagery Restoration Method
## 4 Probabilistic LRMA Model for HSI Restoration
### Clean HSI Estimation Method
First, an intermediate variable \(\mathbf{t}_{ij}=(t_{ij1},\cdots,t_{ijk})\in\{0,1\}^{K}\), which satisfies a multinomial distribution parameterized by \(\boldsymbol{\omega}\), is introduced to represent \(\mathbf{S}\) as  
\[s_{ij}\sim\prod_{k=1}^{K}N(\mathbf{s}_{ij}|\mu_{k},\tau_{k}^{-1})^{t_{ijk}}. \tag{14}\]  
Second, to guarantee Equation (12) tractable, we represent \(\mu_{k}\), \(\tau_{k}\) and \(\boldsymbol{\omega}\) using the following conjugate priors  
\[ \begin{split} p(\tau_{k})&\sim\text{Gam}(\tau_{k}| c_{0},d_{0})\\ p(\mu_{k})&\sim N(\mu_{k}|\mu_{0},(\rho_{0}\tau_{k })^{-1})\\ p(\boldsymbol{\omega})&\sim\text{Dir}(\boldsymbol{ \omega}|\boldsymbol{\alpha}_{0})\end{split}  \tag{15}\]  
where \(c_{0}\), \(d_{0}\), and \(\boldsymbol{\alpha}_{0}=(\alpha_{01},\cdots,\alpha_{0K})\) are the parameters of above distributions [42]. With above representations, we build a full probabilistic low-rank model, which is tractable. Thus, the estimated parameters are transformed into \(\{\mathbf{M},\mathbf{H},\mathbf{t},\omega,\boldsymbol{\mu},\boldsymbol{\tau},\theta\}\) from \(\{\mathbf{M},\mathbf{H},\boldsymbol{\omega},\boldsymbol{\mu},\boldsymbol{ \tau},\theta\}\), where \(\mathbf{t}=\{\mathbf{t}_{ij}\}\). For simplicity, we use \(\Theta\) to denote all variables of \(\{\mathbf{M},\mathbf{H},\mathbf{t},\boldsymbol{\omega},\boldsymbol{\mu}, \boldsymbol{\tau},\theta\}\) in the following. Third, a new simple distribution \(q(\Theta)\) is introduced to approximate \(p(\Theta|\mathbf{X})\) as  
\[\text{KL}(q\|p)=-\!\!\int q(\Theta)\ln\!\left\{\frac{p(\Theta|\mathbf{X})}{q( \Theta)}\right\}\!d\Theta, \tag{16}\]  
where \(\text{KL}(q\|p)\) denotes the KL divergence between \(q(\Theta)\) and \(p(\Theta|\mathbf{X})\). Since minimizing KL divergence between \(q(\Theta)\) and \(p(\Theta|\mathbf{X})\) can guarantee the estimation of \(\Theta\) from \(q(\Theta)\) is close to the one learned from \(p(\Theta|\mathbf{X})\), we estimate \(\mathbf{M}\), \(\mathbf{H}\) together with other parameters by minimizing \(\text{KL}(q\|p)\) iteratively. The details can be seen from [42]. Here we only give the updating equation for \(\mathbf{m}_{i}\)(the \(i\)-th row of \(\mathbf{M}\)) and \(\mathbf{h}_{j}\). (the \(j\)-th row of \(\mathbf{H}\)). By defining \(q(\mathbf{m}_{i})=N(\mathbf{m}_{i}|\boldsymbol{\mu}_{m_{i}},\mathbf{\Sigma}_{ \mathbf{m}_{i}})\) and \(q(\mathbf{h}_{j})=N(\mathbf{h}_{j}|\boldsymbol{\mu}_{h_{j}},\mathbf{\Sigma}_{ \mathbf{h}_{i}})\), the parameters related with \(\mathbf{m}_{i}\) and \(\mathbf{h}_{j}\). can be inferred. \(\boldsymbol{\mu}_{\mathbf{m}_{i}}^{T}=\mathbf{\Sigma}_{\mathbf{m}_{i}}\!\left[ \sum_{k}\!\left(\tau_{k}\right)\!\sum_{j}\!\left(t_{ijk}\right)\!\left(x_{ij} -\!\left\langle\mu_{k}\right\rangle\!\right)\!\left\langle\mathbf{h}_{j} \right\rangle\right]^{T}\) and \(\boldsymbol{\mu}_{\mathbf{h}_{j}}^{T}=\mathbf{\Sigma}_{\mathbf{h}_{j}}\!\left[ \sum_{k}\!\left(\tau_{k}\right)\!\sum_{i}\!\left(t_{ijk}\right)\!\left(x_{ij} -\!\left\langle\mu_{k}\right\rangle\!\right)\!\left\langle\mathbf{m}_{i} \right\rangle\right]^{T}\) are the mean vectors for \(\mathbf{m}_{i}\) and \(\mathbf{h}_{j}\), while \(\mathbf{\Sigma}_{\mathbf{m}_{i}}=\left\{\sum_{k}\!\left(\tau_{k}\right)\!\sum_{j }\!\left\langle t_{ijk}\right\rangle\!\left\langle\mathbf{h}_{j}^{T}\mathbf{h}_ {j}\right\rangle+\boldsymbol{\Gamma}\right\}^{-1}\) and \(\mathbf{\Sigma}_{\mathbf{h}_{j}}=\left\{\sum_{k}\!\left(\tau_{k}\right)\! \sum_{i}\!\left\langle t_{ijk}\right\rangle\!\left\langle\mathbf{m}_{i}^{T} \mathbf{m}_{i}\right\rangle+\boldsymbol{\Gamma}\right\}^{-1}\) are the covariance matrices for \(\mathbf{m}_{i}\) and \(\mathbf{h}_{j}\). \(\boldsymbol{\Gamma}=\text{diag}(\Theta)\). When the stop criterion is satisfied, the optimized \(\mathbf{m}_{i}\) and \(\mathbf{h}_{j}\) are denoted as \(\mathbf{m}_{i}\)\({}^{*}\) and \(\mathbf{h}_{j}\)\({}^{*}\). Then each element in the denoised HSI image \(\mathbf{L}\) can be calculated by \(l_{ij}=\mathbf{m}_{i}\)\({}^{*}\times(\mathbf{h}_{j}\)\({}^{*})^{\text{T}}\).