# Regularized Within-Class Precision Matrix Based PLDA in Text-Dependent Speaker Verification
## 3 Graphical Least Absolute Shrinking and Selection Operator (GLASSO)
### Glasso  
The GLASSO is a variable selection method to estimate a sparse precision matrix using the \(L_{1}\) (lasso) penalty. In other words, it estimates a sparse undirected graphical model. Consider that we have samples (corresponding to speaker embeddings in our case) that follow a multivariate normal distribution \(N(\mathbf{\mu},\ \mathbf{\Sigma})\). Let \(\mathbf{\Theta}=\mathbf{\Sigma}^{-1}\) be the true precision matrix and \(\mathbf{S}\) be the empirical covariance matrix. The GLASSO-regularized precision matrix \(\overset{\wedge}{\mathbf{\Theta}}\) is defined by maximizer of the \(L_{1}\)-penalized Gaussian log-likelihood:  
\[\overset{\wedge}{\mathbf{\Theta}}=\underset{\mathbf{\Theta}}{\operatorname{ argmax}}\{\log\det(\mathbf{\Theta})-\operatorname{tr}(\mathbf{S}\mathbf{\Theta})-p \|\mathbf{\Theta}\|_{1}\} \tag{8}\]  
where \(\det(\cdot)\) and \(\operatorname{tr}(\cdot)\) are the determinant and trace of a matrix, respectively, \(\rho\) (\(>0\)) is a regularization parameter, and \(\|\cdot\|_{1}\) is the \(L_{1}\) norm operator (sum of the absolute values of the elements). We omit diagonal elements from the penalty. Therefore, only off-diagonal elements are penalized. The GLASSO is a biased estimator that shrinks all non-zero elements in the estimated precision matrix toward zero. A higher \(\rho\) results in (i) more regularization, (ii) lower estimation error with an accompanying higher bias in the estimated precision matrix, and (iii) a sparser estimated precision, and vice versa. Thus, the GLASSO requires the sparse assumption of the true precision matrix for the desired good asymptotic properties, such as selection consistency [49; 50]. In addition, the selection consistency requires a strict condition on the covariance matrix \(\mathbf{S}\), the irrepresentable condition [49; 50]. Typically, the GLASSO can detect the conditional independence structure on the considered covariates only when the covariates are not severely dependent. Therefore, a proper value of \(\rho\) must be selected for the performance, and simultaneously, the underlying covariance structure of the considered model should be considered. To summarize, the GLASSO regularization can reduce the estimation error in the precision matrix by pursuing a sparse structure, but the associated improvement depends on the underlying model. In [51], solve (8) using convex duality, taking advantage of the fact that the optimization of (8) is a kind of convex optimization. In other words, [51] solve (8) by estimating \(\mathbf{\Sigma}\), rather than \(\mathbf{\Sigma}^{-1}\), using block coordinate descent algorithm, as follows. Let \(\mathbf{W}\) be the regularization of \(\mathbf{S}\). In general, \(\mathbf{W}\) is initialized to \(\mathbf{S}+\rho\mathbf{I}\). This algorithm optimizes each column (and corresponding row) of \(\mathbf{W}\) iteratively until convergence. For each iteration, \(\mathbf{W}\) and \(\mathbf{S}\) are partitioned as follows:  
\[\mathbf{W}=\left( \begin{array}{cc}\mathbf{W}_{11}&\mathbf{w}_{12}\\ \mathbf{w}_{12}^{T}&w_{22}\end{array} \right) \tag{9}\]  
\[\mathbf{S}=\left( \begin{array}{cc}\mathbf{S}_{11}&\mathbf{s}_{12}\\ \mathbf{s}_{12}^{T}&s_{22}\end{array} \right) \tag{10}\]  
for each \(i\)-th column, where \(\mathbf{W}_{11}\) and \(\mathbf{S}_{11}\) are the submatrix of \(\mathbf{W}\) and \(\mathbf{S}\) (obtained by excluding the \(  
usage of the GLASSO-PLDA for computing the log-likelihood ratio (7) is the same as those of the conventional PLDA. The GLASSO-PLDA is based on our assumption that an optimal solution for the within-class precision matrix (as mentioned in Section 1, we focus on the within-class precision matrix only) exists at some point between a high variance (corresponding to low \(\rho\)) and high bias (corresponding to high \(\rho\)). Notice that there is a trade-off between estimation error and bias, as mentioned in Section 3.2.