## 2 Classifier Design
### Proposed PL-MSMM
\begin{split}&\min_{\mathcal{W}}\ \frac{1}{2}\parallel \mathcal{W}\parallel_{F}^{2}+\tau\sum_{c=1}^{k}\|\mathbf{\mathrm{W}}_{c}\|_{*}+\\ &\max_{\hat{g}_{1},...,\hat{g}_{n}\in\mathcal{Y}^{n}}C\Bigg{\{} \frac{1}{n}\sum_{i=1}^{n}\left\{\Delta(\hat{g}_{i},y_{i})+\langle\mathcal{W}, \delta\mathbf{\Psi}(\mathbf{X}_{i},\hat{g}_{i},y_{i})\rangle\right\},\\ &\ \ -p\frac{1}{n}\sum_{i=1}^{n}\left\{\Delta(\hat{g}_{i},y_{i})+ \langle\mathcal{W},\delta\mathbf{\Psi}(\mathbf{X}_{i},\hat{g}_{i},y_{i})\rangle \right\}\Bigg{\}}.\end{split}  \tag{7}\]  
The objective function in (7) comprises three convex terms--that is, the nuclear and Frobenius norms that satisfy the triangle and homogeneity properties, respectively, the third term being the maximum value of a set of linear functions, making it convex. Consequently, the objective function in (7) is convex but non-differentiable and non-smooth. The ADMM framework can be used to address convex optimization problems by decomposing the objective function into subproblems that are easier to optimize [31; 32]. The optimization problem in (7) can be viewed as a combination of the loss function and regularization term. In other words, the objective function comprises two parts without any coupled constraints. By introducing a new variable \(\mathcal{Z}=\mathcal{W}\) and changing the variable \(\mathcal{W}\) in the second part to \(\mathcal{Z}\) while adding new constraint conditions, the ADMM framework can be used to handle both parts separately. The original problem can be rewritten as follows:  
\[ \begin{split}&\min_{\mathcal{W},\mathcal{Z}}\ \ P(\mathcal{W})+Q(\mathcal{Z})\\ &\text{s.t.}\ \ \mathcal{W}-\mathcal{Z}=0,\end{split}  \tag{8}\]  
where \(\mathcal{Z}\in\mathbb{R}^{p\times q\times k}\) denotes the additional variable that decomposes the original problem into two subproblems. \[ \begin{split} P(\mathcal{W})=&\max_{\hat{g}_{1},\ldots, \hat{g}_{n}\in\mathbb{Z}^{n}}C\Bigg{\{}\frac{1}{n}\sum_{i=1}^{n}\left\{\Delta( \hat{y}_{i},y_{i})+\langle\mathcal{W},\delta\mathbf{\Psi}(\mathbf{X}_{i},\hat{ y}_{i},y_{i})\rangle\right\},\\ &\qquad\qquad\qquad\qquad\frac{-p}{n}\sum_{i=1}^{n}\left\{\Delta( \hat{y}_{i},y_{i})+\langle\mathcal{W},\delta\mathbf{\Psi}(\mathbf{X}_{i},\hat{ y}_{i},y_{i})\rangle\right\}\Bigg{\}},\\ Q(\mathcal{Z})=\frac{1}{2}\parallel\mathcal{Z}\parallel_{F}^{2}+ \tau\sum_{c=1}^{k}\left\|\mathbf{Z}_{c}\right\|_{*},\end{split}  \tag{9}\]  
where \(P(\mathcal{W})\) denotes the loss function and \(Q(\mathcal{Z})\) includes the regular term. The augmented Lagrangian method can be applied to solve (8):  
\[L(\mathcal{W},\mathcal{Z},\Lambda)=P(\mathcal{W})+Q(\mathcal{Z})+\langle \Lambda,(\mathcal{Z}-\mathcal{W})\rangle+\frac{\rho}{2}\parallel\mathcal{Z}- \mathcal{W}\parallel_{F}^{2}, \tag{11}\]  
where \(\Lambda\) denotes the Lagrangian multiplier and \(\rho>0\) denotes the hyper-parameter. Then, we can decompose the optimization problem into two subproblems: \(\mathcal{W}\) and \(\mathcal{Z}\). The ADMM algorithm minimizes and alternates by iteratively solving and then updating the Lagrange multiplier multipliers \(\Lambda\) accordingly. \[\mathcal{Z}^{t+1}=\operatorname*{arg\,min}_{\mathcal{Z}}\,L\big{(}\mathcal{Z},\mathcal{W}^{t},\Lambda^{t}\big{)}, \tag{12}\]  
\[\mathcal{W}^{t+1}=\operatorname*{arg\,min}_{\mathcal{W}}\,L\Big{(}\mathcal{Z} ^{t+1},\mathcal{W},\Lambda^{t}\Big{)}, \tag{13}\]  
\[\Lambda^{t+1}=\Lambda^{t}+\rho\Big{(}\mathcal{Z}^{t+1}-\mathcal{W}^{t+1} \Big{)}, \tag{14}\]  
where \(t\) and \(t+1\) denote the \(t\)-th and \(t+1\)-th iterations, respectively. The derivations of \(\mathcal{Z}^{t+1}\) and \(\mathcal{W}^{t+1}\) are discussed next.