# On the Adaptive Penalty Parameter Selection in ADMM
## 3 Numerical Experiments
### Portfolio Optimization  
In the modern portfolio theory, an optimal portfolio selection strategy has to realize a trade-off between risk and return. Recently, l1-regularized Markowitz models have been  
Figure 1: Relative residual (**left**) and \(\tau^{k}\) history (**right**) against number of iterations for the 4 consensus logistic regression instances. From **top** to **bottom**: a9a, cod-ma, ijcnn1, phishing. considered; the l1 penalty term is used to stabilize the solution process and to obtain sparse solutions, which allow one to reduce holding costs [21; 22; 23; 24]. We focus on a multi-period investment strategy [23] that is either medium- or long-term; thus, it allows the periodic reallocation of wealth among the assets based on available information. The investment period is partitioned into \(m\) sub-periods, delimited by the rebalancing dates \(t_{1},...,t_{m+1}\), at which points decisions are taken. Let \(n\) be the number of assets and \(u_{j}\in\mathbb{R}^{n}\) the portfolio held at the rebalancing date \(t_{j}\). The optimal portfolio is defined by the vector \(u=(u_{1}^{\top},u_{2}^{\top},\ldots,u_{m}^{\top})^{\top}\in\mathbb{R}^{N}\), where \(N=m\cdot n\). A separable form of the risk measure obtained by summing single period terms is considered  
\[\rho(u)=\sum_{j=1}^{m}u_{j}^{\top}C_{j}u_{j},\]  
where \(C_{j}\in\mathbb{R}^{n\times n}\) is the covariance matrix, assumed to be positive definite, estimated at \(t_{j}\). With this choice, the model satisfies the time consistency property. \(l_{1}\)-regularization has been used to promote sparsity in the solution. Moreover, the \(l_{1}\) penalty term either avoids or limits negative solutions; thus, it is equivalent to a penalty on short positions. Let \(\xi_{\text{init}}\) and \(\xi_{\text{term}}\) be the initial wealth and the target expected wealth resulting from the overall investment, respectively, and let \(r_{j}\) be the expected return vector estimated at time \(j\). The l1-regularized selection can be formulated as the following compact constrained optimization problem [23; 24]:  
\[ \begin{array}{ll}\min_{u}&\frac{1}{2}u^{\top}Cu+\lambda\|u\|_{1}\\ \text{s.t.}&Au=b\end{array}  \tag{27}\]  
where \(C=diag(C_{1},C_{2},\ldots,C_{m})\in\mathbb{R}^{N\times N}\) is a \(m\times m\) diagonal block matrix, \(A\) is an \(m\times m\) lower bidiagonal block matrix, with blocks of dimension \(1\times n\), defined as  
\[ \begin{array}{ll}diag(A)&=\ (1_{n}^{\top},1_{n}^{\top},\ldots,1_{n}^{\top}), \\ subdiag(A)&=\ (-(1_{n}+r_{1})^{\top},\ldots,-(1_{n}+r_{m-1})^{\top}),\end{array} \]  
and \(b=(\xi_{\text{init}},0,0,\ldots,\xi_{\text{term}})^{\top}\in\mathbb{R}^{m}\). Methods based on Bregman iteration have proved to be efficient for the solution of Problem (27) as well [23; 24; 25; 26]. Now we reformulate Problem (27) as  
\[ \begin{array}{ll}\min_{u,v}&H(u)+G(v)\\ \text{s.t.}&u-v=0\end{array}  \tag{28}\]  
where \(H(u)=\frac{1}{2}u^{\top}Cu\) restricted to the set \(\{u\,|\,Au=b\}\) and \(G(v)=\lambda\|v\|_{1}\). Scaling the dual variable, the augmented Lagrangian function associated with Problem (28) is  
\[\mathcal{L}_{A}=H(u)+G(v)+\frac{\tau}{2}\bigg{\|}u-v-\frac{\xi}{\tau}\bigg{\|} ^{2}-\frac{\tau}{2}\bigg{\|}\frac{\xi}{\tau}\bigg{\|}^{2},\]  
where \(\xi,\tau>0\). Starting from given estimates \(u^{0},v^{0},\xi^{0}\), and \(\tau^{0}\), at each iteration \(k\), adaptive ADMM updates the estimates as  
\[u^{k+1} = \operatorname*{argmin}_{Au=b}\frac{1}{2}u^{\top}Cu+\frac{\tau^{k }}{2}\bigg{\|}u-v^{k}-\frac{\xi^{k}}{\tau^{k}}\bigg{\|}^{2}, \tag{29}\] \[v^{k+1} = \operatorname*{argmin}_{v}\lambda\|v\|_{1}+\frac{\tau^{k}}{2} \bigg{\|}u^{k+1}-v-\frac{\xi^{k}}{\tau^{k}}\bigg{\|}^{2},\] (30) \[\xi^{k+1} = \xi^{k}+\tau^{k}(u^{k+1}-v^{k+1}). \tag{31}\]Given \((u^{k},v^{k},g^{k})\), the u update (29) is an equality-constrained problem that involves the solution of the related Karush-Kuhn-Tucker (KKT) system (see [1], Section 4.2.5), which results in a linear system with positive definite coefficient matrix.