# On the Adaptive Penalty Parameter Selection in ADMM
## 4 Conclusions  
In this paper, we analyzed different strategies for the adaptive selection of the penalty parameter in ADMM. Exploiting the equivalence between ADMM and the Douglas-Rachford splitting method applied to the dual problem, as suggested in [7], optimal penalty parameters can be estimated at each iteration from spectral stepsizes of a gradient step applied to the Fenchel conjugates of the objective functions. To this end, we selected different spectral steplength strategies based on the Barzilai-Borwein rules, which have been proved to be very efficient in the context of smooth unconstrained optimization.  
We compared the different adaptive strategies on the solution of problems coming from distributed machine learning and multiperiod portfolio optimization. The results show that, while adaptive versions of ADMM are usually more effective than the \"vanilla\" one (using a prefixed penalty parameter), different strategies might perform better on different problems. Moreover, in some cases, the proposed alternation rule might get stuck  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Method** & **Iterations** & **Density** & **Ratio** \\ \hline \multicolumn{4}{c}{\(\tau^{0}=0.1\)} \\ Vanilla ADMM & 1211 & 0.065 & 1.370 \\ Adaptive ADMM [7] & 505 & 0.065 & 1.370 \\ Adaptive ADMM-BB1 & 510 & 0.065 & 1.370 \\ Adaptive ADMM-BB2 & 645 & 0.065 & 1.370 \\ Adaptive ADMM-ABB\({}_{\text{min}}\) & 1573 & 0.065 & 1.370 \\ \hline \multicolumn{4}{c}{\(\tau^{0}=0.5\)} \\ Vanilla ADMM & 3000 & — & — \\ Adaptive ADMM [7] & 997 & 0.065 & 1.370 \\ Adaptive ADMM-BB1 & 691 & 0.065 & 1.370 \\ Adaptive ADMM-BB2 & 1077 & 0.065 & 1.370 \\ Adaptive ADMM-ABB\({}_{\text{min}}\) & 1161 & 0.065 & 1.370 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Performance for portfolio Dow Jones with 10-year simulation. \(\epsilon_{\text{cor}}=0.2\).  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Method** & **Iterations** & **Density** & **Ratio** \\ \hline \multicolumn{4}{c}{\(\tau^{0}=0.1\)} \\ Vanilla ADMM & 1421 & 0.023 & 2.005 \\ Adaptive ADMM [7] & 1140 & 0.023 & 2.005 \\ Adaptive ADMM-BB1 & 906 & 0.023 & 2.005 \\ Adaptive ADMM-BB2 & 3000 & — & — \\ Adaptive ADMM-ABB\({}_{\text{min}}\) & 3000 & — & — \\ \hline \multicolumn{4}{c}{\(\tau^{0}=0.5\)} \\ Vanilla ADMM & 1984 & 0.023 & 2.005 \\ Adaptive ADMM [7] & 1368 & 0.023 & 2.005 \\ Adaptive ADMM-BB1 & 947 & 0.023 & 2.005 \\ Adaptive ADMM-BB2 & 3000 & — & — \\ \hline \multicolumn{4}{c}{\(\tau^{0}=1\)} \\ Vanilla ADMM & 3000 & — & — \\ Adaptive ADMM [7] & 1789 & 0.023 & 2.005 \\ Adaptive ADMM-BB1 & 1307 & 0.023 & 2.005 \\ Adaptive ADMM-BB2 & 3000 & — & — \\ Adaptive ADMM-ABB\({}_{\text{min}}\) & 3000 & — & — \\ \hline \hline \end{tabular}
\end{table}
Table 5: Performance for portfolio NASDAQ100 with 10-year simulation.  
with a fixed penalty parameter, leading to a slower convergence. Future work will deal with the analysis of an improved version of the adaptation strategies, aimed at solving the aforementioned issue and their analysis on wider problem classes.  
All authors have contributed equally to this work. All authors have read and agreed to the published version of the manuscript.  
This work was partially supported by Istituto Nazionale di Alta Matematica-Gruppo Nazionale per il Calcolo Scientifico (INdAM-GNCS), by the Italian Ministry of University.  
Not applicable.  
The authors declare no conflict of interest.