# Load Balancing of Two-Sided Assembly Line Based on Deep Reinforcement Learning
## 6 Conclusions and Future Work Avenues  
In this article, the load balancing-oriented TALBP has been studied and a deep reinforcement learning algorithm combining distributed proximal policy optimization (DPPO) and convolutional neural network (CNN) has been presented. To the best of our knowledge, this is the first attempt to solve the load balancing-oriented TALBP based on deep reinforcement learning. A mathematical model with objectives of optimizing line efficiency (\(LE\)), smoothness index (\(SI\)), and completion time smoothness index (\(CSI\)) is provided. In the proposed deep reinforcement learning algorithm, distributed multiple processes have been proposed to improve the search speed and capability of solutions. A total of 18 task assignment state features for the load balancing-oriented TALBP environment have been considered, which ensure that the agent can obtain more useful information from the environment and perform optimal selection as completely as possible, and different reward functions according to objectives and their implicit information have been proposed to guide good solution direction. Fianlly, the performance of the proposed algorithm has been verified on all scales of benchmark instances via a comparison with the single-process deep reinforcement learning algorithm in terms of model training and solution results.  
Although the proposed algorithm performs better in this study, there is still the possibility of improvement; for example, we could directly obtainin the Pareto-optimal solution set using the deep reinforcement learning algorithm and/or extend the application of the proposed algorithm to other versions of the TALBP or other types of assembly lines, such as linear and parallel assembly lines, as well as similar production planning problems, such as the two-sided disassembly line balancing problem [43], the assembly sequence problem [44], and so on.  
Conceptualization, G.J. and S.S.; methodology, G.J. and Y.Z.; validation, C.W. and B.L.; formal analysis, G.J. and C.W.; investigation, C.W.; resources, S.S. and B.L.; data curation, Y.Z.; writing--original draft preparation, G.J., S.S. and B.L.; writing--review and editing, Y.Z., X.H., and C.W.; supervision, X.H.; project administration, Y.Z. and X.H.; funding acquisition, Y.Z. and X.H. All authors have read and agreed to the published version of the manuscript.  
This research is funded by China National Heavy Duty Truck Group Co., Ltd., (Project No. 22H010100926), and New young teachers research start-up Fund of Shanghai Jiao Tong University (Project No. 22X010503668).  
Not applicable.  
The data presented in this study are available on request from the corresponding author.  
The authors are grateful to the anonymous reviewers, whose valuable comments and suggestions helped a lot to improve this paper, and to the editors for their kind and sincere support.  
The authors declare no conflict of interest.  
\begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Instance**} & \multicolumn{2}{c}{**Training Time (h)**} \\ \cline{2-3}  & **DPPO–CNN** & **PPO–CNN** \\ \hline P9 & 0.15 & 0.10 \\ P12 & 0.20 & 0.12 \\ P16 & 0.35 & 0.20 \\ P24 & 0.50 & 0.35 \\ P65 & 1.00 & 0.68 \\ P148 & 1.50 & 0.95 \\ P205 & 4.00 & 2.34 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Comparison of the offline training times of DPPO–CNN and PPO–CNN.