###### Abstract  
Deep learning, with increasingly large datasets and complex neural networks, is widely used in computer vision and natural language processing. A resulting trend is to split and train large-scale neural network models across multiple devices in parallel, known as parallel model training. Existing parallel methods are mainly based on expert design, which is inefficient and requires specialized knowledge. Although automatically implemented parallel methods have been proposed to solve these problems, these methods only consider a single optimization aspect of run time. In this paper, we present Trinity, an adaptive distributed parallel training method based on reinforcement learning, to automate the search and tuning of parallel strategies. We build a multidimensional performance evaluation model and use proximal policy optimization to co-optimize multiple optimization aspects. Our experiment used the CIFAR10 and PTB datasets based on InceptionV3, NMT, NASNet and PNASNet models. Compared with Google's Hierarchical method, Trinity achieves up to 5% reductions in runtime, communication, and memory overhead, and up to a 40% increase in parallel strategy search speeds. distributed machine learning; deep learning; reinforcement learning +
Footnote †: journal:  
02221508080  
the granularity of the operator. For instance, [PERSON] et al. [PERSON] et al. (2014) used skeleton networks to achieve the random partition of the network and reduce redundancy. [PERSON] et al. (2015) and [PERSON] et al. [PERSON] et al. (2016) further introduced high-performance computing (HPC) matrix portion methods to achieve fine-grained parallelism for matrix multiplication operations inside operators. However, as the granularity of the division becomes finer, the distributed combinations increase exponentially ([PERSON] et al., 2016). It is also challenging to find an optimal parallel strategy using expert experience, which requires knowledge in many fields, such as deep learning, distributed computing and computer architecture; and it is difficult to generalize an optimal parallel strategy to other networks. To address these challenges, researchers proposed automated frameworks for finding efficient parallelization strategies in a limited search space. These frameworks can be roughly divided into two types: one is based on graph algorithms ([PERSON] et al., 2011; [PERSON] et al., 2013; [PERSON] et al., 2014; [PERSON] et al., 2015; [PERSON] et al., 2016; [PERSON] et al., 2017; [PERSON] et al., 2018) and the other type is based on machine learning ([PERSON] et al., 2013; [PERSON] et al., 2014; [PERSON] et al., 2015; [PERSON] et al., 2016; [PERSON] et al., 2017; [PERSON] et al., 2018). For the former, [PERSON] et al. [PERSON] et al. (2018) followed HPC, and proposed Scotch, a method to partition static graphs, including Multilevel Method ([PERSON] et al., 2017), k-way Fiduccia-Mattheyses ([PERSON]) ([PERSON]) ([PERSON]) ([PERSON], 2018), and RPM Recursive Bipartitioning Mapping ([PERSON] et al., 2018). Although these methods balance the communication workloads, they cannot be directly applied to dynamic environments. [PERSON] et al. [PERSON] et al. (2019) proposed FlexFlow, a deep learning framework that uses the guided randomized search of the search space to find a fast parallelization strategy for a specific parallel machine. [PERSON] et al. [PERSON] et al. (2019) built a performance model for Parameter Server (PS) ([PERSON] et al., 2019; [PERSON] et al., 2019; [PERSON] et al., 2019) and minimized the job training time based on predicting the training speed. [PERSON] ([PERSON], 2019) considered a similar problem, using dynamic programming to automatically split dataflow graphs and minimize the total communication costs. However, these approaches have limitations. FlexFlow is suitable only for DNNs, Optimus is only based on a PS, and Tofu only considers the minimal communication overhead. Moreover, their parallel strategy search space usually grows exponentially. Therefore, these limitations make them difficult to further promote.