# A Parameter Communication Optimization Strategy for Distributed Machine Learning in Sensors
## 1 Introduction  
Sensor networks have important applications such as environmental monitoring, industrial machine monitoring, body area networks and military target tracking. The design of sensor networks depends on the intended application, so the design of a body area networks must consider energy consumption [1; 2], network lifetime [3], hardware, the environment and routing protocols [2]. [PERSON] et al. reviewed the major developments in sensor networks and outlined new challenges, surveying the challenges in three diverse categories. The first is the internal platform and underlying operating system. The second is the communication protocol stack. The third is network services, provisioning, and deployment [4]. Machine learning automatically learns programs from data without being explicitly programmed. In the last decade, the use of machine learning has grown from the field of statistics to become a primary mechanism for prediction in diverse applications. In particular, machine learning methods have been widely used in sensors, such as addressing energy-aware communications [5], attack detection and classification [6], task scheduling in wireless sensor networks [7] and generating workloads for cloud computing [8]. Due to the energy and bandwidth constraints of sensors, it is not possible to transmit all the data back to the base station for processing and inference, therefore, it is necessary to apply distributed machine learning in sensors, which can greatly reduce the amount of data in the communication and truly utilize the distributed characteristics of sensors [9]. According to the general terms used in distributed machine learning, we also refer to the sensors used for computation as worker nodes in our paper. The main purposes of distributed machine learning in sensors are as follows: (1) solving the problem of insufficient memory of a single worker node to ensure that the data level of terabyte (TB) and above can be processed; (2) using the parallel acceleration training model to drastically decrease the training time. The most important issue is how to achieve parallelization. Data parallelization based on parameter servers (PSs) is a common parallelization scheme in distributed machine learning, whereby the data samples are divided into small datasets and distributed to each worker node, these worker nodes can share the access model parameters. In each iteration of the training, each worker node computes locally and updates its data subset, then submits the local updates to a PS. This PS updates the global model parameters and distributes the new global model parameters to each worker node. In general, the data parallel algorithm is executed in the form of a Bulk Synchronous Parallel (BSP) strategy [10; 11]. Worker nodes begin to wait after submitting the local updates, until all worker nodes have submitted the local updates and the PS has updated the global model parameters, then worker nodes will start a new iteration. Thus, BSP leads to load imbalance due to the different performance of worker nodes. In order to solve the problems existing in BSP, the distributed machine learning asynchronous iterative strategy has been proposed [12; 13; 14; 15], in which worker nodes can start the next iteration by using the local model parameters before receiving the global model parameters. In this strategy, the fault tolerance is increased, which leads to the machine learning model falling into local optima, and the accuracy may not be guaranteed. [PERSON] [16] proposed the Stale Synchronous Parallel (SSP) strategy, which allows each worker node using non-latest global model parameters to reduce the synchronization overhead of submitting the local updates to the PS and strictly controls the number of iterations to ensure the model will converge. Although using the non-latest global model parameters can improve the training speed, the parallel errors are accumulated and the convergence rate is reduced due to the loss of partial updates [17].