# Internet Traffic Prediction with Distributed Multi-Agent Learning
## 5 Experiment and Analysis  
The Internet traffic dataset is divided into training and test subsets at a ratio of 5:1. The deep learning models are implemented with Python and TensorFlow, with three recurrent layers and 20 neurons in each recurrent layer. Adam is used as the optimizer with an adaptive learning rate starting from 0.01 and mean square error (MSE) is used as the loss function. The batch size is 128 and each model is trained for 200 epochs. For each agent number, ten runs are conducted separately. The weight matrix \(W\) for different agent numbers from the previous study [12] is used in this paper:  
\[W_{2}= \begin{bmatrix}0.9&0.1\\ 0.15&0.85\end{bmatrix}  \tag{9}\]  
\[W_{3}= \begin{bmatrix}0.9&0.05&0.05\\ 0.1&0.8&0.1\\ 0.025&0.025&0.95\end{bmatrix}  \tag{10}\]  
\[W_{4}= \begin{bmatrix}0.8&0.025&0.025&0.1\\ 0.05&0.85&0.025&0.075\\ 0.025&0.025&0.9&0.05\\ 0.05&0.1&0.1&0.75\end{bmatrix}  \tag{11}\]  
Figure 3: The LSTM and GRU cells. (**a**) LSTM; (**b**) GRU. \[W_{5}=\left[ \begin{array}{cccccc}0.8&0.05&0.05&0.05&0.05\\ 0.05&0.85&0.025&0.025&0.05\\ 0.025&0.025&0.85&0.05&0.05\\ 0.05&0.1&0.05&0.7&0.1\\ 0.04&0.11&0.05&0.05&0.75\end{array} \right] \tag{12}\]  
The evaluation results with different agent numbers for the LSTM and GRU models are shown in Figure 4, which are box and whisker plots for ten separate runs. With the increase in agent numbers, both LSTM and GRU demonstrate better performance with decreased RMSE and MAE values. GRU maintains a better performance than LSTM in Figure 4. While adding more agents into training might bring a further potential performance improvement, the marginal improvement becomes increasingly smaller. In addition, it becomes more difficult to coordinate more agents in practice. Thus, an agent number up to 5 is used in this paper. Since the research motivation of this paper is to validate the superiority of the distributed multi-agent learning approach over traditional centralized learning, we leave the exploration of using more agents in the follow-up studies. The mean RMSE and MAE results are further listed in Table 1 for LSTM and GRU models trained with different agent numbers. The baselines for comparison are those so  
Figure 4: The evaluation results with different agent numbers. (**a**) RMSE; (**b**) MAE. phisticated deep learning models from previous studies, including Time Series Transformer (TST) [42], Multilayer Perceptron (MLP) [43], Temporal Convolutional Networks (TCN) [44], Fully Convolutional Network (FCN) [45], Residual Neural Network (ResNet) [45], and InceptionTime [46] in the traditional centralized learning approach. Among these baseline methods, the previous best prediction result is obtained with InceptionTime. While LSTM or GRU trained with a single agent is not competitive with InceptionTime, an improved performance is obtained by adding and involving more agents. Only three agents are enough for defeating InceptionTime when GRU is the base prediction model in individual agents, as indicated in Table 1. A new state-of-the-art performance in the literature is achieved for the considered Internet traffic data by GRU-based distributed multi-agent learning trained with five agents, as shown in bold in Table 1. The results in Table 1 answer our second research question that the proposed distributed multi-distributed learning approach manages to outperform the traditional centralized learning approach and would be worthy a further investigation in the future studies for similar problems. Figure 5 further shows the comparison between true and predicted results for the first week in the test set. For clarity, only a single run with five agents is plotted for LSTM or GRU. From Figure 5, the predictions from both models present a similar pattern with the true values. However, LSTM generates larger predicted values when the true values are small, e.g., from 22 June 2019 to 23 June 2019, which results in a larger error than GRU.