## 3 Demonstration and Validation
### Results Analysis  
Before using the model to make predictions, each skill's processing time and the model's conveyors speed had to be tuned so that the simulation closely matched the physical system's behaviour. For this first calibration, single-product production plans were considered with a station at conveyor D. It was assumed that all the conveyors moved at the same speed, regardless of the station's position. In order to assess this calibration's impact, twelve scenarios were tested on both systems (the simulation model and the physical demonstrator) for each station's possible location:  
* Production pattern--six combinations of ProductType1, ProductType2 and ProductType3. * Production plan size--ten or twenty products. For each conducted test, the total processing time and the two KPIs below were extracted. * Cycle time (CT)--average time spent by products on the line, in _s/product_. * Throughput (Tp)--average amount of exported products per time unit, in _products/min_. The first analysis unveiled some traits of the system's behaviour. The scenarios with a station at conveyor A presented the least accurate predictions, because products continuously communicate with the entry conveyor until they are allowed to enter the network, which makes the MAS become unexpectedly slower. As the station advances from one conveyor to the following, between B and D, the model gradually becomes less accurate. This is due to the fact that, the further the station is from the entry conveyor, the more products get stuck at the network, which leads to an increase in the amount of  
Figure 19: Manufacturing unit simulator’s components. communications and, consequently, to greater delays. However, the change in the station's location from conveyor D to E is a turning point on this regard. From this conveyor on, the accumulation takes longer to occur and there is no such growth in the amount of communications, resulting in better predictions. These discrepancies led to a second calibration. For the simplicity's sake, only two possible locations for the station (conveyors B and D) and the two scenarios below were tested:  
* Production pattern--1\(\rightarrow\)2\(\rightarrow\)3 or 3\(\rightarrow\)2\(\rightarrow\)1. * Production plan size--twenty products. Besides, distinct speeds were chosen for each model's conveyor, through trial-and-error, taking into account the station's position as well. Afterwards, all twelve scenarios were repeated with a station at conveyor B or D, the best and the second worst cases, respectively. As expected, the second calibration managed to improve the overall model's accuracy for both setups. Nevertheless, the physical system still became quite unstable upon the products' accumulation, possibly due to the increase in the amount of agents and, consequently, communications. Moreover, in spite of being possible to adjust both the total time and the throughput for a specific production plan, it was achieved by reducing the conveyors' velocity, which spoiled each product's cycle time. For that reason, it is safe to assume that such change is insufficient. Since modelling the delays transcended the scope of this work, a last verification was performed so as to ascertain whether processing a sufficiently large amount of products helped dissimilaring such latency. For that, the scenarios from the second calibration were rerun with larger production plans, but the KPIs calculations still considered sets of twenty consecutive products. The predictions' relative error for different production plan sizes, with a station at conveyor B, are represented in Figure 20.  
The corresponding values with a station at conveyor D are portrayed in Figure 21.  
Figure 20: Predictions’ relative error as a function of the plan size, with a station at conveyor B. A closer look at the previous plots suggests that, regardless of the station's position, the relative error of each KPI's predictions did not directly increase with the production plan's dimension.