# Obtaining Human Experience for Intelligent Dredger Control: A Reinforcement Learning Approach
## 5 Evaluation and Results
### Intelligent Control of the Cutting Process
#### 5.2.1 Results and Analysis  
In Figure 6, we have compared the performance of the experienced human operator with Q-learning, SARSA, and SARSA(\(\lambda\)) with respect to the slurry density, the motor current of the cutter head, the motor current of the pump, the suction vacuum, the flow velocity and the swing speed. Figure 5: Relative percentage error of the predictions. In the cutting process, the swing speed is the control variable, so its behavior will affect the changes of other variables. For example, if the swing speed is increased, the amount of the soil to be cut (i.e., the slurry density) will grow accordingly. As the swing operation causes the CSD to move around the working spud pole by pulling and slacking on the fore sideline wires of two side anchors, it also produces a reaction force on the cutter head. Thus, the increase of the swing speed will also lead  
Figure 6: Performance comparison between the experienced human operator and three intelligent control models (i.e., Q-learning, SARSA, and SARSA(\(\lambda\))). to the rise of the cutting torque, which will be reflected in the motor current of the cutter head. For the same reason, if the amount of the dredged soil is increased, it will be harder for the underwater pump to sucmany samples of unreasonable manipulations. However, from the results we can conclude that the intelligent control methods can learn from the experience of the human operator, and, in addition, improve the performance to avoid unreasonable manipulations. **Motor Current. **: When we look at the motor current of the cutter head and the motor current of the pump, we can also find that the curves of the Q-learning, SARSA, and SARSA(\(\lambda\)) change more gently then the curve of the human operation. The motor current of the cutter head reflects the cutting forces and the type of soil to be dredged, while the motor current of the pump indicates the slurry density between the cutter head and the underwater pump. In practice, it is hoped that the change of the motor current does not fluctuate too much and can be stabilized within the rated voltage range. However, it is a challenge task even for the experienced operator to achieve such a goal, as can be seen in Figure 6. Comparatively, all the intelligent control models can ensure that the change of the motor current will be stable and gentle, which is conducive to the safe operation of the motor and improve the service life of the motor. The reason the intelligent control models can keep the motor current stable in uncertain environments is also because of the ability of quick responses. Thus, we can conclude that the reward scheme designed in Equation (7) is effective to ensure the safe operation of the motors of the cutter head and the pump. **Suction Vacuum. **: The degree of suction vacuum can indicate the pipeline concentration between the cutter head and the underwater pump, and if the value is too low, it is hard for the underwater pump to suck up the dredged slurry. In this case, the human operator or the intelligent control models should decrease the swing speed. Otherwise, the low vacuum can result in cavitation in the pipeline, which will affect the normal operation of the underwater pump. Thus, in Figure 6 we can see that all the intelligent control models are able to guarantee that the degree of suction vacuum will be kept with the range of \(-68\) to \(-38\) bar. Comparatively, the curve of the suction vacuum produced by the human violently oscillates between \(-81\) to \(-28\) bar. Thus, we can conclude that the adjustment of the swing speed of the intelligent control models takes account of the degree of suction vacuum and ensures that the value will be kept within a suitable range. **Flow Velocity. **: As mentioned before, the flow velocity in the pipeline can also indicate the slurry density, and it is considered to be the key indicator for the prevention of blockage. In other words, if the flow velocity is too low, the solids level in the pipeline must be high, and the pipeline will be in danger of being blocked.