## 4 Proposal--DDQN with Sensor Fusion
### Proposed Fusion Methods  
This article uses two sensory fusion techniques combined with the controller based on reinforcement learning, namely: (i) Late Modified Fusion (Late DDQN) and (ii) Interactive Fusion (Interactive DDQN). The sensors used in AMR navigation are: (i) RGB-D camera, (ii) Global Positioning System (GPS), (iii) Inertial Measurement Unit, and (iv) Endoder. All these sensors are merged with the respective techniques mentioned above. Soon after, the result of the fusion is combined in a neural network linked to the DDQN reinforcement learning method. The output of the DDQN network is the actions applied to the actuators (motors) of the AMR. Figure 2 presents the flowchart referring to the proposed method. The AMR used is a differential robot: the Pioneer 3-DX model. Computer simulations were performed using Coppelia software version 4.3.0 (rev.12). This simulator has a remote API for remote communication to integrate other programming languages, thus allowing the use of the Python language. All algorithms in this article were developed in Python version 3.8.13. Figure 1: Representation of current sensory fusion methods. Figure 2: Reinforcement learning-based sensor fusion—Late Fusion DDQN. The term Modified Late Fusion is adopted in this paper because the method used in this research has a feature extractor before the application of a Fully Connected neural network, that is, a convolutional neural network (CNN) responsible for processing the images captured by the visual sensor. This way, the method intersects with the Feature Fusion illustrated in Figure 1. The FC1 network is a Fully Connected network responsible for processing the data, referring to non-visual sensors. Interactive Fusion, as explained in the related works section, is a method proposed by [7] and originally applied in the medical field. In Interactive Fusion, image-related exams are processed by a CNN network, where the authors select for this purpose an architecture known in the literature as VGG-13. On the other hand, exams related to tabular data, that is, data from clinical exams such as blood and categorical data such as sex, among others, are processed by a Fully Connected network. The fusion, or combination of data, occurs specifically in the intermediate convolutional layers of VGG-13, so that the tabular network (Fully Connected) multiplies its output values by the feature maps obtained by CNN. The author calls this technique the Channel Wise Multiplication and claims that iterative convolutional maps are created, similar to attention mechanisms. The technique surpassed, in terms of performance, a Traditional Late Fusion. In this article, the method initially proposed by [7] is modified in order to carry out the task of sensor fusion in autonomous navigation. The CNN used to extract the characteristics of the image obtained by the visual sensors is shallower than a VGG-13, since the visual image for the perception of the AMR does not have as many characteristics as a clinical MRI exam used by [7]. The same CNN network is adopted for Late Fusion and Interactive Fusion. Table 1 shows the CNN configuration used. The images used in our application are captured by an RGB-D camera, processed in grayscale and then subjected to a technique known as \"skipframe\" for efficient handling and representation. In this process, specific hues in grayscale images represent varying depths, capturing crucial spatial information provided by the RGB-D camera. These depth-integrated grayscale images are then stacked, forming an input of the dimensions \(64\times 64\times 3\), which is analogous to a standard RGB image but instead encapsulates sequential frames with depth information. This stacking serves multiple purposes: it retains temporal context between consecutive frames, which is vital for understanding the dynamics of the environment, and it aligns the input dimensions with the standard input expected by typical CNN architectures.