## 3 The Proposed NeXtFusion 3D Object Detection Network
### Extracting Radar Features  
The proposed multi-modal object detection network in this paper utilizes features of each object within the image to predict all other properties associated with the object. To maximize the utilization of radar point-cloud data associated with the object in this context, it is essential to initially establish a connection between radar detections and their corresponding objects of interest detected within the image. Figure 6: Three sets of examples (**a–c**) where (**a**) represents a perfect overlap between the predicted bounding box (\(A\)) and the ground-truth box (\(B\)), (**b**) represents a partial overlap resulting in 0.5 IoU losses, and (**c**) represents the disjoint problem of IoU when \(A\) and \(B\) do not overlap. Figure 7: Two sets of examples (**a,b**) where (**a**) represents a perfect overlap between the predicted bounding box (\(A\)) and the ground-truth bounding box (\(B\)) and (**b**) represents a non-overlapping case of \(A\) and \(B\), which solves the IoU’s disjoint problem by introducing a third smallest enclosing bounding box called \(C\) that encompasses both \(A\) and \(B\) bounding boxes. An autonomous vehicle's movement can be visualized using the right-handed coordinate system as it travels in the forward direction. A commonly employed method to determine the right-hand rule involves extending the index finger along the positive \(x\)-direction, bending the middle finger (and/or ring and pinky fingers) inward to indicate the positive \(y\)-direction, and raising the thumb to represent the positive \(z\)-direction, as denoted by Figure 8. In this configuration, the \(x\)-axis denotes the direction of motion, the \(y\)-axis runs parallel to the front axle of the vehicle, acting as the reference point, and the \(z\)-axis runs perpendicular to the \(x\)- and \(y\)-axes and points out through the roof of the vehicle. For handling and analyzing point-cloud data representations, i.e., data obtained from radar sensors, a polar coordinate system is being used. In this type of system, a point in space is represented by a distance (\(d\)) from a reference point (origin) and an azimuth angle (\(\infty\)) measured from a reference direction (usually the positive \(x\)-axis). This system is particularly suitable for representing spatial information in scenarios where the distance and angle of objects from a reference point are essential, as is often the case with radar measurements and point-cloud data. By determining the distance (\(r\)) of an object from point \(A\) and its azimuth (\(\infty\)) from the radar, one can make an approximation of the location of the object of interest (\(A\)) within the global coordinate system [34]. Within the global coordinate system of \(x\), \(y\), and \(z\), each radar detection is expressed as a 3D point relative to the sensor's position, thus characterizing it as \(\left(x,\ y,\ z,\ v_{x},\ v_{y}\right)\), where \(x,\ y\), and \(z\) denote the position of the point in a 3D space, and \(v_{x}\) and \(v_{y}\) signify the stated radial speed of the object along the \(x\) and \(y\) axes. For each scenario, three sequential radar point-cloud sweeps are combined, with a 0.25 s interval between each. Each camera within the nuScenes dataset is pre-calibrated, featuring both intrinsic and extrinsic parameters. The intrinsic parameter is a \(3\times 3\) matrix that defines the internal characteristics of the camera, including focal length, principal point, and distortion coefficients, which are typically acquired through specialized calibration procedures that involve checkerboard patterns or similar techniques and can be defined as follows:  
\[Intrinsic_{param}=I_{p}= \begin{bmatrix}f_{x}&0&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{bmatrix}  \tag{3}\]  
Here, \(f_{x}\) and \(f_{y}\) represent the focal lengths in the \(x\) and \(y\) planes, respectively, and \(c_{x}\) and \(c_{y}\) represent the offset points of the camera in the \(x\) and \(y\) planes, respectively.