# When Deep Learning Meets Data Alignment:
### Review Scope  
This paper reviews state-of-the-art learning-based approaches to registration. The ability of deep neural networks to generalize from training data and manage geometric properties has created a new subfield at the intersection between learning and registration algorithms. Although some reviews of registration have been performed [9; 10; 11], no reviews address learning-based approaches for registration without focusing on a specific scope such as medical image registration or image localization. The contributions of this paper are: (1) we provide a global overview of learning-based registration methods by proposing a well-defined framework that encompasses both the traditional and learning-based approaches; and (2) we review the recent learning-based registration approaches, which have been classified according to a proposed taxonomy to foster discussion. Figure 1 graphically summarizes the scope of this paper. The figure shows at the top the four main stages of the traditional pipeline for registering two given inputs (\(P\) and \(Q\)). These stages are: Target Selection (yellow, oblique lines), which defines the fixed input that the other input is going to be aligned to; Feature Extraction (red, dotted pattern), which computes the set of features \(\omega\) and \(\varphi\) for each input; Feature Matching (green, squared lines) to find correspondences between the previously extracted features; and Pose Optimization (blue, vertical lines), which is the process to minimize the distance error between both inputs. The right end is the final result transformation (\([R,t]\)), which is the rotation \(R\) and translation \(t\) parameters that indicate how data should be transformed to be aligned. These stages and terminology are further explained in Section 2. Figure 1: Registration framework. Schematic of the registration process in the traditional pipeline on the top part, and the learning-based approaches reviewed in this paper at the bottom. Learning-based approaches encapsulate the traditional pipeline in a conceptual space, allowing different types of inputs depending on the data and the number of encapsulated steps. The conceptual space is the learned parameters during the training process, and theoretically could also be considered as an input for the registration process. Vertically, the schematic is divided at the top with the traditional stages clearly defined and at the bottom the learning-based approaches that are reviewed here, represented by a neural network, and the possible data for inputs and outputs. Having in mind this information, the inputs for the learning-based proposal could be in different formats, such as point clouds, voxel grids, meshes, etc. In addition to full end-to-end approaches, some methods accept as inputs the result of the Feature Extraction or Matching stages in the traditional pipeline. The output could be some data in a specific format as well as the result of the stages from the traditional pipeline (feature vector, matching vector, and transformation). Nevertheless, with the new learning-based approaches, a new conceptual kind of space appears, containing learned properties about the object, materials, and their behavior that can be registered with the input data (e.g., aligning input data of an inflated ball with its deflated state restricted by the physical behavior of the object, rather than aligning with a final deflated target). The conceptual space is modeled by a neural network and its training process, and, theoretically, it could be considered as an input to the registration process, but it is not an input that one might use in every registration instance since it is an internal representation. This allows the network to encode conceptual models such as physical phenomena (e.g., force vector and symbolic/conceptual information such as \"sporty, comfy\") or mathematical rules. Besides, the neural network could perform one or more phases from the traditional pipeline (represented by the colored rectangles (see Figure 1) shown in the network).