# Addressing the Algorithm Selection Problem through an Attention-Based Meta-Learner Approach
## 2 Background and Related Work
This raises the question of whether exploring more recent neural network architectures can improve the pattern-extraction process to further leverage the meta-features that characterize the problem instances and thus lead to better learning of the mappings of problem instances to algorithms. Moreover, [PERSON] et al. [29] classified neural network meta-learners into three categories: metric-, model-, and optimization-based techniques. Metric-based techniques focus on learning a feature space based on the network's weights, which is then used for new tasks by comparing new inputs with example inputs in the learned space. This learning approach, in which the learned feature space stored in the network's parameters is not updated when new tasks are seen, is the focus of our proposed attention-based meta-learner. Alternatively, model-based meta-learning relies on an adaptive internal state that sequentially processes meta-data to capture task-specific information [29]. Besides, [PERSON] et al. [29] defined optimization-based meta-learning as an approach that optimize for fast learning by treating meta-learning as a bi-level optimization problem, with a base-learner making task-specific updates at the inner level and performance across tasks being optimized at the outer level. In this work, we focus on the VRPTW as an application problem. It is an extension of the VRP in which servicing a particular customer \(i\), out of \(\mathbf{C}\) customers, can only be carried out during a time interval \([t_{max},t_{min}]\) corresponding to the customer's time window and defined by a starting time \(t_{max}\) and ending time \(t_{min}\)[1; 30]. The problem, essentially, consists of identifying \(k\) routes for the vehicles. Such routes together and subject to additional constraints must cover the set of customers where, for each customer \(i\), there is a given demand \(m_{i}\). Furthermore, no vehicle can supply more customers beyond its capacity \(q_{k}\), and each customer must be visited exactly once by a single vehicle [1; 30]. We do not provide further details on the VRPTW since it is only the application problem and not the focus of this work. However, we invite the interested reader to consult the works of [PERSON] et al. [30] and [PERSON] et al. [1], which provide a more detailed description of this problem. Moreover, the modern approach to the ASP from the meta-learning perspective has already shown that simple meta-learners, such as MLPs, can yield good results when used to select the best meta-heuristic for solving the VRPTW [31]. This work, therefore, focuses on the meta-heuristic approach, given that using these types of methods (e.g., SA [32; 33] and GAs [34; 35]) has proven to be an effective strategy to consistently obtain adequate solutions when solving routing problems. As an introductory study, we built a portfolio of algorithms consisting of three different configurations of the simulated annealing (SA) meta-heuristic to test attention-based neural networks as algorithm selectors. From this point on, and for clarity, we will treat each of these configurations as a different algorithm. These algorithms generate meta-data by solving various VRPTW instances. Subsequently, we use the generated meta-data to evaluate the effectiveness of an attention-based meta-learner as an alternative performance mapping mechanism by selecting the algorithm out of the three that best solves a specific VRPTW instance. However, as mentioned before, this research's focus differs from the VRPTW itself; thus, we do not aim to improve on the current state-of-the-art solutions. Alternatively, this work aims to study the use of an attention-based meta-learner in the context of the ASP for selecting the best algorithm for a specific problem instance. We chose the VRPTW to evaluate the proposed meta-learner and compare it against other common meta-learners found in the most relevant and recent literature.