# A General Framework Based on Machine Learning for
## 5 Conclusions and Future Work  
This paper introduced a procedure to facilitate the task of devising algorithm selection strategies. One of the most remarkable features of this procedure is its reduced training time, which is a bottleneck of these methods. Additionally, the effort needed to collect all the training examples remains the same for both traditional algorithm selection methods and those produced with our approach. The difference lies in the way such instances are used to produce an algorithm selector. We noticed that Machine Learning techniques are a suitable option for this purpose. As a general conclusion, in this work, the proposed approach reduced the time needed to produce algorithm selectors and outperformed other metaheuristic-based approaches taken from the literature (GA and SA). Our proposed model helps in lightening the load of building a critical element of algorithm selection methods for CSPs: the learning component that maps instances to heuristics. Addditionally, our implementation of the model increases the credibility of the research, since all elements related to the inner selection algorithm are publicly available. At this point, we consider the importance of highlighting the flexibility of the proposed approach. In this work, we used a few ML methods to implement algorithm selectors for CSP. However, using the layered-architecture proposed, we could easily change the problem domain and maintain the same ML methods. Such a change would allow us to address a different problem domain easily. Conversely, we could modify the layer related to the algorithm selection process, where the ML methods are, and then provide different capabilities to the algorithm selectors. The idea of splitting the logic of the algorithm selection process by layers represents an improvement regarding generality of the algorithm selection process. We identified two exciting paths for future work. The first one is how the Machine-Learning-based algorithm selectors may deal with making decisions as the search progresses. All the algorithm selectors generated in this work make a decision about which heuristic to use only when the search starts and keeps that decision until the instance is solved, it is proven unsatisfiable, or the process stops due to a time-out. Other methods from the literature are allowed to choose a different heuristic while solving an instance--and not only at the beginning of the search. Some examples include algorithm selectors based on genetic algorithms and simulated annealing. Even so, these methods were, in overall terms, outperformed by those we developed in this work. Nonetheless, the algorithm selector obtained through genetic algorithms showed an outstanding performance for set BQWH18, where it even surpassed the Oracle. Hence, it would be interesting to study options for Machine-Learning-based algorithms selectors to explode this capability while maintaining the already observed benefits. Some ideas for achieving this goal might involve hybrid methods that lie on the intersection of Machine Learning and metaheuristics [58; 59]. Another critical path for research might focus on extending the proposed model to cover algorithm generation. In this work, we explored Machine Learning techniques that produce algorithm selectors. However, the community shows a growing interest in methods that automatically generate new solving strategies, such as heuristics [10; 38]. In this regard, we find that extending our model to also consider algorithm generation is an exciting idea to explore as part of future work. Conceptualization, J.C.O.-B. ; methodology, J.C.O.-B., A.E.G.-R. and I.A. ; software, J.C.O.-B. ; validation, I.A. and J.M.C.-D.; formal analysis, J.C.O.-B., A.E.G.-R., I.A. and J.M.C.-D.; investigation, S.E.C.-P.; resources, J.C.O.-B. ; data curation, J.C.O.-B. ; writing--original draft preparation, J.C.O.-B., I.A., A.E.G.-R.; writing--review and editing, J.M.C.-D., H.T.-M. and S.E.C.-P.; visualization, J.M.C.-D.; supervision, H.T.-M.; project administration, J.C.O.-B. and I.A.