# Combinatorial Optimization Problems and Metaheuristics: Review, Challenges, Design, and Development
## 5 Research Gaps and Future Work  
In this study, we explored, from various perspectives, concepts related to combinatorial optimization problems and metaheuristics, with the aim of highlighting the main challenges and issues that do not permit real progress in the field. Consequently, this study highlights several potential future studies that can contribute to the actual development of the area.  
The main challenges and open question that represent interesting research avenues are summarized as follows:  
* Guidelines to organize the field are essential to avoid misinterpretation of concepts and to compare metaheuristics characteristics and behaviors more efficiently and standardly. Furthermore, a classification system is essential to promote effective knowledge management and discussions of algorithms. However, a classification system must be developed, recognized, and adopted by the community in the first place. It is a clear, open question that must be answered soon.
* There is an absence of rigor and baseline tests to compare the performance of metaheuristics. As a result, the studies commonly present a biased or poor/unfair performance analysis. Thus, it demands a research line to develop standard benchmarking tests that can be reused as a reference by studies that compare metaheuristic performance. Moreover, the existence of a benchmarking database can facilitate and improve the process of performance comparison. For example, a new metaheuristic could be compared with all tested metaheuristics across several benchmarking tests.
* There is a necessity to add more statistical analyses--beyond dispersion statistics measures (e.g., mean, median, standard deviation, and outliers' analysis)--to promote more sophisticated comparisons and evaluate other aspects of metaheuristics' performance.
* It is necessary to remedy the absence of formalism to design and develop COP and metaheuristics in a standard way to enable their scalability, extensibility, and reusability. It is also an important aspect to avoid creating algorithms from scratch or developing algorithms that work only with one COP or are hard to adapt to a new situation.
* The demand for mathematical methods to evaluate other aspects of the metaheuristics building blocks such as metrics and parameters for balance ratio of exploitation and exploration, search space coverage, and solutions characteristics is essential. These needs are important to support a dynamic (or more intelligent) behavior in a metaheuristic during the search.  
Nevertheless, we understand that many other research lines are also required, and, unfortunately, we could not cover them due to some limitations and research decisions. Consequently, we also strongly reinforce the necessity of research to evaluate other aspects that could unleash the metaheuristic potential and promote the field's evolution.  
As future work, we are focused on the development of this framework, named ToGO (Towards Global Optimal). In fact, it has already started and soon it will be shared with the community. ToGO is an extensible open-box and open-source framework to solve COP by using metaheuristics. ToGO also was conceived to enable the sharing of COP and metaheuristics experiments; for example, by sharing their performance in a global database that can be used by other studies.  
Therefore, to evolve this framework, the research community must be involved. In fact, it must be maintained by the community, as it is a fundamental assumption and critical factor of success for this project. This is vital, especially considering the wide variety of metaheuristics algorithms and support components that can be shared to improve the quality of problems' solutions. Thus, community cooperation can occur on three levels: (1) the framework core components (version of distribution), (2) extensible components that run above the structure, and (3) sharing data and metrics.