Efficiently sampling training instances when the distribution of practical optimization problems is unknown can be challenging but crucial for the generalization of Automated Algorithm Selection (AAS) models. One approach is to use a diverse set of instances that cover a wide range of problem characteristics. This can be achieved by incorporating instances that represent different problem complexities, structures, and features.
One method is to use clustering techniques to group instances based on their similarities and differences. By selecting instances from different clusters, we can ensure a diverse representation of the problem space. Additionally, active learning strategies can be employed to iteratively select instances that provide the most information gain, helping to improve the model's generalization ability.
Another approach is to incorporate domain knowledge or expert input to guide the selection of training instances. Experts can provide insights into the types of problems that are commonly encountered in practice, helping to tailor the training set to be more representative of real-world scenarios.
Furthermore, techniques such as transfer learning can be utilized to leverage knowledge from related domains or previously seen instances to improve the generalization of the AAS model. By transferring information from known instances to new, unseen instances, the model can adapt more effectively to different problem distributions.