The time-consuming tuning phase is now substituted by an automated process. Let us note that only the final user will typically benefit from an automated tuning process. On the contrary, the algorithm designer faces a longer and harder development phase, with a possible preliminary phase of exploratory tests, followed by the above described exhaustive documentation of the tuning process when the algorithm is presented to the scientific community. On the contrary, the theoretical framework of computational learning theory and machine learning is very different from that of heuristics. In addition, the abundance of negative results in computational learning should warn about excessive hopes.
Some considerations follow in the next sections.
The purpose of learning is generalization . These examples are used in the training process to output a hypothesis concept , but the performance of the learning algorithm is judged by the error rate on different examples, although extracted from the same probability distribution. The analogy with parameter training in heuristics is that, for a problem characterized by a certain probability distribution on instances, some of these instances are used for training but different instances are used to validate the algorithm. Unfortunately this is not the typical case in the heuristics literature. In many cases the performance is judged on the tasks used to tune the algorithm parameters.