Improving Genetic Programming for Image Classification
Authors:   , Fan Qinglan 
Published: October 2024

<p dir="ltr">Image classification is a fundamental task in computer vision. Due to the high dimensionality of the image data and high variations across images such as rotation, scale, illumination, and background variations, developing a good image classification method is challenging. Moreover, it is typically difficult and expensive to collect a large number of labeled images/instances in some computer vision applications, increasing the difficulty of training models to achieve high generalization performance. In addition to improving the classification performance, reducing the computational cost of an image classification method is crucial as many existing methods tend to take a long time to train a model.</p><p dir="ltr">Genetic programming (GP), an evolutionary computation technique, has a flexible representation of variable length and powerful global searchability and can produce potentially interpretable models. GP has been used for image classification. However, the potential of GP in image classification has not been comprehensively investigated in terms of GP representations, i.e., program structures, functions, and terminals. Furthermore, most existing GP-based methods usually require a long computation time for fitness evaluations, posing a challenge to real-world applications.</p><p dir="ltr">The overall goal of this thesis is to address the above issues and further explore the potential of GP in image classification. This goal is achieved by developing GP-based approaches with new representations, new fitness functions, and new genetic operators to improve classification performance and developing new surrogate models to reduce the computational costs of GP-based methods.</p><p dir="ltr">Firstly, this thesis proposes a GP approach with a new program structure, new functions, and new terminals for image classification. These new designs enable GP to perform feature extraction and feature construction, automatically select a suitable classification algorithm from a set of classification algorithms instead of relying on a fixed classifier like in most GP methods, and conduct classification using a single program. Additionally, this thesis develops a new mutation operator for dynamically adjusting the size of the GP programs. The experimental results on eight benchmark datasets show that the new approach achieves significantly higher classification accuracy than almost all compared methods.</p><p dir="ltr">Secondly, this thesis proposes a new image classification approach using GP with a new program structure to achieve flexible feature reuse. The new method can automatically learn various useful image features based on the new functions and terminals. Moreover, the new method is flexible in learning global and/or local features because of the region detection functions. The experimental results on 12 benchmark datasets suggest that the new approach achieves better classification performance than state-of-the-art methods in almost all comparisons.</p><p dir="ltr">Thirdly, this thesis proposes a new ensemble construction approach based on multi-tree GP (i.e., one individual contains multiple trees) for image classification with limited training data. One GP individual forms an ensemble, and its multiple trees are base learners. The new approach assigns different weights to multiple trees using the idea of AdaBoost and performs classification via weighted majority voting. The new approach out performs a large number of benchmark methods including ensemble-based, convolutional neural network (CNN)-based, and GP-based methods on eight image classification datasets containing small training sets.</p><p dir="ltr">Fourthly, this thesis proposes a GP approach with a new fitness function for image classification with small training data. In addition to classification accuracy, the new fitness function containing distance measures can simultaneously minimize the within-class distance and maximize the between-class distance, improving generalization performance accordingly. This thesis also develops a new crossover operator based on the nichingtechnique, which enables better exploitation of the global and local search space. The new approach achieves significantly better generalization performance than almost all comparison methods on eight image classification datasets containing a few training instances.</p><p dir="ltr">Finally, this thesis proposes a new surrogate-assisted GP approach including global and local surrogate models to accelerate the evolutionary learning process of GP methods for image classification. Furthermore, a new surrogate training set is constructed to assist in establishing the relationship between the GP tree and its fitness, and surrogate models can be built accordingly. Experimental results on eleven datasets show that the new approach significantly reduces the computational cost of GP methods without sacrificing the classification accuracy.</p>