title: Advanced techniques for classification of non-stationary streaming data and applications
Creators: name: Luong, Vu A
Publisher: name: Griffith University
Year: 2022
Subjects: subject: dynamic data streams
subject: Heterogeneous Ensemble Selection
subject: machine learning

description: Today we are going through the Industry 4.0, where not only people are connected via social networks, but an enormous number of electronic devices are also connected via the Internet of Things (IoT). With the rapid development of modern technologies like blockchains, 5G, computing chips, software infrastructures, people and application programs are interacting with each other at a very fast pace. As a result, massive amount of data is generated in real time, posing many interesting but challenging problems to the machine learning community. According to the International Data Corporation [73], the total amount of data created, captured, copied, and consumed globally is forecast to increase rapidly, reaching more than 180 zettabytes by 2025, which is approximately 90 times of the data volume in 2010. However, only a minor subset of this newly created data is saved, as just about two percentage of the data created and consumed in 2020 was retained into 2021. One of the reasons for this is that data from numerous real-world applications are often collected in the form of unbounded data streams, including sensor networks, video services, event logs, and traffic monitoring systems, which would exceed the physical storage eventually. Therefore, the two major concerns about unlimited data volume and fast velocity remain unsolved when dealing with data streams. Traditional offline machine learning methods have successfully solved many intelligence tasks in recent years, most notably computer vision and natural language processing. However, they are not efficient when dealing with data streams generated dynamically in real-time from above mentioned applications. Particularly, the offline learning paradigm suffers from many limitations in this context: (1) it is not practical to store the entire data stream in memory; (2) traditional algorithms need to be retrained when new training data instances are available; and (3) the slow training time makes them almost impossible to adapt instantly to real-time data. In this study, we focus on developing new ensemble methods to solve the problems of data stream classification. Although several studies related to ensemble learning have been proposed in the literature, some research gaps still exist. First, most ensemble algorithms developed for evolving data streams are homogenous, which means that all the base classifiers are generated from the same learning algorithm, most frequently Hoeffding Trees. The data stream literature is lacking heterogeneous ensembles, which benefit from having much fewer base learners to obtain comparable prediction performance to homogeneous ensembles. Therefore, we introduce the HEterogeneous Ensemble Selection (HEES) method that dynamically determines an appropriate subset of base learners to make predictions for non-stationary data streams. Though HEES only uses 8 base classifiers, our experiments on 50 datasets shows that its prediction accuracy is higher than other homogeneous ensembles with 40 base classifiers, including OzaBagAdwin, OzaBoostAdwin, BOLE, and LNSE. Second, most existing models in the literature have low expressive capability; hence, we propose a Streaming Deep Forest (SDF) method to fill this gap. Also, an active learning strategy is introduced to save the label query costs and to speed up SDF. As a result, SDF obtains the state-of-the-art accuracy in both immediate and delayed settings. Next, we present a multi-layer heterogeneous ensemble called SMiLE and a selection method to tackle the real-world problem of insect stream classification. In our experiments, SMiLE achieves the best performance on 10/11 datasets and second-best performance on the remaining dataset in comparison to benchmark algorithms. Finally, we propose an incremental framework to combine different segmentation models for medical images. The proposed framework is about 16 times faster than the second-fastest method MLR on both CVC_ColonDB and MICCAI2015 datasets in our experiments.