Adaptive machine learning
By: Aman Singhal
Tags: Modelling, Practical, Online Learning, Machine Learning, Research

Different approaches are used to put machine learning models in production. Quite often models are put into production after one-off training (stationary models). For such a model to keep predicting accurately over time, the data that it is making predictions on must have a similar distribution as the data on which the model was trained. However, distributions can drift over time.

Such deviations are termed concept drift.

(A concept in “concept drift” refers to the unknown and hidden relationship between inputs and output variables. Concepts are also known as covariates.)

Figure 1: Types of concept drift

Techniques that tackle variation in concept over time can be broadly categorized as:

1. Online/Incremental techniques:

It is a type of machine learning that utilizes dynamic inputs (real-time inputs e.g. sensor date) post an initial static model has been assumed. Inputs are processed one by one.

Figure 2: Flow chart of online models

Purely online techniques, however, suffer as a detected change of concept could simply be noise and hence the computation that ensues will turn out to be inaccurate as well

Model performance can only be gauged over a span of multiple inputs where the model must have changed behavior multiple times

2. Batch based non-stationary techniques:

Batch based models allow training and testing samples to accumulate overtime for continuous model enhancement

Figure 3: Flow chart of batch-based non-stationary models

Various batch-based techniques which were explored consist of:

· Observation management: Describes how the new data points are incorporated to update the existing model

· Adaption technique: This lays emphasis on how the model learns from new data

· Forgetting mechanism: Describes how the non-relevant data points over time are faded away from the model’s learned behavior pattern

All machine learning algorithms for non-stationary concepts can be categorized on the basis of their respective learning techniques as:

· Windowing: The idea is to let the window grow till no change has been detected and to shrink its size when a change is detected, ensuring that the observations kept into the window are relevant to the current concept. Windowing does not work well with recurring concept drift.

· Weighting: Weighing algorithm assigns higher weights to more recent observations while training the model. Weighting does not work well with recurring concept drift.

· Sampling: Sampling algorithms summarize the characteristics of the whole stream by retaining into memory observations which have been selected according to a given probability distribution. This probability distribution might be biased in favor of the most recent observations in order to account for the drifts. Sampling does not work well as the information brought by the observations related to past concepts might be very small.

· Ensemble methods: Ensemble methods to maintain a set of models into memory and combine their predictions. The final prediction can be obtained by different aggregation methods. This can prove to be an unuseful strategy when the concepts never reoccur.

Discussed below is an algorithm which learns by combining weighing and ensemble methods to predict in non-stationary environments. Also, this algorithm has been picked since it is widely used beyond theoretical practices as well.

Learn++ NSE (Learn ++ algorithm for Non-stationary concepts)

It is an ensemble-based, non-stationary machine learning algorithm that updates itself by learning on batch data. The ++ is representative of the algorithm’s ability to learn from incrementally available batch data. Such batch data will comprise of labeled data points as well as data points for scoring. Labeled datasets can be further bifurcated into training and testing datasets, just as in the case of stationary ML algorithms.

Hence, training datasets are assumed to be available at certain intervals (which may be non-uniform in time), and are labeled — D1, D2 … Dt corresponding to instances 1,2…t.

Each new dataset represents a snapshot of the most recent concept. The amount of change in the concept since the previous dataset may be minor/substantial, tracked by the performance of the existing ensemble on the most recent labeled dataset.

The steps for creating the Learn++ NSE ensemble of models can be broken down as follows:

· Base model: Base model is trained and optimized once on historic data before the first instance of when the model is put in production

· Ensemble of models: One model is trained per instance of available data. e.g.: One model will be trained only on D1 at first instance, one model trained only on D2 at second instance (t)

· Model error update: Each model from the ensemble of models is evaluated on Dt (most recent training data set) and assigned an error score basis of its performance at instance t

· Overall error score: For each model at instance ‘t’ it is the weighted (higher weight is assigned to more recent error score) average of all its past error rates

· Ensemble importance weights: Models of the ensemble are assigned a weight inversely proportional to their overall error rate (A model with higher error is assigned a lower weight)

· Net ensemble output: Net Ensemble output is the average of the output of each model weighted by the corresponding importance weight

· Ensemble performance: Model performance is evaluated on the most recent training dataset Dt

Mathematically, a classification Learn++ NSE algorithm is as follows:

Figure 4: Pseudocode for Learn++ NSE

1. Calculating performance of existing ensemble on the latest labeled data:

· Equation 2: Each training labeled datapoint is assigned an initial value = 1/mt where mt is the cardinality Dt. The net error is Et

· Et: The net calculated error is the sum of all misclassifications multiplied by 1/mt

· Bt: It is the normalized error for the ensemble of models as calculated as Eqn. 3

· Equation 4: Datapoints in Dt are weighted as shown

· Equation 5: The assigned weights are then normalized to create error distribution for Dt

This error distribution will be used to assign errors to all the models in the ensemble of models.

2. A is trained solely on Dt to create the latest model for adding to the ensemble of models.

3. Calculating performance of individual models on Dt:

· Equation 6: Error of each classifier from the ensemble of models is calculated on Dt

4. Error thresholding:

· Latest model: If the calculated error of the latest model is greater than a threshold, then a model is again trained till threshold criteria are met

· Other models: If the calculated error is greater than 0.5, then it is assigned a value 0.5 corresponding to that dataset Dt, so that such a model may become insignificant at later another iteration (Eqn. 6)

Calculated error is normalized as in Eqn. 7 for each model.

5. Time-weighted average corresponding to each classifier model is calculated:

· Equation 8: Weightage of each error is calculated using the sigmoid function and normalized

· Equation 9: Calculated weightage from Eqn. 8 is multiplied with calculated errors

6. Classifier voting rights are calculated as in Eqn. 10

7. The overall model outputs are generated basis of Equation 11

References

1. Adaptive machine learning algorithms for data streams subject to concept drift: https://tel.archives-ouvertes.fr/tel-01812044v2/document

2. Instance-based vs Batch-based Incremental Learning Approach for Students Classification :

https://pdfs.semanticscholar.org/f90e/42ca8c80c75bc28a74d4419c837871679720.pdf

3. Adaptive machine learning for changing environments:

https://www.turing.ac.uk/research/research-projects/adaptive-machine-learning-changing-environments

4. Introduction to Online Machine Learning: Simplified :

https://www.analyticsvidhya.com/blog/2015/01/introduction-online-machine-learning-simplified-2/

5. Concept Drift: Monitoring Model Quality in Streaming Machine Learning Applications: https://www.youtube.com/watch?v=woRmeGvOaz4

6. Overview of the different approaches to putting Machine Learning (ML) models in production:

https://medium.com/analytics-and-data/overview-of-the-different-approaches-to-putting-machinelearning-ml-models-in-production-c699b34abf86

7. A Survey on Concept Drift Adaptation :

http://users.ics.aalto.fi/indre/surv.pdf

8. An Ensemble Approach for Incremental Learning in Nonstationary Environments:

http://users.rowan.edu/~polikar/RESEARCH/PUBLICATIONS/mcs07b.pdf

9. A collection of resources for concept drift data and software :

https://github.com/gditzler/ConceptDriftResources

10. Learning under Concept Drift, an Overview :

https://arxiv.org/pdf/1010.4784.pdf

11. Learning Flexible Concepts from Streams of Examples: FLORA2 :

https://pdfs.semanticscholar.org/ee21/fe39d99bd233dbf092f5cd21a50df28c7b04.pdf?_ga=2.226304664.306480498.1567516954-319752636.1567076557