# Application of Conformal Predictors to Tea Classification Based on Electronic Nose

Ilia Nouretdinov, Guang Li, Alexander Gammerman, Zhiyuan Luo

► To cite this version:

Ilia Nouretdinov, Guang Li, Alexander Gammerman, Zhiyuan Luo. Application of Conformal Predictors to Tea Classification Based on Electronic Nose. 6th IFIP WG 12.5 International Conference on Artificial Intelligence Applications and Innovations (AIAI), Oct 2010, Larnaca, Cyprus. pp.303-310, 10.1007/978-3-642-16239-8_40. hal-01060632

HAL Id: hal-01060632

https://inria.hal.science/hal-01060632

Submitted on 16 Nov 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Application of Conformal Predictors to Tea
Classification Based on Electronic Nose

Ilia Nouretdinov¹, Guang Li², Alexander Gammerman¹, and Zhiyuan Luo¹

¹ Computer Learning Research Centre, Royal Holloway, University of London
Egham Hill, Egham, Surrey TW20 0EX, UK
zhiyuan@cs.rhul.ac.uk

² Department of Control Science and Engineering, Zhejiang University,
38 ZheDa Road, HangZhou 310027, Zhejiang, P.R. China

**Abstract.** In this paper, we present an investigation into the performance of conformal predictors for discriminating the aroma of different types of tea using an electronic nose system based on gas sensors. We propose a new non-conformity measure for the implementation of conformal predictors based on Support Vector Machine for multi-class classification problems. The experimental results have shown the good performance of the implemented conformal predictors.

**Keywords:** Tea Classification; Electronic Nose; Conformal Predictors; Support Vector Machines; Pattern Recognition.

# 1 Introduction

Tea is by far the most popular drink consumed in Britain today, with over 165,000,000 cups being drunk in the UK every single day of the year [1]. The major quality attributes of tea are flavour, aroma, colour and strength. Aroma means smell of the tea and is considered as the most important of the attributes. However, characterisation of aroma of tea has been a challenge for tea scientists for long. Tea flavour is traditionally measured through the use of a combination of conventional analytical instrumentation and human or ganoleptic profiling panels. These methods are expensive in terms of time and labour and also inaccurate because of a lack of either sensitivity or quantitative information [5].

Electronic nose has been successfully used for tea classification and grading [2, 5, 6, 12]. In general, an electronic nose is a smart instrument that is designed to detect and discriminate among complex odours using an array of sensors. The array of sensors is exposed to volatile odour vapour through suitable odour handling and delivery system that ensures constant exposure rate to each of the sensors. The response signals of these sensors are recorded and processed for analysis and classification. For tea classification, we want to discriminate the aroma of different types of tea.

Different data analysis and machine learning methods have been tried to analyse and classify the signals from the sensor array of the electronic nose system. For example, these signals were processed using Principal Components Analysis (PCA), Fuzzy C Means algorithm (FCM), Self-Organizing Map (SOM) method along with a Radial Basis Function network (RBF), Probabilistic Neural Network classifier, Back-Propagation network and Olfactory model [3, 5, 6, 8, 11, 12]. Good performance has been reported
in the literature. However, the learning techniques used can only provide bare predictions, i.e. algorithms predicting labels for new examples without saying how reliable these predictions are. The reliability of a method is often given by measuring general accuracy across an independent test set. Confidence estimation is a well-studied area of both parametric and non-parametric statistics; however, usually only low-dimensional problems are considered.

This paper describes an investigation into the performance of conformal predictors for discriminating the aroma of different types of tea using an electronic nose system. In particular, we study a recently developed theory of algorithmic learning, namely conformal predictors described in [10]. One of the major advantages of these algorithms is that they can be used for solving high-dimensional problems without requiring any parametric statistical assumptions about the source of data (unlike traditional statistical techniques). The only assumption made is the i.i.d. assumption (the examples are generated from the same probability distribution independently of each other). Another advantage of the method is that it also allows to make estimation of confidence in the classification of individual examples. The remainder of this paper is arranged as follows. We briefly describe conformal predictors and present a new non-conformity measure for the implementation of conformal predictors based on Support Vector Machine for multi-class classification problems in the next section. Section 3 discusses the experiments and results of applying the implemented conformal predictors to tea classification. Finally, Section 4 concludes the paper.

## 2 Conformal Prediction

In machine learning problems, we usually work with examples which are pairs $z_i = (x_i, y_i)$. $x_i \in \mathbf{X}$ is an object, normally represented as a vector and its components (dimensions) are called attributes. $y_i \in \mathbf{Y}$ is its label, which may be real-valued, or taking one of several possible values [4]. For the tea classification problem using electronic nose system, $x_i$ is a vector of response signals from the sensor array and $y_i$ is the finite types of tea.

Let us assume that we are given a training set of $l$ examples $\{(x_1, y_1), \dots, (x_l, y_l)\}$. Our goal is to predict the classifications for testing examples. We make only one assumption (i.i.d.) about the examples: all the examples have been generated independently by some fixed but unknown stochastic mechanism. Suppose that for a new example $x_{l+1}$ and its corresponding real label $y_{l+1}$ is unknown, we are interesting in predicting its label. The basic idea behind conformal predictors is to try every possible label $y \in \mathbf{Y}$ as candidate for the new example and see how well the resulting sequence $\{(x_1, y_1), \dots, (x_l, y_l), (x_{l+1}, y)\}$ conforms the i.i.d. assumption. If it does, we will say it is “random”. The ideal case is where all $y \in \mathbf{Y}$ but one lead to sequences that are not random and $y$ can be considered as the prediction for $x_{l+1}$. The general description of the conformal predictors is given in Algorithm 1 [10].

In conformal predictors, checking whether a possible new label violates i.i.d. assumption is done by a test for randomness based on a non-conformity (or strangeness) measure $\alpha = A(z_{l+1}, \{z_1, \dots, z_{l+1}\})$. This is a measure of disagreement or non-conformity between a finite set (subset of $\mathbf{X} \times \mathbf{Y}$) and its element. Specific form of
this function depends on a particular machine learning algorithm to be used and can be determined for many well-known machine learning algorithm. In this paper, we investigate and propose a non-conformity function for multi-class Support Vector Machine.

**Algorithm 1 Conformal Predictor**

**Input:** example sequence $\{(x_1, y_1), (x_2, y_2), \dots, (x_l, y_l)\}$

**Input:** new example $x_{l+1}$

**Input(optional):** confidence level $\gamma$

**Input:** non-conformity measure $A$

**for** $y \in Y$ **do**

$$z_{l+1} = (x_{l+1}, y)$$

**for** $j$ in $1, 2, \dots, l+1$ **do**

$$\alpha_j = A(z_j, \{z_1, \dots, z_l, z_{l+1}\})$$

**end for**

$$p(y) = \frac{\#\{j=1, \dots, l+1: \alpha_j \ge \alpha_{l+1}\}}{l+1}$$

**end for**

**Output(optional):** predictive set $R_{l+1}^{\gamma} = \{y : p(y) \ge 1 - \gamma\}$

**Output:** forced prediction $\hat{y}_{l+1} = \arg\max_y\{p(y)\}$

**Output:** confidence $conf(\hat{y}_{l+1}) = 1 - \max_{y \neq \hat{y}_{l+1}}\{p(y)\}$

**Output:** credibility $cred(\hat{y}_{l+1}) = \arg\max_y\{p(y)\}$

The conformal predictor calculates $p(y)$ which is a p-value associated with a hypothetical completion $y_{l+1} = y$ for the test example. Using these p-values, predictions made by conformal predictors can be presented in following two ways:

- Either the user inputs a required degree (level) of certainty and the algorithm outputs a predictive set: a list of classifications that meet this confidence requirement.

- Or the algorithm outputs so called forced prediction: an individual predictions together with its confidence, which is the minimal level of certainty at which the output is certain (predictive set consists of only one label) and credibility, which indicates whether the test object is representative of the training set, in order to compare with the other conventional learning methods.

The common way of presenting the prediction results is to choose a significance level ("degree of certainty") $\gamma < 1$ and output the $(\gamma)$-predictive set (region) containing all labels with p-value equal or greater than $1 - \gamma$:

$$R_{l+1}^{\gamma} = \{y : p(y) \ge 1 - \gamma\} \qquad (1)$$

Prediction errors can occur when the prediction set fails to contain the true label. If the size of the predictive set $R_{l+1}^{\gamma}$ is 1 or 0, then this prediction is certain, otherwise it is uncertain and we have multiple predictions. In the context of conformal predictors, uncertain predictions are not errors, but these are indications that the amount of information is not sufficient to make a certain decision at the selected level. Naturally, the higher the confidence level is, the more multiple predictions will appear. If non-conformity measure is specified adequately for the problem at hand, the percentage of
uncertain predictions should not be too high. It has been proved that the conformal predictors have validity property [10, 7] in online mode where the examples are presented one by one. Each time, the predictors take an object to predict its label and then receive feedback of the true label. The validity property implies that the successive predictions will be right 1- $\epsilon$ of the time for a given error probability $\epsilon$, even though they are based on an accumulating dataset rather than on independent dataset.

It is important to note that the prediction region of a conformal predictor in online mode is valid for any used non-conformity measure [10]. However, the quality of prediction depends on the choice of this measure. If it is inappropriate, the confidence in a prediction will be still valid but never high. Thus a non-conformity measure for conformal predictor is usually based on an underlying method of prediction (such as nearest neighbours and Support Vector Machine) which is appropriate enough for the data set.

In the case of finite label space, there is an alternative way to represent the output: single ("forced") prediction and some measure of *confidence* in this prediction. It does not require the confidence level $\gamma$ as input. The single prediction is selected by largest *p*-value. The maximal confidence level (degree of certainty) at which the prediction is certain, is called "confidence". It can be calculated as 1 minus the second largest *p*-value.

## 2.1 Support Vector Machine

Conformal predictors are a generic framework for making prediction and virtually any classification or regression algorithm can be transformed into a conformal predictor [10]. In this section, we consider the implementation of conformal predictors on the top of Support Vector Machine. Support Vector Machines (SVM) are effective learning systems based on statistical learning theory and have been applied successfully to many real problems [9]. For pattern recognition, SVM finds the hyperplane that separates the data set $(x_1, y_1), \dots, (x_l, y_l), x_i \in \mathbb{R}^d, y_i \in \{-1, +1\}$ with maximal margin. If the data are not linearly separable one can introduce slack variables $(\zeta_i)$ in the optimisation,

$$ \min \frac{1}{2} \| \mathbf{w} \| + C \sum_{i=1}^{l} \zeta_i \quad (2) $$

under the constraints $y_i[(\mathbf{w} \cdot \mathbf{x}_i) + b] \ge 1 - \zeta_i$ and $\zeta_i \ge 0$, where $\mathbf{w}$ is a normal vector that is perpendicular to the hyperplane, $\frac{\|\mathbf{w}\|}{b}$ determines the offset of the hyperplane from the origin along the normal vector $\mathbf{w}$ and $C$ is a constant chosen a priori. This optimisation problem can then be translated into its corresponding dual problem [9]

$$ \max \sum_{i=1}^{l} \alpha_i - \frac{1}{2} \sum_{i,j=1}^{n} \alpha_i \alpha_j y_i y_j K(\mathbf{x}_i, \mathbf{x}_j), \quad (3) $$

under the constraints, $0 \le \alpha_i \le C$, $\sum_{i=1}^{l} \alpha_i y_i = 0$, where $K$ is a kernel function defined by $K(\mathbf{x}_i, \mathbf{x}_j) = \mathbf{x}_i \cdot \mathbf{x}_j$. The Lagrangian $\alpha_i$ is assigned to each example in the dual setting of SVM and it is independent on their order. It is positive if the example is a Support Vector (SV), and 0 if it is non-SV. Therefore, these Lagrangian values can be used as non-conformity measure for SVM.
SVMs are inherently two-class classifiers. The common approach to dealing with multi-class SVM is to reduce the multi-class problem into multiple binary classification problems. Then each of the problems can be solved by a binary classifier. In this paper, we propose and use the following non-conformity measure as a tool to link two-class problems together for multi-class classification problems, see Algorithm 2. For example, if we have 4 classes: A, B, C and D, then a binary class non-conformity measure can be applied to each of possible class pairs: A-B, A-C, A-D, B-C, B-D and C-D. When the class pair A-B is considered, non-conformity values are only assigned to those examples from the class A and class B and the examples from other classes are ignored. The same procedure is applied to the other class pairs. Having considered all the possible class pairs, each example now has 3 different non-conformity values. The maximal of them is considered as the final non-conformity measure of this example.

<table><caption>**Algorithm 2** Non-conformity measure for multi-class SVM (for $q > 2$ classes)</caption><tr><td>Input:</td><td>Data set {z<sub>1</sub> = (x<sub>1</sub>, y<sub>1</sub>), . . . , z<sub>l+1</sub> = (x<sub>l+1</sub>, y<sub>l+1</sub>)}<br/>where y<sub>i</sub> ∈ Y = {1, 2, . . . , q}</td></tr><tr><td>Input:</td><td>z<sub>j</sub> (one of examples above)</td></tr><tr><td>for</td><td>r = 1, . . . , q − 1 do</td></tr><tr><td>for</td><td>s = r + 1, . . . , q do</td></tr><tr><td>Apply dual form SVM for examples from classes r and s only</td><td></td></tr><tr><td>Extract Lagrange multipliers α<sup>SVM(r:s)</sup><sub>j</sub> for j such that y<sub>j</sub> ∈ {r, s}</td><td></td></tr><tr><td>α<sup>SVM(r:s)</sup><sub>j</sub> = 0 for j such that y<sub>j</sub> ∉ {r, s}</td><td></td></tr><tr><td>end for</td><td></td></tr><tr><td>end for</td><td></td></tr><tr><td>α<sub>j</sub> := max<sub>r,s</sub> α<sup>SVM(r:s)</sup><sub>j</sub></td><td></td></tr></table>

## 3 Experiments and Results

Metal Oxide Semiconductor (MOS) sensors are commonly used in electronic nose applications for their convenience in operating and steadiness in features. The MOS gas sensors obtained from Figaro Engineering Inc., Japan (http://www.figaro.co.jp) were used in the experiments. In particular, an array of seven gas sensors: TGS2610, TGS2611, TGS800, TGS813, TGS822, TGS826 and TGS880 were chosen for the tea classification experiments. For illustration purposes, four types of tea which are ready available from the market: Japanese Green Tea (JG), Chinese Green Tea (CG), Indian Black Tea (IB) and Chinese Black Tea (CB) were used and we label them as class 1 to 4, respectively.

Each tea sample is heated before data acquisition. There are 68 data samples in total obtained in the experiments (15 for Indian Black Tea, 19 for Chinese Black Tea, 17 for Chinese Green Tea and 17 for Japanese Green Tea). Each data sample is described by 7 features obtained from those 7 gas sensors of the electronic nose system. A typical sensor response is shown in Figure 1. Originally each of these 7 features was represented as time series and we simply take its maximum value in time for the classification problem.
<figure><img src="image_2.png" /><figcaption>Fig. 1. An example of recorded sensor response</figcaption></figure>

A 4-class SVM conformal predictor as described above (see Algorithm 2) has been implemented and applied to the tea classification dataset in leave-one-out mode. Table 1 shows example individual predictions with sample data presented in random order, where “T” is the true label, p(1) ... p(4) are calculated *p-values* for different labels, “P” is the forced prediction, “Conf” and “Cred” are the confidence and credibility associated with the forced prediction, respectively. For example, the true label is 4 and the forced prediction is 1 for the 7th sample data. The calculated confidence for the prediction is 0.937 and credibility is 0.222. High (i.e. close to 100%) confidence means that all labels except the predicted one are unlikely. Low credibility means that the whole situation is suspect; perhaps the test example is very different from all examples in the training set. The overall accuracy of forced prediction is 89.7%, as 61 of 68 predictions are correct according to the confusion matrix shown in Table 2.

<table><caption>Table 1. Examples of individual predictions for leave-one-out cross validation</caption><thead><tr><th>No.</th><th>T</th><th>p(1)</th><th>p(2)</th><th>p(3)</th><th>p(4)</th><th>P</th><th>Conf</th><th>Cred</th><th>No.</th><th>T</th><th>p(1)</th><th>p(2)</th><th>p(3)</th><th>p(4)</th><th>P</th><th>Conf</th><th>Cred</th></tr></thead><tbody><tr><td>6</td><td>2</td><td>0.056</td><td>0.118</td><td>0.063</td><td>0.05</td><td>2</td><td>0.937</td><td>0.118</td><td>40</td><td>1</td><td>1</td><td>0.056</td><td>0.063</td><td>0.05</td><td>1</td><td>0.937</td><td>1</td></tr><tr><td>7</td><td>4</td><td>0.222</td><td>0.056</td><td>0.063</td><td>0.053</td><td>1</td><td>0.937</td><td>0.222</td><td>41</td><td>4</td><td>0.056</td><td>0.056</td><td>0.063</td><td>1</td><td>4</td><td>0.937</td><td>1</td></tr><tr><td>8</td><td>1</td><td>1</td><td>0.056</td><td>0.063</td><td>0.05</td><td>1</td><td>0.937</td><td>1</td><td>42</td><td>4</td><td>0.056</td><td>0.056</td><td>0.063</td><td>1</td><td>4</td><td>0.937</td><td>1</td></tr><tr><td>9</td><td>4</td><td>0.167</td><td>0.111</td><td>0.188</td><td>0.316</td><td>4</td><td>0.812</td><td>0.316</td><td>43</td><td>3</td><td>0.056</td><td>0.056</td><td>1</td><td>0.05</td><td>3</td><td>0.944</td><td>1</td></tr><tr><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td><td>..</td></tr></tbody></table>

The performance of forced prediction of the conformal predictor is compared with KIII model (a novel bionic neural network), a conventional artificial neural network
<table><caption>**Table 2.** Confusion matrix for forced prediction</caption><thead><tr><th>True Label\Forced Prediction</th><th>JG Tea (1)</th><th>CG Tea (2)</th><th>IB Tea (3)</th><th>CB Tea (4)</th></tr></thead><tbody><tr><th>JG Tea (1)</th><td>14</td><td>1</td><td>1</td><td>1</td></tr><tr><th>CG Tea (2)</th><td>1</td><td>16</td><td>0</td><td>0</td></tr><tr><th>IB Tea (3)</th><td>1</td><td>0</td><td>14</td><td>0</td></tr><tr><th>CB Tea (4)</th><td>2</td><td>0</td><td>0</td><td>17</td></tr></tbody></table>

(back-propagation network) and volunteers [11]. 30 volunteers were invited to try tea classification. All the volunteers were trained to remember the odour of each kind of tea. Then they made tea classification by smelling without seeing. Table 3 shows the comparison of forced prediction of conformal predictors with other methods on the same data set. Forced predictions made by conformal predictors have similar performance as those of the KIII model and BP networks. The volunteers performed not so well as the electronic nose, because of some physiological and psychological factors [5].

<table><caption>**Table 3.** Performance comparison (accuracy %)</caption><thead><tr><th>Method</th><th>JG Tea (1)</th><th>CG Tea (2)</th><th>IB Tea (3)</th><th>CB Tea (4)</th><th>Average</th></tr></thead><tbody><tr><th>KIII</th><td>93.3%</td><td>86.7%</td><td>93.3%</td><td>80%</td><td>88.3%</td></tr><tr><th>Back-Propagation</th><td>80%</td><td>100%</td><td>66.7%</td><td>93.3%</td><td>85%</td></tr><tr><th>Volunteers</th><td>80%</td><td>46.7%</td><td>83.3%</td><td>50%</td><td>65%</td></tr><tr><th>Conformal Predictor</th><td>82.4%</td><td>94.1%</td><td>93.3%</td><td>89.5%</td><td>89.7%</td></tr></tbody></table>

The predictive sets for a specific confidence level $\gamma$ can be derived from the list of p-values by Equation (1). For the confidence level of 80%, 85%, 90% and 95%, the performance (in terms of error rate and multiple prediction rate) of the implemented conformal predictor is shown in Table 4. It is clear that the predictions made by the conformal predictor are valid and uncertainty rate becomes larger as the confidence level grows. Validity is true in the sense that error rate is smaller than 1 minus confidence level. The property of validity means that the conformal predictors never overrate the accuracy and reliability of their predictions. From the practical point of view, they allow us to control the error rate by selecting a suitable confidence level. For example, at 80% confidence level, the error rate is just over 16% and less 6% of predictions are uncertain. At 90% confidence level, over 23% of predictions become uncertain.

<table><caption>**Table 4.** Performance of conformal predictor at different confidence level</caption><thead><tr><th>Confidence Level</th><th>Error Rate</th><th>Multiple Prediction Rate</th></tr></thead><tbody><tr><td>80%</td><td>16.2%</td><td>5.9%</td></tr><tr><td>85%</td><td>11.8%</td><td>13.2%</td></tr><tr><td>90%</td><td>5.8%</td><td>23.5%</td></tr><tr><td>95%</td><td>0</td><td>100%</td></tr></tbody></table>
## 4 Conclusion

An investigation into application of conformal predictors to tea classification based on an array of gas sensors has been described. A new non-conformity measure for multi-class tea classification problem using SVM has been discussed. Experiments have been carried out to evaluate the performance of the conformal predictors. Empirical results have showed the good performance of the implemented conformal predictor.

**Acknowledgements.** This work was supported by a Royal Society International Project, Natural Science Foundation of China (NSFC-RS 60911130129) and ERASysBio grant for SHIPREC project.

## References

1. United Kingdom Tea Council, http://www.tea.co.uk/the-home-of-tea-all-you-need-to-know-about-tea (last accessed: June 2010).

2. N. Bhattacharyya, R. Bandyopadhyia, M. Bhuyan, B. Tudu, D. Ghosh and A. Jana. Electronic Nose for Black Tea Classification and Correlation of Measurement with “Tea Taster” Marks. IEEE Transactions on Instrumentation and Measurement, Vol 57, Issue 7, pp. 1313-1321, 2008.

3. S. Borah, E.L. Hines, M.S. Leeson, D.D. Iliescu, M. Bhuyan and J.W. Gardner. Neural network based on electronic nose for classification of tea aroma. Sensing and Instrumentation for Food Quality and Safety, Vol 2, No. 1, pp.7-14, 2008.

4. R.O. Duda, P.E. Hart and D.G. Stork. *Pattern Classification*, Wiley, 2001.

5. R. Dutta, K.R. Kashwan, M. Bhuyan, E.L. Hines, and J.W. Gardner. Electronic nose based tea quality standardization. Neural Networks, Vol 16, Issue 5-6, pp. 847-853, 2003.

6. E. Gonzalez, G. Li, Y. Ruiz, and J. Zhang. A Tea Classification Method Based on an Olfactory System Model. Advances in Cognitive Neurodynamics ICCN 2007, pp.747-751, 2008.

7. I. Nouretdinov, and V. Vovk. Criterion of calibration for Transductive Confidence Machine with limited feedback. Theoretical Computer Science, Vol364, Issue 1, Algorithmic learning theory, pp. 3-9, 2006.

8. B. Tudu, A. Jana, A. Metla, D. Ghosh, N. Bhattacharyya, R. Bandyopadhyay. Electronic nose for black tea quality evaluation by an incremental RBF network. Sensors and Actuators B: Chemical, Vol 138, pp. 90-95, 2009.

9. V. N. Vapnik. *Statistical Learning Theory*, Wiley, New York, 1998.

10. V. Vovk, A. Gammerman, G. Shafer. *Algorithmic Learning in a Random World*, Springer, 2005.

11. X. Yang, J. Fu, Z. Lou, L. Wang, G. Li, and W. J. Freeman. Tea classification based on artificial olfaction using bionic olfactory neural network, Proceedings of Third International Symposium on Neural Networks, pp 343-348, 2006.

12. H. Yu, and J. Wang. Discrimination of LongJing green-tea grade by electronic nose. Sensors and Actuators B, Vol 122, pp.134-140, 2007.