# Automated Tongue Feature Extraction for ZHENG Classification in Traditional Chinese Medicine

Authors: Ratchadaporn Kanawong; Tayo Obafemi-Ajayi; Tao Ma; Dong Xu; Shao Li; Ye Duan
Journal: Evidence-Based Complementary and Alternative Medicine
 (2012)

---

## Abstract

ZHENG, Traditional Chinese Medicine syndrome, is an integral and essential part of Traditional Chinese Medicine theory. It defines the theoretical abstraction of the symptom profiles of individual patients and thus, used as a guideline in disease classification in Chinese medicine. For example, patients suffering from gastritis may be classified as Cold or Hot ZHENG, whereas patients with different diseases may be classified under the same ZHENG. Tongue appearance is a valuable diagnostic tool for determining ZHENG in patients. In this paper, we explore new modalities for the clinical characterization of ZHENG using various supervised machine learning algorithms. We propose a novel-color-space-based feature set, which can be extracted from tongue images of clinical patients to build an automated ZHENG classification system. Given that Chinese medical practitioners usually observe the tongue color and coating to determine a ZHENG type and to diagnose different stomach disorders including gastritis, we propose using machine-learning techniques to establish the relationship between the tongue image features and ZHENG by learning through examples. The experimental results obtained over a set of 263 gastritis patients, most of whom suffering Cold Zheng or Hot ZHENG, and a control group of 48 healthy volunteers demonstrate an excellent performance of our proposed system.


---

## Body

## 1. Introduction

Traditional Chinese Medicine (TCM) has a long history in the treatment of various diseases in East Asian countries and is also a complementary and alternative medical system in Western countries. TCM takes a holistic approach to medicine with emphasis on the integrity of the human body and the close relationship between a human and its social and natural environment [1]. TCM applies different therapeutic methods to enhance the body’s resistance to diseases and prevention. TCM diagnosis is based on the information obtained from four diagnostic processes, that is, looking, listening, and smelling, asking, and touching. The most common tasks are taking the pulse and inspecting the tongue [2]. For thousands of years, Chinese medical practitioners have diagnosed the health status of a patients’ internal organs by inspecting the tongue, especially the patterns on the tongue’s surface. The tongue mirrors the viscera. The changes of tongue can objectively manifest the states of a disease, which can help differentiate syndromes, establish treatment methods, prescribe herbs, and determine prognosis of disease.ZHENG (TCM syndrome) is an integral and essential part of TCM theory. It is a characteristic profile of all clinical manifestations that can be identified by a TCM practitioner. ZHENG is an outcome after analyzing all symptoms and signs (tongue appearance and pulse feeling included). All diagnostic and therapeutic methods in TCM are based on the differentiation of ZHENG, and this concept is as ancient as TCM in China [3]. ZHENG is not simply an assemblage of disease symptoms but rather can be viewed as the TCM theoretical abstraction of the symptom profiles of individual patients. As noted in the abstract, ZHENG is also used as a guideline in TCM disease classification. For example, patients suffering from the same disease may be grouped into different ZHENGs, whereas different diseases may be grouped as the same ZHENG. The Cold ZHENG (Cold syndrome) and the Hot ZHENG (Cot syndrome) are the two key statuses of ZHENG[3].Other ZHENGs include Shen-Yang-Xu ZHENG (Kidney-Yang deficiency syndrome), Shen-Xu ZHENG (Kidney deficiency syndrome), and Xue-Yu ZHENG (Blood Stasis syndrome) [4].In this paper, we explore new modalities for the clinical characterization of ZHENG using various supervised machine-learning algorithms. Using an automated tongue-image diagnosis system, we extract objective features from tongue images of clinical patients and analyze the relationship with their corresponding ZHENG data and disease prognosis (specifically stomach disorders, i.e., gastritis) obtained from clinical practitioners. We propose a system that learns from the clinical practitioner’s subjective data on how to classify a patient’s health status by extracting meaningful features from tongue images using a rich set of features based on color-space models. Our premise is that Chinese medical practitioners usually observe the tongue color and coating to determine ZHENG such as Hot or Cold ZHENG, and to diagnose different stomach disorders including gastritis. Hence, we propose using machine-learning techniques to establish the relationship between the tongue image features and the ZHENG by learning through examples. We are also interested in the correlation between the Hot and Cold patterns observed in ZHENG gastritis patients and their corresponding symptom profiles.Various types of features have been explored for tongue feature extraction and tongue analysis, including texture [5], color [6–8], shape [9], spectrum [8], among others. A systematic tongue feature set, comprising of a combination of geometric features (size, shape, etc.), cracks, and textures, was later proposed by Zhang et al. [10]. Computer-aided tongue analysis systems based on these types of features have also been developed [11, 12]. Our goal is to provide a set of objective features that can be extracted from patients’ tongue images, based on the knowledge of ZHENG, which improves accuracy of an objective clinical diagnosis. Our proposed tongue feature set is based on an extensive color model.This paper is organized as follows: in Section2, we provide a TCM descriptive view of the physiology of the tongue. An overview of the proposed feature extraction and learning framework along with a complete description of the color space model feature set is presented in Section 3. Our experimental results and analysis in a tongue image dataset from gastritis patients with Cold ZHENG and Hot ZHENG are discussed in Section 4 before drawing our conclusions and proposing plans for future work in Section 5.


## 2. Tongue Diagnosis in TCM

TCM believes that the tongue has many relationships and connections in the human body, both to the meridians and the internal organs. It is, therefore, very useful and important during inspection for confirming TCM diagnosis as it can present strong visual indicators of a person’s overall physical and mental harmony or disharmony. In TCM, the tongue is divided into tongue tip, tongue margins, tongue center, and tongue root.Figure1(a) shows each part of the tongue and its correspondence to different internal organs according to TCM while Figure 1(b) illustrates how we geometrically obtain an approximation of these regions from the tongue image. The tongue tip reflects the pathological changes in the heart and lungs, while the bilateral sides of the tongue reflect that of the liver and gallbladders. The pathological changes in the spleen and stomach are mirrored by the center of tongue, while changes in the kidneys, intestines, and bladder section correspond to the tongue root.Tongue areas and their correspondence to internal organs in TCM.
(a)
organ layout of tongue regions(b)
geometrical layout of tongue regionsIn this paper, we focus on the patients with stomach disorders, gastritis. Hence, we are interested in extracting features not only from entire tongue image but also specifically from the middle region, as this corresponds to the stomach organ, according to TCM. We extract the middle rectangular region, illustrated in Figure1(b), as our approximation for the tongue middle region.The practitioner examines the general and local shape as well as the color of the tongue and its coating. According to TCM, the normal tongue is pale red with thin white coating. Some signs of imbalance or pathology are red body, yellow coating, or thick coating like mozzarella cheese, and so forth. Some characteristic changes occur in the tongue in some particular diseases. Most tongue attributes are on the tongue surface. A TCM doctor looks at several attributes of tongue body: color, moisture, size, shape, and coating. These signs not only reveal overall states of health but they also correlate to specific organ functions and disharmonies, especially in the digestive system.The two main characteristics of the tongue in TCM ZHENG diagnosis are the color and the coating. The color of the patient’s tongue color provides information about his/her health status. For example [13], dark red color can indicate inflammation or ulceration, while a white tongue indicates cold attack, mucus deposits, or a weakness in the blood leading to such conditions as anemia [12]. Moreover, a yellow tongue points out a disorder of the liver and gallbladder, and blue or purple implies stagnation of blood circulation and a serious weakening of the part of the digestive system that corresponds to the area of the tongue where the color appears.The coating on the tongue is discriminated by not only its presence but also its color. The color could be yellow, white, and other colors. However, the color in image is not the exact true color of the tongue. To properly identify the color of the tongue coating, we applied the specular component technique presented in our prior work on tongue detection and analysis [2]. Figure 2 illustrates different tongue images of patients and their corresponding ZHENG class.Tongue images of patients with different ZHENG classification. “Normal” represents a healthy person.
(a)
Hot ZHENG(b)
Cold ZHENG(c)
Normal


## 3. Tongue Feature Extraction and Classification Framework

### 3.1. Feature Extraction for Tongue Image Analysis

Our goal is to compute a set of objective featuresF⃗j={Fn} from each tongue image jthat can be fed into our learning system so that we can predict not only the color and coating on the tongue, but also different ZHENGs of the gastritis patients. These features are designed to capture different color characteristics of the tongue. While a single feature may not be very discriminative, our premise is that the aggregation of these features will be discriminative. We leave it to the learning algorithm to determine the weight/contribution of each feature in the final classification.Most color spaces are represented in tuples of number, normally three or four color components. Color components determine the position of the color in the color space used. There are many color spaces defined for different purposes. We designed a set of 25 features that span the entire color-space model. They can be grouped under eight categories: RGB, HSV, YIQ, Y’CbCr, XYZ, L*a*b*, CIE Luv, and CMYK.In this section, we first describe in detail how we compute each featuref⃗i per ith  pixel in the image. Then, we explain how each feature per pixel is aggregated to obtain F⃗j={Fn} per tongue image j.


#### 3.1.1. RGB

RGB is an additive color system, based on trichromatic theory in which red, green, and blue light components are added together to produce a specific pigment. The RGB model encodes the intensity of red, green, and blue, respectively.(Ri,Gi,Bi) for each pixel is an unsigned integer between 0 and 255.  Each RGB feature {fni∣n=1,…,3} represents the normalized intensity value of the red, green, and blue component, respectively, of theith pixel in the image. We denote the normalized value of each component asri=Ri/255,gi=Gi/255,andbi=Bi/255. Thus, f1i=ri;f2i=gi;f3i=bi.All the remaining color-space model features described in our feature set derive their value from the RGB feature set.


#### 3.1.2. HSV

HSV color space represents color using a 3-tuple set of hue, saturation, and value. It separates the luminance component of the color from chrominance information. The HSV model(Hi,Si,Vi) is obtained by a linear transformation of thenormalized RGB color space {ri,gi,bi}.For each pixelpi, let M̃i=max{ri,gi,bi} represent the maximum value of the pixel’s RGB triple set while m̃i=min(ri,gi,bi), the minimum value of the set. We also denote the difference between maximum and minimum values of each RGB tuple by Δi=M̃i-m̃i. The HSV components {Hi,Si,Vi} are computed from RGB color space {ri,gi,bi} as follows:
(1)Vi=Mĩ,Si={0,M̃i=0,ΔiM̃i,otherwise,Hi={0,Δi=0,gi-bi6⋅Δi,M̃i=ri,(bi-riΔi+2)⋅16,M̃i=gi,(ri-giΔi+4)⋅16,M̃i=bi.Thus, the HSV features aref4i=Hi;f5i=Si;f6i=Vi.


#### 3.1.3. YIQ

The YIQ color model is the television transmission color space for a digital standard. TheY component represents the perceived luminance, while I and Q components are the color information. I character is referred to “in-phase” term and Q letter stands for “quadrature.”  I and Q can place color in a graph representing I as X axis and Q as Y axis. The YIQ system takes advantage of human color perceiver characteristics [14, 15].The YIQ model(Yi,Ii,Qi) is obtained by a linear transformation of the normalized RGB color space {ri,gi,bi} as follows:
(2)[YiIiQi]=[0.299+0.587+0.1140.596-0.274-0.3220.211-0.523+0.312][rigibi].The{Yi,Ii,Qi} values are each normalized to obtain {yi,ii,qi}∈[0,1]. Thus, the YIQ features are  f7i=yi;f8i=ii;f9i=qi.


#### 3.1.4. Y’CbCr

Like YIQ, Y’CbCr is the television transmission color spaces but it is in analogue spaces for the NTSC system. YCbCr color space detaches RGB into the luma component, the blue-difference and red-difference chroma components. The transformation equation from RGB (unnormalized) model to YCbCr is defined as(3)[Yi′CbiCri]=[0.299+0.587+0.114-0.169-0.331+0.5000.500-0.419-0.081][RiGiBi].
Similar to the YIQ features, the {Yi′,Cbi,Cri} values are each normalized to obtain{yi′,cbi,cri}∈[0,1]. Thus the YIQ features are f10i=yi′;f11i=cbi;f12i=cri.


#### 3.1.5. XYZ

Brightness and chromaticity are two principal components of color that interact with human vision. XYZ are developed under CIE XYZ color space [16]. The XYZ values can be obtained by a linear transformation of the gamma corrected value of the RGB normalized color space  {ri,gi,bi}.The gamma-corrected function is defined as(4)γ(t)={t12.92,ift≤0.04045,(t+a1+a)2.4,otherwise,
wherea=0.055. Thus, XYZ model consisting of {Xi,Yi′′,Zi} components is given by(5)[XiYi′′Zi]=[0.41240.35760.18050.21260.71520.07220.01930.11920.9505][γ(ri)γ(gi)γ(bi)].The{Xi,Yi′′,Zi} values are each normalized to obtain{xi,yi′′,zi}∈[0,1]. Thus, the XYZ features are defined as f13i=xi;f14i=yi′′;f15i=zi.


#### 3.1.6. L*a*b*

CIE L*a*b* color space is a nonlinear transformation of the CIE XYZ color space [17]. CIE L*a*b* try to imitate the logarithmic response of the human eye. The L* component is designed to match closely with human perception of lightness. The other two components describe the chroma.The forward transformation of CIE XYZ color space to CIE L*a*b* is computed as follows:(6)Li*=116φ(Yi′′δ2)-16,Ai=500[φ(Xiδ1)-φ(Yi′′δ2)],Bi=200[φ(Yi′′δ2)-φ(Ziδ3)],
where
(7)φ(t)={t1/3,ift>(629)3,13(296)2t+429,otherwise,
and {δ} denotes the D65 white point given by {0.950456,1.0,1.088754}.The L*a*b* values{L*i,Ai,Bi} are normalized as{li*,ai,bi}∈[0,1]. Hence, the CIE L*a*b* color features are given by f16i=li*;f17i=ai;f18i=bi.


#### 3.1.7. CIE Luv

CIE Luv, or L*u*v*, is color-space-computed from the transformation of the CIE XYZ color space by International Commission on Illumination (CIE) in order to perceptual uniformity [17]. Similar to CIE L*a*b*, the D65 white point is referred by  {δ}:
(8)Li′′={(293)3(Yi′′δ2),ifYi′′δ2≤(629)3,116(Yi′′δ2)1/3-16,otherwise,Ui=13Li′′(4XiXi+15Yi′′+3Zi-k1),Vi=13Li′′(9Yi′′Xi+15Yi′′+3Zi-k2),
where k1=0.2009,k2=0.4610, under the standard luminance C. The normalized {Li′′,Ui,Vi} values are denoted by {li′′,ui,vi}∈[0,1]. Therefore, f19i=li′′;f20i=ui;f21i=vi.


#### 3.1.8. CMYK

The CMYK color space is a subtractive color system mainly used in the printing industry [16]. The components consist of cyan, magenta, yellow, and neutral black. It is a common way to translate RGB display on monitors to CMYK values for printing.LetM̃i=max{ri,gi,bi} represent the maximum value of the pixel’s RGB triple set. The CMYK color space, denoted by {Ci,Mi,Yi*,Ki}, can be computed from the RGB model as follows:
(9)Ki=1-M̃i,Ci=M̃i-riM̃i,Mi=M̃i-giM̃i,Yi*=M̃i-biM̃i,Thus, the CMYK features are computed asf22i=Ci;f23i=Mi;f24i=Yi*;f25i=Ki.


#### 3.1.9. Aggregate Operators for the Feature Vectors

To train our classification model using this set of features, we need to combine the features per pixel into one composite feature vectorF⃗j={Fn} per tongue image (or region) j. We aggregate the pixel features using two different statistical averages (mean and median) and the standard deviation values. We derive five variations of feature vectors for our automated tongue ZHENG classification system using the following operators: mean, median (medF⃗), standard deviation (σF⃗), “mean plus standard deviation” ({μF⃗,σF⃗}), and “median plus standard deviation” ({medF⃗,σF⃗}).LetN denote the number of pixels in a given tongue image (or region) j. The mean feature vector is denoted byμFj⃗={μFn}, where μFn is given by
(10)μFn=∑i=1NfniN,n=1,…,25.The median feature vector, denoted bymedFj⃗={medFn}, is computed as medFn=mid{sort(Fset)},n=1,…,25. Standard deviation depicts the margin of difference between a given feature value and its average value among all the pixels in the given region. Thus, the standard deviation feature vector is denoted by σFj⃗={σFn}, where σFn is given by
(11)σFn=∑i=1N(fni-μFn)N2,n=1,…,25.The “mean plus standard deviation,” denoted by{μF⃗,σF⃗}, is a concatenation of the mean feature vector and the standard deviation feature vector. Similarly, the “median plus standard deviation” feature vector, denoted by {medF⃗,σF⃗}, is a concatenation of the median feature vector and the standard deviation feature vector. Thus, the total number of features in both concatenated feature vectors is 50 each.


### 3.2. Supervised Learning Algorithms for ZHENG Classification

We apply three different supervised learning algorithms (AdaBoost, support vector machine, multilayer perceptron network) to build classification models for training and evaluating the proposed automated tongue based diagnosis system. Each model has its strength and weakness, which we describe briefly below. We empirically evaluate their performance over our dataset.


#### 3.2.1. AdaBoost

An ensemble of classifiers is a set of classifiers whose individual predictions are combined in some way (typically by voting) to classify new examples. Boosting is a type of ensemble classifier which generates a set of weak classifiers using instances drawn from an iteratively updated distribution of the data, where in each iteration the probability of incorrectly classified examples is increased and the probability of the correctly classified examples is decreased. The ensemble classifier is a weighted majority vote of the sequence of classifiers produced.The AdaBoost algorithm [18] trains a weak or base-learning algorithm repeatedly in a series of round t=1,…,T. Given a training set {xi,yi}i=1,…,n, where xi belongs to some domain X and yi∈Y={-1,+1} (the corresponding binary class labels), we denote the weight of ith example in round t by Dt(i). Initially, all weights are set equally and so D1(i)=1/n,for alli. For each round t, a weak learner is trained using the current distribution Dt. When we obtain a weak hypothesis ht with error ϵt=Pri~Dt[ht(xi)≠yi.], if ϵt>1/2, we end training; otherwise, we set αt=(1/2)ln((1-ϵt)/ϵt) and update Dt+1 as(12)Dt+1(i)=Dt(i)Zt×{e-αtifht(xi)=yi,eαtifht(xi)≠yi,
where Zt is a normalization factor.The final hypothesis is given byH(x)=sign(∑t=1Tatht(x)).


#### 3.2.2. Support Vector Machine

The support vector machine (SVM) [19] is one of the best-known general purpose learning algorithms. The goal of the SVM is to produce a model which predicts target values of data instances in the testing set given a vector of feature attributes. It attempts to maximize the margin of separation between the support vectors of each class and minimize the error in case the data is nonlinearly separable. The SVM classifiers usually perform well in high-dimensional spaces, avoid overfitting, and have good generalization capabilities.For a given a training set{xi,yi}i=1,…,n, the SVM model for an instance xcan be written as [20](13)f(x)=∑i=1nyiαik(xi,x)+b,
where k is the kernel function used (polynomial kernel in this work), αi is the Lagrange multiplier, and b is a constant.In our work, we utilize the sequential minimal optimization (SMO) algorithm [21], which gives an efficient way of solving the dual problem of the support vector machine optimization problem.


#### 3.2.3. Multilayer Perceptron Networks

The multilayer perceptron network (MLP) [22] is a feed-forward neural network with one or more layers that are hidden from the input and output nodes. Neural networks have the ability to learn complex data structures and approximate any continuous mapping [23]. The model of each neuron in the network includes a nonlinear activation function that is differentiable such as the sigmoid. The units each perform a biased weighted sum of their inputs and pass this activation level through the transfer function to produce their output given by(14)φ(x)=f(wTx+θ),
wherew is the synaptic vector, x is the input vector, θ is the bias constant, and T is the transpose operator. For K-class classification, the MLP uses back propagation to implement nonlinear discriminants. There are K outputs with softmax as the output nonlinearity.


### 3.3. Dataset Labeling and Preprocessing

Our proposed system relies on a labeled dataset, to effectively build an automated tongue-based ZHENG classification system. Our dataset is comprised of tongue images from 263 gastritis patients and a control group of 48 healthy volunteers. Most of the gastritis patients have been classified as Hot or Cold ZHENG and are identified with a color label (yellow or white) based on the color of the coating of their tongue, as determined by their Chinese doctors. The doctors also carry out a detailed profile of the ZHENG symptoms for each patient based on clinical evaluations. The list of the main symptom profile terms is summarized in Table1.Table 1
Symptom profile terms of Cold ZHENG and Hot ZHENG.
SubjectsTerms (keywords)Cold-ZHENG related symptomsCold (chill, coldness), hot diet/drink preferred, desires warm environment, pale flushing of face, not thirsty, no bad mouth breath, no acidic saliva, clear urine, loose stool, high and short pitch voice, and feeling cold at limbs.Hot-ZHENGrelated symptomsFever (heat, hot), cold diet/drink preferred, desires cold environment, red flushing of face, thirsty, obvious bad mouth breath, acidic saliva, yellow urine, hard stool, constipation, and feeling hot at limbs.We are also interested in the relationship between TCM diagnosis and Western medicine diagnosis; hence, for a subset of the patients, we are provided with their corresponding Western medical gastritis pathology. They are grouped into two categories: superficial versus atrophic. In Western medicine, the doctors are also interested in knowing whether theHelicobacter Pylori (HP) bacterium found in the stomach is present (positive) or absent (negative) in the patients with chronic gastritis. Thus, we are provided with that information for a subset of the patients. It was not feasibleto obtain all the different information collected per patient. Table 2 summaries the population of each subset for four different labels (ZHENG, Coating, Pathology, and HP).Table 2
Data label summary for the gastritis patients.
Data labelsPopulationZHENG: Hot/Cold132/68Coating: yellow/white147/67Pathology: superficial/atrophic84/144HP bacterium: positive/negative72/167


## 3.1. Feature Extraction for Tongue Image Analysis

Our goal is to compute a set of objective featuresF⃗j={Fn} from each tongue image jthat can be fed into our learning system so that we can predict not only the color and coating on the tongue, but also different ZHENGs of the gastritis patients. These features are designed to capture different color characteristics of the tongue. While a single feature may not be very discriminative, our premise is that the aggregation of these features will be discriminative. We leave it to the learning algorithm to determine the weight/contribution of each feature in the final classification.Most color spaces are represented in tuples of number, normally three or four color components. Color components determine the position of the color in the color space used. There are many color spaces defined for different purposes. We designed a set of 25 features that span the entire color-space model. They can be grouped under eight categories: RGB, HSV, YIQ, Y’CbCr, XYZ, L*a*b*, CIE Luv, and CMYK.In this section, we first describe in detail how we compute each featuref⃗i per ith  pixel in the image. Then, we explain how each feature per pixel is aggregated to obtain F⃗j={Fn} per tongue image j.


### 3.1.1. RGB

RGB is an additive color system, based on trichromatic theory in which red, green, and blue light components are added together to produce a specific pigment. The RGB model encodes the intensity of red, green, and blue, respectively.(Ri,Gi,Bi) for each pixel is an unsigned integer between 0 and 255.  Each RGB feature {fni∣n=1,…,3} represents the normalized intensity value of the red, green, and blue component, respectively, of theith pixel in the image. We denote the normalized value of each component asri=Ri/255,gi=Gi/255,andbi=Bi/255. Thus, f1i=ri;f2i=gi;f3i=bi.All the remaining color-space model features described in our feature set derive their value from the RGB feature set.


### 3.1.2. HSV

HSV color space represents color using a 3-tuple set of hue, saturation, and value. It separates the luminance component of the color from chrominance information. The HSV model(Hi,Si,Vi) is obtained by a linear transformation of thenormalized RGB color space {ri,gi,bi}.For each pixelpi, let M̃i=max{ri,gi,bi} represent the maximum value of the pixel’s RGB triple set while m̃i=min(ri,gi,bi), the minimum value of the set. We also denote the difference between maximum and minimum values of each RGB tuple by Δi=M̃i-m̃i. The HSV components {Hi,Si,Vi} are computed from RGB color space {ri,gi,bi} as follows:
(1)Vi=Mĩ,Si={0,M̃i=0,ΔiM̃i,otherwise,Hi={0,Δi=0,gi-bi6⋅Δi,M̃i=ri,(bi-riΔi+2)⋅16,M̃i=gi,(ri-giΔi+4)⋅16,M̃i=bi.Thus, the HSV features aref4i=Hi;f5i=Si;f6i=Vi.


### 3.1.3. YIQ

The YIQ color model is the television transmission color space for a digital standard. TheY component represents the perceived luminance, while I and Q components are the color information. I character is referred to “in-phase” term and Q letter stands for “quadrature.”  I and Q can place color in a graph representing I as X axis and Q as Y axis. The YIQ system takes advantage of human color perceiver characteristics [14, 15].The YIQ model(Yi,Ii,Qi) is obtained by a linear transformation of the normalized RGB color space {ri,gi,bi} as follows:
(2)[YiIiQi]=[0.299+0.587+0.1140.596-0.274-0.3220.211-0.523+0.312][rigibi].The{Yi,Ii,Qi} values are each normalized to obtain {yi,ii,qi}∈[0,1]. Thus, the YIQ features are  f7i=yi;f8i=ii;f9i=qi.


### 3.1.4. Y’CbCr

Like YIQ, Y’CbCr is the television transmission color spaces but it is in analogue spaces for the NTSC system. YCbCr color space detaches RGB into the luma component, the blue-difference and red-difference chroma components. The transformation equation from RGB (unnormalized) model to YCbCr is defined as(3)[Yi′CbiCri]=[0.299+0.587+0.114-0.169-0.331+0.5000.500-0.419-0.081][RiGiBi].
Similar to the YIQ features, the {Yi′,Cbi,Cri} values are each normalized to obtain{yi′,cbi,cri}∈[0,1]. Thus the YIQ features are f10i=yi′;f11i=cbi;f12i=cri.


### 3.1.5. XYZ

Brightness and chromaticity are two principal components of color that interact with human vision. XYZ are developed under CIE XYZ color space [16]. The XYZ values can be obtained by a linear transformation of the gamma corrected value of the RGB normalized color space  {ri,gi,bi}.The gamma-corrected function is defined as(4)γ(t)={t12.92,ift≤0.04045,(t+a1+a)2.4,otherwise,
wherea=0.055. Thus, XYZ model consisting of {Xi,Yi′′,Zi} components is given by(5)[XiYi′′Zi]=[0.41240.35760.18050.21260.71520.07220.01930.11920.9505][γ(ri)γ(gi)γ(bi)].The{Xi,Yi′′,Zi} values are each normalized to obtain{xi,yi′′,zi}∈[0,1]. Thus, the XYZ features are defined as f13i=xi;f14i=yi′′;f15i=zi.


### 3.1.6. L*a*b*

CIE L*a*b* color space is a nonlinear transformation of the CIE XYZ color space [17]. CIE L*a*b* try to imitate the logarithmic response of the human eye. The L* component is designed to match closely with human perception of lightness. The other two components describe the chroma.The forward transformation of CIE XYZ color space to CIE L*a*b* is computed as follows:(6)Li*=116φ(Yi′′δ2)-16,Ai=500[φ(Xiδ1)-φ(Yi′′δ2)],Bi=200[φ(Yi′′δ2)-φ(Ziδ3)],
where
(7)φ(t)={t1/3,ift>(629)3,13(296)2t+429,otherwise,
and {δ} denotes the D65 white point given by {0.950456,1.0,1.088754}.The L*a*b* values{L*i,Ai,Bi} are normalized as{li*,ai,bi}∈[0,1]. Hence, the CIE L*a*b* color features are given by f16i=li*;f17i=ai;f18i=bi.


### 3.1.7. CIE Luv

CIE Luv, or L*u*v*, is color-space-computed from the transformation of the CIE XYZ color space by International Commission on Illumination (CIE) in order to perceptual uniformity [17]. Similar to CIE L*a*b*, the D65 white point is referred by  {δ}:
(8)Li′′={(293)3(Yi′′δ2),ifYi′′δ2≤(629)3,116(Yi′′δ2)1/3-16,otherwise,Ui=13Li′′(4XiXi+15Yi′′+3Zi-k1),Vi=13Li′′(9Yi′′Xi+15Yi′′+3Zi-k2),
where k1=0.2009,k2=0.4610, under the standard luminance C. The normalized {Li′′,Ui,Vi} values are denoted by {li′′,ui,vi}∈[0,1]. Therefore, f19i=li′′;f20i=ui;f21i=vi.


### 3.1.8. CMYK

The CMYK color space is a subtractive color system mainly used in the printing industry [16]. The components consist of cyan, magenta, yellow, and neutral black. It is a common way to translate RGB display on monitors to CMYK values for printing.LetM̃i=max{ri,gi,bi} represent the maximum value of the pixel’s RGB triple set. The CMYK color space, denoted by {Ci,Mi,Yi*,Ki}, can be computed from the RGB model as follows:
(9)Ki=1-M̃i,Ci=M̃i-riM̃i,Mi=M̃i-giM̃i,Yi*=M̃i-biM̃i,Thus, the CMYK features are computed asf22i=Ci;f23i=Mi;f24i=Yi*;f25i=Ki.


### 3.1.9. Aggregate Operators for the Feature Vectors

To train our classification model using this set of features, we need to combine the features per pixel into one composite feature vectorF⃗j={Fn} per tongue image (or region) j. We aggregate the pixel features using two different statistical averages (mean and median) and the standard deviation values. We derive five variations of feature vectors for our automated tongue ZHENG classification system using the following operators: mean, median (medF⃗), standard deviation (σF⃗), “mean plus standard deviation” ({μF⃗,σF⃗}), and “median plus standard deviation” ({medF⃗,σF⃗}).LetN denote the number of pixels in a given tongue image (or region) j. The mean feature vector is denoted byμFj⃗={μFn}, where μFn is given by
(10)μFn=∑i=1NfniN,n=1,…,25.The median feature vector, denoted bymedFj⃗={medFn}, is computed as medFn=mid{sort(Fset)},n=1,…,25. Standard deviation depicts the margin of difference between a given feature value and its average value among all the pixels in the given region. Thus, the standard deviation feature vector is denoted by σFj⃗={σFn}, where σFn is given by
(11)σFn=∑i=1N(fni-μFn)N2,n=1,…,25.The “mean plus standard deviation,” denoted by{μF⃗,σF⃗}, is a concatenation of the mean feature vector and the standard deviation feature vector. Similarly, the “median plus standard deviation” feature vector, denoted by {medF⃗,σF⃗}, is a concatenation of the median feature vector and the standard deviation feature vector. Thus, the total number of features in both concatenated feature vectors is 50 each.


## 3.1.1. RGB

RGB is an additive color system, based on trichromatic theory in which red, green, and blue light components are added together to produce a specific pigment. The RGB model encodes the intensity of red, green, and blue, respectively.(Ri,Gi,Bi) for each pixel is an unsigned integer between 0 and 255.  Each RGB feature {fni∣n=1,…,3} represents the normalized intensity value of the red, green, and blue component, respectively, of theith pixel in the image. We denote the normalized value of each component asri=Ri/255,gi=Gi/255,andbi=Bi/255. Thus, f1i=ri;f2i=gi;f3i=bi.All the remaining color-space model features described in our feature set derive their value from the RGB feature set.


## 3.1.2. HSV

HSV color space represents color using a 3-tuple set of hue, saturation, and value. It separates the luminance component of the color from chrominance information. The HSV model(Hi,Si,Vi) is obtained by a linear transformation of thenormalized RGB color space {ri,gi,bi}.For each pixelpi, let M̃i=max{ri,gi,bi} represent the maximum value of the pixel’s RGB triple set while m̃i=min(ri,gi,bi), the minimum value of the set. We also denote the difference between maximum and minimum values of each RGB tuple by Δi=M̃i-m̃i. The HSV components {Hi,Si,Vi} are computed from RGB color space {ri,gi,bi} as follows:
(1)Vi=Mĩ,Si={0,M̃i=0,ΔiM̃i,otherwise,Hi={0,Δi=0,gi-bi6⋅Δi,M̃i=ri,(bi-riΔi+2)⋅16,M̃i=gi,(ri-giΔi+4)⋅16,M̃i=bi.Thus, the HSV features aref4i=Hi;f5i=Si;f6i=Vi.


## 3.1.3. YIQ

The YIQ color model is the television transmission color space for a digital standard. TheY component represents the perceived luminance, while I and Q components are the color information. I character is referred to “in-phase” term and Q letter stands for “quadrature.”  I and Q can place color in a graph representing I as X axis and Q as Y axis. The YIQ system takes advantage of human color perceiver characteristics [14, 15].The YIQ model(Yi,Ii,Qi) is obtained by a linear transformation of the normalized RGB color space {ri,gi,bi} as follows:
(2)[YiIiQi]=[0.299+0.587+0.1140.596-0.274-0.3220.211-0.523+0.312][rigibi].The{Yi,Ii,Qi} values are each normalized to obtain {yi,ii,qi}∈[0,1]. Thus, the YIQ features are  f7i=yi;f8i=ii;f9i=qi.


## 3.1.4. Y’CbCr

Like YIQ, Y’CbCr is the television transmission color spaces but it is in analogue spaces for the NTSC system. YCbCr color space detaches RGB into the luma component, the blue-difference and red-difference chroma components. The transformation equation from RGB (unnormalized) model to YCbCr is defined as(3)[Yi′CbiCri]=[0.299+0.587+0.114-0.169-0.331+0.5000.500-0.419-0.081][RiGiBi].
Similar to the YIQ features, the {Yi′,Cbi,Cri} values are each normalized to obtain{yi′,cbi,cri}∈[0,1]. Thus the YIQ features are f10i=yi′;f11i=cbi;f12i=cri.


## 3.1.5. XYZ

Brightness and chromaticity are two principal components of color that interact with human vision. XYZ are developed under CIE XYZ color space [16]. The XYZ values can be obtained by a linear transformation of the gamma corrected value of the RGB normalized color space  {ri,gi,bi}.The gamma-corrected function is defined as(4)γ(t)={t12.92,ift≤0.04045,(t+a1+a)2.4,otherwise,
wherea=0.055. Thus, XYZ model consisting of {Xi,Yi′′,Zi} components is given by(5)[XiYi′′Zi]=[0.41240.35760.18050.21260.71520.07220.01930.11920.9505][γ(ri)γ(gi)γ(bi)].The{Xi,Yi′′,Zi} values are each normalized to obtain{xi,yi′′,zi}∈[0,1]. Thus, the XYZ features are defined as f13i=xi;f14i=yi′′;f15i=zi.


## 3.1.6. L*a*b*

CIE L*a*b* color space is a nonlinear transformation of the CIE XYZ color space [17]. CIE L*a*b* try to imitate the logarithmic response of the human eye. The L* component is designed to match closely with human perception of lightness. The other two components describe the chroma.The forward transformation of CIE XYZ color space to CIE L*a*b* is computed as follows:(6)Li*=116φ(Yi′′δ2)-16,Ai=500[φ(Xiδ1)-φ(Yi′′δ2)],Bi=200[φ(Yi′′δ2)-φ(Ziδ3)],
where
(7)φ(t)={t1/3,ift>(629)3,13(296)2t+429,otherwise,
and {δ} denotes the D65 white point given by {0.950456,1.0,1.088754}.The L*a*b* values{L*i,Ai,Bi} are normalized as{li*,ai,bi}∈[0,1]. Hence, the CIE L*a*b* color features are given by f16i=li*;f17i=ai;f18i=bi.


## 3.1.7. CIE Luv

CIE Luv, or L*u*v*, is color-space-computed from the transformation of the CIE XYZ color space by International Commission on Illumination (CIE) in order to perceptual uniformity [17]. Similar to CIE L*a*b*, the D65 white point is referred by  {δ}:
(8)Li′′={(293)3(Yi′′δ2),ifYi′′δ2≤(629)3,116(Yi′′δ2)1/3-16,otherwise,Ui=13Li′′(4XiXi+15Yi′′+3Zi-k1),Vi=13Li′′(9Yi′′Xi+15Yi′′+3Zi-k2),
where k1=0.2009,k2=0.4610, under the standard luminance C. The normalized {Li′′,Ui,Vi} values are denoted by {li′′,ui,vi}∈[0,1]. Therefore, f19i=li′′;f20i=ui;f21i=vi.


## 3.1.8. CMYK

The CMYK color space is a subtractive color system mainly used in the printing industry [16]. The components consist of cyan, magenta, yellow, and neutral black. It is a common way to translate RGB display on monitors to CMYK values for printing.LetM̃i=max{ri,gi,bi} represent the maximum value of the pixel’s RGB triple set. The CMYK color space, denoted by {Ci,Mi,Yi*,Ki}, can be computed from the RGB model as follows:
(9)Ki=1-M̃i,Ci=M̃i-riM̃i,Mi=M̃i-giM̃i,Yi*=M̃i-biM̃i,Thus, the CMYK features are computed asf22i=Ci;f23i=Mi;f24i=Yi*;f25i=Ki.


## 3.1.9. Aggregate Operators for the Feature Vectors

To train our classification model using this set of features, we need to combine the features per pixel into one composite feature vectorF⃗j={Fn} per tongue image (or region) j. We aggregate the pixel features using two different statistical averages (mean and median) and the standard deviation values. We derive five variations of feature vectors for our automated tongue ZHENG classification system using the following operators: mean, median (medF⃗), standard deviation (σF⃗), “mean plus standard deviation” ({μF⃗,σF⃗}), and “median plus standard deviation” ({medF⃗,σF⃗}).LetN denote the number of pixels in a given tongue image (or region) j. The mean feature vector is denoted byμFj⃗={μFn}, where μFn is given by
(10)μFn=∑i=1NfniN,n=1,…,25.The median feature vector, denoted bymedFj⃗={medFn}, is computed as medFn=mid{sort(Fset)},n=1,…,25. Standard deviation depicts the margin of difference between a given feature value and its average value among all the pixels in the given region. Thus, the standard deviation feature vector is denoted by σFj⃗={σFn}, where σFn is given by
(11)σFn=∑i=1N(fni-μFn)N2,n=1,…,25.The “mean plus standard deviation,” denoted by{μF⃗,σF⃗}, is a concatenation of the mean feature vector and the standard deviation feature vector. Similarly, the “median plus standard deviation” feature vector, denoted by {medF⃗,σF⃗}, is a concatenation of the median feature vector and the standard deviation feature vector. Thus, the total number of features in both concatenated feature vectors is 50 each.


## 3.2. Supervised Learning Algorithms for ZHENG Classification

We apply three different supervised learning algorithms (AdaBoost, support vector machine, multilayer perceptron network) to build classification models for training and evaluating the proposed automated tongue based diagnosis system. Each model has its strength and weakness, which we describe briefly below. We empirically evaluate their performance over our dataset.


### 3.2.1. AdaBoost

An ensemble of classifiers is a set of classifiers whose individual predictions are combined in some way (typically by voting) to classify new examples. Boosting is a type of ensemble classifier which generates a set of weak classifiers using instances drawn from an iteratively updated distribution of the data, where in each iteration the probability of incorrectly classified examples is increased and the probability of the correctly classified examples is decreased. The ensemble classifier is a weighted majority vote of the sequence of classifiers produced.The AdaBoost algorithm [18] trains a weak or base-learning algorithm repeatedly in a series of round t=1,…,T. Given a training set {xi,yi}i=1,…,n, where xi belongs to some domain X and yi∈Y={-1,+1} (the corresponding binary class labels), we denote the weight of ith example in round t by Dt(i). Initially, all weights are set equally and so D1(i)=1/n,for alli. For each round t, a weak learner is trained using the current distribution Dt. When we obtain a weak hypothesis ht with error ϵt=Pri~Dt[ht(xi)≠yi.], if ϵt>1/2, we end training; otherwise, we set αt=(1/2)ln((1-ϵt)/ϵt) and update Dt+1 as(12)Dt+1(i)=Dt(i)Zt×{e-αtifht(xi)=yi,eαtifht(xi)≠yi,
where Zt is a normalization factor.The final hypothesis is given byH(x)=sign(∑t=1Tatht(x)).


### 3.2.2. Support Vector Machine

The support vector machine (SVM) [19] is one of the best-known general purpose learning algorithms. The goal of the SVM is to produce a model which predicts target values of data instances in the testing set given a vector of feature attributes. It attempts to maximize the margin of separation between the support vectors of each class and minimize the error in case the data is nonlinearly separable. The SVM classifiers usually perform well in high-dimensional spaces, avoid overfitting, and have good generalization capabilities.For a given a training set{xi,yi}i=1,…,n, the SVM model for an instance xcan be written as [20](13)f(x)=∑i=1nyiαik(xi,x)+b,
where k is the kernel function used (polynomial kernel in this work), αi is the Lagrange multiplier, and b is a constant.In our work, we utilize the sequential minimal optimization (SMO) algorithm [21], which gives an efficient way of solving the dual problem of the support vector machine optimization problem.


### 3.2.3. Multilayer Perceptron Networks

The multilayer perceptron network (MLP) [22] is a feed-forward neural network with one or more layers that are hidden from the input and output nodes. Neural networks have the ability to learn complex data structures and approximate any continuous mapping [23]. The model of each neuron in the network includes a nonlinear activation function that is differentiable such as the sigmoid. The units each perform a biased weighted sum of their inputs and pass this activation level through the transfer function to produce their output given by(14)φ(x)=f(wTx+θ),
wherew is the synaptic vector, x is the input vector, θ is the bias constant, and T is the transpose operator. For K-class classification, the MLP uses back propagation to implement nonlinear discriminants. There are K outputs with softmax as the output nonlinearity.


## 3.2.1. AdaBoost

An ensemble of classifiers is a set of classifiers whose individual predictions are combined in some way (typically by voting) to classify new examples. Boosting is a type of ensemble classifier which generates a set of weak classifiers using instances drawn from an iteratively updated distribution of the data, where in each iteration the probability of incorrectly classified examples is increased and the probability of the correctly classified examples is decreased. The ensemble classifier is a weighted majority vote of the sequence of classifiers produced.The AdaBoost algorithm [18] trains a weak or base-learning algorithm repeatedly in a series of round t=1,…,T. Given a training set {xi,yi}i=1,…,n, where xi belongs to some domain X and yi∈Y={-1,+1} (the corresponding binary class labels), we denote the weight of ith example in round t by Dt(i). Initially, all weights are set equally and so D1(i)=1/n,for alli. For each round t, a weak learner is trained using the current distribution Dt. When we obtain a weak hypothesis ht with error ϵt=Pri~Dt[ht(xi)≠yi.], if ϵt>1/2, we end training; otherwise, we set αt=(1/2)ln((1-ϵt)/ϵt) and update Dt+1 as(12)Dt+1(i)=Dt(i)Zt×{e-αtifht(xi)=yi,eαtifht(xi)≠yi,
where Zt is a normalization factor.The final hypothesis is given byH(x)=sign(∑t=1Tatht(x)).


## 3.2.2. Support Vector Machine

The support vector machine (SVM) [19] is one of the best-known general purpose learning algorithms. The goal of the SVM is to produce a model which predicts target values of data instances in the testing set given a vector of feature attributes. It attempts to maximize the margin of separation between the support vectors of each class and minimize the error in case the data is nonlinearly separable. The SVM classifiers usually perform well in high-dimensional spaces, avoid overfitting, and have good generalization capabilities.For a given a training set{xi,yi}i=1,…,n, the SVM model for an instance xcan be written as [20](13)f(x)=∑i=1nyiαik(xi,x)+b,
where k is the kernel function used (polynomial kernel in this work), αi is the Lagrange multiplier, and b is a constant.In our work, we utilize the sequential minimal optimization (SMO) algorithm [21], which gives an efficient way of solving the dual problem of the support vector machine optimization problem.


## 3.2.3. Multilayer Perceptron Networks

The multilayer perceptron network (MLP) [22] is a feed-forward neural network with one or more layers that are hidden from the input and output nodes. Neural networks have the ability to learn complex data structures and approximate any continuous mapping [23]. The model of each neuron in the network includes a nonlinear activation function that is differentiable such as the sigmoid. The units each perform a biased weighted sum of their inputs and pass this activation level through the transfer function to produce their output given by(14)φ(x)=f(wTx+θ),
wherew is the synaptic vector, x is the input vector, θ is the bias constant, and T is the transpose operator. For K-class classification, the MLP uses back propagation to implement nonlinear discriminants. There are K outputs with softmax as the output nonlinearity.


## 3.3. Dataset Labeling and Preprocessing

Our proposed system relies on a labeled dataset, to effectively build an automated tongue-based ZHENG classification system. Our dataset is comprised of tongue images from 263 gastritis patients and a control group of 48 healthy volunteers. Most of the gastritis patients have been classified as Hot or Cold ZHENG and are identified with a color label (yellow or white) based on the color of the coating of their tongue, as determined by their Chinese doctors. The doctors also carry out a detailed profile of the ZHENG symptoms for each patient based on clinical evaluations. The list of the main symptom profile terms is summarized in Table1.Table 1
Symptom profile terms of Cold ZHENG and Hot ZHENG.
SubjectsTerms (keywords)Cold-ZHENG related symptomsCold (chill, coldness), hot diet/drink preferred, desires warm environment, pale flushing of face, not thirsty, no bad mouth breath, no acidic saliva, clear urine, loose stool, high and short pitch voice, and feeling cold at limbs.Hot-ZHENGrelated symptomsFever (heat, hot), cold diet/drink preferred, desires cold environment, red flushing of face, thirsty, obvious bad mouth breath, acidic saliva, yellow urine, hard stool, constipation, and feeling hot at limbs.We are also interested in the relationship between TCM diagnosis and Western medicine diagnosis; hence, for a subset of the patients, we are provided with their corresponding Western medical gastritis pathology. They are grouped into two categories: superficial versus atrophic. In Western medicine, the doctors are also interested in knowing whether theHelicobacter Pylori (HP) bacterium found in the stomach is present (positive) or absent (negative) in the patients with chronic gastritis. Thus, we are provided with that information for a subset of the patients. It was not feasibleto obtain all the different information collected per patient. Table 2 summaries the population of each subset for four different labels (ZHENG, Coating, Pathology, and HP).Table 2
Data label summary for the gastritis patients.
Data labelsPopulationZHENG: Hot/Cold132/68Coating: yellow/white147/67Pathology: superficial/atrophic84/144HP bacterium: positive/negative72/167


## 4. Results and Analysis

### 4.1. Experimental Setup

In this section, we evaluated the performance of our proposed ZHENG classification system using the three classification models (AdaBoost, SVM, and MLP) described in Section3.2. We compared the performance of training the classifier models using the set of features extracted from the entire tongue image versus the middle tongue region only. As mentioned in Section 2, in TCM, it is believed that the middle tongue region provides discriminant information for diagnosing stomach disorders. Hence, we extract features from the middle tongue region, as described in Figure1(b), to evaluate the performance compared to extracting features from the entire tongue region. In training and testing our classification models, we employ a 3-fold cross-validation strategy.  This implies that the data is split into three sets; one set is used for testing and the remaining two sets are used for training. The experiment is repeated with each of the three sets used for testing. The average accuracy of the tests over the three sets is taken as the performance measure. For each classification model, we varied the parameters to optimize its performance. We also compare the results obtained using the five different variations of the feature vector (mean = μF⃗, median =medF⃗, standard deviation = σF⃗, mean + standard deviation = {μF⃗,σF⃗}, and median + standard deviation = {medF⃗,σF⃗}), as described in Section 3.1. We also apply Information Gain attribute evaluation on the feature vectors to quantify and rank the significance of  individual features. Lastly, we apply the Best First feature selection algorithm to select the “significant” features before training the classifiers to compare the performance of training the classifiers with the whole feature set against selected features.The performance metrics used are the classification accuracy (CA) and the averageF-measure. CA is defined as the percentage of correctly classified instances over the entire set of instances classified. In our dataset, as described in Table 2, for each data label, the population of both classes (which we denote by {C1,C2}) is not uniformly distributed. Hence, evaluating the performance of our classifiers using simply the classification accuracy does not paint an accurate picture of the discriminative power of the classifier. Since the dataset distribution is skewed, we can achieve a high accuracy but very poor performance in discriminating between both classes. Thus, we judge our classifiers using the average F-measure obtained for both binary classes. The F-measure combines precision and recall. It measures how well an algorithm can predict an instance belonging to a particular class. Let TP represent true positive, which we define as the number of instances that are correctly classified as C1 for a given test set, while TN denotes true negative, the equivalent for C2 instances. Let FP represent false positive, which we define as the number of instances that are incorrectly classified as C1 for a given test set, while FN denotes false negative, the equivalent for C2 instances.  Precision=TP/(TP+FP) and Recall=TP/(TP+FN). Thus, the F-measure is defined as(15)F-measure=2⋅Recall⋅PrecisionRecall+Precision.For both binary classes{C1,C2}, let (|C1|,|C2|) denote the total number of instances belonging to class C1 and C2,respectively, then the average F-measure is defined as(16)F-measure̅=|C1|⋅F-measure(C1)+|C2|⋅F-measure(C2)|C1|+|C2|.In all the tables illustrating the different experimental results, we highlight the bestF-measure̅ obtained along with the corresponding classification accuracy of the classifier.


### 4.2. Classification Results Based on Tongue Coating and ZHENG for Gastritis Patients

The experimental results presented in this section analyze the discrimination among the gastritis patients based on their tongue coating color and ZHENG category. Table3 summarizes the results obtained using our proposed color-space feature vector to train the classifiers to automatically classify the color of the coating of a gastritis patient’s tongue as yellow or white. We can observe from Table 3 that the combination of the median and standard deviation feature values ({medF⃗,σF⃗}) yields the best result for both the entire tongue region and the middle tongue region only. The results for both regions are also very comparable.Table 3
Tongue coating color classification: yellow versus white for gastritis patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.68169.160.75776.640.75276.170.76177.570.79680.840.77378.04{μF⃗,σF⃗}0.74374.770.79279.440.77477.570.76476.640.79980.370.76777.10medF⃗0.75876.640.72874.300.72472.900.73574.770.78979.440.76677.10{medF⃗,σF⃗}0.76376.640.80180.370.76777.100.78178.500.77577.100.81181.31σF⃗0.74775.700.79779.910.78378.500.74774.770.77777.570.78378.97When using the entire tongue region, the top three significant features for the color coating classification, ranked by the information gain attribute, were{σF9,medF12,σF2}, which denote the standard deviation of Q chroma (YIQ model), the median of Cr component (YCbCr), and the standard deviation of Green Channel (RGB), respectively. For the middle tongue region only, the top three were {σF9,σF20,medF4} which denote the standard deviation of Q chroma (YIQ model), the standard deviation of u component (L*u*v*), and the median of the Hue (HSV). It is also interesting to observe that out of the top ten significant features using the entire region versus the middle tongue region, they both have six of those features in common.The result obtained on ZHENG classification between the Hot and Cold groups is shown in Table4. For the ZHENG classification, using the standard deviation feature values (σF⃗) performs best when dealing with the entire tongue region while the {medF⃗,σF⃗} feature vector is the top performer for the middle tongue region only.Table 4
ZHENG classification between Hot and Cold syndromes for gastritis patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.61863.500.71671.500.71071.000.62263.500.71070.500.66367.00{μF⃗,σF⃗}0.75075.000.68067.500.72372.000.66468.000.73573.500.74074.00medF⃗0.64765.500.64964.500.67668.000.68471.000.66167.000.69069.00{medF⃗,σF⃗}0.73874.500.66566.000.72672.500.68570.000.70872.000.76176.00σF⃗0.76376.500.70971.000.70971.000.67669.000.70470.000.71972.00For ZHENG classification between Hot and Coldsyndromes for gastritis patients, when using the entire tongue region, only one feature was considered significant by the information gain attribute: σF9, that is, which is the standard deviation of Q chroma (YIQ model). For the middle tongue region, the most important feature is σF20, the standard deviation of u component (L*u*v*). Even though the noteworthy feature in the entire tongue area and the middle tongue area is not the same, both Q components in YIQ color space and u component in L*u*v* color space show the difference from green to red in chromaticity diagram.Table5 summarizes the results obtained when we train different classifiers to detect the presence of the HP bacteria in a gastritis patient using the color feature vector. The classification result obtained in learning the pathology groups of the patients (superficial versus atrophic) is shown in Table 6. Both cases are not very strong, which illustrates a weak correlation between the western medicine diagnosis and the tongue information utilized by Chinese medical practitioners. No feature was identified as significant in either case.Table 5
Detection of presence of HP bacteria (positive versus negative) in gastritis patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.67971.970.68168.200.67368.200.69671.970.68670.290.68270.29{μF⃗,σF⃗}0.64466.110.68067.780.71371.970.63264.850.68168.200.68167.78medF⃗0.65567.780.66667.360.66667.780.69971.550.64469.040.67668.20{medF⃗,σF⃗}0.65567.780.68668.200.69569.870.63365.270.63164.440.68468.20σF⃗0.66168.200.69571.130.70270.290.59461.090.66966.950.64965.27Table 6
Classification between superficial and atrophic pathology of the gastritis patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.60463.160.64264.470.62763.160.65866.670.63163.160.62262.72{μF⃗,σF⃗}0.63365.350.66265.790.70271.050.60461.400.63063.600.62162.28medF⃗0.63364.470.60162.720.64064.040.62365.790.63263.160.62362.28{medF⃗,σF⃗}0.65766.230.66065.790.69769.740.61362.720.64564.470.66366.23σF⃗0.63764.910.69770.180.65966.230.63164.040.62963.160.63964.47Tables7–10 illustrate how experimental results reflect the analysis of the classification between two pathology types of gastritis patients according to ZHENG category. Table 7 summarizes the results obtained using our proposed color-space feature vector to train the classifiers to automatically classify between Superficial group and Atrophic group for patients labeled as Cold ZHENG. The results obtained on classification between superficial group and atrophic group for Hot ZHENG patients is shown in Table 8. We can observe from Table 7 that the σF⃗ feature vector performed best for the entire tongue region while the {medF⃗,σF⃗} feature vector yielded the best result for the middle tongue region.Table 7
Tongue Classification between superficial and atrophic in Cold syndrome patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.57958.330.65866.670.63363.330.65165.000.63965.000.63363.33{μF⃗,σF⃗}0.71671.670.64765.000.68068.330.64365.000.64965.000.66266.67medF⃗0.60060.000.71471.670.73373.330.63363.330.61366.670.63363.33{medF⃗,σF⃗}0.71771.670.69870.000.70070.000.68468.330.59860.000.66766.67σF⃗0.70170.000.76176.670.74575.000.57958.330.59860.000.60160.00Table 8
Tongue classification between superficial and atrophic in Hot syndrome patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.76877.060.75575.230.73573.390.71071.560.73576.150.68067.89{μF⃗,σF⃗}0.74174.310.84584.400.76476.150.68068.810.77777.060.78077.98medF⃗0.71872.480.70872.480.71871.560.68668.810.70670.640.73673.39{medF⃗,σF⃗}0.71571.560.81781.650.81581.650.67267.890.77477.060.80880.73σF⃗0.77077.060.81881.650.81781.650.67567.890.79278.900.78177.98Table 9
Tongue classification between Hot syndrome and Cold syndrome in superficial patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.58359.680.77377.420.70570.970.70570.970.77377.420.72672.58{μF⃗,σF⃗}0.74074.190.83983.870.76577.420.69069.350.83983.870.75775.81medF⃗0.62862.900.74074.190.74374.190.67567.740.71070.970.65866.13{medF⃗,σF⃗}0.77477.420.83983.870.75575.810.77477.420.83983.870.77477.42σF⃗0.83483.870.75775.810.83883.870.81982.260.79179.030.75075.81Table 10
Tongue Classification between Hot syndrome and Cold syndrome in atrophic patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.53955.140.64263.550.64564.490.57258.880.76275.700.61561.68{μF⃗,σF⃗}0.66267.290.68169.160.69870.090.63864.490.70269.160.68568.22medF⃗0.61261.680.64663.550.66666.360.61162.620.60662.620.63864.49{medF⃗,σF⃗}0.70471.030.65764.490.67768.220.60460.750.70169.160.70370.09σF⃗0.69670.090.69168.220.73473.830.65064.490.67566.360.64563.55Similarly, from Table8 we can observe that for the Hot ZHENG patients, for the middle tongue region, the {medF⃗,σF⃗} feature vector also performed best. However, {μF⃗,σF⃗} feature vector performs best when dealing with the entire tongue region.When using the entire tongue region, the top three significant features for the pathology classification between superficial and atrophic in Cold ZHENG, ranked by the information gain attribute, were{σF9,σF6,σF1} which denote the standard deviation of Q chroma (YIQ model), the standard deviation of value component (HSV), and the standard deviation of Red Channel (RGB), respectively.In Table8, when using the entire tongue region, the top three significant features for the pathology classification between superficial and atrophic in Hot syndrome, ranked by the information gain attribute, were {μF22,μF25,μF3} which denote the mean of Cyan Ink (CMYK model), the mean of Black Ink (CMYK model), and the mean of Blue Channel (RGB), respectively. For the middle tongue region only, the top three were {σF22,σF25,medF25}, which denote the standard deviation of Cyan Ink (CMYK model), the standard deviation of Black Ink (CMYK model), and the median of Black Ink (CMYK model).The next set of experimental results focus on training our classifier using our proposed color-space feature vector to discriminate Hot ZHENG from Cold ZHENG in each pathology group. Table9 summarizes the results obtained to train the classifiers to automatically classify between Hot and Cold ZHENG for superficial gastritis patients. Table 10 reflects the results for gastritis patients. We can observe from Table 9 that both {μF⃗,σF⃗} and {medF⃗,σF⃗} feature vectors perform the best for both the entire tongue region and the middle tongue region. From results in Table 10, using the standard deviation feature values ({μF⃗,σF⃗}) performs best when dealing with the entire tongue region while the ({μF⃗,σF⃗}) feature vector is the top performer for the middle tongue region.When using the entire tongue region, the top three significant features for the ZHENG classification between Hot syndrome and Cold syndrome in the patients who are superficial, ranked by the information gain attribute, were{σF9,medF3,medF18}, which denote the standard deviation of Q chroma (YIQ model), the median of Blue Channel (RGB), and the median of the blue sensitivity Z component, respectively. For the middle tongue region only, the top three were med F24,σF19, and med F5 which denote the median of Yellow Ink (CMYK), the standard deviation of lightness component (Luv model), and the median of saturation (HSV). It is also interesting to observe that by comparing the set of the top five significant features using the entire region versus the set from the middle tongue region, they both have the Yellow Ink (CMYK) in common.When using the entire tongue region, there is only one significant feature difference for the ZHENG classification betweenHot syndrome and Cold syndrome in patients who are atrophic, ranked by the information gain attribute, σF9 which denotes the standard deviation of Q chroma (YIQ model). For the middle tongue region only, there were two significant features: {μF19,μF3} which denote the mean of the blue sensitivity Z component (XYZ) and the mean of the Blue Channel (RGB).


### 4.3. Classification Results for Gastritis Patients versus Control Group

The experimental results presented in this section analyze the discrimination between the gastritis patients and control group. Table11 summarizes the results obtained using our proposed color-space feature vector to train the classifiers to automatically classify patients with coating on tongue versus healthy patients with normal tongue (without coating). We can observe from Table 11 that the {medF⃗,σF⃗} feature vector yields the best result for the entire tongue region while for the middle tongue region, it was the σF⃗ feature vector.Table 11
Classification between normal tongue and tongue with coating.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.80382.820.83182.440.79580.530.77178.630.77477.480.76475.95{μF⃗,σF⃗}0.82983.590.85185.110.84885.500.81281.680.81481.680.81682.44medF⃗0.78580.530.80383.210.81483.210.77680.530.79178.630.78479.39{medF⃗,σF⃗}0.81483.210.83583.590.86186.260.81783.590.82382.060.82482.44σF⃗0.81883.210.83983.590.85185.110.83784.730.78679.390.81882.44When using the entire tongue region, the top three significant features for distinguishing between normal tongue and tongue with coating, ranked by the information gain attribute, were{σF1,σF6,σF25} which denote the standard deviation of Red Channel (RGB), the standard deviation of value component (HSV), and the standard deviation of Black Ink (CMYK) respectively. For the middle tongue region only, there were only two significant features: {σF13,σF14} which denote the standard deviation of lightness component (L*a*b) and the standard deviation of a* component (L*a*b*). It is also interesting to observe that by comparing the set of the top 10 significant features using the entire region versus the set from the middle tongue region, they both have the lightness and a* component (L*a*b*) in common.The results obtained from the classification between the normal group and the entire set of patients with ZHENG syndrome is shown in Table12. The {μF⃗,σF⃗} feature vector performs best when dealing with the entire tongue region while the {medF⃗,σF⃗} feature vector is the top performer for the middle tongue region.Table 12
Tongue classification between normal group and ZHENG gastritis group.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.76578.630.80980.240.78478.630.78179.440.77076.610.76276.61{μF⃗,σF⃗}0.83684.680.85284.680.85785.890.82082.660.79880.650.82682.26medF⃗0.75677.820.79581.450.78478.630.77278.230.81781.450.78578.63{medF⃗,σF⃗}0.80281.450.84584.270.84484.680.77979.440.83783.470.86987.10σF⃗0.82683.470.84984.680.84384.270.79981.050.78077.020.83383.87When using the entire tongue region, the top three significant features for the classification between the normal group and the gastritis group, ranked by the information gain attribute, were{σF1,σF6,σF25} which denote the standard deviation of Red Channel (RGB), the standard deviation of value component (HSV), and the standard deviation of Black Ink (CMYK) respectively. For the middle tongue region only, the top three were: {medF1,medF6,σF13} which denote the median of Red Channel (RGB), the median of Value component (HSV), and the standard deviation of lightness component (L*a*b*).Tables13 and 14 show the results of training our classifiers to discriminate between the normal group and the Hot ZHENG patients only, and then normal group versus Cold ZHENG patients only. Table 13 illustrates the results for normal versus hot ZHENG. We can observe that the σF⃗ feature vector performs best both for the entire tongue region and the middle tongue region. From Table 14, when only the normal versus Cold ZHENG patients is considered, the same feature vector, {μF⃗,σF⃗}, performs best for both cases, however, considering only the middle tongue region outperforms using the entire tongue region.Table 13
Tongue classification between normal group and Hot ZHENG.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.67170.000.78177.780.70872.220.74175.000.77377.220.75576.11{μF⃗,σF⃗}0.80480.560.79279.440.81681.670.78078.890.76477.220.79979.44medF⃗0.72172.780.71172.220.73975.000.72773.890.73973.330.74474.44{medF⃗,σF⃗}0.79680.000.81482.780.79780.000.78179.440.75275.000.79879.44σF⃗0.76877.220.82882.220.82682.780.73675.000.76677.220.80580.56Table 14
Tongue classification between normal group and Cold ZHENG.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.69068.970.75975.860.67668.100.71471.550.74174.140.73173.28{μF⃗,σF⃗}0.74274.140.78578.450.74875.000.82682.760.75975.860.75075.00medF⃗0.68668.970.74575.000.75775.860.67267.240.75075.000.74274.14{medF⃗,σF⃗}0.75975.860.77477.590.73473.280.76876.720.73373.280.81181.03σF⃗0.74174.140.73373.280.73473.280.67968.100.72372.410.70870.69When using the entire tongue region, the top three significant features for the classification between the normal group and the gastritispatients with Hot syndrome, ranked by the information gain attribute, were {σF1,σF6,σF25} which denote the standard deviation of Red Channel (RGB), the standard deviation of value component (HSV), and the standard deviation of Black Ink (CMYK), respectively. For the middle tongue region only, there were only two significant features: {σF13,σF14} which denote the standard deviation of lightness component (L*a*b) and the standard deviation of a* component (L*a*b*).  When the set of the top ten significant features using the entire region versus the set from the middle tongue region are compared, they both have the lightness and a* component (L*a*b*) in common.When using the entire tongue region, the top three significant features for the classification between the normal group and the gastritispatients with Cold syndrome, ranked by the information gain attribute, were {σF25,σF22,σF1} which denotethe standard deviation of Black Ink (CMYK), the standard deviation of Cyan Ink (CMYK), and the standard deviation of Red Channel (RGB), respectively. For the middle tongue region only, the top three were {σF13,μF22,σF14} which denote the standard deviation of lightness component (L*a*b), the mean of Cyan Ink (CMYK), and the standard deviation of a* component (L*a*b*).Table15 show the results of training our classifiers to discriminate between the normal group and the superficial patients while Table 16 shows the result for normal group versus the atrophic patients. When using the entire tongue region, the top three significant features for the classification between the normal group and the superficial group, ranked by the information gain attribute, were {σF1,σF6,σF25} which denote the standard deviation of Red Channel (RGB), the standard deviation of value component (HSV), and the standard deviation of Black Ink (CMYK), respectively. For the middle tongue region, the top three were {medF9,medF1,medF6} which denote the median of Q chromatic component (YIQ), the median of Red Channel (RGB), and the median of Value component (HSV).Table 15
Tongue classification between normal group and superficial patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.65565.910.73774.240.75475.760.69469.700.68768.180.70470.45{μF⃗,σF⃗}0.67968.180.75175.000.77477.270.74975.000.74474.240.71971.97medF⃗0.67567.420.73774.240.73773.480.73373.480.67767.420.73973.48{medF⃗,σF⃗}0.69570.450.75975.760.81181.060.74975.000.76275.760.72672.73σF⃗0.68768.940.73574.240.70670.450.72672.730.74274.240.74975.00Table 16
Tongue classification between normal group and atrophic patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.73375.520.80380.210.78179.170.75477.080.77078.130.69970.83{μF⃗,σF⃗}0.73673.960.77278.130.83783.850.79880.730.78278.650.80280.21medF⃗0.72673.960.75477.080.75175.520.72675.520.74974.480.75375.52{medF⃗,σF⃗}0.73874.480.81682.290.81881.770.75175.520.79278.650.84884.90σF⃗0.76177.080.78779.690.79980.210.77278.130.79880.210.79179.69When using the entire tongue region, the top three significant features for the classification between the normal group and the atrophic group, ranked by the information gain attribute, were{μF25,μF22,μF1} which denote the mean of Black Ink (CMYK model), the mean of Cyan Ink (CMYK model), and the mean of Red Channel (RGB), respectively. For the middle tongue region, the top three were {medF16,σF13,σF23} which denote the median of red sensitivity X component (XYZ), the standard deviation of lightness (L*a*b*), and the standard deviation of Cyan Ink (CMYK).


### 4.4. Analysis of Classification Results

From the experimental results presented in Sections4.2 and 4.3, we can draw the following conclusions. Firstly, concerning the performance of the different classification models, we observe that the MLP and SVM models usually outperformed the AdaBoost model. The multilayer perceptron neural network seems most adequate for learning the complex relationships between the color features of the tongue images and the ZHENG/coating classes. However, both the MLP and SVM models have many parameters to consider and optimize while the AdaBoost is a much simpler model. In the AdaBoost model, we use a decision tree as our base weak learner and vary the number of classifiers to optimize its performance.Secondly, we observe that when making discriminations within the gastritis patients group (hot versus cold ZHENG, yellow versus white coating, etc.), it was more profitable to apply the feature vectors on the entire tongue image. When classifying the normal groups versus the ZHENG groupings, usually, it improved classifier performance to apply the feature vectors to the middle tongue regions only.Thirdly, we also observe that from the evaluation of the variations of the feature vectors used, taking into account both the average and the standard deviation usually resulted in an excellent performance. It seemed like the mean outperformed the median slightly, overall, that is,{μF⃗,σF⃗}. In a few cases, simply considering variation of the spread of the values over the region ({σF⃗}) yielded the best performance. Thus, we can conclude that when deriving a feature vector for the tongue image, the mean (or median) as well as the standard deviation (which takes into account the variation of the spread on the region) is very important.Lastly, we observe that though we were not able to effectively discriminate between the pathology groups (superficial versus atrophic and also the presence of the HP bacterium using our color-space feature vectors, we were able to classify them much better when we took into account the ZHENG classes. This further strengthens the notion that our proposed color-space feature vectors are able to discriminate between the hot and cold ZHENG patients in addition to discerning a ZHENG patient from a non-ZHENG (healthy) patient.


### 4.5. Applying Feature Selection Algorithm

The classification results presented in Sections4.2 and 4.3 were obtained using the whole feature set. For each experiment carried out on the entire tongue region, we also applied information gain attribute evaluation to rank the significance of the features. In this section, we apply feature selection algorithm (Best First) to select only a subset of features, which are deemed significant, before training the classifiers. Our goal is to see if this would yield a better result than using the whole feature set. The Best First algorithm searches the space of attribute subsets by greedy hill climbing augmented with a backtracking facility.The summary of the results obtained is shown in Table17. The normal group refers to the healthy (non-ZHENG) control group. We present the best classification result obtained for each experiment based on using the five variations of the feature vectors (μF⃗, medF⃗, σF⃗, {μF⃗,σF⃗}, {medF⃗,σF⃗}) and the three different classification models (Adaboost, SVM, and MLP). As we can observe from Table 17, using the whole feature set to train the classifiers yielded a better result in all cases except for the Atrophic Patients (Hot versus Cold ZHENG) experiment. Thus, we can conclude the overall, using the aggregate of the proposed feature sets is more discriminative even though some features are more significant than others.Table 17
Comparison between using selected features versus Whole feature set for classification.
Classification experiment typeFeature selectionWhole featureF-measureAccuracyF-measureAccuracyCoating (yellow versus white)0.76477.10%0.80180.37%ZHENG (Hot versus Cold)0.64265.00%0.76376.50%HP Bacteria (positive versus negative)0.63672.38%0.71371.97%Gastritis patients (superficial versus atrophic)0.65668.42%0.70271.05%Cold ZHENG patients (superficial versus atrophic)0.75075.00%0.76176.67%Hot ZHENG patients (superficial versus atrophic)0.77677.98%0.84584.40%Superficial Patients (Hot versus Cold ZHENG)0.80780.65%0.83983.87%Atrophic patients (Hot versus Cold ZHENG)0.78278.50%0.73473.83%Normal tongue versus tongue with coating0.83385.88%0.86186.26%Normal group versus ZHENG patients0.83484.68%0.85785.89%Normal group versus Hot ZHENG0.80881.11%0.82882.22%Normal group versus Cold ZHENG0.75075.00%0.78578.45%Normal group versus superficial patients0.76576.52%0.81181.06%Normal group versus atrophic patients0.76278.13%0.83783.85%


## 4.1. Experimental Setup

In this section, we evaluated the performance of our proposed ZHENG classification system using the three classification models (AdaBoost, SVM, and MLP) described in Section3.2. We compared the performance of training the classifier models using the set of features extracted from the entire tongue image versus the middle tongue region only. As mentioned in Section 2, in TCM, it is believed that the middle tongue region provides discriminant information for diagnosing stomach disorders. Hence, we extract features from the middle tongue region, as described in Figure1(b), to evaluate the performance compared to extracting features from the entire tongue region. In training and testing our classification models, we employ a 3-fold cross-validation strategy.  This implies that the data is split into three sets; one set is used for testing and the remaining two sets are used for training. The experiment is repeated with each of the three sets used for testing. The average accuracy of the tests over the three sets is taken as the performance measure. For each classification model, we varied the parameters to optimize its performance. We also compare the results obtained using the five different variations of the feature vector (mean = μF⃗, median =medF⃗, standard deviation = σF⃗, mean + standard deviation = {μF⃗,σF⃗}, and median + standard deviation = {medF⃗,σF⃗}), as described in Section 3.1. We also apply Information Gain attribute evaluation on the feature vectors to quantify and rank the significance of  individual features. Lastly, we apply the Best First feature selection algorithm to select the “significant” features before training the classifiers to compare the performance of training the classifiers with the whole feature set against selected features.The performance metrics used are the classification accuracy (CA) and the averageF-measure. CA is defined as the percentage of correctly classified instances over the entire set of instances classified. In our dataset, as described in Table 2, for each data label, the population of both classes (which we denote by {C1,C2}) is not uniformly distributed. Hence, evaluating the performance of our classifiers using simply the classification accuracy does not paint an accurate picture of the discriminative power of the classifier. Since the dataset distribution is skewed, we can achieve a high accuracy but very poor performance in discriminating between both classes. Thus, we judge our classifiers using the average F-measure obtained for both binary classes. The F-measure combines precision and recall. It measures how well an algorithm can predict an instance belonging to a particular class. Let TP represent true positive, which we define as the number of instances that are correctly classified as C1 for a given test set, while TN denotes true negative, the equivalent for C2 instances. Let FP represent false positive, which we define as the number of instances that are incorrectly classified as C1 for a given test set, while FN denotes false negative, the equivalent for C2 instances.  Precision=TP/(TP+FP) and Recall=TP/(TP+FN). Thus, the F-measure is defined as(15)F-measure=2⋅Recall⋅PrecisionRecall+Precision.For both binary classes{C1,C2}, let (|C1|,|C2|) denote the total number of instances belonging to class C1 and C2,respectively, then the average F-measure is defined as(16)F-measure̅=|C1|⋅F-measure(C1)+|C2|⋅F-measure(C2)|C1|+|C2|.In all the tables illustrating the different experimental results, we highlight the bestF-measure̅ obtained along with the corresponding classification accuracy of the classifier.


## 4.2. Classification Results Based on Tongue Coating and ZHENG for Gastritis Patients

The experimental results presented in this section analyze the discrimination among the gastritis patients based on their tongue coating color and ZHENG category. Table3 summarizes the results obtained using our proposed color-space feature vector to train the classifiers to automatically classify the color of the coating of a gastritis patient’s tongue as yellow or white. We can observe from Table 3 that the combination of the median and standard deviation feature values ({medF⃗,σF⃗}) yields the best result for both the entire tongue region and the middle tongue region only. The results for both regions are also very comparable.Table 3
Tongue coating color classification: yellow versus white for gastritis patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.68169.160.75776.640.75276.170.76177.570.79680.840.77378.04{μF⃗,σF⃗}0.74374.770.79279.440.77477.570.76476.640.79980.370.76777.10medF⃗0.75876.640.72874.300.72472.900.73574.770.78979.440.76677.10{medF⃗,σF⃗}0.76376.640.80180.370.76777.100.78178.500.77577.100.81181.31σF⃗0.74775.700.79779.910.78378.500.74774.770.77777.570.78378.97When using the entire tongue region, the top three significant features for the color coating classification, ranked by the information gain attribute, were{σF9,medF12,σF2}, which denote the standard deviation of Q chroma (YIQ model), the median of Cr component (YCbCr), and the standard deviation of Green Channel (RGB), respectively. For the middle tongue region only, the top three were {σF9,σF20,medF4} which denote the standard deviation of Q chroma (YIQ model), the standard deviation of u component (L*u*v*), and the median of the Hue (HSV). It is also interesting to observe that out of the top ten significant features using the entire region versus the middle tongue region, they both have six of those features in common.The result obtained on ZHENG classification between the Hot and Cold groups is shown in Table4. For the ZHENG classification, using the standard deviation feature values (σF⃗) performs best when dealing with the entire tongue region while the {medF⃗,σF⃗} feature vector is the top performer for the middle tongue region only.Table 4
ZHENG classification between Hot and Cold syndromes for gastritis patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.61863.500.71671.500.71071.000.62263.500.71070.500.66367.00{μF⃗,σF⃗}0.75075.000.68067.500.72372.000.66468.000.73573.500.74074.00medF⃗0.64765.500.64964.500.67668.000.68471.000.66167.000.69069.00{medF⃗,σF⃗}0.73874.500.66566.000.72672.500.68570.000.70872.000.76176.00σF⃗0.76376.500.70971.000.70971.000.67669.000.70470.000.71972.00For ZHENG classification between Hot and Coldsyndromes for gastritis patients, when using the entire tongue region, only one feature was considered significant by the information gain attribute: σF9, that is, which is the standard deviation of Q chroma (YIQ model). For the middle tongue region, the most important feature is σF20, the standard deviation of u component (L*u*v*). Even though the noteworthy feature in the entire tongue area and the middle tongue area is not the same, both Q components in YIQ color space and u component in L*u*v* color space show the difference from green to red in chromaticity diagram.Table5 summarizes the results obtained when we train different classifiers to detect the presence of the HP bacteria in a gastritis patient using the color feature vector. The classification result obtained in learning the pathology groups of the patients (superficial versus atrophic) is shown in Table 6. Both cases are not very strong, which illustrates a weak correlation between the western medicine diagnosis and the tongue information utilized by Chinese medical practitioners. No feature was identified as significant in either case.Table 5
Detection of presence of HP bacteria (positive versus negative) in gastritis patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.67971.970.68168.200.67368.200.69671.970.68670.290.68270.29{μF⃗,σF⃗}0.64466.110.68067.780.71371.970.63264.850.68168.200.68167.78medF⃗0.65567.780.66667.360.66667.780.69971.550.64469.040.67668.20{medF⃗,σF⃗}0.65567.780.68668.200.69569.870.63365.270.63164.440.68468.20σF⃗0.66168.200.69571.130.70270.290.59461.090.66966.950.64965.27Table 6
Classification between superficial and atrophic pathology of the gastritis patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.60463.160.64264.470.62763.160.65866.670.63163.160.62262.72{μF⃗,σF⃗}0.63365.350.66265.790.70271.050.60461.400.63063.600.62162.28medF⃗0.63364.470.60162.720.64064.040.62365.790.63263.160.62362.28{medF⃗,σF⃗}0.65766.230.66065.790.69769.740.61362.720.64564.470.66366.23σF⃗0.63764.910.69770.180.65966.230.63164.040.62963.160.63964.47Tables7–10 illustrate how experimental results reflect the analysis of the classification between two pathology types of gastritis patients according to ZHENG category. Table 7 summarizes the results obtained using our proposed color-space feature vector to train the classifiers to automatically classify between Superficial group and Atrophic group for patients labeled as Cold ZHENG. The results obtained on classification between superficial group and atrophic group for Hot ZHENG patients is shown in Table 8. We can observe from Table 7 that the σF⃗ feature vector performed best for the entire tongue region while the {medF⃗,σF⃗} feature vector yielded the best result for the middle tongue region.Table 7
Tongue Classification between superficial and atrophic in Cold syndrome patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.57958.330.65866.670.63363.330.65165.000.63965.000.63363.33{μF⃗,σF⃗}0.71671.670.64765.000.68068.330.64365.000.64965.000.66266.67medF⃗0.60060.000.71471.670.73373.330.63363.330.61366.670.63363.33{medF⃗,σF⃗}0.71771.670.69870.000.70070.000.68468.330.59860.000.66766.67σF⃗0.70170.000.76176.670.74575.000.57958.330.59860.000.60160.00Table 8
Tongue classification between superficial and atrophic in Hot syndrome patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.76877.060.75575.230.73573.390.71071.560.73576.150.68067.89{μF⃗,σF⃗}0.74174.310.84584.400.76476.150.68068.810.77777.060.78077.98medF⃗0.71872.480.70872.480.71871.560.68668.810.70670.640.73673.39{medF⃗,σF⃗}0.71571.560.81781.650.81581.650.67267.890.77477.060.80880.73σF⃗0.77077.060.81881.650.81781.650.67567.890.79278.900.78177.98Table 9
Tongue classification between Hot syndrome and Cold syndrome in superficial patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.58359.680.77377.420.70570.970.70570.970.77377.420.72672.58{μF⃗,σF⃗}0.74074.190.83983.870.76577.420.69069.350.83983.870.75775.81medF⃗0.62862.900.74074.190.74374.190.67567.740.71070.970.65866.13{medF⃗,σF⃗}0.77477.420.83983.870.75575.810.77477.420.83983.870.77477.42σF⃗0.83483.870.75775.810.83883.870.81982.260.79179.030.75075.81Table 10
Tongue Classification between Hot syndrome and Cold syndrome in atrophic patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.53955.140.64263.550.64564.490.57258.880.76275.700.61561.68{μF⃗,σF⃗}0.66267.290.68169.160.69870.090.63864.490.70269.160.68568.22medF⃗0.61261.680.64663.550.66666.360.61162.620.60662.620.63864.49{medF⃗,σF⃗}0.70471.030.65764.490.67768.220.60460.750.70169.160.70370.09σF⃗0.69670.090.69168.220.73473.830.65064.490.67566.360.64563.55Similarly, from Table8 we can observe that for the Hot ZHENG patients, for the middle tongue region, the {medF⃗,σF⃗} feature vector also performed best. However, {μF⃗,σF⃗} feature vector performs best when dealing with the entire tongue region.When using the entire tongue region, the top three significant features for the pathology classification between superficial and atrophic in Cold ZHENG, ranked by the information gain attribute, were{σF9,σF6,σF1} which denote the standard deviation of Q chroma (YIQ model), the standard deviation of value component (HSV), and the standard deviation of Red Channel (RGB), respectively.In Table8, when using the entire tongue region, the top three significant features for the pathology classification between superficial and atrophic in Hot syndrome, ranked by the information gain attribute, were {μF22,μF25,μF3} which denote the mean of Cyan Ink (CMYK model), the mean of Black Ink (CMYK model), and the mean of Blue Channel (RGB), respectively. For the middle tongue region only, the top three were {σF22,σF25,medF25}, which denote the standard deviation of Cyan Ink (CMYK model), the standard deviation of Black Ink (CMYK model), and the median of Black Ink (CMYK model).The next set of experimental results focus on training our classifier using our proposed color-space feature vector to discriminate Hot ZHENG from Cold ZHENG in each pathology group. Table9 summarizes the results obtained to train the classifiers to automatically classify between Hot and Cold ZHENG for superficial gastritis patients. Table 10 reflects the results for gastritis patients. We can observe from Table 9 that both {μF⃗,σF⃗} and {medF⃗,σF⃗} feature vectors perform the best for both the entire tongue region and the middle tongue region. From results in Table 10, using the standard deviation feature values ({μF⃗,σF⃗}) performs best when dealing with the entire tongue region while the ({μF⃗,σF⃗}) feature vector is the top performer for the middle tongue region.When using the entire tongue region, the top three significant features for the ZHENG classification between Hot syndrome and Cold syndrome in the patients who are superficial, ranked by the information gain attribute, were{σF9,medF3,medF18}, which denote the standard deviation of Q chroma (YIQ model), the median of Blue Channel (RGB), and the median of the blue sensitivity Z component, respectively. For the middle tongue region only, the top three were med F24,σF19, and med F5 which denote the median of Yellow Ink (CMYK), the standard deviation of lightness component (Luv model), and the median of saturation (HSV). It is also interesting to observe that by comparing the set of the top five significant features using the entire region versus the set from the middle tongue region, they both have the Yellow Ink (CMYK) in common.When using the entire tongue region, there is only one significant feature difference for the ZHENG classification betweenHot syndrome and Cold syndrome in patients who are atrophic, ranked by the information gain attribute, σF9 which denotes the standard deviation of Q chroma (YIQ model). For the middle tongue region only, there were two significant features: {μF19,μF3} which denote the mean of the blue sensitivity Z component (XYZ) and the mean of the Blue Channel (RGB).


## 4.3. Classification Results for Gastritis Patients versus Control Group

The experimental results presented in this section analyze the discrimination between the gastritis patients and control group. Table11 summarizes the results obtained using our proposed color-space feature vector to train the classifiers to automatically classify patients with coating on tongue versus healthy patients with normal tongue (without coating). We can observe from Table 11 that the {medF⃗,σF⃗} feature vector yields the best result for the entire tongue region while for the middle tongue region, it was the σF⃗ feature vector.Table 11
Classification between normal tongue and tongue with coating.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.80382.820.83182.440.79580.530.77178.630.77477.480.76475.95{μF⃗,σF⃗}0.82983.590.85185.110.84885.500.81281.680.81481.680.81682.44medF⃗0.78580.530.80383.210.81483.210.77680.530.79178.630.78479.39{medF⃗,σF⃗}0.81483.210.83583.590.86186.260.81783.590.82382.060.82482.44σF⃗0.81883.210.83983.590.85185.110.83784.730.78679.390.81882.44When using the entire tongue region, the top three significant features for distinguishing between normal tongue and tongue with coating, ranked by the information gain attribute, were{σF1,σF6,σF25} which denote the standard deviation of Red Channel (RGB), the standard deviation of value component (HSV), and the standard deviation of Black Ink (CMYK) respectively. For the middle tongue region only, there were only two significant features: {σF13,σF14} which denote the standard deviation of lightness component (L*a*b) and the standard deviation of a* component (L*a*b*). It is also interesting to observe that by comparing the set of the top 10 significant features using the entire region versus the set from the middle tongue region, they both have the lightness and a* component (L*a*b*) in common.The results obtained from the classification between the normal group and the entire set of patients with ZHENG syndrome is shown in Table12. The {μF⃗,σF⃗} feature vector performs best when dealing with the entire tongue region while the {medF⃗,σF⃗} feature vector is the top performer for the middle tongue region.Table 12
Tongue classification between normal group and ZHENG gastritis group.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.76578.630.80980.240.78478.630.78179.440.77076.610.76276.61{μF⃗,σF⃗}0.83684.680.85284.680.85785.890.82082.660.79880.650.82682.26medF⃗0.75677.820.79581.450.78478.630.77278.230.81781.450.78578.63{medF⃗,σF⃗}0.80281.450.84584.270.84484.680.77979.440.83783.470.86987.10σF⃗0.82683.470.84984.680.84384.270.79981.050.78077.020.83383.87When using the entire tongue region, the top three significant features for the classification between the normal group and the gastritis group, ranked by the information gain attribute, were{σF1,σF6,σF25} which denote the standard deviation of Red Channel (RGB), the standard deviation of value component (HSV), and the standard deviation of Black Ink (CMYK) respectively. For the middle tongue region only, the top three were: {medF1,medF6,σF13} which denote the median of Red Channel (RGB), the median of Value component (HSV), and the standard deviation of lightness component (L*a*b*).Tables13 and 14 show the results of training our classifiers to discriminate between the normal group and the Hot ZHENG patients only, and then normal group versus Cold ZHENG patients only. Table 13 illustrates the results for normal versus hot ZHENG. We can observe that the σF⃗ feature vector performs best both for the entire tongue region and the middle tongue region. From Table 14, when only the normal versus Cold ZHENG patients is considered, the same feature vector, {μF⃗,σF⃗}, performs best for both cases, however, considering only the middle tongue region outperforms using the entire tongue region.Table 13
Tongue classification between normal group and Hot ZHENG.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.67170.000.78177.780.70872.220.74175.000.77377.220.75576.11{μF⃗,σF⃗}0.80480.560.79279.440.81681.670.78078.890.76477.220.79979.44medF⃗0.72172.780.71172.220.73975.000.72773.890.73973.330.74474.44{medF⃗,σF⃗}0.79680.000.81482.780.79780.000.78179.440.75275.000.79879.44σF⃗0.76877.220.82882.220.82682.780.73675.000.76677.220.80580.56Table 14
Tongue classification between normal group and Cold ZHENG.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.69068.970.75975.860.67668.100.71471.550.74174.140.73173.28{μF⃗,σF⃗}0.74274.140.78578.450.74875.000.82682.760.75975.860.75075.00medF⃗0.68668.970.74575.000.75775.860.67267.240.75075.000.74274.14{medF⃗,σF⃗}0.75975.860.77477.590.73473.280.76876.720.73373.280.81181.03σF⃗0.74174.140.73373.280.73473.280.67968.100.72372.410.70870.69When using the entire tongue region, the top three significant features for the classification between the normal group and the gastritispatients with Hot syndrome, ranked by the information gain attribute, were {σF1,σF6,σF25} which denote the standard deviation of Red Channel (RGB), the standard deviation of value component (HSV), and the standard deviation of Black Ink (CMYK), respectively. For the middle tongue region only, there were only two significant features: {σF13,σF14} which denote the standard deviation of lightness component (L*a*b) and the standard deviation of a* component (L*a*b*).  When the set of the top ten significant features using the entire region versus the set from the middle tongue region are compared, they both have the lightness and a* component (L*a*b*) in common.When using the entire tongue region, the top three significant features for the classification between the normal group and the gastritispatients with Cold syndrome, ranked by the information gain attribute, were {σF25,σF22,σF1} which denotethe standard deviation of Black Ink (CMYK), the standard deviation of Cyan Ink (CMYK), and the standard deviation of Red Channel (RGB), respectively. For the middle tongue region only, the top three were {σF13,μF22,σF14} which denote the standard deviation of lightness component (L*a*b), the mean of Cyan Ink (CMYK), and the standard deviation of a* component (L*a*b*).Table15 show the results of training our classifiers to discriminate between the normal group and the superficial patients while Table 16 shows the result for normal group versus the atrophic patients. When using the entire tongue region, the top three significant features for the classification between the normal group and the superficial group, ranked by the information gain attribute, were {σF1,σF6,σF25} which denote the standard deviation of Red Channel (RGB), the standard deviation of value component (HSV), and the standard deviation of Black Ink (CMYK), respectively. For the middle tongue region, the top three were {medF9,medF1,medF6} which denote the median of Q chromatic component (YIQ), the median of Red Channel (RGB), and the median of Value component (HSV).Table 15
Tongue classification between normal group and superficial patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.65565.910.73774.240.75475.760.69469.700.68768.180.70470.45{μF⃗,σF⃗}0.67968.180.75175.000.77477.270.74975.000.74474.240.71971.97medF⃗0.67567.420.73774.240.73773.480.73373.480.67767.420.73973.48{medF⃗,σF⃗}0.69570.450.75975.760.81181.060.74975.000.76275.760.72672.73σF⃗0.68768.940.73574.240.70670.450.72672.730.74274.240.74975.00Table 16
Tongue classification between normal group and atrophic patients.
Feature vectorEntire tongueMiddle tongueAdaBoostSVMMLPAdaBoostSVMMLPF-measCAF-measCAF-measCAF-measCAF-measCAF-measCAμF⃗0.73375.520.80380.210.78179.170.75477.080.77078.130.69970.83{μF⃗,σF⃗}0.73673.960.77278.130.83783.850.79880.730.78278.650.80280.21medF⃗0.72673.960.75477.080.75175.520.72675.520.74974.480.75375.52{medF⃗,σF⃗}0.73874.480.81682.290.81881.770.75175.520.79278.650.84884.90σF⃗0.76177.080.78779.690.79980.210.77278.130.79880.210.79179.69When using the entire tongue region, the top three significant features for the classification between the normal group and the atrophic group, ranked by the information gain attribute, were{μF25,μF22,μF1} which denote the mean of Black Ink (CMYK model), the mean of Cyan Ink (CMYK model), and the mean of Red Channel (RGB), respectively. For the middle tongue region, the top three were {medF16,σF13,σF23} which denote the median of red sensitivity X component (XYZ), the standard deviation of lightness (L*a*b*), and the standard deviation of Cyan Ink (CMYK).


## 4.4. Analysis of Classification Results

From the experimental results presented in Sections4.2 and 4.3, we can draw the following conclusions. Firstly, concerning the performance of the different classification models, we observe that the MLP and SVM models usually outperformed the AdaBoost model. The multilayer perceptron neural network seems most adequate for learning the complex relationships between the color features of the tongue images and the ZHENG/coating classes. However, both the MLP and SVM models have many parameters to consider and optimize while the AdaBoost is a much simpler model. In the AdaBoost model, we use a decision tree as our base weak learner and vary the number of classifiers to optimize its performance.Secondly, we observe that when making discriminations within the gastritis patients group (hot versus cold ZHENG, yellow versus white coating, etc.), it was more profitable to apply the feature vectors on the entire tongue image. When classifying the normal groups versus the ZHENG groupings, usually, it improved classifier performance to apply the feature vectors to the middle tongue regions only.Thirdly, we also observe that from the evaluation of the variations of the feature vectors used, taking into account both the average and the standard deviation usually resulted in an excellent performance. It seemed like the mean outperformed the median slightly, overall, that is,{μF⃗,σF⃗}. In a few cases, simply considering variation of the spread of the values over the region ({σF⃗}) yielded the best performance. Thus, we can conclude that when deriving a feature vector for the tongue image, the mean (or median) as well as the standard deviation (which takes into account the variation of the spread on the region) is very important.Lastly, we observe that though we were not able to effectively discriminate between the pathology groups (superficial versus atrophic and also the presence of the HP bacterium using our color-space feature vectors, we were able to classify them much better when we took into account the ZHENG classes. This further strengthens the notion that our proposed color-space feature vectors are able to discriminate between the hot and cold ZHENG patients in addition to discerning a ZHENG patient from a non-ZHENG (healthy) patient.


## 4.5. Applying Feature Selection Algorithm

The classification results presented in Sections4.2 and 4.3 were obtained using the whole feature set. For each experiment carried out on the entire tongue region, we also applied information gain attribute evaluation to rank the significance of the features. In this section, we apply feature selection algorithm (Best First) to select only a subset of features, which are deemed significant, before training the classifiers. Our goal is to see if this would yield a better result than using the whole feature set. The Best First algorithm searches the space of attribute subsets by greedy hill climbing augmented with a backtracking facility.The summary of the results obtained is shown in Table17. The normal group refers to the healthy (non-ZHENG) control group. We present the best classification result obtained for each experiment based on using the five variations of the feature vectors (μF⃗, medF⃗, σF⃗, {μF⃗,σF⃗}, {medF⃗,σF⃗}) and the three different classification models (Adaboost, SVM, and MLP). As we can observe from Table 17, using the whole feature set to train the classifiers yielded a better result in all cases except for the Atrophic Patients (Hot versus Cold ZHENG) experiment. Thus, we can conclude the overall, using the aggregate of the proposed feature sets is more discriminative even though some features are more significant than others.Table 17
Comparison between using selected features versus Whole feature set for classification.
Classification experiment typeFeature selectionWhole featureF-measureAccuracyF-measureAccuracyCoating (yellow versus white)0.76477.10%0.80180.37%ZHENG (Hot versus Cold)0.64265.00%0.76376.50%HP Bacteria (positive versus negative)0.63672.38%0.71371.97%Gastritis patients (superficial versus atrophic)0.65668.42%0.70271.05%Cold ZHENG patients (superficial versus atrophic)0.75075.00%0.76176.67%Hot ZHENG patients (superficial versus atrophic)0.77677.98%0.84584.40%Superficial Patients (Hot versus Cold ZHENG)0.80780.65%0.83983.87%Atrophic patients (Hot versus Cold ZHENG)0.78278.50%0.73473.83%Normal tongue versus tongue with coating0.83385.88%0.86186.26%Normal group versus ZHENG patients0.83484.68%0.85785.89%Normal group versus Hot ZHENG0.80881.11%0.82882.22%Normal group versus Cold ZHENG0.75075.00%0.78578.45%Normal group versus superficial patients0.76576.52%0.81181.06%Normal group versus atrophic patients0.76278.13%0.83783.85%


## 5. Conclusion and Future Work

In this paper, we propose a novel color space-based feature set for use in the clinical characterization of ZHENG using various supervised machine-learning algorithms. Using an automated tongue-image diagnosis system, we extract these objective features from tongue images of clinical patients and analyze the relationship with their corresponding ZHENG data and disease prognosis (specifically gastritis) obtained from clinical practitioners. Given that TCM practitioners usually observe the tongue color and coating to determine ZHENG (such as Cold or Hot ZHENG) and to diagnose different stomach disorders including gastritis. We propose using machine-learning techniques to establish the relationship between the tongue image features and ZHENG by learning through examples.The experimental results obtained demonstrate an excellent performance of our proposed system.Our future work will focus on improving the performance of our system by exploring additional tongue image features that can be extracted to further strengthen our classification models. We plan to explore ways to improve our methodology to more accurately classify the ZHENGs such as including a preprocessing step of coating separation prior to the feature extraction phase. Lastly, we plan to evaluate the classification of the other ZHENG types mentioned in Section1.



---

*Source: 912852-2012-05-31.xml*