TongueDx: A tongue diagnosis system for personal health careon smartphones
Ini RYU
1(Adviser：Itiro SIIO)

INTRODUCTION

applied when a precise colored picture is taken. Oneof the white balancing methods commonly used is totake a photograph with a gray-colored card next tothe object, and adjust the white balance of the taken
picture so that the gray card appears gray (sameRGB values).
However, the gray card method is not acceptable
in a smartphone application that will be used in everyday situation, because the picture taking process
becomes burdensome and the gray card is not convenient to keep in daily life. In this study, we propose
to utilize user’s teeth, instead of gray card, to calibrate the white balance of tongue pictures. Thismethod is feasible because teeth color is not easily to
be changed in ordinary situations, and their color isclose to white.
To evaluate our method of using teeth color as acolor reference, we carry out an experiment by comparing the white-balancing accuracy between teeth
color and a gray card as calibration methods. According to the result, the handy calibration methodby teeth is proved to be feasible. In order to adjust white balance by teeth in a photograph, wehave developed following teeth recognition algorithm.
Firstly, our program crops the teeth part rectangle
from the picture. Secondly, it marks pixels withblue color when the brightness of the pixel is morethan the speciﬁc threshold value (b0 ). We use thebrightness formula (Brightness = 0.229R+0.587G+0.114B) here. It changes b0 from 50 to 200, until
more than two full horizontal runs of blue pixels arefound in the tooth part rectangle. If the horizontal runs of bright pixels are found, our program considers that teeth are detected successfully, and uses
these pixels for adjusting white balance. We asked 7
people for the teeth recognition test. Most of theirteeth area are successfully detected except when thesurrounding light is unusual or the picture is poorly
taken.

Recently health check applications based on medical health diagnosing technique running on smartphones have become popular. We discover thatTongue Diagnosis, which is one of the important
health diagnosing techniques in Traditional ChineseMedicine (TCM), is a promising medical technique
for these applications [1]. According to TCM, practitioners conclude one’s health condition by examining his tongue color, coating (normally white fur) andbody shapes as well as other features appeared on thetongue. For instance, when tongue looks sort of whiteand pasty, it could mean that your blood is lacking
hemoglobin; a red inﬂamed tongue indicates lack of
certain nutrients, especially iron and vitamins B. In
this research, we utilize the color of tongue coating
and body to exam and indicate health condition.

2TONGUEDX

We have developed a tongue diagnosis application
on smartphones named TongueDx (Fig. 1). On ourTongueDx system, we calibrate the color of tongue
images with respect to the teeth to resolve possible
inﬂuence caused by arbitrary lighting conditions. Kmeans algorithm is used to distinguish tongue coating from body; Daily-tongue-color-graph is made forusers to track their health condition by recording
tongue coating and body color on smartphones.

2.2

From acquired tongue images, our application tries
to identify two major parts of tongue coating andbody (Fig. 2), because healthy color for each part is
diﬀerently deﬁned. Coating and body parts can beseparated by their color, as they have distinctly different colors. Although it is common that coating isbrighter than body, there are exceptions dependingon user’s health. Thus, our system uses “user selection” for separated part to determine tongue coating
and body once for the ﬁrst time of use, and in thelater operation, it chooses the closest color as coating

Figure 1: Illustration of our system. Prior to the use,gray card and teeth picture is taken once (left), andthen, teeth and tongue pictures are taken everyday(right).
2.1

Tongue Color Extraction and Separation

Tongue Color Calibration by Teeth

Sometimes, the color of photograph is deviated because the surrounding light has various color spectrums, and it aﬀects the color of the photo. Color calibration to adjust the “white balance” is commonly1

or body automatically.
To make a distinction between tongue coating andbody, we use K- means algorithm. In fact, we foundthat some tongue pictures have teeth shadow in theupper part, and it can aﬀect the accuracy of tongue
color. Hence, our system cluster pixels into 4 (k = 4)groups to check the teeth shadow. In this clustering,
we expected to diﬀerentiate pixels into coating, body,teeth shadow and background (white color). Compare the pixels area of upper to the lower part, if thepixels area of upper part is larger than the lower partto a speciﬁc threshold value (set to 700), we cluster
pixels into 4 (k = 4) groups. Otherwise, 3 (k = 3)
groups clustering is expected to diﬀerentiate pixels
into coating, body and background (white color).

unstable. In her health condition diary, it referredthat her digestive system was in trouble in that day
because of overeating and over stress, and felt uncomfortable after that day. Therefore, we think ourtongue color graph could give users and doctors some
useful reference value in physical health inspection.
As further feasibility test, we are asking 20s collage
students to use the TongueDx application.

Figure 3: The tongue body (left) and coating (right)
color graph reﬂecting health condition.

4Xu et al. established an automatic recognitionsystem to detect tongue coating and body color.
They have identiﬁed coating and body area by their“splitting-merging” algorithm, which is utilized toseparate two parts of tongue as two continuous areas
by color similarity [2]. In this paper, we prefer to use
K-means algorithm to identify the two areas by color,because we expect optimal separation is achieved by
the established clustering method, and each area is
not alway continuous.

Figure 2: Separated tongue body and coating.
2.3

RELATED WORK

Personal Health Condition Graph

5

The daily-tongue-color-graph was designed forusers to take their health development timely. Users
can keep track of their health condition by recording
the color of tongue coating and body on smartphones(Fig. 3). When the line graph is smooth and thetongue body and coating color is near to the pink
and light white, respectively, it indicates the user ishealthy; if the line ﬂuctuates and the tongue color isfar from pink, it is possible that there is something
wrong with the user’s body, and it is better to ask
for more advice from experienced doctors.

CONCLUSIONSIn this paper, TongueDx - a tongue diagnosis application on smartphones - is developed for users todiagnose their own health condition by taking pictures of their tongue. We are planning to carry out
further user study, to improve the accuracy of tongue
diagnosis, and to collaborate with TCM researchers.

References

PERFORMANCE EVALUATION[1] Backgrounder Traditional Chinese Medicine, National Center for Complementary and Alternative Medicine (NCCAM), National Institute ofHealth, USA, D428 (2013).

In the preliminary user experience, one of the authors (20s female) used the TongueDx application
and write body condition diary for one month. Fig.
3 actually shows the real data about her tongue colorgraph. It shows in one day the tongue body color signiﬁcantly changed from Pale to Purple and appeared[2] Xu Jiatuo, Zhou Changle, Fang Zhaoqin, Zhang
Zhifeng, Wang Zhiguo, Sun Yang: Computerized
Analysis and Recognition of Tongue and Its Coating Color in Tongue Diagnosis, Shanghai Chinese
Medicine University, Vol. 18, No. 3, pp.43–47,
Sep. 2004.

32