Conspicuous Character Patterns
Seiichi Uchida

Ryoji Hattori

Masakazu Iwamura

Shinichiro Omachi

Kyushu Univ., Japan

Kyushu Univ., Japan

Osaka Pref. Univ., JapanTohoku Univ., Japan

Koichi Kise
Osaka Pref. Univ., Japan

Abstract
Detection of characters in scenery images is often a very
difficult problem. Although many researchers have tackled
this difficult problem and achieved a good performance, it
is still difficult to suppress many false alarms andalthough
missings. This paper investigates a conspicuous character pattern, which is a special pattern designed for easierdetection. In order to have an example of the conspicuous character pattern, we select a character font with alarger distance from a non-character pattern distribution
and, simultaneously, with a smaller distance from a character pattern distribution. Experimental results showed thatthe character font selected by this method is actually more
conspicuous (i.e., detected more easily) than other fonts.

1. Introduction
Character detection, or character localization, in scenery
images is a well-known difficult problem. Generally, pastattempts at this problem (e.g., [1, 2, 3, 4]) have utilized
several heuristics. For example, they used contrast, edge
complexity, texture features, pattern regularity, etc. Detection performance, however, is still not satisfactory due toinevitable trade-off of the problem; if we want to detect allcharacters, we often have many false alarms around complex edge regions (such as tree regions). Conversely, if wewant to suppress those false alarms, we may miss most characters. Actually, it will be very difficult to relax this tradeoff in the strategy of the past attempts.
In this paper, we try to escape the trade-off by employing
a new approach on this problem — the new approach is touse character patterns designed for easier detection. In otherwords, instead of trying to detect all character patterns, wetry to detect some special character patterns which are designed to appeal their existence strongly to detectors.
This approach may seem strange from the viewpoint of
the past attempts — it, however, is not so strange. In fact,

we can find a similar approach in OCR/MICR fonts, which
are specially designed to be recognized correctly by machines. In [5, 6], we can find updated versions of OCR fonts
for camera-based character recognition. Two-dimensionalbar codes, such as QR-code, are designed to be detectedeasily by cameras. Characters on signboards are often designed to be detected easily by humans !! (So, they aremeant to catch one’s eye.)
According to this approach, we can set our research purpose as the design of conspicuous character patterns. If theconspicuous character patterns are perfectly conspicuous inany scenery image, it is possible to detect all of them without any false alarm. Thus, we can escape from the above
trade-off.
A naive example of conspicuous character patterns is
red-colored characters; we can detect them in a given
scenery image only by seeking red areas in the image. Another naive example is Times-Roman characters; we can detect them through a template matching-based search technique using original Times-Roman font templates. Although these naive examples are useful to understand theidea of the conspicuous character patterns, they are, clearly,
not the best. It is hard to guarantee that there is no red areaother than the characters. It is also hard to guarantee thatTimes-Roman is most distinguishable from non-characterareas. We must take a more reasonable and systematic
method for designing conspicuous character patterns.
These naive examples suggest an important fact thatconspicuous character patterns should be designed together
with the detection scheme. In other words, we must design
the characters to be detected most easily for the detection
scheme assumed. In fact, the red-colored characters are notconspicuous for the detector based on Times-Roman template matching. Similarly, the Times-Roman fonts are notconspicuous for the color-based detector.
Generally, each character detection scheme is defined by
the following four factors and thus we must find the mostconspicuous character pattern while considering those factors of the assumed detection scheme:character
distribution

Conspicuous
character pattern

Inconspicuous
character pattern

non-character
distribution

Feature space
Figure 1. Character detection scheme assumed in this paper.
(i) How do we determine whether the target area is a character area or not?
(ii) How do we assume the general property of character
patterns?
(iii) How do we assume the general property of noncharacter patterns?
(iv) What features do we use for representing patterns?
In the above example of the red-colored font, we will(i) determine a character area if the color of the area is red
enough, (ii) assume that characters are red-colored, (iii) assume that non-characters are not red-colored, and (iv) use
RGB color feature. It is clear that the most conspicuous
character pattern for this detection scheme is to be designed
as vividly red-colored characters. Every past attempt atcharacter detection also can be described by these four factors and its conspicuous pattern can be defined from them.
In this paper, we will show an example of a reasonable and systematic design method of a conspicuous character pattern for a certain detection scheme. The designing method is rather general and possible to be modified foranother detection scheme. The conspicuousness of the designed characters is verified through several experiments.
Note that we will not create a conspicuous character pattern from scratch. (That is, we will not synthesize it.) Instead, we will select the most conspicuous character patternfrom various fonts. This is just because the selection is far
simpler than the creation as the designing procedure. Thecreation will be considered as our important future work.

2. Assumed detection scheme
In this paper, we assume the following detection scheme
as an example:(i) The target area is determined as the character area
when its distance to a character distribution is small
and its distance to a non-character distribution is large.

(ii) The general property of characters is empirically defined by preparing a large set of font images. In otherwords, the property is defined as the actual distributionof font images.
(iii) The general property of non-characters is also empirically defined by preparing a large set of scenery image
patches.
(iv) A normalized shape feature is used for representing
patterns.
Figure 1 illustrates this detection scheme. Since any
character or non-character pattern p is normalized so thatp = 1, the feature space becomes a hyper-sphere. Distributions of character patterns and non-character patterns
are on the sphere surface. Then the target area is detected
as a character area if it is far to the non-character distribution and near to the character distribution. The former condition is necessary to differentiate the character area from
non-character patterns. The latter condition is introduced toguarantee human readability.
Note that this detection scheme is far more reasonable
than the naive examples in Section 1. This is because theproperties of characters and non-characters are defined empirically by using actual patterns. There is no strange assumption such that each character is red-colored.

3. Designing a conspicuous character pattern
3.1. Conspicuousness
Now, our task is to find the pattern which is the most easily detected by the assumed detection scheme of Section 2.
Formally speaking, we must optimize the pattern p according to the criterion of maximizing its distance from the noncharacter distribution and, simultaneously, minimizing its
distance from the character distribution.
According to the above criterion, we can define theconspicuousness of p by using its distances from thecharacter and non-character distributions as follows. Let{φ1 , . . . , φi , . . . , φI } denote the principal I eigen-vectorsof the covariance matrix of a distribution. Then the distanceof a normalized pattern p from the distribution can be measured by using the I-dimensional subspace whose bases are{φ1 , . . . , φi , . . . , φI }. Specifically, we can have the distance of p as the canonical angle θp given bycos2 θp =II2
p, φi 2=p, φi  .
2 φ 2p
ii=1i=1(1)Using the canonical angle, the conspicuousness of p isdefined as N
θp /(θpN + θpC ) if θpN ≥ ,Cp =(2)0otherwise,where θpN is the canonical angle between p and the noncharacter subspace and θpC is the canonical angle between
p and the character subspace. Both canonical angles arederived by (1). The value θpN /(θpN + θpC ) becomes large if
θpN becomes large and/or θpC becomes small. The maximum
and the minimum values of Cp are 1 (the most conspicuous)and 0 (the least conspicuous), respectively. The threshold (> 0) is introduced so that a small θpN does not give a large
Cp with a very small θpC .
The subspaces of the character and non-character distributions are determined by using empirical covariance matrices of many font images and many scenery image patches,
respectively. Note that the dimensions I of the two subspaces can be different. Also note that it is possible to use
any image feature for representing those images, while wewill use a shape feature, called direction code histogram [7],instead of the bitmap feature. The aim of using the shapefeature is to minimize the effect of color and brightness.
The detail of the shape feature will be given in Section 4.2.

3.2. Selection of most conspicuous character
pattern
As noted in Section 1, we will not synthesize a conspicuous character pattern which globally maximizes Cp . Instead, we will select the most conspicuous character pattern
among various fonts. Specifically, we calculate the conspicuousness Cp for every font image p, and then select the fontimage p with the largest Cp as the most conspicuous character pattern.

4. Experimental result
Using the above method with font images and scenery
image patches, the selection and the evaluation of the mostconspicuous character pattern were done experimentally.
The following is the detail of this experiment.

4.1. Experimental setup
About 8,000 English capital letter images of 308 fonts
were collected (26×308∼ 8,000). Those fonts were selected using their PANOSE [8]. PANOSE is comprised of
10 elements and the first element represents the font family,such as “Latin Text”, “Latin Handwritten”, “Latin Decorative”, etc. The selected 308 fonts were of “Latin Text”,
which is the font family suitable for general documents1 .
As non-character images, about 80,000 image patches(of 64 × 64 pixel size) were collected from scenery imagesof the CalTech’s background database[9], which is available
as a public image database.
1 Theauthors found many “decorative” fonts with “Latin Text”
PANOSE. The authors, however, did not remove them, because there isno objective criterion other than PANOSE for separating Latin Text fonts
from decorative fonts.

11623452345789

Bases of character subspace

Bases of non-character subspace

Figure 2. Bases of character distribution andnon-character distribution.

4.2. Feature space
Each image was represented as the direction code histogram [7]. First, each 64 × 64 image is converted into anedge map and divided into 16 blocks of 16 × 16 pixel size.
The edge pixels in each block were quantized into four directions and then the number of edge pixels of each direction was counted within a block. Thus the dimension of thedirection code histogram is 64 = 16(blocks)× 4(quantizeddirections). Since this feature is totally based on the edge
direction, it is affected by neither color nor intensity.
Then a 64 × 64 covariance matrix was preparedfor each of the character set and the non-character
set and its I principle eigen-vectors were calculated as{φ1 , . . . , φi , . . . , φI }. By observing the cumulative ratio,
the number I was fixed at 5 for characters and 9 for noncharacters. Figure 2 visualizes those eigen-vectors (i.e., thebases of the subspace) of characters and non-characters, respectively. Each eigen-vector is visualized by 64 short linesegments. Each line segment represents the direction andthe value of an element of the vector 2 .

4.3. Selected patterns
According to the procedure in Sections 3.1 and 3.2, themost conspicuous character pattern maximizing Cp was selected for each of 26 letter categories (“A” to “Z”). During
the selection, the threshold  was fixed at 0.4, which wasdetermined experimentally.
Figure 3 (a) shows the most conspicuous character pattern of all categories. Round-shaped characters were selected as the conspicuous character patterns. This is because
the direction code histogram of a round-shaped characterhas almost zero-value elements around the center area and2 In Fig. 2, each element is visualized with its polarity; for example, if
an element of 45◦ has positive (negative) value, it will be visualized as a45◦ (225◦ = 45◦ + 180◦ ) line segment.

(a)

(b)

Figure 3. (a) Example of the most conspicuous pattern and (b) the least conspicuous pattern. Left:original font image. Right: direction code histogram.
thus is different from, especially, the bases φ1 and φ2 ofnon-character subspace. Consequently, θpN becomes large
and thus Cp becomes large for the round-shaped characters.
It will be important to note that the conspicuous pattern is
not limited to the pattern of Fig. 3 (a). Actually, the patternof second-place was a similar round-shaped font. Therefore, we can use various round-shaped fonts as conspicuouspatterns.
Figure 3 (b) shows the least conspicuous character pattern, which has the minimum Cp value by a small θpN .
Those characters often have a direction code histogram similar to one of the basis of non-character subspace. Thus
cos2 θpN becomes large and, equivalently, θpN becomes
small.

4.4. Detection experiment
Actual easiness of detecting the selected conspicuous
patters was examined. As shown in the leftmost figures inFig. 4, the selected most conspicuous patterns were printed
on a paper sheet and put in scene. Similarly, the selected
least conspicuous patterns were also printed and put inthe scene. The detection was performed by exhaustive
search; the conspicuousness Cw was evaluated at all possible square areas, {w}. Note that the size of square areas
was also varied during the search. If we have a large Cw on
a square area w, we can consider that a conspicuous character pattern exists within w.
Figure 4 (a) verifies that the most conspicuous patterns

are really conspicuous; almost all the conspicuous patterns
were detected successfully. (Only “B” was not detected
here.) In addition, there are very few false alarms. This
promising result indicates the importance of designing conspicuous patterns for the detection problem. Another important fact is that none of the least conspicuous patterns
was detected. This fact is also promising because it proves
the conspicuousness is very stable even in real scene.
Figures 4 (b) and (c) still verify the usefulness of the conspicuous patterns in very complex backgrounds. In (b), theconspicuous patterns were successfully detected regardlessof the sizes. In (c), the conspicuous pattern were printed
in different colors. Since the assumed feature (the direction
code histogram) is affected by neither color nor intensity,
the colored conspicuous patterns were detected without obvious degradation. It is interesting to note that most characters in the backgrounds were not detected successfully. This
also indicates that the characters designed to maximize theirconspicuousness are very privileged patterns in the detection problem.

5. Conclusion
Like OCR/MICR fonts for easier character recognition,
conspicuous character patterns were newly introduced foreasier character detection in scenery images. This paper has
discussed the approach of the conspicuous character patternand detailed a systematic methodology of selecting the mostMost conspicuouspatterns
Least conspicuous
patterns(a)
Most
conspicuous
patterns

Least
conspicuous
patterns(b)
Most
conspicuous
patterns

Least
conspicuous
patterns(c)Figure 4. Character detection results on three different scenery images. From left to right: originalscenery image with the most conspicuous and the least conspicuous character patterns (of Fig. 3 ),the locations of those patterns, detected areas, and the detected areas superimposed on the scenery
image.
conspicuous character font among various candidate fonts
for a certain detection scheme. The actual conspicuousnessof the selected pattern was confirmed experimentally; theirdetection was very easier than the detection of other characters.
This paper is the first attempt at developing conspicuous character patterns and therefore there are many future
works. In this paper, we first assume a detection schemeand then design the conspicuous pattern for the scheme. In
future, simultaneous design of the detection scheme and theconspicuous pattern should be investigated. Another future
work is the systematic creation (synthesis) of the most conspicuous pattern instead of selection. For this future work,some parametric font-shape models will be necessary.

References[1] J. Keechul, K. I. Kwang, and A. K. Jain, “Text information
extraction in images and video: a survey, Pattern Recognition,
vol. 37, no. 5, pp. 977-997, 2004.
[2] R. W. Lienhart and F. Stuber, “Automatic text recognition indigital videos,” Proceedings of SPIE, vol. 2666 Image andVideo Processing IV, pp. 180-188, 1996.

[3] X. Chen and A. L. Yuille, “Detecting and reading text in natural scenes,” CVPR 2004, vol. 2, pp. 366-373, 2004.
[4] V. Wu, R. Manmatha, and E. M. Riseman, “An automatic system to detect and recognize text in images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 11, pp. 1224-1229, 1999.
[5] S. Uchida, M. Sakai, M. Iwamura, S. Omachi, K. Kise “Extraction of embedded class information from universal character pattern,” Proc. ICDAR, vol. 1 of 2, pp.437-441, 2007.
[6] S. Omachi, M. Iwamura, S. Uchida, and K. Kise, “Affine
invariant information embedment for accurate camera-based
character recognition,” Proc. ICPR, vol. 2 of 4, pp. 10981101, 2006.
[7] F. Kimura, T. Wakabayashi, S. Tsuruoka, and Y. Miyake, “Improvement of handwritten Japanese character recognition using weighted direction code histogram,” Pattern Recognition,
vol. 30, no. 8, pp. 1329-1337, 1997.
[8] http://www.w3.org/Printing/stevahn.html[9] http://www.vision.caltech.edu/html-files/archive.html