Research Update #9 -- Research in Electronic Color Dropout

As mentioned in previous Research Updates, the United States Census
Bureau is conducting technical market research related to data capture
from paper questionnaires. One focus of this research is expanding the
input tolerances of the data capture system to accommodate a wider range
of printing conditions, which in turn may allow forms to be produced by
multiple vendors or printing methods. This will be important, not only
for the initial questionnaire mailing, but also the Replacement Mailing
now being researched.

One of the goals guiding the design of questionnaires for the 2010
Decennial Census is that they be *respondent-friendly* and *data
capture-friendly* at the same time. Some features that make a
questionnaire respondent-friendly are a printed colored background with
unprinted white write-in fields, and check boxes indicating where
information should be entered. For such a form to be data
capture-friendly also, the scanner or image processing software must be
able to remove, or *drop out*, the colored background without removing
or weakening marks made by the respondent.

During the 2000 Census, *optical* *lamp dropout* was used to remove the
colored form background from the scanned image. The form background was
yellowish-orange, and the document scanners were equipped with red
lamps, making the form background essentially invisible to the scanner.
While the colored background was effectively removed, some of the side
effects were undesirable. In particular, marks made by most red pens or
pencils became invisible to the scanner, requiring data from those
questionnaires to be keyed from the paper form. In addition, since the
chosen background had low visual contrast, it was sometimes difficult
for the respondent to distinguish between the printed background and
blank white paper. This was especially true for respondents with
age-related visual changes, such as yellowing of the eye's lens or
macula. Also, the tungsten lighting that is common in most residential
households is rich in red light content, further reducing the visual
contrast.

Goals for the 2010 Census include improving the respondent-friendliness
of the questionnaire through better choice of the background color,
automating the data capture of red marks, and expanding the acceptable
background color tolerance range. Color document scanners and advanced
electronic image processing software may prove to be key to enabling all
of these improvements.

Early in the research program we recognized that there are no standard
methods for evaluating color dropout. Therefore, part of the effort has
involved developing scientific evaluation methods that can be used to
compare the results achieved by various document scanners and software.
The methods involve scanning specially designed test pages, some of
which are produced via a technique we call [simulated spot
color](http://www.census.gov/procur/www/2010dris/simulated-spot-color.doc),
and counting the number of black pixels in certain areas of the scanned
image.

One of the test pages [(Version
1.21)](http://www.census.gov/procur/www/2010dris/version1.21.doc) was
designed to evaluate the ability to drop out multiple printing
conditions of a given color at the same time. This page contains square
areas of 49 different printing conditions, each characterized by a
screen ruling and fractional coverage. Within each area is an unprinted
rectangle where we can add actual or simulated respondent marks. Marks
can also overlap the printed area. This test page can be produced in any
of a range of colors, with a typical evaluation including a dozen or
more colors. Each page typically is marked with four pen types (standard
ball point, gel ball point, sharpie, and flair), each in four colors
(black, blue, red, and green), and with pencil. Often, a similar number
of simulated marks are digitally printed along with the rest of the
page. This page can be used to evaluate lamp dropout, channel dropout,
selective color dropout, or template based dropout techniques.

Another test page ([Version
5.0](http://www.census.gov/procur/www/2010dris/version5.0.doc)) is being
developed to evaluate dynamic dropout methods, in which the color to be
dropped from a scanned image is determined by automated analysis of that
individual image itself. Because the color dropout performance depends
on the page content, the page was designed to look very much like a
Census form. However, some of the border boxes and segmentation lines
are printed lighter and darker than normal, creating a complete range of
tints in 10% coverage increments. These small areas give an indication
of the range of color that will be dropped.

The goal of color dropout is a bitonal image in which the form
background becomes white, and respondent marks become black. We quantify
the degree of success by comparing two images: a test image created with
the color dropout method, and a reference image prepared with simple
thresholding, as illustrated in this
[example](http://www.census.gov/procur/www/2010dris/example.doc). The
number of black pixels making up the black pen marks are counted, and
the ratio of those in the test image to those in the reference image is
normalized to 1000. A perfect score of 1000 indicates that the color
dropout operation had no effect on black marks. Typical scores for black
pen marks are in the range from 900 to 1000. The same calculation is
made for blue, red, green, and pencil marks, giving a separate score for
each of these five mark colors.

The number of black pixels in the background area of each image is
counted as well, and the ratio is normalized to 1,000,000. A perfect
score of zero is observed fairly frequently, indicating that every pixel
of the background was removed. Nonzero scores tend to be either very
large (probably unacceptable) or less than 500 (probably acceptable).

Results are displayed in a bar graph format like the one shown below.
Each group of six bars represents one background color and one color
dropout method. The black, blue, red, green, and pencil mark bars are in
the positive direction, indicating that a higher score is desirable,
while the background bar (yellow) is in the negative direction, because
a higher score is undesirable. The particular experiment chosen for this
example involved nine background colors and four color dropout methods.
The [background
colors](http://www.census.gov/procur/www/2010dris/background-colors.doc)
were chosen to sample the hue circle, and the dropout methods include
red channel, green channel, blue channel, and a selective color dropout
technique.

+----------+-----------------------------------------------------+-----+
| >        | ![](media/image1.wmf){width="4.322222222222222in"   | > * |
|  **Marks | height="1.40625in"}                                 | *RE |
| > (per   |                                                     | D** |
| > Tho    | red yellow green cyan blue magenta                  | >   |
| usand)** |                                                     | >   |
| >        | # Background Colors                                 |  ** |
| > **Ba   |                                                     | Dro |
| ckground |                                                     | pou |
| > (per   |                                                     | t** |
| > Mi     |                                                     |     |
| llion)** |                                                     |     |
+----------+-----------------------------------------------------+-----+
|          | ![](media/image2.wmf){width="4.303472222222222in"   | >   |
|          | height="1.636111111111111in"}                       | **G |
|          |                                                     | REE |
|          |                                                     | N** |
|          |                                                     | >   |
|          |                                                     | >   |
|          |                                                     |  ** |
|          |                                                     | Dro |
|          |                                                     | pou |
|          |                                                     | t** |
+----------+-----------------------------------------------------+-----+
|          | ![](media/image3.wmf){width="4.300694444444445in"   | >   |
|          | height="1.4527777777777777in"}                      |  ** |
|          |                                                     | BLU |
|          |                                                     | E** |
|          |                                                     | >   |
|          |                                                     | >   |
|          |                                                     |  ** |
|          |                                                     | Dro |
|          |                                                     | pou |
|          |                                                     | t** |
+----------+-----------------------------------------------------+-----+
| ![](m    | ![](media/image10.wmf){width="4.302083333333333in"  | >   |
| edia/ima | height="1.4694444444444446in"}                      |  ** |
| ge4.png) |                                                     | SEL |
| Ba       |                                                     | ECT |
| ckground |                                                     | IVE |
|          |                                                     | >   |
| ![](m    |                                                     | Dro |
| edia/ima |                                                     | pou |
| ge5.png) |                                                     | t** |
| Black    |                                                     |     |
| Marks    |                                                     |     |
|          |                                                     |     |
| ![](m    |                                                     |     |
| edia/ima |                                                     |     |
| ge6.png) |                                                     |     |
| Blue     |                                                     |     |
|          |                                                     |     |
| ![](m    |                                                     |     |
| edia/ima |                                                     |     |
| ge7.png) |                                                     |     |
| Red      |                                                     |     |
|          |                                                     |     |
| ![](m    |                                                     |     |
| edia/ima |                                                     |     |
| ge8.png) |                                                     |     |
| Green    |                                                     |     |
|          |                                                     |     |
| ![](m    |                                                     |     |
| edia/ima |                                                     |     |
| ge9.png) |                                                     |     |
| Pencil   |                                                     |     |
+----------+-----------------------------------------------------+-----+
| Results  |                                                     |     |
| of one   |                                                     |     |
| ex       |                                                     |     |
| periment |                                                     |     |
| showing  |                                                     |     |
| ba       |                                                     |     |
| ckground |                                                     |     |
| r        |                                                     |     |
| etention |                                                     |     |
| as       |                                                     |     |
| negative |                                                     |     |
| and mark |                                                     |     |
| r        |                                                     |     |
| etention |                                                     |     |
| as       |                                                     |     |
| p        |                                                     |     |
| ositive. |                                                     |     |
| RED,     |                                                     |     |
| GREEN,   |                                                     |     |
| BLUE,    |                                                     |     |
| and      |                                                     |     |
| S        |                                                     |     |
| ELECTIVE |                                                     |     |
| dropout  |                                                     |     |
| s        |                                                     |     |
| ettings, |                                                     |     |
| respe    |                                                     |     |
| ctively. |                                                     |     |
+----------+-----------------------------------------------------+-----+

The goal of this market research is *not* to develop a solution for use
in a data capture system for 2010. Rather, it is to identify and
scientifically evaluate color dropout methods that may become part of
the solution. This test methodology helps define the range of input
colors based on the specific imaging system characteristics, and serves
as a comparison between specific systems. We welcome comments and
suggestions from industry on color dropout methods that merit
evaluation, and on our evaluation techniques as well.

**Please note:** The questionnaire colors presented are for
**illustrative purposes only**. They are not likely to be the colors
used by the Census Bureau for the 2010 Census, but are representative of
the type of color that could be selected.

If you have questions, comments, or would like more information, contact
Dr. Brad Paxton at (585) 239-6057 or <brad.paxton@adillc.net>.