**Exploring Measurement and Evaluation Methods For Accessibility**

Shirley A. Becker

Software Engineering

Florida Institute of Technology

150 West University Blvd.

Melbourne, Florida 32901

Ph: (321) 674-8149 Fax: (321) 674-8192

# Background

Our web technology research center, at Florida Tech, has developed a web
usability assessment model and supporting toolset that promotes user
satisfaction as an integral part of the development of online business
applications. This web usability assessment model is an outgrowth of our
collaboration with industry in the pursuit of more effective online
development efforts. Our web usability assessments have included B2C and
B2B e-commerce sites in an international marketplace, the results of
which must remain confidential. We have published generic findings, from
a global perspective, in IEEE Software (Becker & Mottay, 2001).

The usability assessment model, shown in Figure 1, was developed to
identify and measure usability factors that impact a customer's online
experience. The model includes usability factors, which are further
decomposed into usability elements (e.g., font size associated with the
design layout factor). The model takes into account the user profile
such that an assessment is based on the perceived usability of a web
site. The model also includes environmental factors inclusive of monitor
size, browser type, and modem speed. The goals of the organization may
be included, as time-to-market, financial, or other strategic objectives
may impact web site development. Many organizations, for example, did
not initially comply with accessibility guidelines due to cost and
deployment time considerations.

Though not shown, we have expanded these factors into over 100 usability
elements that have been used during usability assessments of commercial
sites (Becker et al., 2000). A prototype web tool, called the Web
Inspector, has been developed for data gathering and assessment
purposes. The Web Inspector allows a user to assess one or more pages
within a site or across sites, based on how the study has been set-up by
an administrator. An assessment is based on one or more usability
factors (design layout, accessibility, design consistency, information
content, navigation, design standards, performance, reliability, and
security) depending on the objectives of the assessment. The tool
summarizes the data and allows for customized reports to be generated
based on one or more assessments.

One of the significant aspects of this tool is that it is a web-based
application. This makes it readily available to be used at
geographically dispersed assessment sites. It has an underlying
relational database so that historical data can be used comparatively as
web sites change.

# Current Research Efforts

The usability assessment model has been modified to include visual
impairment as part of the user profile data (though more accessibility
profile data may be added at a later date). This will require a
usability assessment to include usability elements associated with
textual, audio, imaging, and others, and this work is still underway. We
have also initiated usability work in the area of reading level, as this
usability issue may render sites virtually inaccessible to target users
especially when English is a not the primary language.

We have recently expanded our usability research to address
multicultural issues associated with government and commercial web sites
(Becker & Crespo, 2001). The cultural diversity of the student
population at Florida Tech has presented an opportunity to study
usability from an international perspective. Multicultural usability
issues that have been uncovered include language syntax and semantics
(e.g., direct translation of text into target language resulting in
nonsensical or offensive information, inappropriate hyphenation changing
the meaning of the text, error messages not translated into native
language, search mechanisms on non-English web sites support only
English). We have also identified issues associated with color, symbols,
images, icons, text layout, font size and style, as well, as others.

## Figure 1: Web Usability Assessment Model![](media/image1.png){width="5.9527777777777775in" height="4.46875in"}

The Web Inspector tool is undergoing a transformation in order to make
it a viable tool for usability assessments in terms of the validation of
its usability questions, and the appropriateness of metrics generated.
We have established a web usability lab at Florida Tech, which will
allow us to use the tool by targeted users based on nationality, age,
gender, visual challenges, and other profile data.

# Three Major Issues Uncovered During Our Usability Assessment Work

Three major issues that have been uncovered during our usability
research include: user's age and design layout, internationalization and
design consistency, and information content and accessibility. Each of
these will be briefly described.

*[Age and design layout]{.underline}* - The age of a user may impact
usability due to visual limitations, typing capability, computing
knowledge, and other user profile elements. Visualization of a web site,
based on age, may be impacted by font size, color, and type, background
color and patterns, use of vertical and horizontal white space, and the
position of buttons, labels, images and other web objects, among others.
There are other aspects of usability that may be related to age
inclusive of navigation, design consistency, and others.

*[Multicultural issues associated with design consistency]{.underline}*
-- Design consistency ensures that web objects and pages behave the same
way within and across pages. It also ensures that web objects and pages
have the same look and feel within a web site, regardless of whether
there are international versions of it. Our multicultural studies of
government and commercial e-commerce sites have uncovered a cultural
centricity phenomenon (Becker & Mottay, 2001). The originating web site
tends to have a "better" design in terms of performance, design layout,
navigation, information content, etc., than translated versions. We have
found that translated web sites typically have more graphics than the
original site, thus negatively impacting performance. Another design
inconsistency is the lack of particular words and phrases being
translated from one language to another. Too often, button, links, box,
and other web object labels are not properly translated. A web site, for
example, may have "search," "go," and "cancel" appearing on a
non-English web page.

*[Accessibility and information content]{.underline}* -- Accessibility
requires the availability of meaningful information about the behavior
of a web object, and its relationship with other web objects. A web
object may be related to another web object in a nonlinear fashion, such
that the response of one object impacts another. Visually, this may be
understood by looking at the web objects on a page. When used by a
visually challenged individual, this relationship may go undetected thus
having a negative impact on usability.

To support the accessibility goal, we may have to add information to a
web page regarding the usability requirements of each web object (Becker
& Berkemeyer, 2001). This information would be encapsulated in the
source code, as it is part of the specification of the web object.
Because HTML source code only supports a few usability requirements, we
have developed an enhanced version of HTML called "Self Testing"
Hypertext Markup Language (ST-HTML)[^1]. Since ST-HTML is an extension
to the HTML language, it is "executed" and behaves similarly to the
standard web page.

There are several tags that are proprietary to ST-HTML which allow for
definition and enforcement of usability requirements during testing.
Because these proprietary tags, properties, and values are not part of
the HTML specification, web browsers ignore them. Therefore, ST-HTML
employs the use of a custom browser.

For example, let's say we specified a text box for holding customer
comments about a product. The text box has the following usability
rules:

-   *Rule 1: Data entry is optional* (the customer may leave it blank)

-   *Rule 2: Valid data includes letters, digits, and white spaces*
    (spaces are valid data for this comment field though an email
    address field, for example, would not allow spaces)

-   *Rule 3: User message is displayed* ("Enter your comments here").

The ST-HTML elements, representing these requirements, are in boldface
to show the extension to HTML.

\<TEXTAREA name=CommentTextArea rows=5 cols=35 **DataType**=\"Text\"
**TextContent**=\"Letters, Digits, WhiteSpace, Other\"
**NumericContent**=\"\" **DataLength**=\"\" **Conditions**=\"\"
**DataRequired**=\"No\"\>Enter Your Comments Here\</TEXTAREA\>

 

From an accessibility perspective, these usability requirements are made
accessible in order to provide meaningful information about a web
object. Though not shown in this example, valid and invalid states may
be specified in ST-HTML providing additional information about the
behavior of a web object.

It is important to note that XML would be a powerful tool to support the
addition of meaningful information in source code to support visually
challenged individuals and perhaps non-English speaking individuals. XML
schemas would provide a standardized mechanism for supplementing source
code inclusive of textual descriptions of links, graphics, etc. It would
also support the usability requirements concept, which was developed in
ST-HTML. A team of students at Florida Tech is currently exploring XML
as a viable option to enhance web page source code with this type of
information.

***References provided upon request.***

[^1]: Note: Our original goal with this research was to provide
    automated support for web testing early during the development
    cycle.