## Testing for Accessibility and Usability

**Looking at testing the design and operational characteristics of**

**products** **without formal standards or standardized criteria**

Hewlett-Packard Accessible Solutions (hpas)

**Abstract:**

Standards have not been developed by Industry or the Government to
reflect quantitative parameters, conformance testing or certification
for specific rules associated with Section 508. This paper address
different ways, through functional tests, demonstrations or inspections,
to evaluate a product and to provide information on what is and what is
not accessible in relationship to a technical standard i.e., 1194.23a
and a functional performance standard i.e., 1194.31a associated with
Section 508. The findings from these types of tests can be used to
incorporate new criteria into future designs and operational parameters
of products. Most of the comments are derived from previous testing
accomplished on printers and their associated host computer, but can
apply, in most cases, to any type of product. As the reader progresses
through the concepts of testing for accessibility it becomes apparent
that individuals who have disabilities should accomplish the testing or
at least be very involved in the development and execution of the tests
and assistive technology devices be integrated into the equipment
configurations.

**Background:**

Moving a mouse to select different printing options associated with a
printer driver is considered as a normal, routine operation for a person
without a disability; however, those selections can become very time
consuming and major tasks when the end user has a mobility or dexterity
disability that does not allow for the use of a mouse or other pointing
devices. In some situations people without disabilities, especially
users of laptops, complain that the movement of the mouse is difficult
and "power user" have a great dislike for mice.

Looking for information streamed from a printer to a host Personal
Computer (PC), is a routine function for an end user with no vision
disabilities, but can become very confusing and intimidating to a user
who is blind or who has limited vision.

Knowing what state the printer is in, e.g., power on, paper status, etc.
all constitute basic information that must be provided to the end users,
regardless of their capabilities, to ensure products can be produced and
distributed.

Many users who have no disabilities struggle with some of the
information flow provided by products needed to maintain a normal
operating environment, so it is imperative to also provide all of the
information to the user with a disability. Don't provide just a few of
the items a designer may feel are necessary.

This paper is an attempt to provide an insight into how products can be
tested for accessibility, when formal industry standards and criteria
are not available, which in turn provides a more usable product for the
whole community. Sometimes, due to the lack of a standard the results
become subjective, but the results do provide an insight as to what is
accessible and what is not.

The following are situations to consider when making decisions about the
testing of certain products.

1.  Make a list of the products that need evaluated and the appropriate
    services or output to be tested. This can be accomplished using
    sales information (all products eventually need this evaluation
    regardless of their sales impact), by determining the environment
    the product will normally operate in, evaluating the life cycle of
    the product and any future modifications or enhancement to the
    product that may affect the outcome of your testing and the
    customers desires and comments from previous/similar products.

2.  Once the list has been finalized determine what operating systems
    the products will be tested against. Some of the industry standards
    do not support connectivity to certain assistive technology devices,
    nor do they provide accessible software features allowing the user
    to by-pass mouse induced commands. These limitations should be
    documented as criteria to incorporate changes inti future products.
    Determine the testing environment in relationship to networks,
    servers, etc., any peripherals to be integrated or attached and what
    applicable Printer Control Language (PCL) will be the most widely
    used for the specific products. In some cases all operating systems
    and all PCLs will need to be tested with each product. Ensure the
    environment will support your objectives, but don't "salt" the
    environment to ensure the tests will pass. Establish the environment
    based on actual work situations and operational constraints.

3.  Review customer's comments concerning any specific areas they would
    like evaluated. If the opportunity exists use the exact files and
    configurations employed by the customer. In many cases a product or
    system is determined to be accessible when tested in a sterile
    environment, but when placed into the customer's environment some of
    the accessible features do not respond accordingly.

**Objective:**

Determine the product's accessible characteristics through rigorous
testing and evaluations. Provide changes and solutions as to what can be
done to make it more accessible. Evaluate its compatibility with
assistive technology devices. This should not be limited to one type of
assistive device, but should provide information on several types of
devices for vision, hearing and mobility disabilities. If possible tests
should also include devices cognitive, learning, speech impairments and
seizures (this is especially important for screen flickers). There are
several avenues that must be explored and developed to ensure the
objective is met. They are:

1.  Design a test suite of information that basically touches all
    operational and functional characteristics of the printer.
    Functionality, coupled with accessibility should be the baseline of
    the test suites/cases. Don't limit the test to merely printing a
    document. Exercise as much of the product as possible and include
    some compatibility issues with the hosting PC. Test the product in
    both a networked environment and as a stand-alone printer. Many
    times systematic problems occur within a network that may negate
    some of the accessibility features built into the product, e.g.,
    information being relayed from the printer to the host PC.

2.  The test suite should begin with a basic concept and add additional
    tests, current parameters and environmental changes as the tests
    progress. Design the test suites and test cases to a level that can
    be understood by all, especially the technicians performing the
    test. Make the tests flow in an easy step-by-step format to allow
    complete understanding of what is being exercised and what is the
    expected outcome. Most test cases of this nature should be designed
    to accommodate a tester that may not be very familiar with the
    product under test, but will provide him/her with the necessary
    information to accomplish the desired results. Incorporate all test
    results, defects and pertinent comments into a database. This should
    be accomplished at the closure of each test case/suite or at a
    minimum at the end of every test event. Populating the database on a
    routine basis not only assures the information is captured it also
    provides a very effective tool for re-creating a test or situation
    if needed.

3.  Plan on running the tests several times. Once in a normal
    configuration, e.g., mouse, no assistive devices, etc. Ensure that
    any red line/as run changes to the tests are incorporated into the
    next run and that the configuration is the same as before.

4.  Plan for what the expected results will be. Having an idea as to the
    outcome of a certain test sequence can validate the design criteria
    of the product.

5.  Share all results with individuals responsible for making design
    decisions. Inform them of any changes, large or small, that could
    benefit the product from an accessibility and usability standpoint.
    What may be a minor problem to them may be an enormous hurdle for an
    individual with a special disability.

**Approach:**

These tests were conducted in 2000 while performing accessibility and
usability tests sequences for studies associated with the Customer
Solutions Test Group. The main emphasis of the test was to determine the
compatibility of the PC with the printer using only the PC keyboard as
the input interface to the printer and a screen reader as the assistive
device. All printer drivers and printer functions were controlled by use
of the mouse during the first phase. All PC operating systems and
applications were controlled using a mouse. As the tests progressed the
configuration changed by deleting the mouse and adding assistive devices
to the PC.

> [Test 1. Expected Results:]{.underline}
>
> All functions of the test would perform normally since navigation
> tests had been performed previously on a like product.

The person conducting the test had no disabilities. There were only 5
discrepancies noted during the test. Three of the discrepancies were
compatibility issues with a newer version of a printer driver, and the
other two were associated with selections required during the
installation of a driver. The newness of the test and the unfamiliarity
of the product may have contributed to several of the discrepancies.
Movement of the mouse to those areas requiring action by the technician
was not encumbered in any way and all functions could be selected. The
test time for this particular portion took approximately 4 hours to
complete, which included re-running several of the steps to gain the
desired results.

> [Test 2: Expected Results]{.underline}:

All functions of the test would perform the same as the first test
resulting in the same amount of defects. This thought process was
predicated upon the fact that the only difference between the two
configurations was that the mouse was disconnected and would not be a
factor for input to the printer.

> The test was conducted by an individual with an upper body mobility
> disability that did not allow for use of the mouse or other pointing
> devices. Surprisingly the test was completed in about 3.5 hours
> because the technician was a "Power User" and very familiar with the
> necessary keystrokes required for movement to the desired locations on
> the screen. The same 5 minor defects noted during the first test were
> still present, but there were also 7 more defects noted. Five of the
> new defects were associated with keyboard strokes by not allowing the
> user to access the correct portion of the driver interface and the two
> others were associated with the movement in the "Start" menu of the
> operating system. These discrepancies were not noted during the first
> test and indicated a flaw in some of the design criteria and in some
> of the software logic used with a specific driver. A minor
> modification was made to the printer's driver software and the
> applicable portion of the test was re-accomplished. The keystrokes
> discrepancies associated with the first run of this test were
> corrected; however, the two associated with the 'Start' menu resides
> within the operating system were still present. Documentation was
> provided to the manufacture of the operating system and they have been
> corrected. The ability of the tester to use the keystrokes instead of
> the mouse aided greatly in discovering the discrepancies resulting in
> changes that will enhance the accessibility of the product.
>
> [Test 3: Expected Results:]{.underline}
>
> Since all of the discrepancies had been noted or fixed all functions
> of the test would perform the same as the second minus those fixed
> defects. This particular product had never been evaluated using a
> screen reader so the outcome was not know, even though the software
> had been tested against tools on the WEB to determine its
> compatibility.
>
> A person who has a minor mobility disability, but prefers to use
> keystrokes instead of a mouse, and is also blind conducted the third
> test. Screen reader software, familiar to the tester, was loaded onto
> the host PC and the mouse was left disconnected. The two defects noted
> during the second test with the 'Start" menu were still present as
> well as 6 new defects were noted with the use of the screen reader.
> Four of these new defects were associated with a software product
> residing within the printer that provides printer status back to the
> host PC and the remaining two were associated with the capability of
> the driver providing focus for the screen reader during a driver
> installation. This test was also conducted in about 3.5 hours.
>
> The test was conducted again, but this time the configuration was with
> screen reader software unfamiliar to the tester. The same defects from
> the other tests were still present, but the 6 defects associated with
> the screen reader had increased to 9. It was determined later that the
> three extra defects were the result of using unfamiliar software. The
> time for this test was about 4 hours.

**Discussion:**

1.  Mobility/Dexterity:

> Using accessibility attributes of operating systems can provide
> individuals with the capability to operate products in a normal
> day-to-day operation, but designers need to re-think their position as
> to relying on the operating system. Software systems developed for
> printers and like products should keep accessibility in the forefront
> of their development efforts. Results during the first test revealed
> no discrepancies for conducting day-to-day operations or for several
> unique requirements when using a mouse as the input interface to the
> printer software. As discovered during the second test, moving around
> in the PC/printer environment was successful for the day-to-day
> operations, but lacked the necessary interfaces and compatibility for
> those items required on a non-routine basis.

2.  Vision:

With the third test another layer of interaction/compatibility was
introduced with the incorporation of the screen reader. Once again the
user was able to navigate through the operating system and the
applications used for the test, but only on a normal day-to-day basis.
This test scenario was unique because it encompassed testing for two
disabilities that may be uncommon in most situations.

3\. No tests were conducted for hearing disabilities or other
disabilities.

**Conclusion:**

1.  Developers of software and applications need to be aware that the
    disabled user is not just interested in doing the normal day-to-day
    routine tasks associated with their equipment. They want to be
    self-sufficient and produce their required output without the aid of
    others.

2.  Individuals building specific tests for accessibility need to
    realize that testing products without using the benefit of a person
    with a disability will cause the product to fail in many areas once
    the product enters the market. Technicians/testers with disabilities
    should be used whenever possible. Testing with a non-disabled person
    gives a perspective or benchmark of the product but testing with the
    correct community provides the necessary information ensuring good
    design practices for future products.

3.  Computers, the Internet and business Intranets have increased an
    individual's capability to communicate throughout the world. They
    have improved upon the quality of life and aided in providing a more
    informed world community. Testers need to be aware of all the
    interfaces and capabilities available to persons with disabilities.
    Merely designing a product to produce or print documents using a
    word processing program is not designing for accessibility. They
    need to expand their scope and include accessibility for all aspects
    and functions of any given product.

4.  Information Technology Companies, Designers, developers and testers
    need to form alliances with the assistive technology industry.
    Without the use of an assistive devices in the tests many
    discrepancies would have gone unnoticed until a user in the field
    discovered them. These devices make it possible for individuals with
    disabilities to function in all aspects of our society and become
    more productive citizens. If manufactures of Information Technology
    products don't become familiar with assistive devices and educate
    their work force on how to use these products and how to test with
    them they will lose to those companies that have planned ahead.

Appendix 1

Sample Test Cases

**Installing a driver:**

**[Pre-conditions:]{.underline}**

1\. Clean the system to ensure no print drivers installed

2\. PC Configured per Tier One items 12,19,37,38,39,50, and Tier Two
item 10 Applications (Tiers located in Customer Solutions Test, Test
Description Specification Appendix \"A\")

3\. Install CD

4\. Disconnect the mouse

5 Ensure Printer connected to applicable PC

**[Steps:]{.underline}**

1\. From the START menu select setting

2\. Select printer and add printers

3\. From the add printer wizard menu select my computer, next, LPT1, and
next

4\. From the add printer wizard menu select from the manufactures list
and select \"have disk\"

5\. Install from the CD-ROM (All drivers, i.e., PS, PCL)

6\. Type in the name of the driver in the menu window

7\. Select printer as the default

8\. Select not shared

9\. Print a test page and verify print

# Accessing Accessibility Attributes

**Pre\--conditions:**

1\. Ensure printer driver installed

2\. Disconnect the Mouse

3.Ensure PC Configured per Tier One items 12,19,37,38,39,50, and Tier
Two item 10

4\. Ensure no network configurations are in place

5\. Ensure Printer connected to applicable PC

6\. Delete all previously selected selections for Accessibility Options

**Steps:**

1\. Select Start, Settings, Control Panels, Accessibility Options

2\. Select Keyboard Tab; under sticky keys options check \"Press
modifier key twice to lock\"

3\. Under Notification select \'Make Sound When Modifier Key is Pressed,
then select Okay

4\. Select Sound Tab, under SoundSentry select \"use SoundSentry and
under setting pull down \"Flash Active Window, then select Okay

5\. Under ShowSounds select \"Use ShowSounds\", Select Okay

6\. Select Mouse Tab, Select \'Use Mouse Keys; Under Settings select a
comfortable level, Select \"Use, MouseKeys when NumLock is \"ON\",

then select Okay.,

7\. Select Display Tab: Select Use short Cut, select use high contrast,
select white on black.

8\. Select General Tab: Under automatic reset select \"turn off
accessibility feature after idle for 30 minutes

9\. Under notification select \"make sound when turning a feature on or
off, select apply and Okay.

10\. Repeat TC DA \# 07 and #029. (These are pre-canned test developed
for certain applications)

11\. Verify all controls were accessible by the tab key and shift tab,
that the tab order was down and right, all hot keys worked correctly,
All visual and audio functions for the \"Accessibility Options\"
functioned correctly.

12\. Re-configure the PC to ensure no Accessibility Functions are still
selected