# Presentation: 051147

## ICGI Content Standards

- Usability Test Results
- Content Managers’ Forum
- July 15, 2004

## The Tests:

- Tested eleven content standards
- Terminology
- Provide description—”What would you call this?”
- Rank list of options by preference
- Placement on the web page
- Identify placement on generic web page
- Categorization
- Card sort exercise
- Reaction to “typical” federal page

## Generic Federal Homepage

## The Test Participants:

- Eleven participants
- Mix of races/ethnicities
- Variety of web experience
- Wide age spectrum
- Both genders
- Range of formal education

## Sponsorship of Gov Web Sites

- Participants shown a homepage (HHS/HUD)
**What tells you it's an official govt. site?**
**	**.gov in URL					54.5%
- 	"U.S. Dept .of..." visible			45.5%
- 	"Dept. of/agency name" visible		18.2%
- 	graphic logo/seal				18.2%
- 	other graphics					  9.1%

## Comments

- “ “Mainly checks for the .gov URL such as dot.gov and trusts it more than other sites like .com.”
- “Definitely goes to .gov for government information and checks for the .gov URL.”

## Link to Gov’t-wide Portal

- Which label most accurately depicts:
- An official gateway to all government information in the United States
- A portal containing links to federal and state government sites

## Ranking:

- 					Unadjusted	Adjusted
- _[www.us.gov](http://www.us.gov/)_			1.90		2.05
- _[www.fedinfo.gov](http://www.fedinfo.gov/)_		1.91		1.91
- _[www.usa.gov](http://www.usa.gov/)_		2.10		2.23
- _[www.firstgov.gov](http://www.firstgov.gov/)_		3.89		3.82
- Participants thought that “U.S.” and “fed” meant federal, and therefore, were looking for a more encompassing term that did not seem to exclude state governments.

## Location

- Users expected the link to be at the top of the page:
- 3 participants-			top left menu bar
- 2 participants-			top right menu bar
- 2 participants-			top nav bar
- 2 participants-			bottom of page/ 					footer

## Comments

- “Regarding the site name www.firstgov.gov – doesn’t know what this means at all.”
- “Regarding the site name www.fedinfo.gov – said this doesn’t apply to states because of the fed term.”
- “Suggested govinfo.gov”
- “Said firstgov.gov is ridiculous and looks like a typo”
- “Said to be consistent and put all links on the left.  I don’t want to look all over the page for it.  There should be one spot where I can expect to see everything.”

## About Us

- Which label most accurately depicts a place where you can:
- Learn who operates this web site
- Learn what this organization does
- Find the organizational structure of the group that runs this site

## Ranking

- 					 Unadjusted		Adjusted
- About Us				2.25		2.50
- About [Organization]		2.31		2.56
- Learn About Us			2.50		2.50
- Who We Are			2.94		2.94
- For More Information		4.50		4.50

## Location

- Users were split as to expecting the About Us link to be at the top or the bottom of the page.
- 3 participants-			top left menu bar
- 1 participant-			bottom of page
- 1 participant-			bottom/ middle of page
- 1 participant-			bottom/ right

## Comments

- “Expects basic stuff near the top of the page and lower links to be more explanatory.”
- “Said the test page looked as expected- it has links to specific organizations where you can find more.  Expects to find jobs and then sees employment.”
- “Said the test page looked as expected.  Expects to find email not the mailing address.”
- “A guide to the organization structure is great”
- “Would like to see contact us closer to the top.  At the bottom it makes me think they don’t want me to contact them.”

## Laws and Regulations

- Card Labels
- Guidance
- Policy
- Current Legislation in Congress
- Major Laws & Statutes
- Executive Orders
- Proposed Rules
- Regulatory Actions
- U.S. Code
- Federal Register Documents
- Preambles to Final Rules
- Dockets
- Testimony

- Tester Recommendations
- Legislative Rulings
- Legal/ Legislation
- Building a Law from Start to Finish
- Steps to Creating a Law
- Information Reference/ Resources
- Rules and Regulations
- Governmental Procedures

## Ranking

- Laws and Regulations			1.71
- Policies and Regulations			2.14
- Policy						3.00
- Laws and Regulatory Assistance	3.14
- The majority of the participants (90%) thought “passed and signed laws are different than proposals, so they would not be grouped together” or “these terms should be arranged in a developmental way.”

## Location

- The majority though the Laws and Regs link belonged at the top of the page
- 4 participants-			top left nav bar
- 2 participants-			top right nav bar
- 2 participants-			middle left menu bar
- 2 participants-			bottom of page/ footer

## Comments

- “Divided cards into two groups.  From the two groups in the card sort would call them Preamble (for future laws) and Statutes (for current laws).”
- “Said they should be arranged in the order of the process of becoming a law.”
- “Thought these looked like the steps to creating a law.”
- “Sees the list in a developmental sense such that proposed rules come before laws.  Would arrange the list into three categories and didn’t think “Guidance” fit in the group.”
- “Said terms were related but didn’t all fit together.  Saw a difference between the concrete laws and other terms.”
- “Said the information did not fit together and should be broken into categories: 1. Agency Legislation in Process, 2. Historical Source of Information for Legislation, 3. Executive Orders, 4. Policy & Guidance.”

## Search

- Which label most accurately depicts a place where you can:
- Enter terms in order to find content most relevant to your needs
- Scan a website using keywords to find matching information
- Use single words or phrases to quickly locate information within an agency's web site
- 6 out of 10 (60%) of the participants came up with at least one label suggestion that included the word “Search” (search, keyword search, quick search, etc.).

## Ranking

- Users were provided the following options:
- Find It
- Ask
- Search
- Where do I...
- Select a Topic
- Clear preference here: 80% of the participants tested ranked the label “Search” #1.
- For most participants "Ask" ranked very low, and some commented it sounded like Ask Jeeves or like one could actually ask questions online.  Many participants would like a "Select a Topic" feature and would expect to see a drop-down menu, but said this was obviously different than Search.

## Location

- 90% expected Search to be somewhere in the top menu bar
- Right hand side of the bar		4
- Left hand side of the bar		2
- Center of the bar			0
- No real preference within bar	3

## Search Preferences

- Participants were given color printouts of three home pages:
- example with search box empty (ED.gov, box in left hand menu)
- example with search box filled in (DOT.gov, filled box in header graphic)
- example with no search box, just a link to "Search" (DOD, link in menu bar)
- During this test, 60% of the participants expressed a definite liking for a “search box” (2 with text inside, 2 empty, 2 no preference or didn’t matter).

## Contact Us

- Which label most accurately depicts a place where you can:
- Email the owners of the site
- Get the phone number of the site owners
- Get the address of the site owners
- Report a technical issue or a question about the site
- Suggestions for labels included Public Information (3), Request... (3), For More Info (3), FOIA (2 – one user had worked for govt. and recognized acronym).

## Ranking

- 	Assistance
- 	Comments
- 	Help
- 	Contact Us
- 	For More Information
- There was a strong preference for Contact Us- eight of the nine participants listed it as #1.

## Location

- 70% expected to find the Contact Us link at the bottom of the page
- 3 participants-		top right menu bar
- 3 participants- 	bottom left menu bar
- 2 participants-		bottom of page
- 1 participant-		bottom right menu bar
- 1 participant-		bottom center of page

## Comments

- “Said this should be a link to a phone directory for agencies, divisions, and bureaus . . . you can never find anybody anymore!”
- “Thinks agencies should send at least a boiler plate response in an email.”
- “Contact Us should be a way to email an agency.”

## FOIA

- Which label most accurately depicts a place where you can:
- Learn about your rights to request from a Government Agency public information that in the past had not been available to the public.
- Request information – such as books, charts, emails, and records - on a specific Government Agency’s activities.

## Ranking

- 					Unadjusted		Adjusted
- Public Rights to Gov Info		2.43		3.18
- Public Access to Gov Info		2.50		2.82
- Freedom to Request		2.86		3.59
- Request Gov Info			3.44		3.55
- Freedom of Information Act	3.67		3.77
- FOIA					4.07		4.23

## Location

- Participants were mixed on expected location:
- 5 participants			left menu bar
- 3 participants			page center
- 4 participants			bottom of page
- 2 participants			right menu bar

## Comments

- “Recognized "Freedom of Info" Thinks media/big wigs have this, not people like her... Would not recognize FOIA, Definitely prefers the term "public"  Tells her she, a citizen, has access to it.”
- “[says] If she didn't know FOIA, she would pick public rights/access.”
- “"Rights" suggests these may be problems or roadblocks in getting the info.”
- “... people want to go right to submitting a request.  The example page [USDA FOIA page] is what's expected but info about law is not useful.... Links on the page should be moved up.”
- “Example page: does not like to have to go to another page to find out more about filing a request.”

## Current Information

- Test participants were shown two pages:
- Today’s Date at top of page (DOL)
- Last Modified Date at bottom of page (BLS)
- Is this page up to date?
- What does the date indicate?
- Is this date useful?

## Ranking

- Participants were then asked to rank, in terms of usefulness, the following types of dates:
- Today’s Date
- Posted Date
- Last Modified Date
- Last Reviewed Date
- Last Certified Date
- Last Updated Date
- Creation Date

## Results

- Participants either loved or hated Today’s Date:
- “Today's date doesn't tell whether info is current.”
- “Likes both Today's Date and Last Modified..  Depends on info he's looking for.”
- “Date up on top left corner is not updated date, not useful.”
- “Date by itself could mean today. Many people may not know what "modified" means - but there has to be a label with a date.”
- “Today's date is important.  Indicates that page is kept current all day. Stats should be current!”

## Results

- Participants seemed to like either “Last Modified” and “Last Updated” to indicate the currency of content on a particular page.  Several ranked them equally or close to equal.
- “Stats and Statements of fact- wants a last modified date.  If no date no way of knowing how  accurate or up to date is the info.”
- “Last modified-content updated. TP likes the last modified date info.”

## Location

- Most participants expect the date to be at the top of the page:
- 5 participants-			top left of page
- 2 participants gave an expected location vs. actual (desired) location--“prefer top of page, expect upper right”

## FAQs or Common Questions

- Which label most accurately depicts a place where you can find information requested most often from the agency?
- 4 out of 11 (36%)- 	Frequently Asked Questions
- 3 out of 11 (27%)-	FAQs
- Other suggestions: Frequently Requested..., Most Often Requested....
- Strong preference for “standard” labels of Frequently Asked Questions or FAQs. Note: Throughout testing, users often expressed dislike for, or confusion from, acronyms such as FAQ, FOIA, etc.

## Location

- Participants were mixed on expected location:
- 3 participants-			top left of page
- 2 participants-			bottom of page
- 3 participants-			right menu bar (1 bottom, 2 unspecified)
- 2 participants-			middle of left menu bar

## Comments

- “Doesn't like to hunt.  Uses it as a last resort.  Looking for questions/answers. On test page [EPA FAQ on Radon]...a lot on one page.”
- “Want to reach a human being. Not machine.”
- “Usually get the answers I want”
- “Would like contact info or help if you can't find what you need.
- “Does use FAQs depending on what he is looking for, if he doesn't see info on first page and he thinks it is common information.”

## No Fear Act

- Which label most accurately depicts a place where you can:
- See a summary of any grievances related to unfair treatment of a person or group on the basis of prejudice filed against an agency
- Find statistical information about any acts of reprisal against government workers who reported unfair treatment at an agency
- Suggestions for labels included: Grievances, Government Workers Disputes, Report on Grievances, Discrimination Against Government Employees, Employee Actions & Grievances, Reprisal, Discriminatory Site, Unfair Employment...

## Ranking

- Discrimination and Retaliation Complaints	2.20	2.41
- Equal Employment Opportunity Complaints	2.44	2.82
- Equal Employment Opportunity Data		3.25	4.45
- Employee Complaint Data			3.56	3.73
- Federal Employee Antidiscrimination		4.29	4.64
-  	and  Retaliation Act
- EEO Data					4.75	5.05
- Antidiscrimination & Whistleblower		4.88	4.91
- 	Protection Reporting

## Location

- Top locations selected on generic page layout:
- 5 of 11 (45%) of participants- 	look for it at bottom of the page, (no predominance for left/right/center).
- 4 of 11 (36%) of participants- 	look for it in top nav bar, (2 left side, 2 right side)

## Comments

- “Didn't like Federal Employee Antidiscrimination & Retaliation Act and Antidiscrimination & Whistleblower Protection Reporting because they were "very wordy"”
- “Employee Complaint Data is too generic and vague, not very familiar with EEO or EEOC”

## Web Site Policies and Important Notices

- Card Sort Items
- Privacy & Security Statement
- Freedom of Information Act (FOIA)
- Information Quality Guidelines
- Portable Document Format (PDF)
- Accessibility Policy
- Linking to Outside Websites
- Disclaimer of Endorsement
- Copyright Information
- No Fear Act
- Photo Credits
- Schedule for Publishing Information
- Nondiscrimination Statement
- Legal Notices
- Free Commercial Downloads

- Participant Suggestions
- Policies
- Legal
- Overall Info
- Site Info
- Additional Info
- Information Guidelines
- Information Statement

## Card Sort Results

- Users had a lot of difficulty creating one group that covered all topics. Some users suggested two groupings of “legal” and “information”.
- The following content items were singled out by multiple (3+) users as either irrelevant or confusing with relationship to grouping the other items: PDF, Linking To Outside Websites, Free Commercial Downloads, and Photo Credits.
- Participant comment: “stuff you don’t even know what it is. PDF? Photo Credits?”

## Ranking

- 						Unadjusted	Adjusted
- Notices and Disclaimers	1.50		1.55
- Site Notices and Links	2.44		2.50
- Site Policies and Links	2.71		2.85
- Important Notices		3.00		3.09
- Note: 54% of the participants ranked “Important Notices” at or near the bottom.

## Location

- Whatever we call it, users expect to find it at the bottom of the page:
- 7 of 11 (63%) of participants- 	look for it at bottom of the page (0 left side, 2 right side, 1 center, 4 not specified)
- No other location got more than 1 suggestion.

## Comments

- “Important Notices (this doesn't say anything)”
- “Does not know what "Accessibility Policy" is.  Accessibility does not fit.”
- “Keep policies/legal info separate from free stuff .”
- “Card Sort: PDF, Linking to Outside Websites, Free Commercial Downloads -these three don't fit with others”
- “Free Commercial Downloads - What does that mean! Nothing to do with govt.”
- “"I like policies more than notices. " "They are legal policies not all are site policies." Why is "Photo Credits" here.”
- “No to [suggested label] Important Notices (Not important)”

## What testing could be done next

- Two tests were already designed by the testing subcommittee but were dropped from this round of testing because of time constraints
- Label, location and expected content for “Jobs”, “Privacy”, “Site Index” and “Site Map”.
- Best practices for Linking to Outside Websites.

## Possible additional tests

- Label, location and content for “Information Quality Guidelines”
- Appropriate content and best practices for cross-government portal implementation
- Plain language (writing content for the web)
- Limited English Proficiency  (translation services / cross-language content delivery)