# Presentation: 514650

## Improving Satisfaction with www.gao.gov

**E-Government Citizen Satisfaction Summit**
**Session: Acting on Customer Survey Results**
**Sharon Dooley, GAO****September 21, 2004**

## Why Change

- Current site not so current
- Web site design was “old”
- Needed more focus on users
- GAO’s challenge:
- Serve a wide range of audiences
- Limited information about site visitors
- “Portal” to vast amount of information

**Notes:**
Technology has changed...site needed changes to take advantage of newer technologies (e.g., content/format separation)
Site needed more of a focus on user tasks (a growing web best practice: user-focused) including
changes in homepage content and design (like reduce logo, have just one; make robust for sporadic user)
Information architecture focused on user tasks (more on that later)

We knew this from work we’d done talking to users a user-centered site review, but we didn’t have a lot of good web analytics type information.  We knew that organizationally we have a very important client—the Congress, but we also knew that we have many ways of communicating with the Congress, of getting our message to them. The web site is just one of many ways. But what about others for whom our site is the only or main way they find out what we have to say? We didn’t know much about them.

So, our challenge was to redesign to better serve our multiple audiences, but we needed to know more about them.
Enter the ACSI.

## Step 1: Learn About Site Visitors

- February 2003 implemented American Customer Satisfaction Index (ACSI)
- Custom questions identify and segment site audiences
- Who they are
- What they come for
- How does satisfaction vary among them
- Quarterly benchmarks compare us to others

**Notes:**
ACSI customer satisfaction survey provides us a baseline measure for visitor satisfaction. The benchmarking capability is really what sold us on this tool. We have survey research experts at GAO. We could have designed questions and measures of satisfaction. But, those wouldn’t let us compare to any other sites. We wouldn’t have much useful context for whatever the measure is. So, we opted for the ACSI web survey tool.

Information about our audience components important for redesign
one-quarter newcomers (less satisfied),
20-25 percent researchers, 20-25 percent from universities, 20 –25 percent students, 20 percent federal executive branch agencies.
Information about prevalent audience tasks
40 percent research a topic, 40 percent find a specific product

The quarterly e-gov benchmarks showed us how satisfied our visitors were compared to visitors of other sites. The published e-gov benchmarks were an important driver in getting us to move ahead on redesigning our site. 

What did the benchmarks show us?....

## Step 2: Benchmark Current Satisfaction

- Quarterly benchmarks: we were average
- Content and site performance rank high
- First-time visitors less satisfied than others

**Notes:**
E-gov quarterly benchmarks for March 2004:
GAO scored 67 in June 2004 (66 in December 2003 benchmarks, 68 in March 2004)
Pretty steadily in the middle of the pack—not where we wanted to be, although we did well in future behaviors (likely to recommend and likely to return).  One of our four goals in GAO’s strategic plan is to “maximize the value of GAO by being a model federal agency”  and being in the middle of the pack of portal sites doesn’t quite meet that goal.

BUT, In some areas we do very well, surpassing many sites. Looking just at the period before redesign (these are fairly constant over time)
    Content ranks High: good marks for accuracy, quality and freshness of the information
   Site performance high too.
     Future behaviors well-rated

Look and feel, navigation, searching were consistently lower rated

First-time visitors are about a quarter of our visitors. Through segmentation, we know that first time visitors are consistently in the low 60s in satisfaction, compared to the high 60s for infrequent visitors and the high 60s and low 70s for frequent visitors.

So, where to start? What changes should we try to make?...

## Step 3:  Identify and Prioritize Needed Improvements

- Priority areas forimprovement
- Searching
- Navigation
- Look and feel
- Open-ended commentsgave us tips for quick hits

**Notes:**
This is the priority map that comes with the ACSI. What it does is lets you focus your efforts where you will get the most satisfaction improvement. For us—our top priorities were in navigation, search, and to a lesser extent look and feel.

Besides the priority map, we had an open-ended question asking people to tell us “anything else about your experience with the GAO web site.”
This gave us some hints on things we could do quickly that would probably help with overall satisfaction. For example,
realizing why our search engine was simply unavailable to some users.
And how some of our databases could be changed to improve responsiveness.

But navigation, search, look and feel are pretty big areas...where would we start? That is where drilling down into the components helps to further prioritize and guide actions....

## Step 4: Drill Down for Actionable Guidance

- Architecture and design changes
- Number of steps to find information
- Ability to find information
- Clarity of site organization
- First time visitors’ experience
- Short term search improvements
- Make search accessible to more users
- Improve responsiveness of databases
- Add memory
- Future search efforts to focus on narrowing results

**Notes:**
We knew from the pieces of the priority elements that information architecture and design changes were in order. And improvements in the architecture and design should improve the first time visitor’s experience as well. 
In navigation, it was the ability to find information, and, to a lesser extent, the number of steps it took to find things. 
In the look and feel, it was the clarity of site organization that was lowest rated.
From segmentation, we knew that first time visitors were less satisfied than other visitors.
In searching, the particular item of focus was the ability to narrow search results

So much to do, we adopted a phased approach. Some things we could do quickly—improving database responsiveness by writing more efficient queries, changing how the search engine was configured--had positive impacts on satisfaction. At the same time we added more memory—and that had a positive impact as well.

The architecture and design changes required more time and effort. 
Goal to focus on user tasks
highlight some existing functions.
Establish community pages for newcomers and others, gathering relevant content together.

Future search changes focus on integrating document metadata with full text searching. The goal here is to let visitors use the metadata information about the products to narrow their search results. The first step is to let the database fields influence the search results.

How’d we do? Well, here is the previous home page....and then the new home page

## Slide 7

## Slide 8

**Notes:**
Redesigned site: Major content areas in the middle, including community pages.
Spotlight section mainly for things the agency wants to highlight for people. 
Right column is other sections and services
A combination of the middle and right columns becomes persistent navigation on the lower level pages
More white space, less explanatory text

The new information architecture was a major effort that focused on what people were trying to do on the site
Make it simpler to get to our products (fewer home page categories, focus on reports and testimony)
In look and feel: Removing the frames and incorporating cascading style sheets and other newer technology for the infrastructure. We had webmaster mail and open-ended comments from the ACSI indicating that frames was a sore spot for a segment of our visitors. And we had known for some time that we needed to transition to a more efficiently supportable approach.
Improving the first-time visitor experience with
Especially for pages: putting relevant content in one place
Clearer architecture with better categorization, improved labeling

## Step 5: Assess Results

|  | SatisfactionPre-Redesign | SatisfactionPost-Redesign |
| --- | --- | --- |
| Quarterly e-gov benchmark | 67 | 71 |
| First time visitors | 64 | 72 |
| Frequent visitors | 70 | 72 |
|  | PercentagePre-Redesign | PercentagePost-Redesign |
| Find what you were looking for? | 63 | 72 |
| Not using search | 10 | 25 |

**Notes:**
We have gone from 67 in the June 2004 quarterly benchmark to 71 in the September 2004 benchmark, the most recent. (We were at 66 a year ago, in September 2003.) So, while it is still early, it appears we have improved satisfaction.

First time visitors satisfaction increased 8 points 
64 before redesign (April 30-June 16, 2004)
72 since redesign (June 29-August 11, 2004)

Frequent visitors increased from 70 to 72. Frequent visitors are more familiar with the old design and tend to be at least temporarily more dissatisfied when a site is re-launched. This doesn’t seem to be the case for GAO.

More are finding what they are looking for (another custom question): increased 9 percentage points from 63 percent to 72 percent.

More are not using search; increased 15 percentage points: from 10 percent not using any searching to 25 percent.

Given that more people are not using search and more are finding what they want, it looks like the revised information architecture is having a positive impact.

## Step 5: Assess Results (cont.)

- Top priorities are navigation and search

**Post- Redesign June 29 - August 11, 2004**

**Pre- Redesign April 30 - June 16, 2004**

**Notes:**
Look and feel increased by 5 points, no longer a priority 
Navigation and search remain top priorities
Navigation increased by 5 points, so it looks like we are on the right track there.
Search even went up 2 points and we didn’t do much yet to improve searching.

We have additional work underway in search.

And we have Phase 2 work underway in navigation.

Which leads us to Step 6

## Step 6:  Continue Identifying Improvements

- Planned enhancements to other sections of the site
- Additional “especially for” pages
- Most popular/requested feature for the homepage
- Monitor open-ended comments for other quick hits and clues

**Notes:**
Continue identifying improvements
Planning architecture improvements for legal products, From the Comptroller General, other sections that we didn’t tackle in Phase 1

Additional Especially for pages for other audience groups
Federal agencies
Researchers
Legal community

Most popular/requested feature for the homepage

## Contact Information

- Sharon DooleyDirector, Web Content and Electronic Media ServicesKnowledge ServicesUnited States Government Accountability Officedooleys@gao.gov