# Presentation: 844791

## Web Sites That Work!!!

- Carlynn Thompson
- Director, Research Development and Acquisition Information Support
- Defense Technical Information Center
- April 17, 2001

- The Defense Technical Information Center (DTIC), an element of the Defense Information Systems Agency (DISA), is the central Department of Defense facility for providing access to and facilitating the exchange of mission-related Defense information, and a primary provider of Information Technology support to upper-echelon DoD component organizations.

**Web Sites That Work!!!**
**The Role of Evaluation at DTIC**

## Web Sites that Work 

- Reason for being -- clear mission -- focus
- Defined User community -- public, domain...
- Senior management commitment
- Defined decision makers/focal point
- Dedicated staff
- Competent technical support
- Content - logical organization/searchable
- Consistent look and feel
- Adherence to standards
- Metrics

## DTIC WWW Communities

-  DoD
-  Intel Community
-  US Government
-  Contractors
-  Education
-  Media
-  US Public
-  International

**User Communities    Weekly Activity**

- January 2001

-  8,221,347  Accesses
-  392,140  Unique visitors
-  458,155  MB data delivered

-  8 Separate Web Servers
-  90 Primary Web Sites
-  7 X 24 Operation
-  Public Access
-  Limited Access

**DTIC WWW Communities**

**Notes:**
Weekly statistics reflects the activity of  7 of DTIC’s major  websites.

## DTIC WWW Services

-  Dynamic Web Pages
-  Libraries
-  Comment Forms
-  Collaborative Tools
-  Document Review &
-    Coordination
-  Multimedia (live
-    broadcasts, audio, video)
-  Full Text Search
-  Relational Databases

** ****Services Provided           Example Sites**

-  DefenseLINK
-  JCSLink
-  Air Force Link
-  GulfLINK
-  S&T InfoWeb
-  Early Bird
-  Critical Infrastructure
-  Comptroller
-  Personnel & Readiness
-  PerDiem
-  ArmyLINK

**DTIC WWW Services**

## Partnerships for 

** **Information content
-  Communities of interest

**Partnerships for **
**Innovative Information Technologies**

** ****DTIC Web Support Team**
** **Human/Systems Interface
-  Graphic Design and Layout
-  Integration with client environment
-  Site Design
-  Automated Processes
-  Search Tools
-  Reuse and Shared Resources
-  Technical Advice and Guidance
-  Leverage infrastructure investments

## Resources

**Web Usage**

**Web Costs**

**DTIC Staffing**

**DTIC Funding**

**Resources**

**Notes:**
Should try to update two left charts.

## Web Site Management Practical Issues and Concerns

- Network Bandwidth
- Systems Availability
- External Evaluations
- Direct Customer Feedback
- Web Server Statistics
- Privacy of Users
- Security, security, security...
- What Next?

**Web Site Management ****Practical Issues and Concerns**

## Monitoring Network Response

- Compare response times to other sites
- Diagnose performance problems
- Transient or permanent
- Localize problems
- server
- access provider
- users' geographic locations
- Measuring Quality of Service
- Performance and availability
- Commercial Product

**KEYNOTE**

## Web Site Performance by Metro Area

## Monitoring Network Bandwidth

- Bits Per Second

- Web Spiders at work!

-     T1  =  1,600 K
- 15Mb =15,000 K

**Notes:**
Could update but could also show as past performance.  Could provide input on decision to upgrade and why.

## Monitoring Systems Availability

- Big Brother
- Monitoring and Notification for Systems and Networks

- Web Display
- Email and Pager Notification

**Monitoring Systems Availability**

## External Heuristic Evaluation

-  Match between system and the real world
-  User control and freedom
-  Consistency and standards
-  Error prevention
-  Recognition rather than recall
-  Flexibility and efficiency of use
-  Aesthetic and minimalist design
-  Help users recognize, diagnose, and recover
-  Help and documentation

**Nielsen guidelines for interface usability:**

**External Heuristic Evaluation**

**Goal is to cite violations of usability guidelines**

**Notes:**
Visibility of system status:  The system should always keep users informed about what is going on, through appropriate feedback within reasonable time. 
Match between system and the real world:  The system should speak the users’ language, with words, phrases, and concepts familiar to the user, rather than system-oriented terms.  Follow real-world conventions, making information appear in a natural and logical order. 
User control and freedom:  Users often choose system functions by mistake and will need a clearly marked “emergency exit” to leave the unwanted state without having to go through and extended dialogue.  Support undo and redo. 
Consistency and standards:  Users should not have to wonder whether different words, situations, or actions mean the same thing.  Follow platform conventions. 
Error prevention:  Even better than good error messages is a careful design which prevents a problem from occurring in the first place. 
Recognition rather than recall:  Make objects, actions, and options visible.  The user should not have to remember information from one part of the dialogue to another.  Instructions for use of the system should be visible or easily retrievable whenever appropriate. 
Flexibility and efficiency of use:   Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user to such an extent that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions. 
Aesthetic and minimalist design:  Dialogues should not contain information which is irrelevant or rarely needed.  Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. 
Help users recognize, diagnose, and recover from errors:  Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution. 
Help and documentation:  Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation.  Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large.

## Information did not flow in any obvious order but seemed scattered.
There was an excessive use of animation throughout the site. 
The addition of color may make the page more aesthetically pleasing.
Icons did not always represent same action. 
Navigation skipped levels in a hierarchy returning the user to the home page for no obvious reason.

- There was an excessive use of animation throughout the site.
- The addition of color may make the page more aesthetically pleasing.
- Icons did not always represent same action.
- Navigation skipped levels in a hierarchy returning the user to the home page for no obvious reason.

**Results:**

**OUCH!!!**

**Specific recommendations to improve site were made by the study team.**

**External Heuristic Evaluation**

## How well can users accomplish tasks site designers have in mind
Subjects completed 10 tasks
Required searching for information 
Questions representative site users interests
What time does the first walk-in tour of the Pentagon start?
How often is DefenseLINK News Updated?
What organization publishes personnel statistics for the DoD?

- Subjects completed 10 tasks
- Required searching for information
- Questions representative site users interests
- What time does the first walk-in tour of the Pentagon start?
- How often is DefenseLINK News Updated?
- What organization publishes personnel statistics for the DoD?

**External Usability Study**

## Multiple systems and browers types used
Braille reader needed by one subject
Time limits for tasks
Subjects were observed
Response times recorded
Asked to think out loud
Reasoning behind actions
Frustrations voiced

- Braille reader needed by one subject
- Time limits for tasks
- Subjects were observed
- Response times recorded
- Asked to think out loud
- Reasoning behind actions
- Frustrations voiced

**External Usability Study**

## Subjects found the search feature “frustrating” and “useless” 
unable to understand the search results 
unsure of whether the search completed was an AND search or an OR search when  more than one word was entered
dreaded performing any task requiring the search engine 

- unable to understand the search results
- unsure of whether the search completed was an **AND** search or an **OR** search when  more than one word was entered
- dreaded performing any task requiring the search engine

**Sample Feedback:**

- Time for a redesign!

**External Usability Study**

## Direct Customer Interaction

- Establish Memoranda of Understanding
- Defines responsibilities
- Web Policy, Guidelines and Best Practices
- Establishes processes
- Renew annually
- Listen to customers, users and staff
- Monitor email traffic

- Listen, listen, listen...

## Web Server Summary Statistics

- (Note: this number excludes all graphics, images, and icons files)
- AirForceLINK (Total Server Usage) was accessed from:
-           149,046 unique hosts.
-           2,350,920 (99.7%) were from external hosts.
-           7,908,540 files of all types (html, gif, jpg, etc.) were requested.
-           127,010 errors in completing user requests,
-           110,794.51 megabytes of information were passed to users.
-           140,40.7 html pages per hour.
-           336,977 html pages each day.(Average estimate)

**Web Server Summary Statistics**

## Web Server Summary Statistics

**Web Server Summary Statistics**

## Web Server Summary Statistics

- Around the clock service!

**Web Server Summary Statistics**

## Frequently Requested Pages

- Accesses by Domain

- Most Frequent Accesses by Host

## Web Server Summary Statistics

**User  Capability**

**Web Server Summary Statistics**

## Privacy and SecurityThe Balancing Act

- Privacy

- Security

**Privacy and Security****The Balancing Act**

- Citizens

- Hackers

## Taking on the Privacy Issue

**Taking on the Privacy Issue**

- DTIC met with DoD, DoJ and OMB to address privacy and web logs

- “Anecdotal evidence suggest the collection of session information poses privacy concerns to WWW site visitors.  This concern is appropriate as over time the accumulation of this data can reveal a visitor’s personal preferences and particular interests.  Release of this information could have a chilling effect on citizens’ perceptions of the role the Government plays in monitoring the distribution of Government information.”
- WWW Federal Consortium, WWW Home Page Guidelines and Best Practices, Nov. 1996

**Notes:**
Are all of these charts necessary or could they be reduced and get the point across through comment/discussion.

## Privacy and Security Notice

**Privacy and Security Notice**

## Privacy and Security Notice

**Privacy and Security Notice**

## Privacy - To Cookie or    Not to Cookie

**Session cookies **(expires when browser is shut down)
-  Session cookies are OK
**Persistent cookies **(persists to the next browser session)
-  Clear and conspicuous notice
-  Compelling need
-  Appropriate safeguards for protecting stored data
-  Approval of the agency’s top official

**Privacy - To Cookie or    Not to Cookie**

- http://cio.gov/docs/Documents.htm#security

## Architectural Overview

**Data Originators**

**DTIC Data**
**Upload Services**

**AUSPEX**
- NFS Server

- Security is a primary driver in Network architecture.
- Content delivery and access are segregated.
- Application software is segregated on separate servers.

**Front End **
**Web Servers**
**(Netscape Enterprise )**
**Web Objects Adaptor**

**Application Servers**
**Running WebObjects, Search, media & **
**Other Applications **

**Cisco Routers**

**Big IP**

**Internet**

**NIPRNET**

## Customized Information Portal

- http://www.af.mil/

## Balance service with security 
Practice risk management not risk avoidance
Ensure user privacy
Be skeptical of vendor promises
Educate Staff/Customers
Seek sufficient funding
Best Practices are changing
Establish reasonable metrics

- Practice risk management not risk avoidance
- Ensure user privacy
- Be skeptical of vendor promises
- Educate Staff/Customers
- Seek sufficient funding
- Best Practices are changing
- Establish reasonable metrics

**Lessons Learned**