Review of Web-Based Technical Documentation Processes. FY07 NAEP-QA Special Study Report. TR-08-17
Author: Gribben, Monica
Date: 2008-03-20
Type: Reports - Research
Subjects: Focus Groups,National Competency Tests,Change Strategies,Web Sites,Documentation,Layout (Publications),Intermode Differences,Computer Software Evaluation,Program Effectiveness,Program Evaluation,Research Reports,Audits (Verification),Benchmarking,Scientific and Technical Information,Academic Achievement,Academic Standards,Achievement Rating,Usability,Navigation (Information Systems),Content Analysis

Beginning with the 2000 and 2001 National Assessment of Educational Progress (NAEP) assessments, the National Center for Education Statistics (NCES) has made technical documentation available on the worldwide web at http://nces.ed.gov/nationsreportcard/tdw/. The web-based documentation is designed to be less dense and more accessible than prior printed versions. The current study reviewed the web-based version of NAEP technical documentation, compared it to prior hardcopy versions, and identified possible improvements to both the content and development processes used for the web-based documentation. The study addressed three specific questions about the new web-based version of NAEP technical documentation. The study questions were: (1) How does the new web-based technical documentation for NAEP assessments compare with previous hardcopy versions with respect to comprehensiveness and accessibility? What advantages and disadvantages may be realized via the web-based approach?; (2) To what extent have changes to the development and review processes for technical documentation led to improved timeliness?; and (3) How will updates to the web-based documentation be handled? Will changes be clear to users who may be familiar with previous versions? Panelists provided recommendations focusing on (a) the content of the documentation, (b) the development process, and (c) areas for further research. Recommendations concerning the content of the web-based technical documentation include: (1) Provide a summary of the content and develop a User's Guide; (2) Increase/emphasize search capabilities within the technical documentation; (3) Expand customization of the technical documentation; and (4) Provide data tables in exportable format (Excel). Recommendations concerning the development process include: (5) Develop a design document/plan with specifications for producing, reviewing, and publishing the technical documentation on the web; (6) Initiate a schedule/timeline for release of technical documentation; (7) Collect data to assess the efficiency of the review process; (8) Document review and revision decisions; and (9) Limit changes in standards, requirements, and formats. Recommendations for further research include: (10) Systematically collect feedback from users--via focus groups or surveys; (11) Collect information about users (using Webtrends); (12) Conduct a cost-benefit analysis of print-based and web-based technical documentation; and (13) Explore differences between page navigation models for web-based technical documentation. (Contains 2 tables.) [This work was prepared for the U.S. Department of Education National Center for Education Statistics.]