Appraisal System for Superior Engineering Education Evaluation - Instrument Sharing and Scholarship (ASSESS)

Abstract Appraisal System for Superior Engineering Education Evaluation-Instrument Sharing and Scholarship (ASSESS)AbstractAlthough engineering educators implement many educational innovations to improve studentachievement, few evaluate the impacts of their innovations to support confident adoption of theirinnovations by others. A national panel of 30 engineering education and evaluation professionalshas called for a national resource to enable effective engineering education project evaluation.This paper reports on the process and framework for creating a library of superior evaluationinstruments that supports scholarly innovation in engineering education, referred to as theASSESS system.The overarching goal of the ASSESS project work is to create, thoroughly test, and strategicallycommunicate the system so it is positioned for successful adoption and implementation by theengineering education community. It seeks to build the evaluation capacity of the engineeringeducation community by achievement of two sub-goals: 1. Instrument utilization. Enable engineering educators to locate and effectively deploy superior evaluation instruments to enhance engineering education project discoveries and successes, and 2. Instrument development. Support the engineering education evaluation community in the identification and refinement of evaluation instruments with potential to become major assets to the profession.A team of engineering education and evaluation professionals and project consultants haveestablished a framework for evaluation instrument characterization, database structure, and webinterface that supports desired user functionality. Each candidate instrument is characterized by adescriptive summary, applications to engineering education, underlying theoretical foundations,technical specifications (e.g., reliability, validity data), and implementation features, anddirections for accessing the instrument. Published information on the instrument and its testingare referenced. Instruments are described in ways sensitive to the needs of novice users as wellas needs of evaluation experts.The evaluation instrument inventory is contained in a web-based database that supportsinformation storage, retrieval, and input. The database contains prose textual fields fordescriptive information, as well as quantitative and selected item fields for searchable data on theinstruments. Users are able to search the database for instruments of interest based on keywordsearches and selected item characteristics (content, instrument type, technical specifications, etc.)that fit their evaluation needs. The interface will also support expert review and user feedback oninstruments to facilitate instrument selection and to guide instrument improvement.Because broad utilization of the ASSESS database in required for evaluation capacity building,adoption research is being conducted to determine factors that influence the use of the database.User needs and adoption preferences will be determined and used to guide the final deploymentof the ASSESS system. Evaluation experts from the engineering education evaluationcommunity will continue to review, guide, and contribute superior instruments that are importantto the engineering education community.As the ASSESS system is deployed, the engineering education community will be able to locateand effectively implement evaluation processes that credibly evaluate impacts of engineeringeducation innovations. This will support community adoption of course materials, instructionalpractices, recruitment and retention programs, and novel ideas for future research investigations– all of which advance engineering education.