National STEM Consortium Evaluation. Final Report
Author: Stewart, Sarah
Date: 2015-09-29
Type: Reports - Research
Subjects: STEM Education,Program Evaluation,Consortia,Community Colleges,Certification,Mixed Methods Research,Data Collection,Data Analysis,Grants,Grantsmanship,Partnerships in Education,Educational Policy,Educational Finance,Curriculum Development,Remedial Programs,School Business Relationship,Models,Success,School Holding Power,Academic Persistence,Enrollment,Gender Differences,Education Work Relationship,Employment Potential,Outcomes of Education,Sustainability,Questionnaires,Interviews

Acting as the lead agency for the &quot;National STEM Consortium&quot; (NSC), Anne Arundel Community College (AACC) engaged Hezel Associates to provide an independent program and impact evaluation of the U.S. Department of Labor (USDOL)-funded STEM certificate initiative. This report is comprehensive and covers the findings from all 4 years of the grant. The 4 years of the mixed method evaluation included a variety of data collection activities, including annual interviews with NSC members, advisory board members, employer partners, workforce agency partners, and current and past NSC students; and student questionnaires. Program documents were also reviewed, as well as extant student data. Data analyses consisted of qualitative (using a preordinate scheme) and quantitative (descriptive statistics) methods. Findings generated from the analyses provide the basis for conclusions and recommendations for program improvement throughout the project. The final year's evaluation activities included project staff interviews, employer interviews, a student questionnaire, document review, and analysis of extant student data. High-level evaluation findings for the NSC grant project include the following: (1) Project staff and the management team successfully completed all grant strategies and activities outlined in their work plan; (2) Throughout the grant period, the management team provided necessary policy and fiscal guidance, as well as frequent and useful communication to the project coordinators, making for a well-managed grant; (3) Technical track curricula were completed as a collaborative effort and are publicly available online; (4) STEM Bridge has been popular outside of the NSC, with thousands of users from nonNSC colleges, high schools, and middles schools; (5) The NSC grant has helped to establish new relationships between the colleges and local and regional employers, as well as to strengthen existing relationships; (6) The NSC education model, which included block scheduling, cohort enrollment, compressed classroom time, employer linkages, hybrid delivery, one-on-one advising with a navigator, and contextualized remediation and refreshers through STEM Bridge, appears to have fostered student success in terms of retention and completion; (7) Enrollment targets were met, while the number of completions was slightly lower than the target; (8) NSC female participants were less likely than male participants to complete their technical track within the expected timeframe; (9) NSC students who completed their programs have been successful in finding relevant employment; and (10) The staff involved with the NSC have created curricula and implemented an educational model that advances student success; however, the future of the consortium is not clear. Informal relationships will likely continue, but a formal structure has not been established. This final report describes findings from all data collection activities and analysis throughout the grant, with an emphasis on previously unreported data from Year 4. Conclusions and recommendations for moving forward are also included. An appendix details the Year 4 Instruments: Document Review Framework; Student Questionnaire; Consortium Member Interview Protocol; and Employer/Industry Stakeholder Interview Protocol.