Measuring progress in entrepreneurship education

Abstract This article presents the results of a survey that ranked university entrepreneurship programs. The survey also explored how universities determined what courses constituted a program in entrepreneurship and how they determined the criteria that impact an entrepreneurship program's quality. We conclude the article with a discussion of the education pilot criteria for the Malcolm Baldrige National Quality Award that may be useful for measuring progress in entrepreneurship education. A mail survey was undertaken in late 1994. This survey was sent to deans at 941 business schools in the United States, 42 in Canada, and 270 overseas. Of the 311 replies, 233 came from U.S. business schools, 16 from Canadian schools, and 62 from schools in other countries. The top seven criteria suggested for ranking entrepreneurship programs were courses offered, faculty publications, impact on community, alumni exploits, innovations, alumni start-ups, and outreach to scholars. The most frequently offered entrepreneurship courses at both the undergraduate and graduate levels in the entrepreneurship programs surveyed were entrepreneurship or starting new firms, small business management, field projects/venture consulting, starting and running a firm, venture plan writing, and venture finance. The survey uncovered a number of problems with how academics ranked other entrepreneurship programs. Evaluators did not specify the criteria they used to rank entrepreneurship programs. Evaluators did not offer their specific weights for each criterion used to judge a program. Finally, evaluators were not asked to provide a judgment of their depth of knowledge of other programs. Since the criteria for determining what constitutes a high-quality entrepreneurship program is, at present, rather fluid and indeterminate, we thought it appropriate to borrow insights from a highly successful and visible evaluation effort in higher education, the education pilot criteria for the Malcolm Baldrige National Quality Award (MBNQA). For an MBNQA evaluation, organizations are assessed across 28 requirements that are embodied in seven categories. • Leadership . This category examines senior administrators' commitment and involvement in creating and sustaining performance excellence that has a student focus, clear goals, and high expectations. In the context of entrepreneurship education, the leadership category entails describing the involvement and commitment of entrepreneurship program directors, business school deans, university administrators, advisory board members, and student representatives. • Information and Analysis . This category examines how data and information are used to support the overall mission of the program. The focus of this category is towards identifying and specifying information and data that would be appropriate for evaluating the quality of an entrepreneurship program, as well as for making comparisons with other programs. We suggest that entrepreneurship programs begin to systematically collect information about issues such as demographic and performance measures of incoming students enrolling in entrepreneurship courses; comparative information on entrepreneurship, business school and university students; descriptions of the outcomes that specific entrepreneurship courses intend to generate and the measures of the efficacy of each course; and measures of the intended outcomes of the entrepreneurship program in terms of student performance, student satisfaction, and impact on the community (i.e., number of start-ups, students employed in new firms, students working in positions assisting new firms). • Strategic and Operational Planning . This category focuses on how a program sets strategic directions and key planning requirements. For an entrepreneurship program, such a requirement would entail generating a strategic plan that specifies the purpose and mission of the program, key student and overall program performance requirements, external factors impacting the implementation of the plan, internal resources and university barriers to change, and key critical success factors. • Human Resource Development and Management . This category examines how faculty and staff are supported and developed so as to satisfy the strategic goals of the program. While an entrepreneurship program might typically measure “faculty productivity” as an indicator of this category, the intention is actually towards specifying the resources and systems that impact the ability of staff and faculty to be productive. • Educational and Business Process Management . This category specifies key aspects of the design and delivery of the educational research and service components of a program, as well as an examination of the processes involved in improving these components. Rather than programs being compared to each other by the quantity of courses offered, this category requires that programs be measured on the logic, coherency, and efficacy of the educational experience that entrepreneurship students undertake. • School Performance Results . This category examines the outcomes of a program, such as student performance and improvement, improvement in services provided by the program, and faculty productivity. This category accounts for 23% of the total evaluation score. The primary focus of this category is determining the improvements in student performance. Such key measures might include student performance in specific courses, student demonstrations of key skills and knowledge through portfolios of original work that they create, measures of student satisfaction, and impact on the community (i.e., number of start-ups, students employed in new firms, students working in positions assisting new firms). • Student Focus, and Student and Stakeholder Satisfaction . This category describes the process for determining student and stakeholder needs and expectations, as well as making comparisons of student and stakeholder satisfaction among other programs. This category accounts for 23% of the total evaluation score. The MBNQA evaluation scheme forces us to become aware of the implicit goals, objectives, and pedagogical perspectives of our programs. We must not lose sight of the fact that entrepreneurship programs are and will be evaluated, and that we must, therefore, be ready to offer criteria that we want our programs to be evaluated on. If university entrepreneurship educators do not step forward to assume leadership of our own field, others will surely come to the forefront to determine the rules of the game.