CHAPTER I: INTRODUCTION Introduction to the Problem Centers for Teaching and Learning (CTLs) are a place where faculty and instructors seek support through professional development (PD) programing to promote continued improvement within teaching and learning practices. This topic is particularly pertinent when examining the importance of teaching and learning within higher education today. American society has placed increased emphasis on student learning, student success, and its effects on our society at the secondary level, but how or where should instructor training be supported in higher education (HE)? More importantly, how do CTLs measure PD transfer so that they can provide additional proof of supporting the educational mission of a university? Evaluations are necessary, however, to account for the direct and indirect effects of educational development activities on the quality of teaching provided [additional assessment measures need to be explored]. Furthermore, they are necessary to legitimize the presence of a TLC [or CTL] in an institution, particularly in the context of budget cuts. We [PD leaders] are expected to demonstrate the relevance of our activities while ensuring that they meet the needs of the university community. (Belanger, Belisle, and Bernatchez, 2011, p. 132) This study, thus, seeks to measure a strategically designed, reflectively driven PD curriculum instead of weighing “the sum of its parts.” Aristotle was correct in saying, “the whole is great than the sum of its parts;” the measurement of individual elements of PD programs might not provide an adequate measure of the impact of a CTL. Background of the Study Accrediting bodies, administration, and participating faculty all acknowledge the need for CTLs, but the impact of programing as translated to student learning and achievement is often an elusive measure. The complexities of classroom environments
How do the change scores differ between completers and FTA How do the change scores compare, based on the two factors of (a) completion – completers and FTA non-participants and (b) academic discipline – STEM and non-STEM? How do the change scores compare, based on the two factors of (a) completion – completers and FTA non-participants and (b) initial t score distribution placement – (1) higher, (2) similar, and (3) lower. Hypotheses Faculty completers will yield a greater average change score than FTA The FTA curriculum yields the greatest impact for improvement within teaching and learning. The difference in change scores will yield a greater average for completers within STEM fields than in the remaining three groups: (1) completers within non-STEM fields, (2) FTA non-participants within STEM fields, and (3) FTA non-participants within non-STEM fields. The difference in change scores will the greatest average for completers in the lower initial placement than the remaining five groups: (1) completers in the similar initial placement, (2) completers in the higher initial placement, (3) FTA non-participants in lower initial placement, (4) FTA non-participants in similar initial placement, and (5) FTA non-participants in higher initial placement. A lower initial t score distribution placement provides the greatest opportunity for change scores because there is room for more growth on a finite scale. participants.
Teaching Portfolios The teaching portfolio, also discussed in Chapter II, was introduced in teaching autobiographies and the teaching portfolio. These reflective portfolios represent a learning product, a binder containing: (a) products of good teaching; (b) materials from oneself; and (c) information from others. These products represent the participant’s artifacts of learning that demonstrate reflection on and redesigning of content the next time the course is taught. The portfolios for each FTA completer was collected and preserved for his or her yearly employee reviews and could be added to future submitted promotion and tenure (P&T) documents. The outcome associated with the teaching portfolio was to provide the beginning practice of creating a personalized professional development plan. As for the FLC, active participation requires engagement in small group learning communities that share the goal of solving common teaching and learning problems within a supportive and collaborative environment. Each small group was also required to produce an artifact of learning, which was either presented publically or institutionally archived for collegial future reference. These two elements are the key reflective practice elements of the FTA completion process. “Change Measurement of Independent Variables The summative IDEA score used to compare PD transfer was labeled the Adjusted Summary Evaluation within the Converted Average When Compared to All Classes in the IDEA Database (Appendix C). This score was chosen for comparison purposes because it provides the best control data for instructor comparison across all disciplines (Hoyt & Lee, 2002, p. 36). It is important to clarify that the Converted Average begins as an adjusted measure on the 5-point IDEA range (scale) and is then converted to an adjusted
STEM courses 32 Non-STEM 42 Lower – initial score 12 Similar – initial score 36 Higher – initial score 26 Table 4.2. Descriptive Statistics for Mean Values, N=74 Min Max M SD Avg. Pre. 29 67 51.78 7.79 Avg. Post 30 66 53.15 7.04 Avg. Diff. -18 23 1.36 7.87 Results: Research Question One Research question one queries, “How do the change scores differ between completers and FTA non-participants?” Based on the results of a t-test comparing mean change scores, those who completed had significantly higher improvements than the FTA non-participants (t(72) = -2.83, p > 0.01); equal variance was assumed. Table 4.3. Average Change Scores by Completion Status, N=74 Completion Status N M SD Completers 37 3.84 7.99 Non-participants 37 -1.11 7.00 The mean change scores for completers’ ratings (FTA faculty, Table 4.3.) indicate an increase after completing the training; while FTA non-participants’ ratings change scores indicate a decrease. These results support hypothesis 1: “Faculty completers will yield a greater average change score than FTA non-participants. The FTA curriculum yields the greatest impact for improvement within teaching and learning.” In other words, student
ratings of instruction reveal an improvement for faculty who reflect on their approach to instruction through the completion of the FTA curriculum. Results: Research Question Two Research question two queries, “How do the change scores compare, on the two factors of (a) completion – completers and FTA non-participants and (b) discipline – STEM and non-STEM?” A Two Way ANOVA was used to evaluate the effects of completion status and academic disciplines on change scores, as well as the effect of the interaction between the two. The effect of completions was significant (F(1) = 8.61, p = 0.005), though the effects of both discipline (STEM vs. non-STEM) and the interaction were not significant (F(1) = 0.98, p = 0.33 and F(1) = 0.93, p = 0.34, respectively). Table 4.4. Average Change Scores by Completion Status and Academic Discipline Completion Status Academic Discipline N M SD Completers non-STEM 21 3.86 8.49 Completers STEM 16 3.81 7.57 Non-participants non-STEM 21 .38 7.06 Non-participants STEM 16 -3.06 6.64 Table 4.4. shows that the greatest mean value loss belongs to the FTA who belong within the STEM discipline. Minimal gains for FTA within non-STEM academic disciplines were also found. Overall, the completers within the non-STEM and STEM academic disciplines both showed almost four point gains in their student ratings over time. These results give only partial support for hypothesis 2, that “The difference in change scores will yield a greater average for completers within STEM fields than in the remaining three groups: (1) completers within non-STEM fields, (2) FTA non-participants within STEM fields, and (3) FTA within non-STEM fields.”
Results: Research Question Three Research question three queries “How do the change scores compare, based on the two factors of (a) completion – completers and FTA non-participants and (b) initial t score distribution placement – (1) higher, (2) similar, and (3) lower.” An ANOVA was used again for these analyses. Table 4.5. Average Change Scores by Completion Status and Initial Summary Evaluation Scores Completion Status Initial Sum. Eval. Score N M SD Completers Higher 12 -1.67 5.61 Completers Similar 15 2.87 5.28 Completers Lower 10 11.90 7.71 Non-participants Higher 14 -1.86 4.98 Non-participants Similar 21 -1.43 7.83 Non-participants Lower 2 7.50 7.78 4.5. illustrates that the greatest average gains belong to completers who begin with initial summary evaluation scores in the lower range. This increase was followed by FTA non-participants who also began with initial summary evaluation scores in the lower range; however, the sample size (N=2) was less than desirable for this group. The only other reported gains belong to completers who begin with initial summary evaluation scores in the similar range. Groups with the greatest gains begin with initial summary evaluation scores in the lower range, which is sensible since these groups have the most room for growth. The lack of improvement – and even small decreases in scores – for those in the higher group is not also completely surprising, given the lack of room for improvement and the potential for regression toward the mean. Completion status was not statistically significant (F(1) = 2.15, p = 0.15) in its effects on change scores. In contrast, significant statistical difference was discovered in
May). Federal, science, technology, engineering, and mathematics (STEM) education 5-year strategic plan. Retrieved from . Cox, M. D. (2013). What is a faculty and professional learning community? Miami University: Oxford, OH. Retrieved from . Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Boston: Pearson. Davis, B. G. (2009). Tools for teaching, (2nd ed.). San Francisco: Jossey-Bass. Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process (1910), revised edition. New York: D.C. Heath. Di Stefano, G., Gino, F., Pisano, G., & Staats, B. (2014). Learning by thinking: How reflection aids performance (Working Paper No. 14-093). Retrieved from Harvard Business School . Feldman, K.A. (2007). Identifying exemplary teachers and teaching: Evidence from student ratings. In R.P. Perry & J.C. Smart (Eds.), The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective (pp. 93-129). Dordrecht, The Netherlands: Springer. Freire, P. (1993). Pedagogy of the oppressed. New rev. 20th-Anniversary ed., New York: Continuum. Gooblar, D. (2015, April 1). Getting students thinking about thinking. Chronicle of