teacher participants. The 'Dynamic Integrated Approach' (DIA) to professional development is used both as the basis for the CPD course and as an evaluation framework. This is a complex five-stage model, which sets out teacher development as a hierarchy. The authors claim an association between teachers operating at the higher DIA levels and higher student attainment. The 'before' evaluation consisted of observation, teacher questionnaire, student tests, student questionnaire. 'After' evaluation used the same methods and measures plus a feedback meeting with teachers. Students of teachers in the higher DIA levels made greater gains than those of teachers assessed as in the lower levels. Students of teachers in the DIA group made statistically significant gains while those of a control group did not. The follow-up revealed no further improvement and no decline. In a study over 4 semesters with 74 academics and over 5000 students, Han and Finkelstein (2013) found evidence that CPD for teachers on using clicker assessment and feedback (CAF) tools influenced student perceptions of the technology. Rutz et al. (2012) identify improvement in student critical thinking as evidenced in written assignments. This mixed methods study investigates the impact of CPD (mainly unaccredited) at two different US HEIs over 3 years. Willett et al. (2014) is a detailed paper on the same study and advances a more tentative conclusion that student learning (clearly defined here with reference to writing, quantitative reasoning and critical thinking) is enhanced. Shim and Walczak (2012) also suggest a relationship between teacher practices and student development of critical thinking skills. Additionally, Willett et al. (2014) focus on the development of a methodology for assessing impact using student work and teacher assignments. The Haswell paired-comparison rubric was used to analyse student texts and staff assessment prompts before and after CPD events to establish evidence of impact. The findings indicate clear change/enhancement in academics’ practice and, as suggested above, some evidence of impact on student learning, although the latter findings are less strong. Finally, as suggested in section 1 above, Lumpe et al. (2012) found that the teachers’ self-efficacy was positively correlated to an improvement in students’ standardized science test scores. The
2.2.5.1 Peer-review (N=114) 83% reported that their institution had a policy or expectation around peer-review of teaching. This trend was found within categories of institutions as well, as shown in Table 2.7. Research-intensive (N=40) Teaching-focused (N=44) College-based (N=13) 95% 80% 77% Table 2.7: Percentage of institutions which have an existing policy/expectation around (by institutional category) 34% of respondents reported peer-review was not linked to appraisal in their institution with another 42% indicating that it was linked to appraisal only in some cases. Comparing types of institutions, Table 2.7 indicates that CBHE providers appear to be different from the rest of the sector in this regard. In FE colleges, peer-review was more consistently linked to appraisal. tests did not reveal significant associations between institutional category and whether or not peer-review was linked to annual appraisal/performance reviews. Table 2.8: Peer-review and annual appraisal/performance review (N=39) Teaching-focused (N=39) College-based (N=12) Not linked to appraisal Linked in some cases Linked in all cases Not sure 31% 51% 5% 13% 44% 33% 10% 13% 17% 42% 42% 0% 2.2.5.2 Promotion pathways The literature suggests that having separate promotion routes for teaching, research, and ‘teaching and research’ roles enhance the chances that specific teaching and learning criteria will be included in promotion policies (Cashmore et al., 2013). This, in turn, impacts the culture of CPD within the institution as CPD becomes instrumental as a means to gain qualifications and evidence engagement with teaching and learning. In the research-intensive category of institutions, a majority of the institutions had separate promotion routes for research and teaching roles, with 35% reporting a three-track and 28% a path (see Figure 2.7). A three-track pathway might include separate promotion criteria for track
Future plans Involving students with Peer Review is something that Jo Peat (Head of Academic Professional Development) would like to see developed beyond the URCert. She regards it as important that teaching staff aren’t primarily receiving their feedback from educational developers. Resources constrain development in relation to the training of students, and ensuring that both students and staff are protected. 2.4.3 Case Study – Sheffield Hallam University Sheffield Hallam is the 4th largest university in the UK, with 34,718 students (25,988 undergraduates and 7,114 postgraduates). As of 2013/14, staff totalled 4,494, including 2,114 academic staff and income generated was £257.1m. The University has four teaching faculties – including 18 academic departments - based across two campuses (City and Collegiate). The University encourages work-based learning, with over half of all courses offering integrated practice or work placement opportunities. Approach to CPD for academic staff The approach to CPD for academic staff at Sheffield Hallam University draws upon a peer enhancement strategy informed by the following principles: focussed on teaching and learning practice; authentic; facilitated by using expertise to scaffold learning and application; peer supported; team based; evidence based; and generates reflection and feedforward. The key components of appraisal, CPD and Peer Review and Enhancement continue to be central to this strategy. “Measuring the impact of this work on the staff and student experience remains a challenge, although we routinely reflect on our approach, trial new ways of measuring what we do and adapt our practice in light of experience and feedback” Jackie Cawkwell, CPD Manager
in understanding issues around transfer to practice (De Ridjt et al., 2013). The toolkit could therefore usefully provide advice about what sorts of data are appropriate to use for different audiences. Table 3.6: The intended uses of the evaluation data Use of evaluation data N Informing future CPD including development of metrics 5 Reported to senior management 3 Presented at committees 2 Reported to HR 2 Reported to QAA 1 Reported to respondents 1 Seven pilots chose to use interviews as their preferred methodology to collect evaluation data from the CPD completers, with the remaining two using open questionnaires. These approaches follow trends in evaluation practice which advocate the importance of qualitative approaches to data collection and the potential to develop greater insights into the impacts of CPD through narrative accounts (Bamber, 2013; Rust, 1998). However, the intended audiences of the evaluation data are mixed (Table 3.6), and it is likely that some, such as university committees and senior managers, who may be more interested in a headline summary of satisfaction or numerical overviews of impacts. Therefore mixed method approaches should be considered in order to be able to communicate the outcomes of evaluation work in both qualitative and quantitative formats. The CPD providers identified numerous benefits following the use of the tool to evaluate their provision. They acknowledged its flexibility and the ease at which it aligned with, but also enhanced, their current evaluation practices, particularly in terms of promoting a longitudinal approach to evaluation. This was a current gap in the evaluation provision of most of the pilot institutions. ‘One of the first challenges you often have is coming up with the evaluation questions, so actually having a readymade resource that has started to categorise things into different evaluation scenarios, and starting to give you different questions for different levels of evaluation, we found that really helpful…. you’ll end up with better evaluation tool for whatever you design because some of that thinking and working through and revision have been done already..’ (Pilot 7) In line with the concerns from the CPD completers regarding the language used in the tool the CPD providers felt that where necessary there was the potential to customise the tool to ensure it was sensitive to their own institutional context. The recognition of the need for this by CPD providers is important, as it could mediate potential accessibility and language issues raised by the CPD completers.
23. What are the key aims of this CPD offer? Please tick all that apply. Teacher-focused - Develop pedagogical knowledge, teaching approaches, skills, strategies; Encourage reflective practice; Change teaching behaviour Student-focused - Improve students' approach to learning; Engage students' in learning; Enhance student experience Institutional-focused – Orient participants to understand and implement culture and policies regarding teaching and learning; Leadership development 24. Over what duration does this CPD offer usually run? 1 day or less Less than a month Between 1 - 6 months Between 7 - 12 months More than 12 months 25. How is this CPD offer primarily delivered? Face to face Flexible/on-line Both (blended learning) 26. Who is involved in the delivery and support of this CPD offer? Please tick all that apply Staff dedicated to providing teaching-related CPD Academic staff within faculties Professional service staff (student support, librarians, learning technologists) Previous participants
CPD Activity 1: Evaluation questions for participants of accredited teaching courses for new lecturers CPD Activity Timeframe for evaluation Prior to the CPD activity During CPD activity Immediately after the CPD activity 6 months after the CPD activity 12-24 months Motivation/Reaction Why have you chosen to undertake the accredited teaching course? What do you expect to gain from attending this course? How were these expectations identified, discussed and recorded? How have your expectations of the course changed since you started? How have your individual aims been addressed and accommodated in the course? What would you like to change about the course? How did your experience of undertaking the accredited teaching course align with your expectations? What three things from the accredited teaching course which were most useful to you? What recommendations to you have to improve future courses? What do you remember most about the accredited teaching course and why? What do you remember most about the accredited teaching course and why? Changes in conceptions of teaching & Learning How long have you been in a teaching and / or supporting learning role within HE? In what ways may you have you previously engaged with pedagogic What have you learnt so far? In what ways have you engaged with pedagogic literature and theory to What new information or ideas were you introduced to? What have you done as a result of completing the accredited teaching course? Has the process of undergoing the accredited teaching course led to How has your thinking on [XXXX] shifted following the course? What impact did attending the accredited teaching
outcomes have on your approach to student learning? student learning? have on your approach to student learning? accredited teaching course impacted on students? How can you evidence this impact? accredited teaching course impacted on students learning? How can you evidence this impact? Impacts on institutional culture How did you find out about this course? Are you expected to attend teaching related CPD as part of your academic development? How well do the aims of this course align with departmental priorities? What support is currently available to you? What would assist you in completing the course? How well did the topics covered in the course align with priorities? How do you anticipate sharing information within your department? How have you been recognised and rewarded for completing the accredited teaching course? Has the process of engaging in the Accredited teaching course led to the emergence of communities of practice around teaching and learning? How has this change been supported in your department / institution? How have the topics covered in the course contributed to pursuing departmental priorities? How have you disseminated the information or ideas you were introduced to during the accredited teaching course?
CPD Activity 2: Evaluation questions for participants of an in-house teaching accreditation framework for established academics CPD Activity Timeframe for evaluation Prior to the CPD activity During CPD activity Immediately after the CPD activity 6 months after the CPD activity 12-24 months In-house teaching accreditation framework Why have you chosen to apply for accreditation? How long have you been in a teaching and / or supporting learning role within HE? How did you find out about the accreditation framework? Are you expected to obtain teaching accreditation as part of your academic development? What do you expect to gain from this process? How have your expectations of the accreditation process changed since you started? What have you learnt so far? What support is currently available to you? What would assist you in completing the accreditation process? In what ways have you engaged with pedagogic literature and theory to develop your application? How did your experience of undergoing the accreditation process align with your expectations? What new information or ideas were you introduced to whilst preparing your application for accreditation? What impact do you think undergoing the accreditation process will have on your knowledge, skills or practice? In what ways have you engaged with pedagogic literature and theory to develop your application? What have you done as a result of undergoing the accreditation process? How have you applied the knowledge, skills and ideas reflected on through the accreditation process? What have been the impacts of undergoing accreditation on your school / department? How have you been recognised and rewarded for undergoing accreditation? How has undergoing accreditation impacted on What do you remember most about the accreditation process and why? What impact did undergoing accreditation have on your knowledge, skills or practice? How has your thinking on [XXXX] shifted following accreditation? What, if anything, have you changed as a result of reflecting on your experiences of teaching and/or supporting learning?
How important do you think it is that staff are accredited to teach in HE? What is your level of understanding of the UKPSF? What impact do you think gaining accreditation will have on your knowledge, skills or practice? What impact do you think gaining accreditation will have on your career advancement and employability? What role, if any, have students from your In what ways has reflecting on your experience had an impact on your practice? In what ways has reflecting on your experience had an impact on your ideas about teaching, learning and the student experience? What support did you access to undergo the accreditation process? How do you anticipate sharing information gained through the accreditation process? How has the process helped you to support other staff? What do you think the benefits will be for students following your engagement the accreditation process? How has undergoing the accreditation process been students? Has the process of undergoing accreditation led to new areas of T&L activity / research? Has the process of undergoing accreditation led to the emergence of communities of practice around teaching and learning? How has this change been supported in your department / institution? What challenges have you experienced as you undertook these changes? How do these changes align with departmental priorities? How has the accreditation process helped you to develop your network of teaching and learning contacts? How has you’re your engagement in the accreditation process impacted on students
CPD Activity 3: Evaluation questions for participants of teaching and learning conferences CPD Activity Timeframe for evaluation Prior to the CPD activity During CPD activity Immediately after the CPD activity 6 months after the CPD activity 12-24 months T&L Conference Open questions Why have you chosen to attend this conference? How did you find out about this conference? How long have you been in a teaching and / or supporting learning role within HE? Are you expected to attend teaching related CPD events as part of your academic development? How well do the aims of this conference align with departmental priorities? N/A Open questions How did your experience of the conference align with your expectations? How well did the topics covered in this conference align with departmental priorities? What new information or ideas were you introduced to at this conference? What impact do you think attending this conference will have on your knowledge, skills or practice? What do you think the Open questions What do you remember as being the most useful session and why? topics covered in this conference contributed to pursuing departmental priorities? What, if anything, have you done in response to the information or ideas you were introduced to at this conference? What impact did attending this conference have on your knowledge, skills or practice? N/A
What do you expect to gain from attending this conference? What impact do you think attending this conference will have on your knowledge, skills or practice? What role, if any, have students from your institution had in this conference? What support have you been given to attend this conference? What expectations, if any, are there for you to disseminate conference information? benefit was for students involved in this conference? How do you anticipate sharing the conference information? What three things from the conference were most useful to you? What recommendations to you have to improve future events? How do you think attending this conference has extended your network of teaching and learning contacts by attending this conference? How have you disseminated the information or ideas you were introduced to at this conference? How has attending this conference had an impact on students? How did the workshop develop your network of teaching and learning contacts?
CPD Activity 4: Evaluation questions for participants of teaching and learning workshops CPD Activity Timeframe for evaluation Prior to the CPD activity During CPD activity Immediately after the CPD activity 6 months after the CPD activity 12-24 months T&L workshop Why have you chosen to attend this workshop? How long have you been in a teaching and / or supporting learning role within HE? How did you find out about this workshop? Are you expected to attend teaching related CPD events as part of your academic development? How well do the aims of this workshop align with departmental priorities? N/A How did your experience of the workshop align with your expectations? How well did the topics covered in this workshop align with departmental priorities? What new information or ideas were you introduced to at this workshop? What impact do you think attending this workshop will have on your knowledge, skills or practice? What do you think the benefit was for students What do you remember most about the workshop and why? What impact did attending this workshop have on your knowledge, skills or practice? How has your thinking on [XXXX] shifted following this workshop? What, if anything, have you changed in response to the information or ideas you were introduced to at this workshop? How has this change been supported in your What do you remember most about the workshop and why? What impact did attending this workshop have on your knowledge, skills or practice? How has your thinking on [XXXX] shifted following this workshop? What, if anything, have you changed in response to the information or ideas you were introduced to at this workshop? How has this change been supported in your
What do you expect to gain from attending this workshop? What impact do you think attending this workshop will have on your knowledge, skills or practice? What role, if any, have students from your institution had in this workshop? What support have you been given to attend this workshop? What expectations, if any, are there for you to disseminate workshop involved in this workshop? How do you anticipate sharing the workshop information? What three things from the workshop were most useful to you? 1 What recommendations to you have to improve future workshops? department / institution? What challenges have you experienced as you undertook these changes? How have the topics covered in this workshop contributed to pursuing departmental priorities? How have you disseminated the information or ideas you were introduced to at this workshop? How did the workshop develop your network of teaching and learning contacts? department / institution? What challenges have you experienced as you undertook these changes? How have the topics covered in this workshop contributed to pursuing departmental priorities? How have you disseminated the information or ideas you were introduced to at this workshop? How did the workshop develop your network of teaching and learning contacts?
CPD Activity 5: Evaluation questions for participants of peer review activity CPD Activity Timeframe for evaluation Prior to the CPD activity During CPD activity Immediately after the CPD activity 6 months after the CPD activity 12-24 months Peer Review If a previous review has taken place, what, if any, goals were set at the end of the previous review? To what extent have your previous goals been achieved? What is the focus of this review? What materials are available as a focus for reflection and discussion? What are your aims or intentions in this review, or (for example) using technology? Are there any specific skills or What, if anything, have you learned: About your students? About your teaching or your role as a teacher? About teaching and learning more generally? Has the review process resulted in any changes in your thinking? What changes will you make as a result of the review process? How confident are you that you can address the concerns that you raised at the start? Has the review process raised How has your thinking changed as a result of the review process? Have you made any changes to your teaching approach or methods? How have students responded? How do you know? How confident are you that you can address the concerns you raised during the review process? What did you gain from the review? Is your answer to this the same as it was immediately afterwards? Assuming this is a cyclical process, probably in annual cycles, return to the “Before” questions
CPD Activity 6: Evaluation questions for recipients of teaching and learning development grants CPD Activity Timeframe for evaluation Prior to the CPD activity During CPD activity Immediately after the CPD activity 6 months after the CPD activity 12-24 months T&L Development grants Before the start of a project but after funding notification received; therefore we assume SOTL / teaching development grants are tied to funding bestowed institutionally. This will make it easier to track the aims of the funding scheme with the impact on practice etc. We draw on the work of Admunson & Hum (2012: 4) to define the scope of work undertaken by teaching development grants / projects as those that result in ‘systematic inquiry’ into an aspect of teaching practice, which may results in changes in ‘conceptions of self as a teacher and researcher, changing understanding about How long have you been in a teaching and / or supporting learning role within HE? Why did you submit an application to undertake the proposed project? What was the rationale for undertaking the proposed project? What informed the development of this proposal? How well do the aims of this project align with departmental / Is the project on course? How have your expectations of the project changed? What challenges have you encountered? What successes have you experienced? If relevant, how have students been involved in the project? What professional development How did your experience of the project align with your expectations? How well did the project align with departmental / institutional priorities? What new information or ideas did you gain as a consequence of undertaking this project? What knowledge, experience or skills of pedagogic research and development work do you feel you gained as a consequence of this project? What knowledge, experience IV question successfully used to open dialogue Initially we would like you to reflect on your experience of undertaking pedagogic research and development work through the teaching fellowship scheme. How have you applied or used the knowledge gained through the project since its completion? What were the impacts on your professional development of undertaking this project? Have you applied for funding to continue with this work? How have you built on the findings of this project? Have the findings from this project led to new areas of T&L activity / research? Networking Impacts on practice
student learning and the application of project findings to teaching practice’. This definition aligns with the model of SoTL advocated by Trigwell and Shale (2004) where they include student learning as an outcome that can be documented. institutional priorities? What do you expect to gain from undertaking this project? What impact do you think undertaking this work will have on your knowledge, skills or practice? What do you anticipate the impact of the proposed project will be? What experience do you have of pedagogical research and / or teaching development work? opportunities have you identified that would assist you in undertaking this work? Are you more aware of scholarship and pedagogic literature as a consequence of this project? or skills of teaching practice and student learning do you feel you gained as a consequence of this project? What recommendations for teaching, learning and the student experience resulted from this work and is it possible to implement them? What do you think the benefit was for students involved in this project? How do you plan to disseminate the outcomes of this work? Who supported you in completing this work? What have been the responses of colleagues to the outcomes of the work? What have been the impacts of this work on your practice, your school, and where relevant more widely? If relevant, what have you undertaken since that has followed on from or built on the work that was undertaken through this project? How has undertaking this project had an impact on students?
CPD Activity 7: Evaluation questions for participants involved in teaching and learning mentoring CPD Activity Timeframe for evaluation Prior to the CPD activity During CPD activity Immediately after the CPD activity 6 months after the CPD activity 12-24 months T&L Mentoring Questions here assume mentoring will focus on developing an individual's academic career as a whole, not just their teaching. In some institutions, teaching may not be considered the most important area in which to support development. How long have you been in a teaching and / or supporting learning role within HE? Are there any targets that you are expected to reach in the next 12-24 months? What are your own goals in relation to: Your teaching? Your academic career? Are there any milestones you hope to reach in the next 12-24 months (e.g. completing a probationary period)? What do you hope to achieve in the next 12-24 Has anything in your practice changed as a result of discussion with your mentor? Do you feel any more able to address your main concerns? Are any new issues or concerns being raised? Has there been any change in what you hope to gain from working with your mentor? What, if anything, have you learned: About your students? About your teaching or your academic identity? About teaching and learning generally? Has the mentoring process resulted in any changes in your thinking? Do you think you will make any changes to the way you approach your teaching as a result of mentoring? (Yes)(No). If Yes, what do you intend to change? Is the mentoring relationship continuing? Has your thinking about your teaching changed as a result of the mentoring process? Have you made any changes to your teaching approach or methods? How have students responded? How do you know? Has there been any other effect of the mentoring process? Is the mentoring relationship continuing? Has your thinking about your teaching changed as a result of the mentoring process? Have you made any changes to your teaching approach or methods? How have students responded? How do you know? Has there been any other effect of the mentoring process?