evident —one researcher with an emphasis in teaching and assessment and the other with an emphasis in district-level administration and strategic reform initiatives. To identify both common and contrasting interpretations, this section will draw from the independent coding, analysis, and theoretical development that each of the two researchers conducted. Contrasts in independent coding Each of the two researchers began from the same coding procedures to produce a unique set of codes (see Figure 1) in order to arrive at distinct coding development, theory, and analysis. Researcher A created a positive versus negative coding structure related to teacher evaluation. Many of the codes could have been negatively or positively coded depending on the framework (e.g. discord, control issues, inequality, and time consuming versus goal-setting). This categorization analyzed effect within themes and distinguished between positive, negative, or neutral association with a summary of focused codes within themes. Researcher A isolated some codes as neutral (neither positive or negative) and context dependent neutral (could be positive or negative depending on the context within the interview). For Researcher A, some codes were labeled as one category (i.e. negative); yet, during analysis, recoded as another category (i.e. neutral) dependent upon the interviewee’s context. Researcher A generated the two themes of power and teacher support; a third unidentified theme categorized context dependent codes, not directly associated with the other two. For example, the code control issues could have been negatively labeled with an interviewee who is a district administrator yet neutral with a non-district administrator; the district administrator may provoke the control issues while a non-district administrator may be constrained by the control issues. <Figure 1 here>
References Armstrong, D., Gosling, A., Weinman, J., & Marteau, T. (1997). The place of inter-rater reliability in qualitative research: An empirical study. Sociology, 31:3, 597-606. Babbie, E. (2007). The practice of social research. Belmont, CA: Thomson Wadsworth. Baker, B. D., Oluwole, J. O., & Green III, P. C. (2013). The Legal Consequences of Mandating High Stakes Decisions Based on Low Quality Information: Teacher Evaluation in the Race-to-the-Top Era. Education Policy Analysis Archives, 21(5), 1-64. Burawoy, M., (1998). The extended case method. Sociological Theory, 16(1), 5-33. Charmaz, (2014). Constructing Grounded Theory (Introducing Qualitative Methods series). California: SAGE. Danielson, C. (2001). New trends in teacher evaluation. Educational leadership, 58(5), 12-15. Boston, MA: Allyn & Bacon. Davey, J. W., Gugiu, P. C. & Coryn, C. L. S. (2010). Quantitative methods for estimating the reliability of qualitative data. Journal of MultiDisciplinary Evaluation, 6(13), 140-162. Friese, D. S. (2013). ATLAS.ti 7. GmbH, Berlin: Scientific Software Development. Glaser B. & Strauss, A. (1967) The discovery of Grounded Theory: Strategies for qualitative research. Chicago: Aldine. Grayson, K. (2001). Interrater reliability. Journal of Consumer Psychology, 10 (1&2), Mahwah, NJ: Lawrence Erlbaum Associates, 71-73. Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin; Psychological Bulletin, 112(3), 527. Landis, J. R., & Koch, G. C. (1977). The measurement of observer agreement for categorical data. Biometrics, 33,159-174.
Table 1. Main theories developed by each researcher. Overall theory Researcher A theory Researcher B theory Support for teachers within teacher evaluation Support for teachers, school, and students should be made a priority when developing new (or updating existing) policies. Purposeful partnering depends on leadership levers to set the stage for trust and ownership that are pivotal to make policy implementation meaningful. Importance of teacher evaluation Teacher evaluation is critical in training, assessment, and teaching and learning of students; therefore, the importance placed on the implementation and development of teacher evaluation should be made a priority. Teacher evaluation is a lever of one critical element to an intact system’s structural integrity. It is used as both a lever of professionalization for improvement and as a publicly visible demonstration of accountability. distribution within teacher evaluation When discussing power, it is important to identify strengths and weaknesses among participants to avoid an unbalanced system. Structural integrity depends on purposeful partnering to maintain a balanced fulcrum of power as well as a shared understanding and investment among stakeholders. Figure Captions Figure 1. Themes and codes developed by each researcher. Figure 2. Connections and distinctions made between the themes and codes of each researcher. Similarities in themes are italicized outside of figure or highlighted in grey inside figure. Connections between codes are identified by arrows; distinctions are underlined with dashed lines.