# Appendix B

**Data Collection Instruments**1



# Appendix B **- 1**  **Math Professional Development Study End**  **- of** **- Year**  **Teacher Survey**

**June 201**  **4**

**Paperwork Burden Statement** According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid Office of Management and Budget (OMB) control number. The valid OMB control number for this informat ion collection is **XXXX -** **XXXX** . The time required to complete this information collection is estimated to average 3 0 minutes per response . This information collection is voluntary. If you have any comments concerning the accuracy of the time estimate(s) or su ggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202 – 4651. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: Thomas Wei, U.S. Department of Education, Institute of Education Sciences, Room 500 H , 555 New Jersey Ave. NW, Washington, DC 2020 8 - 5500 .2



## Dear Teacher: The Math Professional Development Study (MPD) is an important national study designed to test innovative models of professional develo pment for mathematics instruction in the fourth grade. Your participation is important and appreciated, but you do have the right to skip any question that you do not wish to answer. The following are responses to some general questions concerning your participation. **What is the purpose of this survey?** The purpose of this survey is to obtain background information about the teachers within the study as well as information about their instructional contexts and professional development experiences in mathematics. **Who is conducting this survey?** The U.S. Department of Education’s Institute of Education Sciences commissioned the MPD study, and American Institutes for Research is administering the study survey. **Why should you participate in this survey?** Policymakers and educational leaders rely on findings from studies, such as the MPD study, to inform their decisions. The current study will fill a critical gap in the mathematics and professional development research base regarding what is effective for i mproving fourth

-grade mathematics achievement. **Will your responses be kept confidential?** Your responses are protected from disclosure by federal statute (P.L. 107

- 279, Title I, Part E, Sec.183). All responses that relate to or describe identifiable charac teristics of individuals may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purposes, unless otherwise compelled by law. **How will your information be reported?** The information you provide will be combined with the information provided by others in statistical reports. No individual data that link your name or e

-mail address with your responses will be included in the statistical reports.

**Thank you for your cooperation in this very important effort!**3



## Professional Development Experiences **Related to Ma** **thematics and Mathematics Teaching and Learning**  **Professional development** refers to a variety of activities intended to enhance your professional knowledge and skills, including workshops, seminars, institutes, college courses, coaching, mentoring, teacher networks, getting observed and receiving feedback on your teaching, and committee work. In this section

, you will be asked aboutprofessional development in which you participated during **summer 2013 and school year 2013**  **- 2014.**  **1. Considering ALL your professional development activities during**  **summer 2013 and the following 2013**  **- 2014 school year** **, did you participate in any professional development related to**  **mathematics or**  **mathematics t** **eaching and l** **earning ?**   Yes [Skip Pattern: respondent directed t o question 2]  No [Skip Pattern: respondent directed to question 9] 

### 2. **During summer 2013 and the following 2013**  **- 2014 school year** **, what is the total number of hours you spent in each of the following**  **mathematics or**  **mathematics t** **eaching and l** **earning professional development activities?**  For each activity, write the total number of HOURS you spent, rounding up to the nearest whole hour. Mark “ 0 ” if you participated in none. Please do NOT count the same hours in more than one row.

**Professional Development Activity Hours of**  **Professional Development**  a. Attended short, stand

- alone professional development session(s) lasting two days or less (including partial days) in mathematics or mathematics teaching and learning hours b. Attended longer professional development program(s) lasting three or more days in mathematics or mathematics teaching and learning (e.g., Intel Math, Mathematics and Science Partnership Program) hours
c. Was observed and given feedback related to mathematics and/or mathematics teaching and learning by someone with knowledge of mathematics instruction (e.g., a coach , department chair, or mentor) hours d. Attended mathematics or math ematics education course(s) for credit, such as a university course(s), that extended throughout several weeks hours e. Participated in teacher study groups, networks, or collaborations supporting professional development in mathematics (e.g., Mathematics Learning Community [MLC], Professional Learning Community) hours f . Other (e.g., participated in committees or task forces related to mathematics, observed mathematics instruction in other schools) hours4



### 3. **H ow much did the**  **professional development you received in**  **mathematics and**  **in mathematics teaching and learning**  **during summer 2013 and the following 2031 -** **2014 school year**  **emphasize each of the following topics?** Check one boxfor each row

. **Topic Not an**  **E mphasis**  **Minor**

**E mphasis**  **Moderate**

**E mphasis**  **Major**

**E mphasis**  a. Deeper understanding of mathematic s topics covered in elementary s chool (Grades K

–5)     b. Deeper understanding of mathematics topics covered in Grades 6

– 12     c . How students think about and learn mathematics (including common student difficulties)     d . How to use repres entations to convey mathematics concepts     e . How to analyze student s’ work to understand their thinking     f . Other (

please specify) :

________________________    5



### 4. **In your mathematics**  **and mathematics**  **teaching and learning professional development**  **activities during**  **summer 2013 and the following 2013 -** **2014 school year , how often did**  **you do the following activities?** Check one boxfor each row.

**Activity During Professional Development**  **Never /**  **Rarely Sometimes**  **Often Most or**  **All of the Time**  a. Solve mathematics problems     b. Share your solutions to mathematics problems with other teachers in small

- or large

-group settings    

c. Discuss mathematics topics that appear in mathematics curricula above or below the current grade you teach     d. Explore the conceptual underpinning of K

– 8 mathematics concepts    
e. Practice using a variety of representations to illustrate mathematics concepts     f. Practice analyzing student work on mathematics problems     g. Practice writing story problems for students you teach     h. Explore common student approaches to a variety of mathematics problems     i. Explore connections between mathematics topics (e.g., connections between arithmetic and algebra)    6



### 5. **To what extent did each of the following improve as a result of your participation in mathematics**  **and in**  **mathematics teaching and learning professional development**  **activities during summer 2013 and**  **the following 2013 -** **2014 school year ?**  Check one box for each row.

**Area of Improvement No**  **Improvement Improved** 

**a Little Improved** 

**a Moderate Amount**  **Improved**

**a Large Amount**  a. Your understanding of addition, subtraction, multiplication, and division     b. Your understanding of connections between addition, subtraction, multiplication, and division    
c. Your understanding of fractions and operations with fractions     d. Your understanding of algebra     e. Your understanding of connections between algebra and arithmetic     f. Your understanding of decimals     g. Your understanding of exponents     h. Your ability to see connections between multiple representations for mathematics     i. Your ability to connect different approaches to mathematics problems     j. Your understanding of common student approaches to mathematics problems     k. Your ability to analyze student work to gain insight into student understanding    7



### 6. **In your mathematics**  **and mathematics teaching and learning**  **professional development activities during**  **summer 2013 and the following 2013 -** **2014 school year , how often was**  **your experience...** Check one boxfor each row.

**Y our Experience W**  **as... Never** **/ Rarely**  **Sometimes Often**  **Most or All of the**  **Time** a. Consistent with your own goals for your professional development     b. Aligned with state or district standards and/or assessments     c. Supportive of the use of district

- adopted curricular materials     d. Supportive of your school’s improvement efforts    
e. Relevant to the mathematics you taught this year     f. Focused on material at the right level of difficulty, given your prior knowledge of mathematics and mathematics teaching     g. Logically connected from one day or session to the next     h. Clear about what you should learn from the professional development experience     i. Clear about how you could use what you learned from the professional development experience in your classroom     j. Focused on practices that you were expected to demonstrate in your classroom    8



### [Skip Pattern: If respondent added hours to **Question 2 Row c** , the respondent will be directed to answer questions 7 and 8. Otherwise, the respondent will be directed to answer question 9.] **7. To what degree did the feedback you received on your instruction following a video or**  **in -** **person observation emphasize the foll owing topics?**  Check one box in each row **Topic Not an**  **emphasis Minor** 

**emphasis Moderate** 

**emphasis Major** 

**emphasis** a. The degree to which the lesson made connections between mathematics topics and/or representations     b. The degree to which students discussed or were presented with the reason “why” a series of steps works    

c. The degree to which activity in the lesson exposed students to a variety of approaches and solution methods     d. The mathematical language you used in the lesson.     e. The degree to which the mathematics and mathematical notation you presented was correct.     f. The way in which you worked with students who had errors in their work or thinking.     

### 8. **To what extent do you agree or disagree with the following statements about the**  **observation feedback you received this year?** Check one box in each row **These experiences... Strongly**  **disagree Disagree**  **Agree Strongly**  **agree** a. Gave me opportunities to work on aspects of my teaching.     b. Provided me with useful feedback about my teaching.     c. Made me pay closer attention to particular things I was doing in the classroom.     d. Led me to think about one or more aspects of my teaching in a new way.     e. Helped me to become a better mathematics teacher.    9



## Mathematics Instruction **9. How much total time did you spend teaching mathematics each day to your regular**  **fourth -** **grade class LAST WEEK?** If last week was not a typical week or did not have a full schedule of classes, answer for the most recent week that did. **Monday:** Minutes teaching math ematics **Tuesday:** Minutes teaching math ematics **Wednesday:** Minutes teaching math ematics **Thursday:** Minutes teaching math ematics **Friday:** Minutes teaching math ematics 

### 10. **In teaching your regular fourth**  **- grade mathematics classes, how much emphasis did you**  **give to each of the following during the 2013 –** **2014 school year?** Check one box for each row.

**Topic Not an**  **E mphasis**  **Minor**

**E mphasis**  **Moderate**

**E mphasis**  **Major**

**E mphasis**  a. Teaching mathematical facts and rules     b. Presenting and exploring the reasoning behind math ematical rules    
c. Using multiple representations(charts, pictures, symbols) when teaching a new topic     d. Examining student work to gain insight into how students are thinking    
e . Having your students justify their solutions or explain their reasoning     f . Encouraging your students to explore alternative methods for solving problems     g. Having your students practice mathematical procedures     h. Explicitly making connec tions among mathematics topics    10



## Beliefs A**bout Mathematics Teaching and Learning 11.**  **Please indicate the degree to which you agree (or disagree) with each of the following.** Check one boxfor each row

. **Belief Strongly**  **Disagree Disagree**  **Agree Strongly**  **Agree** a . Getting the one, correct answer is the most important part of elementary mathematics .     b . Being able to successfully use a rule or formula in mathematics is more important to me than understanding why and how the rule or formula works .    

c. Learning mathematics mainly involves memorizing procedures and formulas.     d . A lot of things in math ematics simply must be accepted as true and remembered .     e. I like to use math ematics problems that can be solved in many different ways .     f. When I teach math ematics, I often find it difficult to interpret students’ wrong answers .     

## Certification **12. Which of the following describes the teaching certificate**  **did you hold in June 2013**  **(i.e., the beginning of the summer**  **prior to this past school year)**  **in this state?** Check only one box.  Regular or standard state certificate or advanced professional certificate  Probationary certificate (issued after satisfying all requirements except the completion of a probationary period)  Provisional or other type of certific ate given to persons who are still participating in what the state calls an “alternative certification program”  Temporary certificate (requires some additional college coursework, student teaching, and/or passage of a test before regular certification c an be obtained)  Waiver or emergency certificate (issued to persons with insufficient teacher preparation who must complete a regular certification program in order to continue teaching)  I do not have any of the above certifications in THIS state.11



## Education and Preservice Training **13. By**  **June 2013 (i.e., the beginning of the summer**  **prior to this past school year)**  **had you**  **earned any of the following degrees, diplomas ,**  **or certificates?** Please check No or Yes in each row then, if applicable, write in the major code from the List of Codes that follows. If you have earned more than one credential per row, report the one most relevant to your current teaching assignment (

e.g., if you have earned two different master’s degrees, use row b to report the master’s degree that is most relevant to your current teaching assignment) . **Degree Earned**  **Major Field**  **Code ( F** **rom List of Codes** **)**
a. Bachelor’s degree  No  Yes  b. Master’s degree  No  Yes  c. Educational specialist, professional diploma, or certificate of advanced graduate studies (at least one year beyond master’s level)  No  Yes  d. Doctorate or professional degree (e.g., Ph.D., Ed.D., M.D . , L.L.B, J.D . , D.D.S.)  No  Yes  

### List of Codes: **Major Field of Study Code**  **Major Field** 01 Elementary education or early childhood education 02 Secondary or middle grades education, general 03 Mathematics education 04 Mathematics 05 Other mathematics

- related subject (e.g., statistics, economics, engineering ) 06 English language arts 07 Natural science (e.g., biology, chemistry, physics, earth science) 08 Social science (e.g., anthropology, geography, history, psychology) 09 Special education 10 English as a second language 11 Other (e.g., foreign language, health education, business, computer science)12



### 14. **By June 2013**  **(i.e., the beginning of the summer prior**  **to this past school year) had you**  **completed at least one college course (undergraduate or graduate) in any of the following**  **mathematics or mathematics -** **related subjects?** Please check the number of courses completed in each row. Do not count any course twice. Include Advanced Placement courses completed in high school for which you received college credit.

**Type of Course Number**  **of Courses Completed 0**  **1 2**  **3 or M**  **ore** a. College algebra (not abstract or linear algebra)     b. Geometry or trigonometry    

c. Pre calculus     d. Calculus or differential equations     e. Math ematics for elementary teachers (i.e., a course for teachers in which the majority of time was focused on mathematics, not on how to teach math ematics or how students learn math ematics)     f. Mathematics education or methods of teaching mathematics (i.e., a course for teachers in which the majority of time was focused on how to teach math ematics and how students learn math ematics , not mathematics, itself)     g. Statistics or probability, including statistical methods in the social sciences (e.g., econometrics or psychometrics)     h. Other mathematics or mathematics

- related courses (e.g. , abstract algebra, linear algebra, differential equations, number theory, mathematical logic, set theory, foundations of mathematics) (please specify) :

_______________________________________    1 3



## Experience
**15. How many years of teaching experience**  **do you have in each of the following settings?** For each row, write the number of years, using whole numbers only. Count school year 2013

-2014 as one full year. **Setting Number of Years**  a. Teaching in total Years b. Teaching at **this** school Years
c. Teaching an elementary school class where your responsibilities included instruction in multiple subjects, including mathematics Years d . Teaching an elementary class where your responsibilities included solely mathematics instruction Years
e. Teaching mathematics content that would be considered by your district to be above the fourth

-grade curriculum Years 

## Demographics **16. What is your gender?**   Male  Female 

### 17. **Are you of Hispanic or Latino origin?**  Yes  No14



### 18. **What is your race?** Check one or more box.  White  Black or African American  Asian  Native Hawaiian or other Pacific Islander  American Indian or Alaska Native15



## Treatment Teacher A

**ddendum 19.**  **Please indicate the degree to which you agree (or disagree) with each of the following.**  Check one boxfor each row

. **Program Strongly**  **Disagree Disagree**  **Agree Strongly**  **Agree** a. I am satisfied with the **Intel** professional development opportunities I have participated in this year.     b. I am satisfied with the **MLC** professional development opportunities I have participated in this year.    
c. I am satisfied with the **Video Feedback** professional development opportunities I have participated in this year.    16



# Appendix B **- 2**  **Impact Evaluation of Math Professional**

**Development District Archival Records Collection Protocol** 

**Paperwork Burden Statement** According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection dis plays a valid OMB control number. The valid OMB control number for this information collection is **XXXX –** **XXXX** . This information collection is voluntary. The estimated time required to complete this information request is 20 hours per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate(s) or su ggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202 –4537. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: Thomas Wei, U.S. Department of Education, Institute of Education Sciences, Room 500H, 555 New Jersey Avenue, NW, Washington, DC 20208 - 5500.17



## District **Ar chi**  **va l Records Collection Protocol**  The Math Professional Development Study requests district administrative records for the students in participating teachers’ classrooms. In this protocol we specify the data elements requested for all students in the participating teachers’ classrooms at three points in time: (1) summer/fall 2013 when classroom assignments are formed; (2) in March 201 4; and (3) at the time that the spring 2014 state assessment is administered. We will work with the district to determine the most efficient process for providing the requested data. The data elements that will be requested are listed below. 

## 1. **Classroom Roster Records**  At three points in time: (1) summer/fall 2013 when classroom assignments are formed; (2) in March 2014; and (3) at the time that the spring 2014 state assessment is administered, the study team will request file s that include the following :
 Date  School  Teacher ID (participating teachers)  Student ID (students in participating teachers’ classrooms) Student records will be requested for the students in these classroom roster records, as specified in the next section. 

## 2. **Student Records** Student assessment and demographic records will be used to describe the student sample, conduct baseline equivalence analyses, and conduct impact analyses for study reports. More specifically, the purpose of the archival student achievement and demographic records is to provide:  Demographic variables used to describe the characteristics of students who were in the participating teachers’ classes;  Demographic and prior achievement variables used to test for baseline equivalence and as covariates (i.e. contro l variables) in analyses of the impact of the study’s PD intervention on student achievement scores.  Achievement variables (i.e., students’ fourth grade mathematics achievement on state assessment) used as outcome measures in analyses of the impact of the study’s PD intervention on student achievement scores. **Requested Data Elements in Student Achievement and Demographic Records A.**  **Identifiers**18

A student identifier is needed to link student data across years and to link data across files if achievement, dem ographic, and other data are provided in separate files. The identifier need not be the actual identification (ID) number used by the district. We anticipate creating a “crosswalk” file, which contains the study ID linked to local IDs and would remain with in the district. This file would permit the necessary links while at the same time ensuring that student names and local IDs do not leave the district office. In addition, by making it possible for the project team to carry out the linking of files, an ide ntifier would reduce the burden on the district. We plan to work with the districts involved in the study to develop a strategy for collecting the required achievement data that meets the needs of the study and is feasible for the districts, while meeting confidentiality requirements. **B. Student Background Variables**   Month and year of birth (MM/YYYY)  Gender  Ethnicity (current U.S. census categories)  Race  Enrolled grade in SY2012 - 13 and SY2013 - 14  National School Lunch Program status in SY2012 - 13 and SY2013 - 14  Free  Reduced  Neither  English language learner status in SY2012 - 13 and SY2013 - 14  Individualized Education Plan status in SY2012 - 13 and SY2013 - 14  School entry and exit dates in SY2012 - 13 and SY2013 - 14  Attendance for SY2012 - 13 and SY2013 - 14  Number of days absent  Number of days attended **C. Student Achievement Variables for SY2012**  **- 13**  **and SY2013**  **- 14**   Mathematics score code (e.g., A = absent, D = no info for student)  Mathematics language version (e.g., E = English, S = Spanish)  Mathematics test version (e. g., S = regular, M = modified, T = alternate)  Mathematics performance level (provide corresponding level description)  Mathematics scale score  Mathematics scale score standard error of measurement Student records will be requested for all students in the participating teachers’ classrooms at the three specific timepoints: (1) summer/fall 2013 when classroom assignments are formed; (2) in March 2014; and (3) at the time that the spring 2014 state assessment is administered. We know that the SY2013 - 14 achievement and attendance data will not be available until summer 2014. As noted earlier, we will work with each district to determine the most efficient process for providing the requested data.