# Appendix B

**Data Collection Instruments**1



# Appendix B **- 1**  **Math Professional Development Study End**  **- of** **- Year**  **Teacher Survey**

**June 201**  **4**

**Paperwork Burden Statement** According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid Office of Management and Budget (OMB) control number. The valid OMB control number for this informat ion collection is **XXXX -** **XXXX** . The time required to complete this information collection is estimated to average 30 minutes per response . This information collection is voluntary. If you have any comments concerning the accuracy of the time estimate(s) or su ggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202 – 4651. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: Thomas Wei, U.S. Department of Education, Institute of Education Sciences, Room 500H, 555 New Jersey Ave. NW, Washington, DC 20208 - 5500.2



## Dear Teacher: The Math Professional Development Study (MPD) is an important national study designed to test innovative models of professional develo pment for mathematics instruction in the fourth grade. Your participation is important and appreciated, but you do have the right to skip any question that you do not wish to answer. The following are responses to some general questions concerning your participation. **What is the purpose of this survey?** The purpose of this survey is to obtain background information about the teachers within the study as well as information about their instructional contexts and professional development experiences in mathematics. **Who is conducting this survey?** The U.S. Department of Education’s Institute of Education Sciences commissioned the MPD study, and American Institutes for Research is administering the study survey. **Why should you participate in this survey?** Policymakers and educational leaders rely on findings from studies, such as the MPD study, to inform their decisions. The current study will fill a critical gap in the mathematics and professional development research base regarding what is effective for i mproving fourth

-grade mathematics achievement. **Will your responses be kept confidential?** American Institutes for Research follows the confidentiality and data protection requirements of IES (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183). **Responses to this data collection will be used only for**  **statistical purposes.**  The reports prepared for the study will summarize findings across the sample and will not associate responses with a specific school or individual. We will not provide information that identifies respondents to anyone outside the study team, except as required by law. Additionally, no one in your school or district will see your responses. **How will your information be reported?** The inf ormation you provide will be combined with the information provided by others in statistical reports. No individual data that link your name or e

- mail address with your responses will be included in the statistical reports.

**Thank you for your cooperation in this very important effort!**3



## Professional Development Experiences **Related to Ma** **thematics and Mathematics Teaching and Learning Professional**  **development** refers to a variety of activities intended to enhance your professional knowledge and skills, including workshops, seminars, institutes, college courses, coaching, mentoring, teacher networks, getting observed and receiving feedback on your tea ching, and committee work. **1. Counting only professional development activities that focused on**  **mathematics or**  **mathematics teaching and learning**  **, what is the total number of hours you**  **participated in each of the following**  **activities d uring**  **summer 2013 and the 2013**  **- 2014 school year**  **?** For each activity, write the total number of HOURS you spent, roundin g up to the nearest whole hour. Mark “ 0 ” if you participated in none. Please do NOT count the same hours in more than one row.

**Professional Development Activity Hours of**  **Professional Development**  a. Attended traditional course/workshop/seminar

- style professional development that focused on mathematics or mathematics teaching and learning (e.g., Intel Math) hours b. Participated in collaborative professional learning communities or planning groups with other teachers that focused on mathematics or mathematics teaching and learning (e.g., Mathematics Learning Communities) hours

c. Classroom teaching was observed and was given feedback that focused on mathematics or mathematics teaching and learning by someone with knowledge of mathematics instruction (e.g., a coach, department chair, or mentor) hours d. Participated in other types of professional development that focused on mathematics or mathematics teaching and learning. Please explain:

______________________________________________________ hours 

### [ Skip Pattern: If respondent added 0 hours to **every row in Question 1**, the respondent will be directed to answer Question 12. Otherwise move to Skip Pattern for Question 2.]4



### [ Skip Pattern: If respondent added hours to **Question 1, Row a**, the respondent will be directed to answer Questions 2, 3, and 4. Otherwise move to Skip Pattern for Question 5.] **2. In your traditional course/workshop/seminar**  **- style professional development activities**  **that focused on mathematics or mathematics teaching and learning during**  **summer 2013 and the 2013**  **- 2014 school year**  **, how often did y ou do the following activities?**  Check one boxfor each row.

**Activity During Professional Development**  **Never /**  **Rarely Sometimes**  **Often Most or**  **All of the Time**  a. Solve mathematics problems     b. Share your solutions to mathematics problems with other teachers in small

- or large

-group settings    
c. Discuss mathematics topics that appear in mathematics curricula above or below the current grade you teach     d. Explore the conceptual underpinning of K

– 8 mathematics concepts    
e. Practice using a variety of representations to illustrate a given mathematic al concept     f. Practice analyzing student work on mathematics problems     g. Practice writing story problems for students you teach     h. Explore the connections between mathematics topics and solution methods     i. Explore the ways students commonly approach mathematics problems     j. Explore common student misconceptions and errors in mathematics    5



### 3. **In your traditional course/workshop/seminar -**  **style professional development activities**  **that focused on mathematics or mathematics teaching and learning during**  **summer 2013 and the 2013**  **- 2014 school year**  **, how often were the following topics a focus?** Check one box for each row.

**Topic Area Never/**  **Rarely Sometimes**  **Often Most or All**  **of the Time** a. Addition, subtraction, multiplication, and division     b. C onnections between addition, subtraction, multiplication, and division    
c. Fractions and operations with fractions     d. Algebra     e. Connections between algebra and arithmetic     f. D ecimals     g. Exponents    6



### 4. **To what extent do you agree or disagree with the following statements about your experiences i** **n your traditional course/workshop/seminar**  **- style professional**  **development activities that focused on mathematics or mathematics teaching and learning during**  **summer 2013 and the 2013 -** **2014 school year ?**  Check one boxfor each row.

**Y our Experience W**  **as... Strongly**  **D isagree**  **Somewhat**

**Disagree Somewhat** 

**Agree Strongly** 

**A gree**  a. Consistent with your own goals for your professional development     b . Complementary to your use of district

adopted curricular materials    
c . Related to the mathematics content you taught this year     d . Logically connected from one day or session to the next    
e. Clear about how you could use what you learned from the professional development experience in your classroom     f . Focused on practices that district or school leaders expect you to demonstrate in your classroom    7



### [ Skip Pattern: If respondent added hours to **Question 1, Row b**, the respondent will be directed to answer Questions 5, 6, and 7. Otherwise move to Skip Pattern for Question 8.] **5. In**  **your collaborative professional learning communities or planning group activities that focused on mathematics or mathematics teaching and learning during**  **summer 2013 and the 2013**  **- 2014 school year**  **, how often did you do the f ollowing activities?**  Check one boxfor each row.

**Activity During Professional Development**  **Never /**  **Rarely Sometimes**  **Often Most or**  **All of the Time**  a. Solve mathematics problems     b. Share your solutions to mathematics problems with other teachers in small

- or large

-group settings    
c. Discuss mathematics topics that appear in mathematics curricula above or below the current grade you teach     d. Explore the conceptual underpinning of K

– 8 mathematics concepts    
e. Practice using a variety of representations to illustrate a given mathematic al concept     f. Practice analyzing student work on mathematics problems     g. Practice writing story problems for students you teach     h. Explore the connections between mathematics topics and solution methods     i. Explore the ways students commonly approach mathematics problems     j. Explore common student misconceptions and errors in mathematics    8



### 6. **In your collaborative**  **professional learning communities or planning group activities that**  **focused on mathematics or mathematics teaching and learning during**  **summer 2013 and the 2013**  **- 2014 school year**  **, how often were the following topics a focus?** Check one box for each row.

**Topi c Area**  **Never/ Rarely**  **Sometimes Often**  **Most or All of the Time**  a. Addition, subtraction, multiplication, and division     b. C onnections between addition, subtraction, multiplication, and division    
c. Fractions and operations with fractions     d. Algebra     e. Connections between algebra and arithmetic     f. D ecimals     g. Exponents    9



### 7. **To what extent do you agree or disagree with the following statements about your experiences i** **n collaborative professional learning communities or planning groups that**  **focused on mathematics or mathematics teaching and learning during summer 2013 and**  **the 2013 -** **2014 school year ?**  Check one boxfor each row.

**Y our Experience W**  **as... Strongly**  **D isagree**  **Somewhat**

**Disagree Somewhat** 

**Agree Strongly** 

**A gree**  a. Consistent with your own goals for your professional development     b . Complementary to your use of district

adopted curricular materials    
c . Related to the mathematics content you taught this year     d . Logically connected from one day or session to the next    
e. Clear about how you could use what you learned from the professional development experience in your classroom     f . Focused on practices that district or school leaders expect you to demonstrate in your classroom    10



### [Skip Pattern: If respondent added hours to **Question 1**  **Row c** , the respondent will be directed to answer Question 8. Otherwise move to Skip Pattern for Question 9.] **8. In the**  **feedback you received after a**  **video or in -** **person observation of your classroom**  **teaching that focused on mathematics or mathematics teaching and learning during summer 2013 and the 2013**  **- 2014 school year**  **, how often were the following topics a focus of the fe**  **edback ?**  Check one box in each row

**Topic Area Never/**  **Rarely Sometimes**  **Often Most or All**  **of the Time** a. How you made connections between mathematics topics and/or representations     b. How you conveyed the meaning of mathematical procedures     c. How you remediated student errors     d. How you responded to student thinking     e. The clarity, precision, and correctness of your mathematics presentations     f. Providing you with useful feedback about your teaching of mathematics    11



### [Skip Pattern: If respondent added hours to **Question 1**  **Row d**  , the respondent will be directed to answer Questions 9, 10, and 11. Otherwise, the respondent will be directed to answer question 12.] **9. In**  **your other types of professional development (i.e., excluding traditional**  **course/workshop/seminar -**  **style professional development, collaborative professional**  **learning communities or planning group activities, and classroom observations and feedback) that fo**  **cused on mathematics or mathematics teaching and learning during summer 2013 and the 2013**  **- 2014 school year**  **, how often did you do the following activities?**  Check one boxfor each row.

**Activity During Professional Development**  **Never /**  **Rarely Sometimes**  **Often Most or**  **All of the Time**  a. Solve mathematics problems     b. Share your solutions to mathematics problems with other teachers in small

- or large

-group settings    
c. Discuss mathematics topics that appear in mathematics curricula above or below the current grade you teach     d. Explore the conceptual underpinning of K

– 8 mathematics concepts    
e. Practice using a variety of representations to illustrate a given mathematic al concept     f. Practice analyzing student work on mathematics problems     g. Practice writing story problems for students you teach     h. Explore the connections between mathematics topics and solution methods     i. Explore the ways students commonly approach mathematics problems     j. Explore common student misconceptions and errors in mathematics    12



### 10. **In your other types of professional development (i.e., excluding**  **traditional course/workshop/seminar**  **- style**  **professional development, collaborative professional learning communities or planning group activities, and classroom observations and**  **feedback) that focused on mathematics or mathematics teaching and learning during**  **summer 2013 and the 2013 -** **2014 school ye ar** **, how often were the following topics a focus?** Check one box for each row.

**Topic Area Never/**  **Rarely Sometimes**  **Often Most or All**  **of the Time** a. Addition, subtraction, multiplication, and division     b. C onnections between addition, subtraction, multiplication, and division    
c. Fractions and operations with fractions     d. Algebra     e. Connections between algebra and arithmetic     f. D ecimals     g. Exponents    13



### 11. **To what extent do you agree or disagree with the following statements about your**  **experiences i n**  **other types of professional development (i.e., excluding traditional**  **course/workshop/seminar -**  **style professional development, collaborative professional**  **learning communities or planning group activitie s, and classroom observations and**  **feedback) that focused on mathematics or mathematics teaching and learning during summer 2013 and the 2013**  **- 2014 school year**  **?** Check one boxfor each row.

**Y our Experience W**  **as... Strongly**  **D isagree**  **Somewhat**

**Disagree Somewhat** 

**Agree Strongly** 

**A gree**  a. Consistent with your own goals for your professional development     b . Complementary to your use of district

adopted curricular materials    
c . Related to the mathematics content you taught this year     d . Logically connected from one day or session to the next    
e. Clear about how you could use what you learned from the professional development experience in your classroom     f . Focused on practices that district or school leaders expect you to demonstrate in your classroom    14



## Mathematics Instruction **12. How much total time did you spend teaching mathematics each day to your regular**  **fourth -**  **grade class LAST WEEK?** If last week was not a typical week or did not have a full schedule of classes, answer for the most recent week that did. **Monday:** Minutes teaching math ematics **Tuesday:** Minutes teaching math ematics **Wednesday:** Minutes teaching math ematics **Thursday:** Minutes teaching math ematics **Friday:** Minutes teaching math ematics15



## Certification **13. Which of the following describes the teaching certificate**  **you held in June 2013**  **(i.e., the beginning of the summer**  **prior to this past school year)**  **in this state?** Check only one box.  Regular or standard state certificate or advanced professional certificate  Certificate issued after satisfying all requirements except the completion of a probationary period  Certificate that requires some additional coursework, student teaching, or passage of a test before regular certification can be obtained  Certificate issued to persons who must complete a certification program in order to continue teaching  I do not have any of the above certifications in THIS state16



## Education and Preservice Training **14. By**  **June 2013 (i.e., t** **he beginning of the summer prior**  **to this past school year) had**  **you earned any of the following degrees, diplomas**  **, or certificates?**  Please check No or Yes in each row then, if applicable, write in the major code from the List of Codes that follows. If you have earned more than one credential per row, report the one most relevant to your current teaching assignment (

e.g., if you have earned two different master’s degrees, use row b to report the master’s degree that is most relevant to your current teaching assignment) . **Degree Earned**  **Major Field**  **Code ( F**  **rom List of Codes** **)**
a. Bachelor’s degree  No  Yes  b. Master’s degree  No  Yes  c. Educational specialist, professional diploma, or certificate of advanced graduate studies (at least one year beyond master’s level)  No  Yes  d. Doctorate or professional degree (e.g., Ph.D., Ed.D., M.D . , L.L.B, J.D . , D.D.S.)  No  Yes  

### List of Codes: **Major Field of Study Code**  **Major Field** 01 Elementary education or early childhood education 02 Secondary or middle grades education, general 03 Mathematics education 04 Mathematics 05 Other mathematics

- related subject (e.g., statistics, economics, engineering ) 06 English language arts 07 Natural science (e.g., biology, chemistry, physics, earth science) 08 Social science (e.g., anthropology, geography, history, psychology) 09 Special education 10 English as a second language 11 Other (e.g., foreign language, health education, business, computer science)17



### 15. **By June 2013**  **(i.e., the beginning of the summer prior**  **to this past school year) had you**  **completed at least one college course (undergraduate or graduate) in any of the following mathematics or mathematics**  **- related subjects?**  Please check the number of courses completed in each row. Do not count any course twice. Include Advanced Placement courses completed in high school for which you received college credit.

**Type of Course Number of Courses Completed**  **0 1**  **2 3 or**  **M ore**  a. College algebra (not abstract or linear algebra)     b. Geometry or trigonometry    

c. Pre c alculus     d. Calculus or differential equations     e. Math ematics for elementary teachers (i.e., a course for teachers in which the majority of time was focused on mathematics, not on how to teach math ematics or how students learn math ematics)     f. Mathematics education or methods of teaching mathematics (i.e., a course for teachers in which the majority of time was focused on how to teach math ematics and how students learn math ematics , not mathematics, itself)     g. Statistics or probability, including statistical methods in the social sciences (e.g., econometrics or psychometrics)     h. Other mathematics or mathematics

- related courses (e.g. , abstract algebra, linear algebra, differential equations, number theory, mathematical logic, set theory, foundations of mathematics) (please specify) :

_______________________________________    18



## Experience
**16. How many years of**  **teaching experience do you have in each of the following settings?** For each row, write the number of years, using whole numbers only. Count school year 2013

-2014 as one full year. **Setting Number of Years**  a. Teaching in total Years b. Teaching at **this** school Years
c. Teaching an elementary school class where your responsibilities included instruction in multiple subjects, including mathematics Years d . Teaching an elementary class where your responsibilities included solely mathematics instruction Years
e. Teaching mathematics content that would be considered by your district to be above the fourth

-grade curriculum Years 

## Demographics **17. Are you male or female?**   Male  Female 

### 18. **Are you of Hispanic or Latino origin?**  Yes  No19



### 19. **What is your race?** Check one or more box.  White  Black or African American  Asian  Native Hawaiian or other Pacific Islander  American Indian or Alaska Native20



# Appendix B **- 2**  **Impact Evaluation of Math Professional** 

**Development District Archival Records Collection Protocol** 

**Paperwork Burden Statement** According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection dis plays a valid OMB control number. The valid OMB control number for this information collection is **XXXX –** **XXXX** . This information collection is voluntary. The estimated time required to complete this information request is 20 hours per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate(s) or suggesti ons for improving this form, please write to: U.S. Department of Education, Washington, DC 20202 –4537. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: Thomas Wei, U.S. Department of Educa tion, Institute of Education Sciences, Room 500H, 555 New Jersey Avenue, NW, Washington, DC 20208 - 5500.21



## District **Ar chi**  **va l Records Collection Protocol**  The Math Professional Development Study requests district administrative records for the students in participating teachers’ classrooms. In this protocol we specify the data elements requested for all students in the participating teachers’ classrooms at three points in time: (1) summer/fall 2013 when classroom assignments are formed; (2) in March 2014; and (3) at the time that the spring 2 014 state assessment is administered. We will work with the district to determine the most efficient process for providing the requested data. The data elements that will be requested are listed below. 

## 1. **Classroom Roster Records** At three points in time : (1) summer/fall 2013 when classroom assignments are formed; (2) in March 2014; and (3) at the time that the spring 2014 state assessment is administered, the study team will request file s that include the following:

 Date  School  Teacher ID (participating teachers)  Student ID (students in participating teachers’ classrooms) Student records will be requested for the students in these classroom roster records, as specified in the next section. 

## 2. **Student Records** Student assessment and demographic records wi ll be used to describe the student sample, conduct baseline equivalence analyses, and conduct impact analyses for study reports. More specifically, the purpose of the archival student achievement and demographic records is to provide:  Demographic variables used to describe the characteristics of students who were in the participating teachers’ classes;  Demographic and prior achievement variables used to test for baseline equivalence and as covariates (i.e. control variables) in analyses of the impact of the study’s PD intervention on student achievement scores.  Achievement variables (i.e., students’ fourth grade mathematics achievement on state assessment) used as outcome measures in analyses of the impact of the study’s PD intervention on student achieveme nt scores. **Requested Data Elements in Student Achievement and Demographic Records A.**  **Identifiers**22

A student identifier is needed to link student data across years and to link data across files if achievement, demographic, and other data are provided in separate files. The identifier need not be the actual identification (ID) number used by the district. We anticipate creating a “crosswalk” file, which contains the study ID linked to local IDs and would remain within the district. This file would permit t he necessary links while at the same time ensuring that student names and local IDs do not leave the district office. In addition, by making it possible for the project team to carry out the linking of files, an identifier would reduce the burden on the di strict. We plan to work with the districts involved in the study to develop a strategy for collecting the required achievement data that meets the needs of the study and is feasible for the districts, while meeting confidentiality requirements. **B. Student Ba**  **ckground Variables**  Month and year of birth (MM/YYYY)  Gender  Ethnicity (current U.S. census categories)  Race  Enrolled grade in SY2012 - 13 and SY2013 - 14  National School Lunch Program status in SY2012 - 13 and SY2013 - 14  Free  Reduced  Neither  English language learner status in SY2012 - 13 and SY2013 - 14  Individualized Education Plan status in SY2012 - 13 and SY2013 - 14  School entry and exit dates in SY2012 - 13 and SY2013 - 14  Attendance for SY2012 - 13 and SY2013 - 14  Number of days absent  Number of days attended **C. Student Achievement Variables for SY2012**  **- 13**  **and SY2013**  **- 14**   Mathematics score code (e.g., A = absent, D = no info for student)  Mathematics language version (e.g., E = English, S = Spanish)  Mathematics test version (e.g., S = regular, M = modified, T = alter nate)  Mathematics performance level (provide corresponding level description)  Mathematics scale score  Mathematics scale score standard error of measurement Student records will be requested for all students in the participating teachers’ classrooms at th e three specific timepoints: (1) summer/fall 2013 when classroom assignments are formed; (2) in March 2014; and (3) at the time that the spring 2014 state assessment is administered. We know that the SY2013 - 14 achievement and attendance data will not be available until summer 2014. As noted earlier, we will work with each district to determine the most efficient process for providing the requested data.