### 2015 **- 16** 

**NATIONAL TEACHER AND PRINCIPAL SURVEY (NT**  **PS)**

**FULL -**  **SCALE DATA COLLECTIO N**  OMB SUPPORTI

NG STATEMENT PART B

OMB# 1850
- 0 598 v . 11 February 2015 Rev. April 27, 2015 **National Center for Education Statistics (NCES)** 

**TABLE OF CONTENTS 1.**  **Respondent Universe ................................**  **................................ ................................**  **................................ ....................**  **1** 1.1. S CHOOLS ................................ ................................ ................................ ................................ ............................ 1 1.2. T EACHERS ................................ ................................ ................................ ................................ ........................... 1 **2. Statistical Procedures for Collecting Information**  **................................ ................................**  **................................ ..........**  **1** 2.1. S AMPLING ................................ ................................ ................................ ................................ .......................... 1 **Sampling –**  **Public Schools ................................**  **................................ ................................**  **................................ .............**  **1 Sampling**  **– Teachers Within All Schools**  **................................ ................................**  **................................ ........................**  **4 Sampling**  **– Principals Within All Schools**  **................................ ................................**  **................................ .......................**  **4** 2.2. S URVEY W EIGHTS ................................ ................................ ................................ ................................ ................ 4 2.3. R ESPONSE R ATES ................................ ................................ ................................ ................................ ................. 4 2.4. P ROCE DURES FOR C OLLECTION OF I NFORMATION ................................ ................................ ................................ ....... 4 **3. Methods for Maximizing Response Rates ................................**  **................................ ................................**  **........................... 5**  **4. Tests of Procedures and Methods ................................**  **................................ ................................**  **................................ ......**  **7 5. Reviewing Statisticians**  **................................ ................................**  **................................ ................................**  **....................... 7** 1

**B. COLLECTION**  **OF INFORMATION**  **EMPLOYING STATISTICAL**  **METHODS 1.**  **Respondent Universe 1.1.**  **S chools**  The respondent universe for the 2015
- 16 full
- scale data collection consists of 96,405 public schools in the 50 U.S. states and the District of Columbia (DC) that offer instruction in any o f grades K
- 12. To be eligible for inclusion in the sample, schools must provide classroom instruction to students, have one or more teachers to provide instruction, serve students in at least one of grades 1
- 12 or the ungraded equivalent, must be located in one or more buildings, and must be located in the U.S. and not in the outlying areas or U.S. territories. NCES’ 2013
- 14 Common Core of Data (CCD) will be used to construct the public school sampling frame. The respondent universe for charter schools will be identified as those public charter schools that meet the NTPS definition of an eligible school f ound on the CCD. Table 1.1 below presents the number of public schools on the 2012
- 13 CCD by urbanicity

and school level. The universe has been adjusted to remove K
-terminal schools, which are not eligible for NTPS. **Table 1.1**  **. Respondent**  **universe by**  **urbanicity and**  **school level**  **for the**  **proposed public**  **school sample, based on the 2012**  **- 13 CCD**  **School level Region**  **Primary Middle**  **High Combined**  **Total** City 15,233 3,584 5,124 2,182 26,123 Suburban 17,865 4,980 5,746 1,515 30,106 Town 6,223 2,381 3,430 924 12,958 Rural 13,608 3,989 7,266 2,355 27,218 Total 52,929 14,934 21,566 6,976 96,405 SOURCE: 2012
- 13 CCD. **1.2. T**  **eachers** Teachers will be randomly s ampled within the second design stage from roster information provided by each participating sampled school. Teachers within the sampled school are classified as ineligible for NTPS if they

are a short
-term substitute teacher, student teacher, a teacher’s aide, or do not teach any o f grades K
- 12 or comparable ungraded levels. This information is obtained from the Teacher Questionnaire
. **2. Procedures f** 

**or Collecting Information 2.1.**  **Sampling** The final 2015
- 1 6 NTPS samples will include no more than:
 8,300 schools and school principals (7,000 traditional public and 1,300 charter schools);  40,000 teachers (34,300 traditional public , and 5,700 charter school teachers); plus  An additional 1,000 schools and principals will be selected for an internet response experiment following similar procedures as the main NTPS sample **Sampling**

**– Public Schools**  The level of precision achieved by the 2011
- 12 SASS was evaluated to inform the sample design2

decisions for the 2015
- 16 NTPS. The precision analysis was based on important analysis variables and on generic proportions to address other important SASS characteristics. The following variables and values were evaluated:  by state (public schools);  by school type (public charter, traditional public, overall public);  by urbanicity

within school ty

pe;  by grade level (primary, middle, high, and combined);  by grade l evel and urbanicity within school ty

pe;  by poverty status (more than 75% of students eligible for free or reduced price lunch, 75% or less);  by poverty status, grade level, and urbanicity within school type; and  by teachers’ years of experience, subject, race/ethnicity
. The desired level of precision for NTPS estimates was defined in terms of a 95% confidence interval half
width (corresponding to 1.96 times the standard error). The desired goal was to achieve a 95% confidence interval half

-width of 2.5% on a generic 20% characteristic. Table 2.1.a presents a portion of the analysis for public schools by s chool ty

pe, grade level, urbanicity, and poverty status. Presented are the anticipated number of responding schools or principals for the NTPS design and the expected precision based on the analysis of SASS 11
- 12. **Table 2.1a**  **. School -** **level precision analysis results for major characteristics of interest -**  **NTPS 2015 -**  **16**

**Characteristic Frame** 

**Schools Expected** 

**Completed Interviews**  **Expected**

**Standard Error**  **95%**

**Confidence Interval Half** 

**** **Width** 1 **Design**

**Effect Studies Needed**  **to Achieve 2.5% Half**  **- width**  **Criterion** All 95,464 5,300 2 0.68% 1.33% 1.51 1 Charter 6,254 707 1.75% **3.43%** 1.35 2 Non
- charter 89,210 4,593 0.72% 1.40% 1.46 1 Primary 52,868 2,418 0.95% 1.86% 1.35 1 Middle 14,912 940 1.52% **2.97%** 1.35 2 High 21,199 1,340 1.47% **2.86%** 1.78 2 Combined 6,485 602 2.15% **4.20%** 1.73 3 City 25,818 1,599 1.24% 2.41% 1.51 1 Suburban 29,900 1,670 1.18% 2.30% 1.44 1 Town 12,785 771 1.72% **3.38%** 1.44 2 Rural 26,961 1,260 1.40% **2.75%** 1.55 2 High poverty 23,731 1,321 1.33% **2.59%** 1.44 2 Low/medium poverty 71,733 3,979 0.79% 1.54% 1.54 1 1 The bold values represent domains that do not meet the 2.5% criterion f or the confidence interval half
- width in one study. 2 The small difference in this total (5,300) as compared to the estimated number of respondents in Part A’s burden table (5,312 ) is due to rounding. Table 2.1.b provides the analogous precision analysis for public school teachers. The expected standard errors were calculated based on the SASS 11
- 12 and scaled for the expected NTPS 20 15
- 16 number of respondents.3

**Table 2.1.b**  **. Teacher -** **level precision analysis results for major characteristics of interest**

**– NTPS 2015**  **- 16** 

**Characteristic Frame**  **Full -**  **Time Equivalent**  **Teachers (In 1000s) Expected**  **Teacher Completed**  **Interviews Expected** 

**Standard Error**  **95% Confidence**

**Interval Half**

  ** Width** **Design**

**Effect** All 3,088.3 26,500 0.41% 0.80% 2.82 Charter 132.7 3,022 1.18% 2.30% 2.60 Non
- charter 2,955.5 23,478 0.43% 0.84% 2.68 Primary 1,490.6 10,951 0.64% 1.25% 2.75 Middle 543.2 4,869 0.94% 1.84% 2.68 High 908.4 7,924 0.74% 1.46% 2.70 Combined 146.0 2,755 1.24% 2.43% 2.63 City 904.8 8,163 0.76% 1.48% 2.94 Suburban 1,187.4 9,210 0.69% 1.36% 2.75 Town 364.2 3,626 1.09% 2.13% 2.66 Rural 631.8 5,501 0.90% 1.76% 2.78 High poverty 724.7 6,317 0.85% 1.68% 2.89 Low/medium poverty 3,088.3 20,183 0.47% 0.93% 2.79 Based on this analysis, the sampling frame will be partitioned into 50 sampling strata for public schools. Public schools are divided into two categories to create the sampling strata: charter schools (18 strata) and traditional public schools (32 strata). Charter schools are identified from the CCD. The 18 charter school sampling strata are constructed by crossing the school grade level variable (primary, middle, high, and combined) with urbanicity ( city, suburban, town, and rural) and poverty status (high, low/medium percent of students eligible f or free or reduced price lunch). The 32 sampling strata for the traditional public schools , excluding charters schools, are defined by crossing the four
- category s chool grade level variable (primary, middle, high, and combined) with urbanicity

(city, suburban, town, and rural) and poverty status (more than 75% of students eli gible for free or reduced price lunch, 75% or less percent of students eligible for free or reduced price lunch).
**Grade level (# Groups)**  **State Grade level**  **(# Groups) State** 

Primary (6) City high poverty City low/medium poverty Suburban high poverty Suburban low/medium poverty Tow n Rural High (4) City high poverty City low/medium poverty

Suburban Tow n and rural

Middle (3) City high poverty City low/medium poverty Suburban, town, and rural Com bined (5) City high poverty

City low/medium poverty

Suburban Tow n Rural4

**Sampling**

**– Teachers**  **w ithin All Schools**  Teachers will be randomly s ampled from roster information provided by e ach participating sampled school. No stratification will be used as the precision analys is indicated that no oversampling i s needed. The average number of teachers sampled per school will be five for primary and combined schools, and 5.375 for middle and high schools. The larger cluster size for middle and high schools allows for greater precision of teachers by

subject area. The maximum number of teachers per school will be set at 20 to avoid overburdening the schools. **Sampling**

**– Principals**  **w ithin All Schools**  For each sampled traditional public and public charter schools, the principal will be included in the survey as a result of the school being selected. **2.2. S**  **urvey Weights**  Schools, principals, and teachers will be weighted by the inverse of the probability o f selection. The final weight contains adjustments for nonresponse and any other sampling or field considerations that arise after the sample has been drawn. **2.3. Response Rates**  We expect the response rates of the 2015
- 16 NTPS to approximate those of the 2011
- 12 SASS or to fall lower given the long
-term trend in declining response rates for federal surveys . Table 2.3 provides the bas e -weighted response rates for the 2011
- 12 SASS. **Table 2.3**  **. Base** **- weighted response rates for the 2011**  **- 12 SASS**  **by respondent type and school type School Type**  **Unit of Observation Teacher**  **Principal School**  Traditional Public 77.92% 72.90% 72.68% Charter 70.36% 69.67% 69.15% **2.4. Procedures for Collection o**  **f Information** The data collection methods for the 2015
- 16 NTPS will be based on those used in the 2014
- 15 NTPS Pilot Test and past cycles of SASS. The 2015
- 16 NTPS will seek school coordinators and will utilize clerical look
-ups of school websites for teacher lists and email addresses along with commercial vendor sources . School coordinators were utilized in past SASS cycles but not in the 2014
- 15 NTPS Pilot Test
. B eginning in August 2015 , all schools will receive an initial mailout package addressed to the principal at the school address
. The package will contain a letter to the principal, a letter to the school coordinator (including instructions for completing online a brief screener inter view using the NTPS Respondent Status Center), a Teacher Listing Form , a School Questionnaire, and a Principal Questionnaire
. The completion of the questionnaires is to be facilitated by the school coordinator
. The internet
-based Respondent Status Center will allow schools to upload or manually enter the Teacher Listing Form , as well as check the status of all of the school’s questionnaires
.5

From September 2015 through October 2015, follow
-up operations will be conducted for schools that have not returned the Teacher Listing Form. These operations will consist of f ield staff contacting the schools by

telephone or personal visit
. Schools will be assigned follow up priority at the time of sampling based on their likelihood to respond and impact on final estimates. Additional reminder letters to the school and school coordinator will be sent in October and November 201 5. Field staff will conduct telephone and personal visit follow
-up of remaining nonresponding schools in December 2015 through May 2016
. **3 .**  **Methods f** 

**or Maximizing Response Rates** A variety of procedures will be employed to ensure high response rates at both the level of the responding unit (i.e., sample member) and at the level of the individual survey items in each survey

questionnaire. The final NTPS design is based on results from the 2014
- 15 pilot test , which examined effectiveness of different contact strategies ( OMB# 1850
- 0803 approved in July 2014 )
. The entire survey

process, starting with securing research cooperation from key public school groups and individual sample members and continuing throughout the distribution and collection of individual questionnaires, is designed to increase survey response rates. In addition, we believe that the following elements of the data collection pl an, in particular, will contribute to overall success of the survey and will enhance the survey

response rates. (1) Visible support from top

- level Federal, State, and local education officials
. Without the support of high
- level officials in the U.S. Department of Education, State Education Agenc ies , and the sampled local school districts , surveys of public school principals and teachers cannot be successfully implemented. Obtaining endorsements from these officials is a critical factor in the success of the data collection procedures. Top
- level Education Department officials will need to fully support the data collection by endorsing the survey

in writing and sending advance letters and notices to sampled districts' Superintendents, and individual survey participants (principals and teachers) to encourage participation. (2) Endorsements from key public school groups . The level of interest and cooperation demonstrated by key groups can often greatly influence the degree of participation of survey respondents. Endorsements are viewed as a critical factor in soliciting cooperation from state and local education officials. The NTPS is seeking endorsement by the following organizations or agencies: American Association of School Administrators Association of American Educators American School Counselors Association Association of Supervision and Curriculum Development American Federation of Teachers American Counseling Association Association for Middle Level Education Council of Chief State School Officers Council of the Great City Schools National Association of Elementary School Principals National Association of Secondary School Principals National Education Association American Association of School Librarians American Montessori Society National Parent Teacher Association As more endorsements are received, they will be added to questionnaires’ cover page (Attachment A).6

(3) Stressing the importance of the survey and the respondents' participation
. Official letters will be used to motivate respondents to return surveys. For 2015 we plan to send an initial letter from the Director of the Census Bureau. Follow up letters will be sent from the Commissioner or Associate Commissioner of NCES. The additional personalization of survey materials (cover letters and survey packets with teachers' names) is also expected to have positive effects on the response rates. (4) Minimize the survey burden on school

- level authorities
. The procedures for the surveys are designed to minimize the survey burden on schools and sampled individuals (principals and teachers) and the survey

instruments have been designed to be completed as quickly and easily as possible. Good questionnaire design techniques have been employed to minimize item nonresponse. All completed questionnaires from the 2011
- 12 SASS have been carefully analyz ed to determine which items had the highest levels of item nonresponse. This information guided NCES in reviewing the clarity of item wording, definitions, and instructions. Items that were not considered to be effective or useful were deleted so as to streamline the questionnaire s and ease the response burden. NTPS also plans to provide links to or incorporate data from other NCES collections such as EDfacts and the Civil Rights Data Collection (CRDC) into final datasets to allow researchers and policy makers to include additional data in their analyses . This will further reduce the need to collect data from schools that have already been collected at the state or district level. (5) Seeking the recruitment of a school coordinator
. An important procedural measure for helping to maximize response rates is the plan to establish a school
-based "surv ey coordinator" to serve as a primary point of contact for NTPS staff
. The use of a school coordinator is expected to help keep response rates high, provide some minimal data quality checks, and simplify

the follow
-up process by having one point of contact
. (6) Tailoring non response follow up strategies. After sample selection, cases will be assigned a “priority” flag based on the weighted response influence of the case. The weighted response influence takes into account both the response propensity and the base weight of a school to create a measure of a school’s potential effect on nonresponse weighting adjustments and final estimates. The weighted response influence can be calculated as: φ = log ( ) 1 where: φ is the final weighted response influence for a school,

is the baseweight for a school, and is the estimated response propensity for a school As the formula shows, a case with either an extremely high weight or an extremely low response propensity

will have a large response influence, re flecting the fact that if they are a nonrespondent, they will disproportionately

affect the nonresponse adjustment cell in which they are located. Missing that particular school’s information may result in biased estimates (if variables in the propensity model are related to outcomes of interest), and will certainly result in increased variance in the estimates (due to more variable final weights). In order to avoid having extreme weights drive the value of weighted response influence, we take the natural log of the base weight in the formula. Using data from the 2011
- 2012 SASS and the 2014 NTPS Pilot Test, including frame variables and response statuses, we calculated a weighted response propensity model, keeping in mind7

domains of interest and estimates o f interest for the 2015 NTPS. Specific categories of variables available for evaluation include geography, urbanicity, racial/ethnic makeup, enrollment, grades levels, free lunch recipiency, and type of school. These variables are available in the 2011
- 201 2 SAS sample files, the NTPS Pilot Test sample files, and the 2015 NTPS sample file, allowing us to leverage past experience in creating the response propensity models. The priority

flag will be used throughout data collection to make non response follow up decisions. For example, cases with the highest weighted response influence will receive early follow up by

field interviewers rather than progressing through multiple data collection operations that are likely to be unproductive. Weighted response influence was also used, with great success, b y the National Survey of College Graduates (NSCG) for identifying high influence sample persons. These high influence cases were targeted during data collection with an experimental incentive to increase sample balance and overall response rate. Response influence has been implemented by the NSCG in production as a way of identifying high influence sample persons for data collection interventions. More information about response influence can be found in: Särndal, C., Lundström, S. (2008). Assessing auxiliary

vectors for control of nonresponse bias in the calibration estimator. Journal of Official Statistics 24, 167
- 191. (7) Increased use of email and internet for collection of teacher lists and survey reminders
. The 2014 NTPS pilot test demonstrated that email was an effective tool to drive participation in the NTPS teacher survey . The pilot test also showed that email addresses and teacher lists could be collected from school websites. The 2015
- 16 NTPS includes c lerical operations to look up principal email addresses and lists of teachers as well as their email addresses during data collection. The teacher lists will be collected for schools that do not return a completed Teacher Listing Form or do not include teacher email addresses on the returned form. Depending on results of the experiment conducted during the NTPS pilot test, commercial vendor lists may also be used to identify

and sample teachers at schools that did not complete a TLF. Emails will be sent to sampled teachers with links to the NTPS teacher survey
. Additionally, principals of schools in the internet experiment will receive email invitations and reminders to complete the survey. **4 . Tests of Procedures and Methods**  The 201 7
- 18 NTPS will be built on the experience of previous rounds of SASS and NTPS . Results from the 2015
- 16 NTPS Schools and Principals Internet T est will impact the final 2017
- 18 NTPS design. These results and the final design will be described in the 201 7
- 18 NTPS data collection clearance package expected to be submitted in early 2017
. **5 .**  **Reviewing Statisticians** The following statisticians have been contributing to the NTPS sample design : Randall Parmer, U.S. Census Bureau , Suitland, MD David Marker, Westat, Rockville, MD Lou Rizzo, Westat, Bethesda, MD Sharon Lohr, Westat, Bethesda, MD Edward Mulrow , NORC , Bethesda, MD Kirk Wolter, NORC , Chicago, IL Andrew Zukerberg, NCES, Washington, DC Marily n Seastrom, NCES, Washington, D C