Analysis of Alternative Poverty Measures Applied to the Case of Montana
Authors: Clausen Robin 
Published: December 2024

Poverty measures are used in the field of education to promote public policy and enable research and evaluation activities. The question remains which poverty measures to choose in what context. Since the 1970’s researchers have been using free and reduced priced lunch eligibility (NSLP Eligibility) as a measure of proxy and choice. NSLP Eligibility data has many emerging insufficiencies, including over identification of students, inaccurate income information, and inaccurate accounting of economically disadvantaged students in Community Eligible Provision schools (Geverdt &amp; Nixon, 2018). The arrival of Covid and constraints placed on schools made these insufficiencies more apparent. Nonetheless, any alternative poverty measures would need to consider policy continuity and historical precedence. There are eight poverty measures under consideration in this study. An example of an alternative poverty measure is the Spatially Interpolated Demographic Estimates (SIDE) provided by the US Department of Education and the Census Bureau. In this study we use three SIDE measures: the School Neighborhood Poverty index, a school level measure created for this study, and a measure based on the geolocation of student addresses. By comparing alternative poverty measures to the free and reduced meal data, the Montana Office of Public Instruction asks how correlated are measures of school poverty to the NSLP measures for March 2019 (policy continuity)? Second, are the same schools classified relatively similarly as the NSLP measure? Third, understanding the impact of poverty measures on the analysis of student outcome and institutional variables is also important to policy continuity. It allows an analysis of the relative strength of a poverty measure and enables comparisons between measures. Fourth, the study also looks to better understand how much variation in satisfactory attendance is explained by each poverty measure and whether there are differences in the direction, significance, and magnitude of the estimates. By holding all factors equal, we can use the model to make further comparisons between poverty measures. In short, all things held equal, do the alternative poverty measures meet or exceed the values found with the NSLP Eligibility data and confirm based on sign and significance.Overall, the most highly correlated poverty measures are NSLP Participation and Longevity. The Longevity measure is construct from the number of years a student has participated in NSLP. SIDE measures are highly correlated in a similar grouping. Participation is the count of those students actually participating in the school meals program, which as research notes is different from NSLP Eligibility. Of these, the SIDE estimates based of student address show the highest correlation. SAIPE and Direct Certification data are moderately correlated. To further measure the fidelity of each poverty measure with the NLSP data, we analyzed the quartiles of the NSLP eligibility data in comparison to the quartiles of each poverty measure. This looks at whether a poverty measure quartile (for example schools with more students closest to the poverty level) corresponds with an eligibility quartile 4 (mostly participating in NSLP). Not surprisingly, the strongest matches were with Direct Certification and Participation rates (Quartile 4). When regressing student outcome measures and institutional variables by each poverty measure, we found that the NSLP eligibility data explained the variation with many student outcomes and institutional variables to a greater degree than the alternative poverty measures. By and large direct certification matched the magnitude of Eligibility more reliably than Participation and the other alternative poverty measures. Most Direct Certification analyses explained at least 30% of the variation in the student outcome and institutional variables. SAIPE and Longevity proved to explain little of the variation in student outcome or institutional variables. In a model, we analyzed the degree to which variation in Satisfactory Attendance is predicted by student outcome measures while controlled by the poverty measures. We then separately regressed each combination of measures by exchanging the values for each poverty measures (all things held equal). Nearly all poverty measures showed stronger associations than seen with the naive condition (no control). Participation, Direct certification, and Longevity showed the most regression values that met or exceeded those found with Eligibility. We then look to the sign, significance, and magnitude of the regression coefficients. The magnitude of the β coefficients were similar with the alternative poverty measures compared with the magnitude of the NSLP eligibility and the naïve condition. This confirms the finding of a RAND study which found similar variation. (Doan, S., Diliberti, M., Grant, D, 2022, p. 18). There are important differences based on significance. For example, for the Superintendent salary measure, the significance is stronger with the student SIDE measures than with either the Eligibility condition or the naïve condition. The signs remain the same with the student point estimates and Eligibility or naïve conditions. By noting differences in the same context, for example by adding/removing an alternative poverty measure from the model, the study concludes that use of a poverty measure is a choice dependent on policy factors. There are differences between how the measure correlate with NSLP Eligibility, explain variation in student outcome and institutional variables, and function in a model where all things are held equal except for the poverty measures (controls). Nonetheless, no single alternative poverty measures have consistent values that meet or exceed the magnitude of the NSLP Eligibility measure. In fact, NSLP eligibility consistently explains more of the variation in the student outcome and institutional variables. The lack of consistency of the alternative poverty measures to meet or exceed NSLP eligibility values, leads to the conclusion that decisions about use of alternative poverty measures depend on the various constructs, policy or otherwise, of the poverty measures.