Poverty Measurement: Adjusting for Geographic Cost-of-Living Difference
Released: 1995-03-29T00:00:00

Letter

Background

In 1969, the federal government officially adopted a measure to ascertain how many people across the country had incomes that were inadequate to meet expenses for basic needs. This poverty measure was based on the finding of the U.S. Department of Agriculture’s (USDA) 1955 Survey of Food Consumption that, on average, families of three or more persons spent one-third of their income on food. Poverty for a family of three was computed as three times the cost of the economy food plan, the least costly food plan designed by USDA. The poverty measure has been updated annually with a COL index to adjust for the change in prices nationwide, but the poverty measure has not been adjusted for differences in prices by geographic area. Thus, in 1993, a family of three with a cash income of less than $11,522 was considered to be living in poverty, regardless of place of residence.

The concept of geographic COL adjustments of poverty measurement has been seen as problematic. A 1976 report to Congress on the measurement of poverty stated that “one of the most troublesome concepts of poverty measurement” was making adjustments for geographic differences in COL.It ultimately concluded that unresolved conceptual issues, such as the development of generally accepted market baskets of goods and services representative of the needs of the poor in various geographic areas, and data limitations precluded satisfactory geographic adjustments. More recently, in a 1992 report, we noted that there was insufficient data on which to base geographic adjustments to the measure of poverty.

Some economists contend that adjusting the poverty measure for geographic differences in COL would be inappropriate, irrespective of the methodology used. They say that any such adjustment to reflect regional differences in market baskets would fail to recognize other regional differences that are relevant to a definition of poverty or the needs of the poor. For example, a COL index probably would not reflect differences among geographic areas in the level of support or assistance available to low-income families.

Scope and Methodology

To address our first two objectives, describing the function of a market basket and identifying potential methods for calculating a COL adjustment, we reviewed the relevant literature on measuring poverty and on geographic adjustment for COL and discussed these issues with specialists. These specialists included individuals associated with poverty measurement or COL data at the Bureau of Labor Statistics (BLS) and the Bureau of the Census, as well as private organizations and academic institutions. On the basis of these reviews and discussions, we identified 12 methodologies that might have potential for adjusting poverty measures to reflect geographic differences in COL. We consider these 12 methodologies to be illustrative for a wide range of potential approaches to determine geographic COL differences, but recognize that the list is not, and cannot be, exhaustive. (A more detailed account of our scope and methodology is contained in app. I.)

To meet our third objective of obtaining expert opinion on the ability of the methodologies to adjust the poverty measure for geographic differences in COL, we identified experts and asked them to review the methodologies. From our list of more than 40 potential experts compiled during our literature review and initial discussions with specialists, we selected 15 experts to review the methodologies. (See app. II for a list of the selected experts.)

We sent a questionnaire to these experts in which we described each methodology briefly. We asked the experts to review each of the 12 methodologies and to categorize the methodology’s potential for use in adjusting the poverty measurement for geographic difference in COL. Additionally, we asked them to discuss the strengths and weaknesses of each methodology. (See app. III for a copy of the information and questionnaire sent to each expert.) All 15 experts responded and we tabulated their ratings for each methodology to determine the ones the experts considered most and least promising. We also analyzed the written responses on strengths and weaknesses.

We did our work in Washington, D.C., between September 1994 and January 1995 in accordance with generally accepted government auditing standards. Because we did not evaluate the policies or operations of any federal agency to develop the information presented in this report, we did not seek comments from any agency.

Market Baskets Are Necessary, but a Uniform National Market Basket Is Neither Used Nor Considered Desirable by Experts

Market baskets of goods and services form the basis for determining a COL index. Of the methodologies we examined that calculate a COL index, none uses a uniform national market basket in which the same quantities of identical goods and services are used in all locations. In fact, these methodologies all used market baskets that have different measures for at least one component—for example, transportation or housing. Several of the experts, in their comments on COL methodologies, said that market baskets for COL indexes should vary to reflect differences in local standards of living.

Market Baskets Are the Foundation of a COL Index

Market baskets of goods and services provide the foundation for determining COL. The composition of the market baskets, such as the items included or the quantity of one item included in relation to other items, affects the dollar values that are determined to represent COL. Conceptually, market baskets for a COL index would accurately reflect differences in tastes, as well as needs, such that an individual would derive equal satisfaction from the various market baskets priced in different geographic locations. For example, food preferences in southeastern states for low-cost cereals, such as rice and corn, lowers COL in these areas, while climatic differences necessitates the expenditures for heating a home and warm clothing and increases the COL in northern states.

Obtaining a consensus on what should go into a COL index’s market baskets and on how to update them would be difficult. The method generally preferred by the experts we contacted to determine the items to include in market baskets is to use expert judgment to specify the requirements for physical health and social well-being. But standards have not been identified for the majority of components of a COL index’s market baskets. Even if consensus were obtained on the specific items and their quantities to include in a COL index’s market baskets, another problem would be how to keep the market baskets up to date to reflect a constant standard of living.

A Uniform National Market Basket Is Not Used in COL Indexes

Of the methodologies we examined that calculate a COL index, all used market baskets that reflected regional differences in standards of needs and/or actual consumption patterns. Most notably, these methodologies varied in how they determined the housing and transportation components of the market baskets by adjusting for regional variation.

Experts Say That Local Standards of Living Are Necessary in a COL Index’s Market Baskets

We received numerous comments about market baskets for a COL index from the experts from whom we solicited assessments of the methodologies. Several experts noted the need to adjust the composition of the market baskets for differences in local standards of living among geographic areas. One expert commented that it is nearly impossible to obtain reliable evidence or credible expert judgments about the composition of market baskets to reflect specific local standards of living. This expert suggested that market baskets should be changed as acceptable standards are developed. The problem of keeping market baskets up to date was noted by other experts in their comments about the use of outdated data and concepts. For example, one expert specifically wanted a child care component to be included in the market baskets.

Methodologies Exist That Potentially Could Lead to a Geographic COL Index

We identified 12 generic methodologies that, in some part, could contribute to the development of a COL index that potentially could be used to adjust the poverty measurement for geographic differences. Four methodologies identified baseline data, or developed a market basket that could be the basis for constructing a COL index by geographic area. Six methodologies calculated a COL index from existing cost data or a previously defined market basket. Two methodologies developed an original market basket, collected data, and calculated a COL index with those data. Table 1 provides descriptions of the 12 methodologies. (Detailed descriptions of these methodologies are found in app. III.)

A few of the methodologies are now used as COL indexes, but most have not been. For example, the norms, local indexes, and economic modeling methodologies are used in the private sector as COL indexes to make geographic COL adjustments for pay and relocation decisions. Until their discontinuance in 1981, estimates from the family budgets methodology had been used by policymakers to set income eligibility criteria for employment programs and to geographically adjust wages and salaries. Several of the methodologies that identify baseline data are used in ways other than to show differences in COL. For example, USDA uses the consumption data methodology to estimate expenditures on a child, which then are used to determine payments for the support of children in foster families. Many of the methodologies were developed by researchers to develop indexes to reflect COL differences, such as those categorized under the estimation models, interarea price index, and the consumer price index methodologies; but none of these are used to make geographic COL adjustments. (See app. III for detailed descriptions of how the data and indexes from the 12 methodologies are used.)

We identified two additional methodologies but could not locate research that delineated how the methodologies could be implemented to develop a COL index. For example, administrative data from public assistance programs, such as the food stamp program, have been proposed as baseline data for developing a COL adjustment that would indicate the incidence of need within a geographic location. However, in our review of the relevant literature and discussions with specialists, we did not locate appropriate data that could be translated into an index to demonstrate geographic variation. Another approach to identify baseline data for a COL index would be to use information obtained from grocery stores’ universal product code scanners. As in the case of administrative program data, we could not locate information that indicated how the product code data could be used to develop a geographic index or ratio.

During the process of obtaining experts’ ratings of promise for the 12 generic methodologies we identified, some experts indicated that we had not identified and presented all possible methodologies to make such a COL adjustment. A number of the experts suggested using a combination of several attributes from the methodologies that they reviewed. In addition, they identified four other methodologies that could be considered for doing geographic COL adjustments. One was a modification of the local indexes methodology, and another was a modeling technique to develop regional variables to obtain baseline data. The other two focused on ways to revise the current poverty measurement. One methodology included the most basic levels of shelter and food as the basis for measuring poverty. The other methodology, according to an expert, is what the National Academy of Sciences panel is expected to recommend in its forthcoming report. None of these methodologies was identified by more than one of the experts, however.

We recognize that our list of 12 methodologies is not exhaustive, but consider it to provide a fair overview of the wide range of alternatives. The fact that the experts suggested further methodologies, and that no alternative was proposed by more than one expert, suggests that no agreement now exists among experts as to the best way to adjust the measurement of poverty for geographic differences in COL. This is discussed in the next section.

Experts Differed on the Methodologies’ Ability to Make Adjustments

The observation in a 1976 report to Congress that “although there may be geographic differences in the cost of living, there is no known way to make satisfactory geographic adjustments to the poverty cutoffs,” still seems valid. The experts who we asked to assess the methodologies differed about how best to make adjustments because of numerous data and conceptual problems that they identified. Overall, the experts’ ratings of each methodology’s promise for geographically adjusting COL were mixed, and our content analysis of the experts’ comments about each methodology’s strengths and weaknesses yielded diverse and sometimes conflicting perspectives.

Experts Viewed Methodologies as Having Mixed Degrees of Promise

Overall, the experts’ ratings of methodologies were mixed. Although the majority of experts rated certain methodologies as showing little or no promise for adjusting the poverty measurement for geographic differences in COL, no clear consensus was observed overall in the ratings the experts gave regarding the methodologies’ promise for making adjustments. A majority of the experts regarded local indexes, polling, family budgets, consumption data, and the consumer price index methodologies as showing little or no promise for making adjustments. The comparable pay methodology was found by more than two-thirds of the experts to be not promising at all. (See table 2 for experts’ ratings of methodologies.)

No methodology was rated by the majority of experts as showing great or very great promise to adjust the poverty measurement for geographic differences in COL. However, three methodologies—budgets, norms, and housing data—received a rating of at least moderate promise by a majority of the experts. The budgets methodology appeared to have the most promise, but less than half of the experts rated it as having great or very great promise.

Experts Identified Strengths and Weaknesses for Promising Methodologies

Our content analysis of the experts’ comments on each methodology’s strengths and weaknesses showed that the experts shared few common views on any specific methodology. When three or more experts did express a similar comment, it most often concerned a weakness rather than a strength of the methodology being rated. Some experts identified an attribute but expressed different perspectives as to whether it constituted a strength or weakness. Examples of mixed responses included one expert indicating that a strength of a particular methodology was its adaptability for use by government, while another expert characterized the same methodology as not being adaptable for use by government. In some instances, experts agreed about a methodology’s attribute—e.g., its emphasis on children—but differed as to whether the presence of this attribute should be viewed as a strength or weakness. (See figure 1 for strengths and weaknesses of the methodologies.)

Our content analysis of the experts’ comments on the strengths and weaknesses of the three methodologies that received a rating of at least moderate promise by the majority of experts illustrates both the diverse and occasionally contradictory comments of the experts. The strengths of the budgets methodology lie in its representation of low-income families and its use of health and social well-being standards in the determination of the market basket. However, its eclectic approach of using these standards from various sources, which makes it difficult to explain to laypersons, was viewed as a weakness. Another weakness of the budgets methodology cited by the experts is that it fails to make adjustments for regional differences in transportation and some of the other market basket components. The experts who commented about its use of expenditure data were evenly split between those who viewed this as a strength and those who said it was a weakness. This methodology was viewed as capturing both contemporary and outdated concepts of consumption needs. For example, one expert cited the use of current standards as a strength, whereas other experts cited the use of 1981-based data to determine the importance given to items in the market basket as a weakness.

The norms methodology was generally rated as promising because the COL index was frequently updated. The experts, however, differed in their comments about the methodology. For example, more than one-half of the experts said that the lowest income level for which the index was provided was well above poverty and was therefore unrepresentative of low-income families. Conversely, one expert, noting the degree of variation in income levels provided in the index, described it as “more relevant to the poor than other available sources.” Mixed responses of both strengths and weaknesses were indicated for the (1) appropriateness of the items in the market basket, (2) degree of geographic variation shown in the index, (3) ability of the methodology to be adapted and implemented by the government, and (4) cost associated with such implementation.

The housing data methodology was regarded as strong in its focus on what the experts considered the major source of variation in COL. The fact that housing was the only cost measured was also cited as this methodology’s major weakness. As shown in table 3, the experts had mixed views about the representation in the baseline data of families living in poverty. The experts also lacked agreement on whether the housing concepts were appropriate. For example, one expert said the methodology had the “merit of focusing on rents for a specified type of apartment,” while another said that “decent, safe, and sanitary” qualities of housing should be controlled in the measure to prevent downward bias in low-income areas.

Weaknesses Identified for Methodologies With Little or No Promise

A content analysis of the experts’ comments revealed that the local indexes methodology had many weaknesses resulting from its price data collection methods, which involve volunteers from chambers of commerce collecting and averaging prices that are representative of purchases of middle-management households in their local areas. This methodology was viewed to be an unsuitable representation of the consumption needs of the poor. Another weakness of the local indexes methodology was its exclusion of nonmetropolitan and rural areas.

The polling methodology was regarded by several experts as a means to validate the measurement of poverty, rather than as an approach to make geographic COL adjustments. These experts said that this methodology provided insight into the relationship between an absolute measure of poverty, such as the current official measure, and a measure that is relative—that is, a measure that changes with growth in the economy or according to society’s perception of an adequate level of income. According to the experts’ comments, the main weakness of polling was in the quality of the data obtained through a public opinion survey. It was thought that the respondents would be biased in providing their estimates. For example, one expert wrote: “If respondents knew the survey results would be used to adjust poverty thresholds with implications for program expenditures and income taxes, then some may intentionally deflate or inflate their response, in their own self-interest.” The experts had mixed views about the costs associated with this method; some experts said it would be cost effective, while others said it would be costly.

According to the experts’ comments, the main weakness of the comparable pay methodology was its reliance on employers’ labor costs. Many experts said that such a measure included influences other than COL and that as a consequence it was inappropriate and an unsuitable substitute for COL, especially as a representation of the needs of the poor. For example, one expert said, “Geographic variations in quality of life affect the relationship between wages/salaries and living costs. Use of employer costs as a measure of living costs would introduce significant regional bias.” Many weaknesses, as well as several mixed responses, were noted for the remaining three methodologies—consumption data, family budgets, and consumer price index.

Conclusion

The concept of adjusting the measurement of poverty for geographic differences in COL has been seen as problematic, and remains so. We asked recognized experts to review 12 methodologies that illustrate the range of alternative approaches to adjust poverty measurement for geographic COL differences, and there was no consensus among these experts that any one methodology was the most promising for making such an adjustment. The fact that several of these experts suggested additional methodologies, but that no additional methodology was suggested by more than one of the experts, suggests to us that a consensus on any one approach does not exist. Where there does appear to be agreement, however, is that several of the methodologies offer little or no promise of appropriately adjusting the measurement of poverty for geographic COL differences. Further, obtaining a consensus on what items should go into a COL index’s market baskets to reflect regional differences in consumption would be difficult.

As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 20 days after its issue date. At that time, we will send copies of the report to the Secretary of Commerce, the Secretary of Labor, the Director of the Office of Management and Budget, and other interested parties. We will also make copies available to others on request.

If you have any questions concerning this report, please call me on (202) 512-8676. Major contributors to the report are listed in appendix IV.

Scope and Methodology

To address the first two objectives of this job—describing the function of a market basket in determining a COL index, and identifying potential methods for calculating a COL adjustment—we first reviewed the relevant literature and held discussions with specialists in the field. These specialists included individuals associated with poverty measurement or COL data at the Bureau of Labor Statistics (BLS) and the Bureau of the Census, as well as private organizations and academic institutions. We also included individuals who did not support geographic adjustment of the poverty measurement, as well as those who have proposed methodologies to achieve this objective.

On the basis of our literature review and preliminary discussions with specialists, we described the function of a market basket and identified an initial set of methodologies that might have potential for adjusting poverty measurement for geographic differences in the COL. We grouped similar methodologies into 12 categories and gave a generic name to each. We excluded potential methodologies if they did not identify existing data that could be turned into a geographically adjusted index. Two methods, one based on use of data from administrative records and one relying on data scanning of uniform product codes, were eliminated because they did not meet this criterion.

To meet our third objective of obtaining expert opinion on the ability of these methodologies to adjust the poverty measure for geographic differences in COL, we selected a panel of 15 experts and surveyed them using a data collection instrument that contained brief descriptions of each of the 12 generic methodologies we identified. We asked the panel to review each description and rate each methodology in terms of its promise for use in adjusting the poverty measurement for geographic differences in COL.

The description of each methodology identified data sources, discussed the cost and time needed to develop an index with the methodology, and provided an example of how the calculations would be made and the index could be used. We asked the developer or someone very familiar with each methodology to review our brief description to ensure that it accurately conveyed the essence of the methodology.

We asked the selected experts to rate each methodology on a five-point scale that ranged from “not promising at all” to “shows very great promise,” and then briefly discuss the strengths and weaknesses of the methodology. The experts were also asked to identify any additional methodology we may have overlooked and provide their views on the major challenges and costs associated with developing COL data that could be used to geographically adjust the poverty measure.

We randomly chose 15 individuals to serve as experts from a candidate list of more than 40 names. To obtain a diverse candidate pool reflective of the different interests involved, we asked for nominations of potential experts from those specialists in the field and representatives of major statistical agencies that we met with during our initial discussions and literature review. To avoid potential conflicts of interest, we excluded individuals from the list who are currently serving on the National Academy of Sciences’ Panel on Poverty and Family Assistance or who are political appointees. We recognize that the responses we received reflect only the views of the experts included.

Several of the experts initially selected were unable to participate. We replaced these individuals with alternates from the remaining pool of candidates. (See app. II for a list of the participating experts.)

Before contacting our initial selections, we asked congressional staff and officials from Census, BLS, and the Office of Management and Budget to review the list for balance and to identify any additional experts they believed should be included. No additions were suggested.

The selected experts received a package containing a letter of introduction, an instruction sheet, descriptions of all the methodologies, and response sheets (see app. III). The package was sent on November 14, 1994.

Responses were received from all 15 experts by January 6, 1995. We tabulated the ratings for each methodology to obtain an overall assessment of the experts’ opinions of how promising each methodology was for use in adjusting the poverty threshold for geographic differences in COL.

We also did a content analysis of the experts’ responses to the strengths and weaknesses question for each methodology. From an initial reading of the responses, we developed a list of cited strengths and weaknesses. We used this list to code the responses of all experts for each methodology. The coding of the responses was verified by a second coder, and a third person checked coding reliability. As a method of focusing our analysis on the recurring comments made by the experts in their discussions of each methodology’s strengths and weaknesses, we adopted a decision rule to report only those comments made by three or more experts for a particular methodology’s attribute.

Experts’ comments on market baskets were identified separately and were used in our description of the function of the market basket. Additionally, we used experts’ general comments on major challenges and costs associated with geographically adjusting poverty measures to illustrate our results.

Experts and Their Affiliations

Mark C. Berger University of Kentucky Dixie Blackley Le Moyne College Tom Carlin Department of Agriculture Lawrence Gibson Eric Marder Associates, Inc.

Data Collection Instrument and Descriptions of the Methodologies

This appendix contains copies of the cover letter, instruction sheet, answer sheets, and brief descriptions of the 12 methodologies that we sent to the 15 experts we selected to review the methodologies.

Major Contributors to This Report

Related GAO Products

Federal Aid: Revising Poverty Statistics Affects Fairness of Allocation Formulas (GAO/HEHS-94-165, May 20, 1994).

Poverty Trends, 1980-88: Changes in Family Composition and Income Sources Among the Poor (GAO/PEMD-92-34, Sept. 10, 1992).

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (301) 258-4097 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Address Correction Requested