September 5, 2018 

VIA ELECTRONIC MAIL 

Director, Information Collection Clearance Division 

Office of the Chief Privacy Officer 

Office of Management 

U.S. Department of Education 

400 Maryland Avenue S.W. 

LBJ, Mailstop L-OM-2-2E319 

Washington, D.C. 20202 

OMInformationQuality@ed.gov 

Principal Deputy Assistant Secretary Office of Management 

U.S. Department of Education 

400 Maryland Avenue S.W. 

LBJ, Room 2W311 

Washington, D.C. 20202 

OMInformationQualityRequests@ed.gov 

ATTN: Information Quality Request 

To whom it may concern: 

This is a Petition for Correction and Disclosure (“Petition”) in accordance with the Information 

Quality Act (“IQA”), the information and quality guidelines issued by the Office of Management 

and Budget (“OMB”), and the IQA Guidelines1 issued by the U.S. Department of Education (the 

“Department”).2 

This Petition focuses on the Department’s recent publication of a Notice of Proposed Rulemaking 

(“NPRM”) that proposes to “rescind” the Department’s Gainful Employment regulation.

3 The 

NPRM includes an abundance of factual claims without disclosing the underlying sources or 

methodologies, a clear failure to comply with the IQA. Where the NPRM does cite sources, it still 

violates the IQA by repeatedly stating conclusions that are not clearly supported by the evidence. 

These failures render meaningless the entire purpose of the public comment period—i.e. , to allow 

the Department to properly determine how to ensure that certain institutions of higher education— 

as a condition of their eligibility to receive Title IV, Higher Education Act (“HEA”) program 

3 

), the Department instructs the public to submit IQA 

correction requests to the Director, Information Collection Clearance Division. Out of an abundance of caution, we are 

providing it to both recipients. 

https://www2.ed.gov/policy/gen/guid/iq/iqg_5a.html

), the Department instructs 

the public to submit IQA correction requests to the Principal Deputy Assistant for Management. In the non-PDF 

version (

https://www2.ed.gov/policy/gen/guid/iq/infoqualguide.pdf

The ED Guidelines do not provide clear instructions on how to submit this sort of petition. In the PDF 

version of the ED Guidelines (

2 

https://www2.ed.gov/policy/gen/guid/iq/infoqualguide.pdf) (hereinafter the “ED Guidelines”)). 

See Exh. A (U.S. Dep’t of Educ., “Information Quality Guidelines” (2002), available at: 1 

83 Fed. Reg. 40,167 (Aug. 14, 2018). 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 2 of 14 

September 5, 2018 

funds—prepare students for gainful employment in a recognized occupation. As a result, potential 

commenters, including NSLDN,

4 will not be able to provide effective feedback on the Department’s 

proposed rescission. 

Because NSLDN stands to suffer harm from the continued dissemination of this information, it is 

an “affected person” under the IQA and may, therefore, submit this Petition. Although the ED 

Guidelines do not define “affected,”5 NSLDN will be harmed by the disseminated information, 

particularly because it uses the published information for a variety of purposes. Not only does 

NSLDN intend to submit written comments in response to the NPRM, but also NSLDN actively 

studies, researches, and proposes student-focused policies at the state and local level. By 

disseminating information that fails to meet the basic standards of the IQA, the Department is 

infringing upon NSLDN’s significant interest in ensuring that the Department relies upon and 

publishes only accurate and reliable data in its communications with the public. 

Given the current inaccuracies in the Department’s NPRM on Gainful Employment, NSLDN 

requests that the Department rescind this NPRM immediately and, if the Department desires, 

correct and reissue it with information that complies with the IQA. 

1. Background on the 2014 Gainful Employment Regulation 

Finalized in 2014, the Gainful Employment regulation was expressly “intended to address growing 

concerns about educational programs” that “are required by statute to prepare students for gainful 

employment in a recognized occupation . . .[,] but instead are leaving students with unaffordable 

levels of loan debt in relation to their earnings.”6 More specifically, the Department was concerned 

that covered “gainful employment programs”—also known as “GE programs”—“[d]o not train 

students in the skills they need to obtain jobs in the occupation for which the program purports to 

provide training.”7 The Department also worried that these programs “experience a high number of 

withdrawals or ‘churn’ because relatively large numbers of students enroll, but few, or none, 

complete the program, which can often lead to default.”8 In other words, the Gainful Employment 

regulation aimed to crack down on GE programs that were both ineffective and expensive, often 

leaving students with debts that they could never hope to repay. 

Motivated by these concerns, the Department established a regulatory framework with two key 

components: accountability and transparency. The “accountability framework” defined “what it 

means to prepare students for gainful employment in a recognized occupation by establishing 

https://www2.ed.gov/policy/gen/guid/iq/infoqualguide.pdf

See generally U.S. Dep’t of Educ., “Information Quality Guidelines” (2002), available at: 5 

NSLDN is a non-profit, nonpartisan organization that works, through litigation and advocacy, to advance 

students’ rights to educational opportunity and to ensure that higher education provides a launching point for economic 

mobility. 

4 

6 

Id. 

7 

8 

Id. 

79 Fed. Reg. 64,890 (Oct. 31, 2014). 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 3 of 14 

September 5, 2018 

measures by which the Department will evaluate whether a GE program remains eligible” to receive 

federal student aid funds.9 The “transparency framework” aimed to “increase the quality and 

availability of information about the outcomes of students enrolled in GE programs” based on the 

assumption that “[b]etter outcomes information will benefit . . . [s]tudents, prospective students, and 

their families, as they make critical decisions about their educational investments,”10 as well as the 

public, taxpayers, the government, and institutions of higher education.11 

The Department relied upon extensive peer-reviewed, statistical information to inform its design of 

the 2014 Gainful Employment rule. For example, the Department provided data on the high tuition 

costs, poor outcomes, and deceptive practices at some institutions in the for-profit sector to 

illustrate why its concern with GE programs at those institutions in particular was justified, rather 

than biased.

12 In addition, the Department relied upon multiple studies and authorities to set the 

D/E rates measures at twenty percent for discretionary income and eight percent for annual 

earnings.13 Finally, the Department conducted its own regression analyses to explore the influence 

of demographic factors—such as socioeconomic status, race and ethnicity, gender, marital status, 

and family history of pursuing higher education—on annual earnings.

14 These analyses showed that 

student demographics were “not strong predictors” of which programs would pass or fail the D/E 

rates measures.15 Because of this reliance on peer-reviewed, scientific evidence-based research, the 

Department easily met its obligations under the IQA. 

In addition, the Department met its obligations to engage in reasoned decision-making under the 

Administrative Procedure Act (“APA”). Shortly after the promulgation of the 2014 Gainful 

Employment rule, two separate lawsuits attacked the rule’s validity. Ultimately, both federal district 

courts that presided over these lawsuits, as well as the United States Court of Appeals for the D.C. 

Circuit that heard one of the lawsuits on appeal, upheld the Gainful Employment regulation in its 

entirety.

16 

9 79 Fed. Reg. at 64,890. 10 Id. at 64,890. 11 Id. 12 79 Fed. Reg. at 64,904-08. 13 Id. at 64,919-22. 14 Id. at 64,910, 65,037-74. 15 Id. at 64,910. 16 See, e.g., Ass'n of Private Sector Coll. & Univ. v. Duncan, 110 F. Supp. 3d 176 (D.D.C. 2015), aff'd, 640 F. App'x 5 

(D.C. Cir. 2016); Ass'n of Proprietary Colleges v. Duncan, 107 F. Supp. 3d 332, 341 (S.D.N.Y. 2015). Two years later, 

however, in June 2017, a federal judge granted narrow relief to a trade association for cosmetology schools, which 

argued that the GE rule was arbitrarily applied to cosmetology school graduates because those graduates tended to 

underreport their income earned from tips. Although the judge narrowly tailored the decision to members of the 

association only so as to avoid “upending the entire GE regulatory scheme,” Am. Ass'n of Cosmetology Sch. v. DeVos, 258 

F. Supp. 3d 50 (D.D.C. 2017), the Department later cited this decision to justify delaying enforcement of the entire 

Gainful Employment regulation, see 82 Fed. Reg. 39,362 (Aug. 18, 2017). That delay is currently being challenged by a 

group of state Attorneys General. See State of Maryland v. Dep’t of Educ., No. 1:17-cv-0239 (D.D.C. 2017). 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 4 of 14 

September 5, 2018 

The NPRM stands in stark contrast to the Department’s earlier rulemaking efforts, however. As 

explained further below, the Department’s proposed rescission of the 2014 rule relies upon 

inaccurate, misleading, and unsourced information in violation of the IQA. 

2. Grounds for Disclosure and Correction under the IQA 

The IQA and its implementing guidelines require that information disseminated to the public by 

federal agencies be accurate, reliable, and unbiased. Indeed, the IQA—passed by Congress in 

2001—directed the OMB to require that each applicable federal agency “issue guidelines ensuring 

and maximizing the quality, objectivity, utility, and integrity of information (including statistical 

information)” that the agency disseminated.17 In response to the statute, OMB issued final 

guidelines implementing the IQA and requiring agencies to publish their own guidelines no later 

than October 1, 2002.18 

Similar to OMB’s Guidelines, the ED Guidelines apply to “information,” i.e., “any communication 

or representation of knowledge, such as facts or data, in any medium or form” that is 

“disseminat[ed]” to the public.19 Exh. A at 1 (emphasis added). The ED Guidelines affirm that, 

“[t]o make sound decisions, the Department intends to accept and use only information that is 

accurate and reliable.” Exh A at 2. Furthermore, the ED Guidelines make clear that it is similarly 

“important that the information the Department [itself] disseminates be accurate and reliable.” Id. at 

1. 

The Department uses three factors to assess the quality of information it disseminates: “utility, 

objectivity, and integrity.” Exh. A at 4. As relevant here, the ED Guidelines define objectivity as 

follows: 

Objectivity refers to the accuracy, reliability, and unbiased nature of 

information. It is achieved by using reliable information sources and 

appropriate techniques to prepare information products. Objectivity 

involves both the content and the presentation of the information. 

Content should be complete, include documentation of the source of 

any information used, as well as, when appropriate, a description of 

the sources of any errors in the data that may affect the quality of the 

information product. 

Id. at 5 (emphasis removed). The ED Guidelines then go on to list what each dissemination of 

general information should include in order to be considered “objective,” such as: 

• “[D]raw[ing] upon peer-reviewed, scientific evidence-based research that is appropriately 

documented;” 

17 Consolidated Appropriations Act, Pub. L. No. 106-554, § 515(a), 114 Stat. 2763 (2001). 18 67 Fed. Reg. 8,452 (Feb. 22, 2002). 19 See also 67 Fed. Reg. at 8,453 (establishing that “information” means “any communication or representation of 

knowledge such as facts or data”). 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 5 of 14 

September 5, 2018 

• “Clearly identify[ing] data sources;” [and] 

• “Confirm[ing] and document[ing] the reliability of the data, and acknowledg[ing] any 

shortcoming or explicit errors in any data that is included.” 

Id. at 5. Additionally, the ED Guidelines state that, to be considered “objective,” each dissemination 

of research and evaluation information should: 

• “Have a research study approach or data collection technique that is well thought out, 

designed to use state of the art methodologies in the data collection, and be clearly 

described;” 

• Present conclusions that are strongly supported by the data;” [and] 

• “Undergo peer review.” 

Id. at 6. 

Beyond objectivity, the Department has also imposed heightened requirements for information 

quality when that information is deemed “particularly influential.” See Exh. A at 9 (“Government 

information that is particularly influential needs to meet higher quality standards, and in particular 

must be reproducible.”). Per the ED Guidelines, information is “influential” if the Department 

determines “that the information is reasonably likely to have a clear and substantial impact on public 

policies or private sector decisions if disseminated.” Id. In the instant case, the Department has 

already determined that the NPRM’s proposed rescission constitutes an “economically significant 

regulatory action” under Executive Order 12,866.20 Thus, the information contained in the NPRM 

is “influential.” Pursuant to the ED Guidelines, then, that information “must be accompanied by 

supporting documentation that allows an external user to understand clearly the information and be 

able to reproduce it, or understand the steps involved in producing it.” Exh. A at 10. 

Despite the clear standards set forth in both the ED Guidelines and the IQA, the Department’s 

NPRM is filled with examples of information that are not supported by sources, do not stand for 

the proposition cited, fail to explain the methodology used, or otherwise are not “accompanied by 

information that allows an external user to understand clearly the information and be able to 

reproduce it, or understand the steps involved in producing it.” 

3. Specific IQA Violations 

The chart below provides a specific description of information disseminated in the NPRM that 

violates the IQA, as well as the basis for each IQA violation, including, where appropriate, an 

explanation of why a particular statement contains inaccurate, unreliable, or misleading information. 

NPRM STATEMENT IQA VIOLATION 

“The first D/E rates were published in 2017, 

and the Department’s analysis of those rates 

1. Fails to clearly describe the research study 

approach or data collection technique 

20 See 83 Fed. Reg. at 40,177. 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 6 of 14 

September 5, 2018 

raises concerns about the validity of the metric, 

and how it affects opportunities for Americans 

to prepare for high-demand occupations in the 

healthcare, hospitality, and personal services 

industries, among others.”21 

2. Fails to clearly identify data sources 

3. Fails to confirm and document the 

reliability of the data and acknowledge any 

shortcomings or explicit errors 

4. Fails to undergo peer review 

5. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“In promulgating the 2011 and 2014 

regulations, the Department cited as 

justification for the 8 percent D/E rates 

threshold a research paper published in 2006 by 

Baum and Schwartz that described the 8 

percent threshold as a commonly used 

mortgage eligibility standard. However, the 

Baum & Schwartz paper makes clear that the 8 

percent mortgage eligibility standard ‘has no 

particular merit or justification’ when proposed 

as a benchmark for manageable student loan 

debt. . . . Upon further review, we believe that 

the recognition by Baum and Schwartz that the 

8 percent mortgage eligibility standard ‘has no 

particular merit or justification’ when proposed 

as a benchmark for manageable student loan 

debt is more significant than the Department 

previously acknowledged and raises questions 

about the reasonableness of the 8 percent 

threshold as a critical, high-stakes test of 

purported program performance.”22 

1. Fails to present conclusions that are 

strongly supported by the data 

a. This failure has been highlighted 

recently by Sandy Baum, the co-author 

of the 2006 study cited by the 

Department. In that post, Baum stated 

that “the Department of Education has 

misrepresented my research, creating a 

misleading impression of evidencebased policymaking. The Department 

cites my work as evidence that the GE 

standard is based on an inappropriate 

metric, but the paper cited in fact 

presents evidence that would support 

making the GE rules stronger.”23 

b. Baum further asserts that “[the 

Department is] correct that we were 

skeptical of [the 8 percent] standard for 

determining affordable payments for 

individual borrowers, but incorrect in 

using that skepticism to defend 

repealing the rule. In fact, our 

examination of a range of evidence 

about reasonable debt burdens for 

students would best be interpreted as 

supporting a stricter standard.”24 That 

is because Baum and her co-author’s 

21 

22 

Id. at 40,171. 

Id. 23 Sandy Baum, “DeVos Misrepresents the Evidence in Seeking Gainful Employment Deregulation,” Urban Wire: 

24 

Education and Training (Aug. 22, 2018), https://www.urban.org/urban-wire/devos-misrepresents-evidence-seekinggainful-employment-deregulation. 

Id. 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 7 of 14 

September 5, 2018 

“research set a guideline for a level of 

debt payments no individual 

student should exceed. Under GE, half 

of a program’s graduates could exceed 

this limit before sanctions would kick 

in.”25 

c. A complete copy of Baum’s blog post is 

attached hereto as Exhibit B. 

“Research published subsequent to the 

promulgation of the GE regulations adds to the 

Department’s concern about the validity of 

using D/E rates as to determine whether or not 

a program should be allowed to continue to 

participate in title IV programs.”26 

1. Fails to identify data sources, including 

whether it is peer-reviewed and scientific 

evidence-based 

2. Fails to confirm and document the 

reliability of the data and acknowledge any 

shortcomings or explicit errors 

3. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“[T]he highest quality programs could fail the 

D/E rates measures simply because it costs 

more to deliver the highest quality program and 

as a result the debt level is higher.”27 

1. Fails to identify data sources 

2. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“Other research findings suggest that D/E 

rates-based eligibility creates unnecessary 

barriers for institutions or programs that serve 

larger proportions of women and minority 

students. Such research indicates that even 

with a college education, women and 

minorities, on average, earn less than white men 

who also have a college degree, and in many 

cases, less than white men who do not have a 

college degree.”28 

1. Fails to draw upon peer-reviewed sources 

2. Fails to acknowledge any shortcomings or 

explicit errors in the data 

3. Fails to present conclusions that are 

strongly supported by the data 

a. Indeed, the source cited by the 

Department does not draw this same 

conclusion. For example, the cited 

table appears to relate to graduates of 

bachelor’s degree programs, and not gainful 

employment programs. 

4. Fails to “be accompanied by supporting 

documentation that allows an external user 

25 Id. 26 83 Fed. Reg. at 40,171. 27 Id. 

Id. (citing Jennifer Ma, Matea Pender, & Meredith Welch, “Education Pays 2016: The Benefits of Higher 

Education for Individuals and Society,” CollegeBoard Trends in Higher Education Series Figure 2.4 (2016), available at: 

28 

https://trends.collegeboard.org/sites/default/files/education-pays-2016-full-report.pdf). 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 8 of 14 

September 5, 2018 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“[D]ue to a number of concerns with the 

calculation and relevance of the debt level 

included in the rates[,] we do not believe that 

the D/E rates measure achieves a level of 

accuracy that it should alone determine whether 

or not a program can participate in title IV 

programs.”29 

1. Fails to clearly describe the research study 

approach 

2. Fails to identify data sources 

3. Fails to confirm and document the 

reliability of the data 

4. Fails to undergo peer review 

5. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“[I]ncreased availability of [income-driven] 

repayment plans with longer repayment 

timelines is inconsistent with the repayment 

assumptions reflected in the shorter 

amortization periods used for the D/E rates 

calculation in the GE regulations.”30 

1. Fails to rely upon peer-reviewed, scientific 

evidence-based research 

2. Fails to identify data sources 

3. Fails to confirm and document the 

reliability of the data 

4. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“[A] program’s D/E rates can be negatively 

affected by the fact that it enrolls a large 

number of adult students who have higher 

Federal borrowing limits, thus higher debt 

levels, and may be more likely than a 

traditionally aged student to seek part-time 

work after graduation in order to balance family 

and work responsibilities.”31 

“[I]t is the cost of administering the program 

that determines the cost of tuition and fees.”32 

1. Fails to rely upon peer-reviewed, scientific 

evidence-based research 

2. Fails to identify data sources 

3. Fails to confirm and document the 

reliability of the data 

1. Fails to rely upon peer-reviewed, scientific 

evidence-based research 

2. Fails to identify data sources 

3. Fails to confirm and document the 

reliability of the data 

4. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

29 Id. 30 Id. at 40,172. 31 

32 

Id. 

Id. 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

Department of Education 

Page 9 of 14 

September 5, 2018 

U.S. 

be able to reproduce it, or understand the 

steps involved in producing it” 

“Programs that serve large proportions of adult 

learners may have very different outcomes 

from those that serve large proportions of 

traditionally aged learners.”33 

1. Fails to rely upon peer-reviewed, scientific 

evidence-based research 

2. Fails to identify data sources 

3. Fails to confirm and document the 

reliability of the data 

4. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“[T]he first set of D/E rates, published in 2016, 

revealed that the D/E rates, and particularly 

earnings, vary significantly from one 

occupation to the next, and across geographic 

regions within a single occupation.”34 

1. Fails to clearly describe the research study 

approach 

2. Fails to confirm and document the 

reliability of the data 

3. Fails to undergo peer review 

4. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“Data discussed during the third session of the 

most recent negotiated rulemaking 

demonstrated that even a small change in 

student loan interest rates could shift many 

programs from a ‘passing’ status to ‘failing,’ or 

vice versa, even if nothing changed about the 

programs’ content or student outcomes.”35 

1. Fails to clearly describe the research study 

approach and data collection technique 

2. Fails to identify data sources 

3. Fails to confirm and document the 

reliability of the data 

4. Fails to undergo peer review 

5. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“Table 1—Number and Percentage of GE 1. Fails to clearly describe the research study 

2015 Programs That Would Pass, Fail, or Fall approach and data collection technique 

into the Zone Using Different Interest Rates”36 2. Fails to identify data sources 

3. Fails to confirm and document the 

reliability of the data 

4. Fails to undergo peer review 

33 Id. 34 Id. 35 Id. 36 Id. 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 10 of 14 

September 5, 2018 

5. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“[T]he Department now recognizes that 

assigning a 10-year amortization period to 

graduates of certificate and associate degree 

programs for the purpose of calculating D/E 

rates creates is an unacceptable and unnecessary 

double standard since the REPAYE plan 

regulations promulgated in 2015 provide a 20-

year amortization period for those same 

graduates.”37 

1. Fails to draw upon peer-reviewed, scientific 

evidence-based research 

“There is significant variation in methodologies 

used by institutions to determine and report infield job placement rates, which could mislead 

students into choosing a lower performing 

program that simply appears to be higher 

performing because a less rigorous 

methodology was employed to calculate in-field 

job placement rates.”38 

1. Fails to clearly describe the research study 

approach and data collection technique 

2. Fails to clearly identify data source 

3. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

be able to reproduce it, or understand the 

steps involved in producing it” 

“The Department also believes that it 

underestimated the burden associated with 

distributing the disclosures directly to 

prospective students. . . . A negotiator 

representing financial aid officials confirmed 

our concerns, stating that large campuses, such 

as community colleges that serve tens of 

thousands of students and are in contact with 

many more prospective students, would not be 

able to, for example, distribute paper or 

electronic disclosures to all the prospective 

students in contact with the institution.”39 

1. Fails to draw upon peer-reviewed, 

scientific-evidence based research 

2. Fails to confirm and document the 

reliability of the data 

“The Department believes that the best way to 

provide disclosures to students is through a 

data tool that is populated with data that comes 

directly from the Department, and that allows 

prospective students to compare all institutions 

1. Fails to draw upon peer-reviewed, scientific 

evidence-based research 

2. Fails to identify data sources 

a. Specifically, in the 2014 rule, the 

Department stated that it “would 

37 Id. at 40,172-73. 38 Id. at 40,173. 39 Id. 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 11 of 14 

September 5, 2018 

through a single portal, ensuring that important 

consumer information is available to students 

while minimizing institutional burden.”40 

conduct consumer testing” to 

determine how to make student 

disclosures as meaningful as possible.41 

The NPRM fails to acknowledge 

whether such testing occurred, 

including the results of that testing. 

The NPRM also fails to state any other 

basis for the Department’s 

conclusions. 

“[T]he Department does not believe it is 

appropriate to attach punitive actions to 

program-level outcomes published by some 

programs but not others. In addition, the 

Department believes that it is more useful to 

students and parents to publish actual median 

earnings and debt data rather than to utilize a 

complicated equation to calculate D/E rates 

that students and parents may not understand 

and that cannot be directly compared with the 

debt and earnings outcomes published by nonGE programs.”42 

1. Fails to draw upon peer-reviewed, scientific 

evidence-based research 

2. Fails to identify data sources 

“The Department has reviewed additional 

research findings, including those published by 

the Department in follow-up to the Beginning 

Postsecondary Survey of 1994, and determined 

that student demographics and socioeconomic 

status play a significant role in determining 

student outcomes.”43 

1. Fails to identify data sources 

a. Specifically, the website cited by the 

Department links to the Beginning 

Postsecondary Survey of 1994’s 

findings, and not the “additional 

research” mentioned by the 

Department, including the 

Department’s own “follow-up.” 

2. Fails to confirm and document the 

reliability of the data 

3. Fails to “be accompanied by supporting 

documentation that allows an external user 

to understand clearly the information and 

40 Id. 41 See, e.g., 34 C.F.R. §§ 668.410(a)(3), 668.412(a). See also 79 Fed. Reg. 64,890, 64,966 (Oct. 31, 2014) (“The 

regulations include text for the student warnings. The Secretary will use consumer testing to inform any modifications 

to the text that have the potential to improve the warning’s effectiveness. As a part of the consumer testing process, we 

will seek input from a wide variety of sources[.]”); id. at 64,969 (noting that while “direct delivery” of warnings to 

students “make it most likely that students receive . . . and review” the information, the Department would conduct 

consumer testing regarding the “most effective delivery methods”). 42 83 Fed. Reg. at 40,174. 43 Id. 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 12 of 14 

September 5, 2018 

be able to reproduce it, or understand the 

steps involved in producing it” 

“The GE regulations failed to take into account 

the abundance of research that links student 

outcomes with a variety of socioeconomic and 

demographic risk factors.”44 

“The GE regulations underestimated the cost 

of delivering a program and practices within 

occupations that may skew reported earnings. 

According to Delisle and Cooper, because 

public institutions receive State and local 

taxpayer subsidies, ‘even if a for-profit 

institution and a public institution have similar 

overall expenditures (costs) and graduate 

earnings (returns on investment), the for-profit 

institution will be more likely to fail the GE 

rule, since more of its costs are reflected in 

student debt.’ Non-profit, private institutions 

also, in general, charge higher tuition and have 

students who take on additional debt, including 

enrolling in majors that yield societal benefits, 

but not wages commensurate with the cost of 

the institution.”45 

“In the case of cosmetology programs, State 

licensure requirements and the high costs of 

delivering programs that require specialized 

facilities and expensive consumable supplies 

may make these programs expensive to operate, 

which may be why many public institutions do 

not offer them. In addition, graduates of 

cosmetology programs generally must build up 

their businesses over time, even if they rent a 

chair or are hired to work in a busy salon.”46 

1. Fails to identify data sources 

2. Fails to confirm and document the 

reliability of the data 

1. Fails to present conclusions that are 

strongly supported by the data 

a. The Delisle and Cooper study cited by 

the Department does not support its 

conclusion that the GE regulations 

“underestimated the cost of delivering a 

program and practices within 

occupations that may skew reported 

earnings.” 

2. Fails to identify data sources 

1. Fails to identify data sources 

2. Fails to confirm and document the 

reliability of the data 

“[S]ince a great deal of cosmetology income 

comes from tips, which many individuals fail to 

accurately report to the Internal Revenue 

Service, mean and median earnings figures 

produced by the Internal Revenue Service 

underrepresent the true earnings of many 

1. Fails to present conclusions that are 

strongly supported by the data 

a. The IRS tax gap study cited by the 

Department does not support the 

Department’s specific conclusions 

about cosmetology graduates. The 

44 Id. 45 Id. 46 Id. 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 13 of 14 

September 5, 2018 

workers in this field in a way that institutions 

cannot control.”47 

study is from 2012 and covers tax year 

2006 only. 

2. Fails to confirm and document the

reliability of the data

1. Fails to present conclusions that are

strongly supported by the data

a. Despite asserting that the alternate

appeals process is “time-consuming and

resource-intensive, with great variations

in the format and completeness of

appeals packages,” the Department then

“estimates that it would take

Department staff [only] 10 hours per

appeal to evaluate the information

submitted.”49 

2. Fails to “be accompanied by supporting

documentation that allows an external user

to understand clearly the information and

be able to reproduce it, or understand the

steps involved in producing it”

a. Since March 2018, NSLDN has

attempted to obtain copies of

institutions’ alternate earnings appeals.

50 

To date, the Department has not

provided such information.

“While the GE regulations include an alternate 

earnings appeals process for programs to 

collect data directly from graduates, the process 

for developing such an appeal has proven to be 

more difficult to navigate than the Department 

originally planned. The Department has 

reviewed earnings appeal submissions for 

completeness and considered response rates on 

a case-by-case basis since the response rate 

threshold requirements were set aside in the 

AACS litigation. Through this process, the 

Department has corroborated claims from 

institutions that the survey response 

requirements of the earnings appeals 

methodology are burdensome given that 

program graduates are not required to report 

their earnings to their institution or to the 

Department, and there is no mechanism in 

place for institutions to track students after they 

complete the program. The process of 

Departmental review of individual appeals has 

been time-consuming and resource-intensive, 

with great variations in the format and 

completeness of appeals packages.48 

“We believe that the analysis and assumptions 

with respect to earnings underlying the GE 

regulations are flawed.”51 

1. Fails to draw upon peer-reviewed, scientific

evidence-based research

2. Fails to confirm and document the

reliability of the data

“There are student costs and benefits 

associated with enrollment in a program that 

would have otherwise lost eligibility to 

participate in the title IV, HEA programs under 

the GE regulations; however, the actual 

1. Fails to draw upon peer-reviewed, scientific

evidence-based research

2. Fails to identify data sources

3. Fails to confirm and document the

reliability of the data

47 Id. 48 Id. at 40,174-75. 49 Id. at 40,179. 50 See generally, Complaint, Nat’l Student Legal Def. Network v. U.S. Dep’t of Educ., No. 1:18-cv-01209-TSC (D.D.C. 

83 Fed. Reg. at 40,175. 

May 23, 2018). 51 
Principal Deputy Assistant Secretary Office of Management & 

Director, Information Collection Clearance Division 

U.S. Department of Education 

Page 14 of 14 

September 5, 2018 

 

 

outcome for students enrolled in failing or zone 

programs under the GE regulations is 

unknown.”52 

 

 

 

 

 

 

 

 

 

 

 

* * * 

Given the importance and immediacy of the public comment period for the NPRM proposing to 

rescind the Gainful Employment regulation in its entirety, and the lack of quality information that 

the Department is disseminating as a part of that process, NSLDN requests that the Department 

rescind the NPRM immediately and, if the Department desires, correct and reissue it with 

information that complies with the IQA. 

Sincerely, 

Robyn Bitner, Counsel53 

52 Id. at 40,178. 53 Ms. Bitner is a member of the New York Bar only. She is currently practicing in the District of Columbia 

under the supervision of members of the D.C. Bar while her D.C. Bar application is pending. 
U.S. Department of Education 

 Information Quality Guidelines 

Introduction 

Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 

(Public Law 106-554) directed the U.S. Office of Management and Budget (OMB) to issue 

government-wide guidelines that “provide policy and procedural guidance to Federal agencies 

for ensuring and maximizing the quality, objectivity, utility, and integrity of information 

(including statistical information) disseminated by Federal agencies.” Information, as defined by 

OMB, includes any communication or representation of knowledge, such as facts or data, in any 

medium or form, including textual, numerical, graphic, cartographic, narrative, or audiovisual 

forms. Dissemination refers to any distribution of information to the public that is initiated or 

sponsored by a federal agency. (OMB, Guidelines for Ensuring and Maximizing the Quality, 

Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, February 22, 

2002, 67 FR 8452-8460). 

In summary, OMB’s guidelines, issued on February 22, 2002, direct agencies to: 

• Develop and implement their own agency-specific information quality guidelines by 

October 1, 2002; 

• Adopt a basic standard of quality (including objectivity, utility, and integrity) as a 

performance goal and incorporate the standard into the agency’s operations; 

• Develop a process for reviewing the quality of information to ensure quality before 

information is disseminated; 

• Establish a process for affected persons to request correction of information that may 

not comply with OMB’s or the agency’s guidelines; and 

• Report annually to the Director of OMB, beginning January 1, 2004, the number and 

nature of complaints received by the agency regarding the agency’s compliance with 

its guidelines concerning the quality, objectivity, utility, and integrity of information, 

and how such complaints were resolved. 

Background 

Information quality is important to the Department of Education because educators, researchers, 

policymakers, and the public use information that the Department disseminates for a variety of 

purposes. Thus, it is important that the information the Department disseminates be accurate and 

reliable. 

The Department’s Strategic Plan emphasizes the importance of information quality. For 

example, goal four of the six strategic goals is to “Transform Education into an Evidence-based 

Field.” Under this goal, the Department seeks “to ensure that research funded or published by the 

Department is of the highest quality.” The Department places priority on ensuring “... that highquality research – whether or not it is funded by the Department – is synthesized, publicized, and 

disseminated widely.” 

1 
The Department also relies on high quality information in the administration of its grant 

programs. For example, Objective 1.1 under the Strategic Plan requires the Department to 

“...link federal education funding to accountability for results.” Consequently, programs that 

cannot demonstrate evidence of effectiveness will be candidates for reform or elimination. High 

quality information is required to demonstrate evidence of effective programs. To make sound 

decisions, the Department intends to accept and use only information that is accurate and 

reliable. 

To serve the public, the Department of Education prepares and disseminates information 

products that describe the condition of American education and the Department’s policies, 

programs, and services. The Department also disseminates profiles of the learner populations 

served by Department programs, evaluations of Department programs, and research products 

describing what works in American education. In addition, the Department reports statistical data 

describing the educational achievement, attainment, and the demographic and socioeconomic 

characteristics of America’s students; the characteristics of the education labor force in the 

United States; the financing of education in the United States; and international comparisons of 

education systems and their students. The Department disseminates most of its information 

products in both printed and electronic formats, as well as in oral presentations. Many 

information products are announced on the Department’s Web site (www.ed.gov), where they 

can be accessed and downloaded. 

Purpose and Scope 

Consistent with the guidance from OMB, the Department of Education’s Information Quality 

Guidelines (Guidelines) described below reflect the Department’s policy and procedures for 

reviewing and substantiating the quality of information it disseminates, (e.g., reports, studies, and 

summaries), as well as provide an administrative mechanism allowing affected persons to seek 

and obtain, where appropriate, correction of information not complying with the Guidelines. 

These Guidelines, along with those issued by OMB, represent a performance goal for the 

Department and are intended only to improve the internal management of the Department. They 

do not create any private right of action to be used by any party against the government in a court 

of law or in an administrative hearing. 

The Guidelines are applicable to information that the Department of Education disseminates on 

or after October 1, 2002, including the review of information to ensure quality before it is 

disseminated to the public. Some previously released information products continue to be used 

for decision-making or are relied upon by the Department and the public as official, 

authoritative, government information; this information is, in effect, constantly being redisseminated and are thus subject to these Guidelines. Previously released information products 

that do not meet these criteria are considered archived information and thus are not subject to 

these Guidelines. 

In addition, individual offices within the Department of Education may have more detailed 

guidelines that are tailored to specific information needs. An example of program-specific 

guidelines are the National Center for Education Statistics (NCES) Statistical Standards, which 

may be accessed at the following url: http://nces.ed.gov/statprog/. Other individual offices 

within the Department of Education also may develop guidelines tailored to their specific needs; 

2 
however, individual office guidelines would be consistent with the Department of Education 

Guidelines described below. 

Under these Guidelines, information disseminated by the Department of Education is divided 

into four categories: 

• General Information about Education Programs, such as fact sheets, descriptions of

programs and services and guidance on who is eligible and how and where to apply

for services or assistance. General information might also include public service

reports on evaluations of specific programs and services, and descriptions of findings.

• Research Studies and Program Evaluation Information, such as detailed reports of

research findings and methodologies and technical reports describing the procedures

employed and the results of program evaluations.

• Administrative and Program Data, such as aggregates of records from schools,

school districts, and states.

• Statistical Data, such as data collections of nonadministrative data and special

purpose surveys that are designed to fill data gaps or information needs.

These Guidelines, however, do not govern all information of the Department, nor do they cover 

all information disseminated by the Department. For example, the Guidelines generally do not 

cover: 

• Internal information such as employee records;

• Internal procedural, operational, or policy manuals prepared for the management and

operations of the Department of Education that are not primarily intended for public

dissemination;

• Information collected or developed by the Department that is not disseminated to the

public, including documents intended only for inter-agency or intra-agency

communications;

• Research findings and other information published by grantees, unless the

Department –

o Represents, uses, or relies upon the information as the official position of the

Department, or in support of the official position of the Department;

o Has authority to review and approve the information before release; or

o Directs that the information be disseminated;

• Opinions that are clearly identified as such, and that do not represent facts or the

agency’s views;

• Electronic links to information on other Web sites;

• Correspondence with individuals;

• Responses to requests for information under the Freedom of Information Act, the

Federal Advisory Committee Act, and the Privacy Act;

• Press releases, unless they contain new substantive information that was not

previously released;

• Congressional testimony that includes data that has previously been disseminated;

• Comments received from the public in response to Federal Register notices;

• Distributions intended to be limited to subpoenas or adjudicative processes, i.e., the

findings and determinations made in the course of adjudications;

3

Appeal

Exh. B
• Information collected during the course of a Departmental investigation that is not 

intended to be disseminated to the public, e.g., data collected through resolution of an 

OCR or OIG investigation; and 

• Archival records, including previously released information products that are not 

being relied upon, used for decision-making, or held out as authoritative data. 

Information Quality 

These Guidelines assess information quality using three factors: utility, objectivity, and integrity. 

These elements are intended to ensure that information the Department disseminates is useful, 

accurate, reliable, unbiased, and secure. Department staff will treat information quality as 

integral to the creation, collection, maintenance, and dissemination of information, and will 

review products before they are disseminated to ensure that they are consistent with these 

Guidelines. In particular, information products from the Department will follow the Guide to 

Publishing at the U.S. Department of Education, and all clearance submissions under the 

Paperwork Reduction Act will explain how the proposed collection of information will yield 

high quality, objective, and useful data, consistent with OMB’s guidelines. Furthermore, the 

Guidelines provide that the level of quality assurance for information must be tied to its level of 

importance. Influential Information, that is information that will or does have a clear and 

substantial impact on public policies or private sector decisions, must meet a higher level of 

quality as described on page 9 of these Guidelines. 

Utility 

Utility refers to the usefulness of the information to its intended users. Utility is achieved by 

staying informed of information needs and developing new products and services where 

appropriate. To maximize the utility of influential information, care must be taken in the review 

stage to ensure that the information can be clearly understood and, where appropriate and to the 

extent practical, an external user of the information can reproduce the steps involved in 

producing the information. 

Ultimately, the Department intends to ensure that the information it disseminates meets the needs 

of intended users. All information products should be grammatically correct and clearly written 

in plain English. The target audience should be clearly identified, and the product should be 

understandable to that audience. 

To ensure the usefulness of Department products, all information products should provide 

information that will help the Department fulfill its mission “to ensure equal access to education 

and to promote educational excellence throughout the nation.” When appropriate, Department 

products should include contact information for users who seek clarification or further 

information, or who want to provide feedback. 

In particular— 

• General Information should provide clear and readable descriptions of the 

Department’s programs and services and, where applicable, guidance and assistance, 

including who is eligible and how and where to apply for services or assistance. It 

also may include information pertaining to evaluations of specific programs and 

services, and descriptions of findings. 

4 
• Research Studies and Program Evaluations should be designed and reviewed to fill 

the information needs that are identified through internal review, legislative 

mandates, or input from data users outside the Department. 

• Administrative and Program Data, e.g., aggregate data (or information) derived from 

records at the school, school district, and state levels, should be carefully described 

and documented in all reports and products released by the Department. 

• Statistical Data, e.g., data collections of non-administrative data and special purpose 

surveys should be designed to fill data gaps or information needs that are identified 

through internal review, legislative mandates, or input from data users outside the 

Department, and should be reviewed for how well they fulfill that purpose. 

The usefulness of information the Department disseminates will be evaluated from the 

perspective of the Department, educators, education researchers, policymakers, and the public. 

The Department relies upon internal reviews and analyses, along with feedback from advisory 

committees, educators, education researchers, policymakers, and the public to achieve this. 

Consistent with OMB’s guidance, the Department’s goal is to maximize the usefulness of the 

information and minimize the cost to the government and the public. When disseminating its 

information products, the Department will utilize varied dissemination channels so that the 

public, education researchers, and policymakers can locate Department information in an 

equitable and timely fashion. 

Objectivity 

Objectivity refers to the accuracy, reliability, and unbiased nature of information. It is achieved 

by using reliable information sources and appropriate techniques to prepare information 

products. Objectivity involves both the content and the presentation of the information. Content 

should be complete, include documentation of the source of any information used, as well as, 

when appropriate, a description of the sources of any errors in the data that may affect the 

quality of the information product. The presentation of the information should be clear and in a 

proper context so that users can easily understand its meaning. 

The Department strives to present information to the public in an accurate, clear, complete, and 

unbiased manner. In keeping with the OMB Information Quality Guidelines, all information 

products should undergo editorial and technical peer review to assist the Department in meeting 

this performance goal. 

General Information 

Department of Education information products should be appropriate for the target audience. 

Each product should: 

1. Clearly state the goals or purpose of the information product; 

2. Include an unbiased presentation of the topic in question; 

3. If applicable, draw upon peer-reviewed, scientific evidence-based research that is 

appropriately documented; 

4. Clearly identify data sources, if applicable; and 

5. Confirm and document the reliability of the data, and acknowledge any shortcomings or 

explicit errors in any data that is included. 

5 
Research and Evaluation Information 

Department of Education research and evaluation information products should, at a minimum: 

1. Clearly state the goals or purpose of the topic in question; 

2. Pose the research or evaluation question in a balanced and unbiased manner; 

3. Provide an unbiased test of the question; 

4. Have a research study approach or data collection technique that is well thought out, 

designed to use state of the art methodologies in the data collection, and be clearly described 

in the study documentation; 

5. Present conclusions that are strongly supported by the data; 

6. Clearly identify data sources, if applicable; 

7. Confirm and document the reliability of the data, and acknowledge any shortcomings or 

explicit errors in any data that is included; and 

8. Undergo peer review. 

Department of Education research and evaluation information products documenting cause and 

effect relationships or evidence of effectiveness should meet the quality standards that will be 

developed as part of the What Works Clearinghouse. 

Administrative and Program Data 

The Department of Education reports data that rely upon information provided by third parties. 

These data draw upon aggregates from student record systems or other administrative data (e.g., 

universe studies, including censuses, and other reports based on aggregate administrative data). 

These data rely upon information provided by third parties. Because of this, the Department does 

not have full control over the quality of the reported data; the Department intends to, however, 

identify the source of the information and any shortcomings or limitations of the data if we rely 

upon it for decision-making purposes. This will facilitate the public’s understanding of the 

strengths and potential weaknesses of these data. Furthermore, as an additional assurance of 

quality, these data should meet the criteria that are being developed as part of an ongoing 

Department-wide data standardization and coordination initiative. At a minimum, these standards 

will require the following: 

1. In formulating a data collection plan the goals of the study should be clearly described; 

2. The subjects to be studied and the data to be collected should be clearly defined, using 

broadly understood concepts and definitions that are consistent with Department data 

definition handbooks; 

3. The research study approach or data collection techniques should be well thought out and 

designed to use state-of-the-art methodologies in the data collection, and should be clearly 

described in the study documentation; 

4. In designing the work, every effort should be made to minimize the amount of time required 

for responding institutions; 

6 
5. The source of research information or data should be reliable. Data should be collected with

survey instruments that have been properly developed and tested;

6. Response rates should be monitored during data collection. When necessary, appropriate

steps should be taken to ensure the respondents are representative of the population;

7. When applicable, care should be taken to ensure the confidentiality of personally identifiable

data, as required by law, during the collection, processing, and analysis of the data;

8. Upon completion of the work, the findings and data should be processed in a manner

sufficient to ensure that the data are edited to help ensure that the data are accurate and

reliable;

9. The findings and data collection should be properly documented and stored, and the

documentation should include an evaluation of the quality of the data with a description of

any limitations of the data. In particular, any known limitations of the information should be

documented (e.g., missing values, amount of nonresponse);

10. The analysis should be selected and implemented to ensure that the data are correctly

analyzed using modern statistical techniques suitable for hypothesis testing. Techniques may

vary from simple tabulations and descriptive analysis to multivariate analysis of complex

interrelationships. Care should be taken to ensure that the techniques are appropriate for the

data and the questions under inquiry;

11. All work should be conducted and released in a timely manner;

12. Reports using these data should identify the source(s) of the information, including a citation.

Reports should also include:

a) The reason the information is provided, its potential uses, and cautions as to

inappropriate extractions or conclusions.

b) Descriptions of any statistical techniques or mathematical operations applied to the

data.

c) The identification of other possible sources of potentially corroborating or conflicting

information; and

13. Prior to dissemination, all reports, data, and documentation should undergo editorial and

technical review to ensure accuracy and clarity.

Statistical Data 

Department of Education reports and data collections that draw upon sample survey data should 

be clearly written, and should follow these Guidelines: 

1. In formulating a data collection plan, the goals of the study should be clearly described;

2. The subjects to be studied and the data to be collected should be clearly defined, using

broadly understood concepts and definitions that are consistent with Department data

definition handbooks;

3. The research study approach or data collection techniques should be well thought out and

designed to use state-of-the-art methodologies in the data collection and should also be

clearly described in the study documentation;

4. In designing the work, every effort should be made to minimize the amount of time required

for study participants;

Appeal

Exh. B

7 
5. The source of data should be reliable. The sample should be drawn from a complete list of 

items to be tested or evaluated, and the appropriate respondents should be identified, 

correctly sampled, and queried with survey instruments that have been properly developed 

and tested; 

6. Response rates should be monitored during data collection. When necessary, appropriate 

steps should be taken to ensure that the respondents are a representative sample; 

7. Care should be taken to ensure the confidentiality of personally identifiable data, as required 

by law, during research/data collection, processing, and analysis of the resulting data; 

8. Upon completion of the work, the data should be processed in a manner sufficient to ensure 

that the data are cleaned and edited to help ensure that the data are accurate and reliable; 

9. The findings and data collection should be properly documented and stored, and the 

documentation should include an evaluation of the quality of the data with a description of 

any limitations of the data. In particular, any known limitations of the information should be 

documented (e.g., missing values, amount of nonresponse); 

10. Data should be capable of being reproduced or replicated based on information included in 

the documentation, such as: 

a) The source(s) of the information; 

b) The date the information was current; 

c) Any known limitations on the information; 

d) The reason that the information is provided; 

e) Descriptions of any statistical techniques or mathematical operations applied to 

source data; and 

f) Identification of other sources of potentially corroborating or conflicting information. 

11. If secondary analysis of data is employed, the source should be acknowledged, the reliability 

of the data should be confirmed and documented, and any shortcomings or explicit errors 

should be acknowledged (e.g., the representativeness of the data, measurement error, data 

preparation error, processing error, sampling errors, and nonresponse errors); 

12. The analysis should be selected and implemented to ensure that the data are correctly 

analyzed using modern statistical techniques suitable for hypothesis testing. Techniques may 

vary from simple tabulations and descriptive analysis to multivariate analysis of complex 

interrelationships. Care should be taken to ensure that the techniques are appropriate for the 

data and the questions under inquiry; 

13. Reports should include the reason the information is provided, its potential uses, and cautions 

as to inappropriate extractions or conclusions, and the identification of other sources of 

potentially corroborating or conflicting information; 

14. Descriptions of the data and all analytical work should be reported in sufficient detail to 

ensure that the findings could be reproduced using the same data and methods of analysis; 

this includes the preservation of the data set used to produce the work; 

15. Prior to dissemination all reports, data, and documentation should undergo editorial and 

technical review to ensure accuracy and clarity. Qualified technical staff and peers both 

inside and outside the Department should do the technical review; 

16. All work should be conducted and released in a timely manner; and 

8 
17. There should be established procedures to correct any identified errors. These procedures 

may include the publication of errata sheets, revised publications, or Web postings. 

Integrity 

Integrity refers to the security or protection of information from unauthorized access or revision. 

Integrity ensures that the information is not compromised through corruption or falsification. 

The Department strives to protect the information it collects, uses, and disseminates to the public 

from unauthorized disclosure, alteration, loss, or destruction. Statutory and administrative 

guidelines to protect the integrity of Department information include the following: 

• Privacy Act; 

• Freedom of Information Act; 

• OMB Circulars A-123, A-127, and A-130; 

• Federal Policy for the Protection of Human Subjects; 

• Family Educational Rights and Privacy Act; 

• Computer Security Act of 1987; 

• Government Information Security Reform Act; and 

• National Education Statistics Act, as amended by the USA Patriot Act. 

Under the Privacy Act, the Department safeguards personally identifiable information that it 

gathers and maintains about individuals in a system of records. The Department is also highly 

protective of administrative records and sample survey data that include personally identifiable 

information, especially survey data that are collected under pledges of confidentiality. 

Under the Computer Security Act of 1987, the Department of Education has identified all federal 

computer systems that contain sensitive information and has implemented security plans to 

protect these systems, so as to protect sensitive information against loss, misuse, disclosure or 

modification. In this context, sensitive information includes data covered under the Privacy Act 

and information that could affect the conduct of federal programs. 

Influential Information 

Government information that is particularly influential needs to meet higher quality standards, 

and in particular must be reproducible. Per the OMB guidelines, information is designated as 

influential if the Department determines that the information is reasonably likely to have a clear 

and substantial impact on public policies or private sector decisions if disseminated. Scientific, 

financial, and statistical information all may be considered influential. Individual programs 

within the Department of Education may designate certain classes of scientific, financial, and 

statistical information as influential. 

For example, institutional data on the total number of student borrowers who enter repayment on 

Stafford loans during a specific fiscal year, and related data on the subset of students who default 

before the end of the next fiscal year are used in the calculation of cohort loan default rates of 

Stafford loan borrowers at each postsecondary institution. These default rates are compared to 

established thresholds for high and low default rates, resulting in sanctions for institutions with 

high default rates and reduced administrative burden for institutions with low default rates. 

9 
Given this use, these data and the calculations used in computing the rates and in setting the 

thresholds are influential. Similarly, the data and formulas used in determining program 

allocation of funds in areas such as special education, adult education, and Title I are influential. 

As specified in the OMB guidelines, influential information must be accompanied by supporting 

documentation that allows an external user to understand clearly the information and be able to 

reproduce it, or understand the steps involved in producing it. With respect to original and 

supporting data related thereto, the Department will assure reproducibility for such data 

according to commonly accepted scientific, financial, or statistical standards for that type of data, 

taking into account any ethical and confidentiality constraints. In the case of influential analytic 

results, the mathematical and statistical processes used to produce the report must be described in 

sufficient detail to allow an independent analyst to substantially reproduce the findings using the 

original data and identical methods. In situations where the public cannot access the data and 

methods due to other compelling interests such as privacy, intellectual property or other 

confidentiality protections, the Department will apply especially rigorous robustness checks to 

analytic results and document what checks were undertaken. 

Information Correction Requests and Appeals 

Effective October 1, 2002, the Department of Education will allow any affected person to request 

the correction of information the Department disseminates that does not comply with applicable 

OMB and Department of Education information quality guidelines. An affected person is an 

individual or an entity that may use, benefit or be harmed by the disseminated information at 

issue. 

Most Department information products include the names of knowledgeable staff that can assist 

users in understanding the information presented, and in determining whether there is an error 

that warrants action using the correction process described in this section. Users of the 

Department’s information should consult with the contact person listed in the product before 

filing a formal request for correction. 

Information Correction Requests 

In the Department of Education’s correction request process, the burden of proof rests with the 

requester. An affected person who believes that information the Department disseminates does 

not adhere to the information quality guidelines of OMB or the Department, or an office of the 

Department that has issued program-specific guidelines, and who would like to request 

correction of specific information, needs to provide the following information: 

• Identification of the requester (i.e., name, mailing address, telephone number, and 

organizational affiliation, if any); 

• A detailed description of the information that the requester believes does not comply 

with the Department’s or OMB’s guidelines, including the exact name of the data 

collection or report, the disseminating office and author, if known, and a description 

of the specific item in question; 

• Potential impacts on the requester from the information identified for correction (i.e., 

describe the requestor’s interest in the information and how the requestor is affected 

by the information in question); and 

10 
• An explanation of the reason(s) that the information should be corrected (i.e., 

describe clearly and specifically the elements of the information quality guidelines 

that were not followed). 

This information should be provided to the Principal Deputy Assistant Secretary for the Office of 

Management at the following address: 

Principal Deputy Assistant Secretary 

Office of Management 

U.S. Department of Education 

RE: Information Quality Request 

Room 2W311, LBJ 

400 Maryland Avenue, SW 

Washington, DC 20202 

Alternatively, requesters may submit e-mail requests to the following address: 

“OMInformationQualityRequests@ed.gov.” Requesters should indicate that they are submitting 

an Information Quality Request in the subject line of the e-mail. 

Review 

The Principal Deputy Assistant Secretary (PDAS) will review the request and determine whether 

it contains all the information required for a complaint. If the request is unclear or incomplete, 

the Department will seek clarification from the requester. 

If the request is clear and complete, the PDAS will forward it to the appropriate program 

office(s) for a response to the requester. The responsible office(s) will determine whether a 

correction is warranted, and if so, what corrective action it will take. Any corrective action will 

be determined based on the nature and timeliness of the information involved, as well as the 

significance of the error on the use of the information, the magnitude of the error, and the cost of 

undertaking a correction. 

Comments about information on which the Department has sought public comment, such as 

rulemaking or studies cited in a rulemaking, will be responded to through the public comment 

process, or through an individual response if there was no published process for responding to all 

comments. The Department may choose to provide an earlier response, if doing so is 

appropriate, and will not delay issuance of the final action in the matter. 

The Department is not required to change the content or status of information simply based on 

the receipt of a request for correction. The Department may reject a request that appears to be 

made in bad faith or without justification, and is only required to undertake the degree of 

correction that is appropriate for the nature and timeliness of the information involved. In 

addition, the Department need not respond substantively to requests that concern information not 

covered by the information quality guidelines. 

11 
Response 

The Department will respond to all requests for correction within 60 calendar days of the PDAS’ 

receipt of the request, including requests that the Department elects not to process further. For 

requests that merit review – 

• If the request is clear and complete, the Department’s response will explain the 

findings of the review, or will inform the requester if more time is needed to complete 

the review, the reason(s) for the additional time, and an estimate of the time it will 

take to respond. The appropriate program office will be responsible for determining 

what action is necessary and, if an error was made, it will determine the appropriate 

level of correction. 

• If the request is incomplete or unclear, the PDAS will seek clarification from the 

requester. In the case of an unclear or incomplete request, the requester may submit 

additional clarifying information if he or she so chooses. However, the deadline for 

the Department’s review and response will be based upon the date the clarifying 

information is received. 

Once a decision is made, the response will explain to the requester that he or she has a right to 

appeal the decision. Copies of all Department correspondence related to Information Quality 

Requests will be maintained by the PDAS. 

Appeals 

If a requester is not satisfied with the Department’s decision on the request (including the 

corrective action, if any), he or she may appeal to the Department’s Principal Deputy Assistant 

Secretary within thirty (30) calendar days of receipt of the Department’s decision. This 

administrative appeal must include a copy of the initial request, a copy of the Department’s 

decision, and a letter explaining why he or she believes the Department’s decision was 

inadequate, incomplete, or in error. 

This appeal information should be provided to the Department’s Principal Deputy Assistant 

Secretary at the following address: 

Principal Deputy Assistant Secretary 

Office of Management 

U.S. Department of Education 

RE: Information Quality Request 

Room 2W311, LBJ 

400 Maryland Avenue, SW 

Washington, DC 20202 

Alternatively, requesters may submit an appeal by e-mail to the following address: 

“OMInformationQualityRequests@ed.gov.” 

Requesters should indicate that they are submitting an Information Quality Appeal in the subject 

line of the e-mail. Such e-mail requests must include all of the information specified for an 

appeal submitted by regular mail. 

12 
The Department will ensure that all appeals are subjected to an impartial review that is 

conducted by parties other than those who prepared the Department’s decision. The Department 

will respond to all appeals within 60 calendar days of the Principal Deputy’s receipt of the 

appeal, or will inform the requester if more time is needed to complete the review of the appeal, 

and the reason(s) for the additional time. 

13 
9/4/2018 DeVos misrepresents the evidence in seeking gainful employment deregulation | Urban Institute

Sandy Baum | August 22, 2018

Urban Wire :: Education and Training

The voices of Urban Institute's researchers and staff

DeVos misrepresents the evidence in seeking

gainful employment deregulation

Secretary of Education Betsy DeVos and the Trump administration have announced their

intention to eliminate the gainful employment (GE) regulations, a move that would strip

students and taxpayers of protection from exploitation by postsecondary institutions

that do not provide meaningful educational opportunities.

To justify this change, the Department of Education has misrepresented my research,

creating a misleading impression of evidence-based policymaking. The Department cites

DeVos misrepresents the evidence in seeking gainful employment deregulation

 

   

 

 

 

 

 

 

 

 

  my work as evidence that the GE standard is based on an inappropriate metric, but the

paper cited in fact presents evidence that would support making the GE rules stronger.

https://www.urban.org/urban-wire/devos-misrepresents-evidence-seeking-gainful-employment-deregulation 1/5
9/4/2018 DeVos misrepresents the evidence in seeking gainful employment deregulation | Urban Institute

The GE rules, which cover all programs in the for-profit sector and nondegree 

programs at public and private nonprofit institutions, restrict the flow of federal dollars 

to programs with graduates who don’t earn enough relative to their loan payments. 

The regulations are designed to prevent institutions from taking advantage of students 

who spend time and energy in programs where they accumulate large amounts of 

debt that they are unable to ever repay, leaving the taxpayers on the hook for the bill 

and the students without a decent education. 

The current rule requires that the median debt level for a program’s graduates not 

exceed 20 percent of the discretionary income of the mean or median graduate, 

whichever is higher. To make the regulations less stringent, the rule’s authors added a 

second standard, allowing programs to qualify if they meet the first standard or if the 

average graduate’s debt payment is less than or equal to 8 percent of their total 

income.

Our examination of a range of evidence about reasonable

debt burdens for students would best be interpreted as

supporting a stricter standard.

In its move to dismantle the regulation, the administration cites a paper on manageable

student debt levels I co-authored with Saul Schwartz in 2008 that raises questions about

the 8 percent threshold. In the notice of proposed rulemaking, the Department of

Education claims that “We base our proposal to rescind the GE regulations on a number

of findings, including research results that undermine the validity of using the

regulations’ debt-to-earnings (D/E) rates measure to determine continuing eligibility for

title IV participation.”

They are correct that we were skeptical of this standard for determining affordable

payments for individual borrowers, but incorrect in using that skepticism to defend

repealing the rule. In fact, our examination of a range of evidence about reasonable debt

burdens for students would best be interpreted as supporting a stricter standard.

In brief, our paper looked at a variety of potential reference points for setting guidelines

for how much individual students could reasonably afford to spend repaying their loans.

Noting that the common standard from the mortgage industry of 8 percent of income

DeVos misrepresents the evidence in seeking gainful employment deregulation

https://www.urban.org/urban-wire/devos-misrepresents-evidence-seeking-gainful-employment-deregulation 2/5
9/4/2018 DeVos misrepresents the evidence in seeking gainful employment deregulation | Urban Institute

We concluded that no borrower could reasonably afford to spend more than 18–20 

percent of their income on student loans and proposed 20 percent of discretionary 

income (income exceeding 150 percent of the poverty level) as a standard on which to 

base an income-based loan repayment plan. (This became the standard for GE.) 

Citing my research and its judgment that 8 percent of income is not the best way to 

measure the affordability of loan payments as a reason to rescind the rules is illogical. 

The GE rules are, if anything, too permissive. 

Our research set a guideline for a level of debt payments no individual student should 

exceed. Under GE, half of a program’s graduates could exceed this limit before 

sanctions would kick in. If this research is going to be brought into the debate, it should 

be used to support eliminating eligibility for federal student aid for more programs—not 

eliminating the rules. 

The Trump administration expresses concerns about blaming institutions for the 

outcomes of their students, in light of the reality that students’ characteristics, in addition 

to their educational attainment levels, contribute to their earnings levels. But these rules 

are in place not to accommodate institutions, but to diminish the number of students who 

spend time and money in programs that have almost no chance of helping them meet 

their goals and to protect taxpayers from being left holding the bag when such students 

are unable to repay the loans they took to enroll in these programs. 

A true evidence-based approach to policymaking would do more than cite a few phrases 

from a scholarly article. It would carefully review evidence about the causes and costs of 

students enrolling in institutions and programs that fail large shares of their students. It 

would design policies that would ameliorate these problems, strengthening existing 

policies and making them more effective rather than erasing years of hard-won 

progress. 

Education Secretary Betsy DeVos, second from left, joined by Education Department Budget Service 

Director Erica Navarro, left, turns to speaks to people behind her before she testifies on Capitol Hill in 

Washington, Wednesday, May 24, 2017. Photo by Carolyn Kaster/AP. 

SHARE THIS PAGE

https://urbn.is/2OSzbLS SHARE

DeVos misrepresents the evidence in seeking gainful employment deregulation

https://www.urban.org/urban-wire/devos-misrepresents-evidence-seeking-gainful-employment-deregulation 3/5