1

UNITED STATES DEPARTMENT OF EDUCATION

+ + + + +

GAINFUL EMPLOYMENT

NEGOTIATED RULEMAKING COMMITTEE 2017-2018

+ + + + +

SESSION 2

+ + + + +

WEDNESDAY,

FEBRUARY 7, 2018

+ + + + +

The Negotiated Rulemaking Committee 

met in the Barnard Auditorium, U.S. Department of 

Education, 400 Maryland Avenue, S.W., Washington, 

D.C., at 9:00 a.m., Ramona Buck, Rozmyn Miller, 

and Javier Ramirez, Facilitators, presiding.

PRESENT

RAMONA BUCK, Federal Mediation and Conciliation 

Service, Facilitator

ROZMYN MILLER, Federal Mediation and 

Conciliation Service, Facilitator

JAVIER RAMIREZ, Federal Mediation and 

Conciliation Service, Facilitator

JEFF ARTHUR, Vice President of Regulatory 

Affairs & Chief Information Officer, ECPI 

University

WHITNEY BARKLEY-DENNEY, Senior Policy Counsel, 

Center for Responsible Lending

JESSICA BARRY, President, School of Advertising

Art

JENNIFER L. BLUM, ESQ., Senior Vice President, 

External Relations and Public Policy, 

Laureate Education, Inc.

STEPHEN CHEMA, Ritzert & Leyton, PC

JENNIFER DIAMOND, Program Associate, Maryland 

Consumer Rights Coalition
2

DANIEL ELKINS, Legislative Director, Enlisted 

Association of the National Guard of the 

United States

RYAN FISHER, Intergovernmental Relations 

Division, State of Texas Office of the 

Attorney General

PAMELA FOWLER, Executive Director of Financial 

Aid, University of Michigan - Ann Arbor

CHRISTOPHER GANNON, Vice President, United 

States Student Association

ANDREW HAMMONTREE, Director of Financial Aid and 

Scholarships, Francis Tuttle Technology 

Center

NEAL HELLER, CEO/President, Hollywood Institute 

of Beauty Careers

MARC JEROME, President, Monroe College

C. TODD JONES, President, Association of 

Independent Colleges & Universities in 

Ohio

ROBERTS JONES, President, Education & Workforce 

Policy, LLC

JOHN KAMIN, Assistant Director, The American 

Legion's National Veterans Employment & 

Education Division

KIRSTEN KEEFE, Senior Attorney, Consumer Finance 

and Housing Unit, Empire Justice Center

CHRISTOPHER MADAIO, Assistant Attorney General, 

Office of the Attorney General of Maryland

JORDAN MATSUDAIRA, Nonresident Fellow, Urban 

Institute; Assistant Professor, Cornell 

University

MARK MCKENZIE, Executive Director, Accreditation 

Commission for Acupuncture and Oriental 

Medicine

LAURA METUNE, Vice Chancellor of External 

Affairs, California Community Colleges

ANTHONY MIRANDO, Executive Director, National 

Accrediting Commission of Career Arts and 

Sciences

MATTHEW MOORE, Director of Financial Aid and 

Scholarships, Sinclair Community College

KELLY MORRISSEY, Director of Financial Aid, 

Mount Wachusett Community College 

CHAD MUNTZ, Director of Institutional Research, 

Office of Administration and Finance, The 
3

University System of Maryland

JONATHAN K. PIERRE, Vice Chancellor for 

Institutional Accountability and Evening 

Division, Southern University Law Center

TIM POWERS, Director of Student Aid Policy, 

National Association of Independent 

Colleges and Universities

THELMA L. ROSS, Interim Director of Student 

Financial Aid, Prince George's County 

Community College

SANDY SARGE, SARGE Advisors

AHMAD SHAWWAL, Student, University of Virginia

DAVID SILVERMAN, Chief Financial Officer and 

Director of Business Affairs, The American 

Musical and Dramatic Academy

JOHNSON M. TYLER, Senior Attorney, Consumer and 

Foreclosure Units, Brooklyn Legal Services

CHRISTINA WHITFIELD, Associate Vice President, 

State Higher Education Executive Officers 

Association

STAFF PRESENT

STEVEN FINLEY, Office of General Counsel

GREGORY MARTIN, Office of Postsecondary 

Education

AGENDA

PAGE

Opening Comments .............................. 5
4

Comments from Negotiators

or Alternates............................ 5

Issue 4 - Sanctions ........................... 7

Public Comments ............................. 143

Adjourn ..................................... 151
5

1 P-R-O-C-E-E-D-I-N-G-S

2 (9:00 a.m.)

3 MR. RAMIREZ: All right, so let's go 

4 ahead and get started now for the day three of 

5 session two for the gainful employment regulatory 

6 negotiations for the Department of Education. 

7 And I want to apologize to the public 

8 who showed up early. I had no way of emailing you 

9 all, but I guess for future reference, if there 

10 is any concern with that, just OPM has it on their 

11 website if there's any delayed starts, so just take 

12 a peek at that in the morning if there's potential 

13 for any concern. They actually posted it pretty 

14 late that it was going to be a two-hour delay in 

15 the start.

16 So let's go ahead and start off with 

17 any comments from the negotiators or alternates. 

18 Mark?

19 MR. JEROME: So, good morning, 

20 everyone. So just to open, I just have a, it 

21 occurred to me last night, a fundamental question. 

22 I've been putting proposals out as have 

23 many of us in good faith, and some of the proposals 
6

1 tighten the rule and make it actually stricter, 

2 including out of my institution, but I was doing 

3 this with the understanding that number one, the 

4 Department is serious in its proposal to apply at 

5 least the debt to earnings to all sectors, and two, 

6 that the Department would eventually give us 

7 informational rates which absolutely will inform 

8 what the policy is, and it's been hard for me to 

9 think about what to propose today not knowing what 

10 the data would look like, so I'm just asking the 

11 Department what their response is to that?

12 MR. MARTIN: You know, that question 

13 and what you just said, Marc, I will address because 

14 I had planned to address it anyway in my opening 

15 in my first remarks, so why don't we just let Javier 

16 go through what we're going to do with public 

17 comment and all that, then when we begin, I will 

18 address those issues.

19 MR. RAMIREZ: Excellent, great, thank 

20 you, Greg. All right, so are there any comments 

21 from the public? All right, hearing none, so 

22 yesterday we were having some robust discussion 

23 on Issue Paper 4 on sanctions, and where we started 
7

1 to get bogged down were what were some of the metrics 

2 that we would use to see if a school was performing 

3 at an acceptable level or a program was performing 

4 at an acceptable level. And so we did 

5 some work last night to try to figure out a way 

6 to try and focus the conversation, and we'll share 

7 that with you after Greg makes some opening remarks 

8 on where we're at with that, so, Greg?

9 MR. MARTIN: Thanks, Javier. For the 

10 record, this is Greg, and, yeah, so last night, 

11 we did some thinking on this, talked with our senior 

12 people, so what I wanted to do was just lay out 

13 where the Department is on this so we can continue 

14 today, and the idea, you know, so with that 

15 knowledge, you can come up with some ideas about 

16 how we move forward with this.

17 So on the issue of sanctions, as you 

18 know, our proposed rules contain no program 

19 sanctions or no loss of eligibility based on the 

20 D/E rates. And the question was raised yesterday, 

21 I think, in many forms, "Is the Department's 

22 position on that firm or are we willing to consider 

23 some type," and I hesitate to use the word sanction. 
8

1

2 I don't know if that's the right word 

3 and I don't want to use that as if that would be 

4 what we'd call it, but some type of consequence, 

5 let's call it that, for having poor metrics, 

6 whatever those wind up being.

7 High debt to earnings. We'll use it 

8 that way, high debt to earnings. So, yes, we are 

9 willing to enter -- what we're -- I guess I should 

10 say right off what we're not willing to entertain, 

11 and what we're not willing to entertain is a, would 

12 be any type of protocol where there would, for 

13 instance, be, you know, debt to earnings calculated 

14 for all programs, but the consequences would only 

15 devolve onto a certain segment of the community. 

16 We're not going to do that. So we remain committed 

17 to doing the rates for all programs at all types 

18 of institutions.

19 What we are willing to entertain is 

20 something other than the current rule, which is, 

21 of course, the loss of program eligibility based 

22 on unacceptable debt to earnings. If we were going 

23 to retain that, we wouldn't be here, so obviously 
9

1 we're moving away from that. 

2 The Department, as Steve talked about 

3 yesterday, there are issues with our authority to 

4 extend loss of program eligibility beyond the GE 

5 programs. However, we do have other areas we might 

6 be able to go. So what I would task all of you 

7 with today is thinking about what -- two things. 

8

9 Another thing, we don't intend to drop 

10 debt to earnings, but we're not opposed to 

11 entertaining other types of metrics that might be 

12 considered along with those, with debt to earnings, 

13 or as the point's been made here, there are many 

14 that might not have a very good debt to earnings 

15 ratio, but they have high graduation rates. If 

16 you did repayment rates, it would show excellent 

17 repayment rates. 

18 So we are going to consider anything 

19 you want to put forward regarding other types of 

20 metrics that might be used in conjunction with debt 

21 to earnings, and also what types of, I'll just use 

22 the word ramifications might result from having 

23 poor metrics for a given period of time. Let's 
10

1 just put it that way just to keep it open. 

2 So hopefully I've, you know, clarified where 

3 we're coming from and what we would be willing to 

4 consider, and it's certainly maybe broader than 

5 what the original proposed rules contained.

6 MR. RAMIREZ: Okay, I am going to put 

7 a hard ten-minute on this Q&A here or for Greg on 

8 this just so that way we could get into the actual 

9 content because we are dealing on, working on a 

10 compressed time schedule here. So let me start 

11 off with Daniel, then Chris, Whitney, Jordan, and 

12 then Laura, so Daniel?

13 MR. ELKINS: This is Daniel. Yeah, so 

14 I wanted to just kind of ask the group as a whole 

15 one or two questions, and this might not be the 

16 appropriate venue to have this discussion, but the 

17 lead up to the question is during the last

18 negotiation, I believe that we all in good faith 

19 sat around the table with the intent to protect 

20 students, and I think that we all can agree on that.

21 Now, how protecting students looks, we 

22 all have a very different understanding, and it 

23 seemed that during the last negotiation, we 
11

1 realized that there were some issues with how GE 

2 was calculated. It excluded some programs that 

3 it shouldn't and it wasn't inclusive of other 

4 programs that it should. 

5 We also realized that there needs to 

6 be a different appeal process so when a metrics 

7 that is not going to be a one-size-fits-all model 

8 misses the mark, those schools have a way of dealing 

9 with that. 

10 So if we don't have the current metrics, 

11 I was wondering, and I don't know Javier, if it's 

12 appropriate or not to discuss what those other 

13 metrics look like. It seems like Greg may be 

14 alluding to that. 

15 But maybe if we moved away just at some 

16 point during the day from one of the point papers 

17 and just had a high-level discussion, maybe put 

18 a time frame on it of what could be some metrics 

19 that we could actually use, and then see if that 

20 gets us any closer to something that is going to 

21 protect students across the board, but also allow 

22 the institutions that possibly wouldn't meet that 

23 metric, but are good for students, to still be 
12

1 successful.

2 MR. RAMIREZ: Yeah, so actually what 

3 I was hoping to do with you all today was to take 

4 you step by step through some of that, and if I 

5 understand Greg correctly, what he was saying is 

6 they are open to other metrics, but D/E has to be 

7 one of them, right? So they are open to it. 

8 I don't think he put any type of 

9 restriction on what that is. It was more of let's 

10 just see what else is out there and see if we come 

11 to agreement, but D/E has to be one of them, okay? 

12 Let me get Chris next.

13 MR. MADAIO: Thank you, Chris Madaio. 

14 So to the Department, you know, I'll just speak 

15 about, to the Department globally. I mean, that's 

16 kind of a shocking statement because I don't think 

17 any of the negotiators would come here and say that, 

18 you know, "We're not willing to entertain X," and 

19 the fact that the Department is saying that it's 

20 not willing to entertain, you know, anything really 

21 in the negotiation of this rule, I mean, that's 

22 a pretty shocking statement.

23 MR. MARTIN: I disagree with you. I 
13

1 don't think it's a shocking statement. I think 

2 that the reason why we're here is that it's obvious 

3 from the way the current rules are that we're back 

4 here to renegotiate those rules, that putting into 

5 place -- that if we come up --

6 As I said, we want to entertain program, 

7 other types of program sanctions, but if that's 

8 going to be that any type of a sanction based on 

9 these rates, that these rates are going to be 

10 calculated for all programs, all institutions, but 

11 the consequences, any consequence that results in, 

12 shall I say, a pejorative consequence or a negative 

13 consequence is only going to be affecting one type 

14 of school. We're not going to do that. 

15 I don't think that's -- I don't in any way think 

16 that's negotiating in bad faith. That's simply 

17 saying that our position is that we're not going 

18 to have a bifurcated rule where the consequences 

19 of it devolve onto one type of school. 

20 So, and I think from our perspective, 

21 that's just being honest and not having people sit 

22 around and think about something that we wouldn't 

23 find acceptable, and I don't think that -- I know 
14

1 I can't speak for any of the negotiators, but I 

2 don't think --

3 There's certainly a lot of negotiators 

4 from a certain segment that probably would find 

5 that a non-starter as well, so I don't think --

6 I think we're just setting the stage for something 

7 that would produce a rule that, where we can reach 

8 consensus. 

9 I disagree with the assertion or 

10 characterization of it as being shocking or taking 

11 anything off the table. I think it's just an 

12 acknowledgment of why we're here to begin with.

13 MR. MADAIO: Okay, so I guess my 

14 follow-up question then is obviously the Higher 

15 Education Act has language regarding what a gainful 

16 employment program needs to do, right? The program 

17 needs to -- those applicable programs as defined 

18 under that section of the HEA, and I know you know 

19 which section I'm talking about, need to prepare 

20 students for gainful employment. I 

21 mean, that language exists, obviously, and I guess 

22 my question to the Department is if a program is 

23 not preparing students for gainful employment in 
15

1 a recognized field, what would the Department 

2 intend to do if there is no sanction specific for 

3 that field?

4 I'm asking -- obviously that statute 

5 differentiates. I mean, that aspect of the 

6 statutory cite contemplates that there's a 

7 difference between sectors. I'm asking for that 

8 sector. I mean, why is it that the Department here 

9 will not differentiate when the statutory language 

10 does differentiate?

11 PARTICIPANT: So I think the goal here 

12 is to come up with a set of regulations that work 

13 for every institution, that looks at the debt to 

14 earnings in the program. I think we've got 

15 certification requirements to the extent that --

16 I think even there in the certification 

17 to the extent any institution is offering programs 

18 that have to meet certain state or accrediting 

19 requirements, we expect those to be met as well. 

20

21 I guess we're left going forward 

22 thinking about what the additional requirements 

23 are for GE programs, but in these regulations, the 
16

1 goal is to try to come up with a system that makes 

2 sense for all programs where the debt to earnings 

3 of the students that complete that program are one 

4 factor that would be considered along with many 

5 others.

6 And just like you might talk about 

7 looking at something like repayment rates, or very 

8 low cohort default rates, or something as an 

9 indication of a good program, there comes a point 

10 where if you get a lot of bad information about 

11 a program, you know, high D/E rates, terrible CDRs, 

12 low completion or something, that that program is 

13 actionable because it's just a bad program, not 

14 solely because of the D/E rates, but also because

15 it's bad for other reasons, and the Department is 

16 willing to look at the concept of sanctions in that 

17 area here as well, all right. 

18 It's not just finding other ways to show 

19 that a program with high D/E rates is good. It's 

20 a system that would also say D/E rates are one of 

21 several factors that may show a program should not 

22 be permitted to keep enrolling students. 

23 It should have enrollment caps. Maybe 
17

1 there should be a prohibition on enrolling new 

2 students if it's failed to improve over time. I 

3 mean, those are the kind of things that we think 

4 there's room for a very profitable discussion here 

5 today.

6 MR. RAMIREZ: Okay, Whitney?

7 MS. BARKLEY-DENNEY: Yes, thank you. 

8 I just wanted to, well, Chris said what I was going 

9 to say about, you know, I don't think that there's 

10 a desire to divide things among institution types. 

11 I think the issue is program types, which of course 

12 any institution can have.

13 And I also just wanted to reiterate my 

14 request backing up what Daniel said that we talk 

15 or think about doing this in a full caucus, not 

16 breaking off into smaller caucuses because nothing 

17 seems to get done there. We all come back to the 

18 table with our own ideas and then we don't even 

19 get to discuss them, so that might be a way of, 

20 you know, being able to move this forward.

21 MR. RAMIREZ: Jordan? And we have two 

22 minutes on this thing, so we've got to move it along.

23 MR. MATSUDAIRA: Yeah, again, I'm kind 
18

1 of surprised by the Department's kind of statement 

2 about just dismissing the idea of having sanctions 

3 in the GE sector out of hand, and I think it brings 

4 it back to, you know, we're going to have a 

5 conversation about what metrics are appropriate. 

6

7 But if we're not in the business of, 

8 you know, kind of thinking through having sanctions 

9 as a potential option, it really, for me, makes 

10 me question what the goal of this regulation really 

11 is, and I just want to ask the Department again 

12 to express how they see the goal of changing the 

13 regulation here.

14 So I remember hearing something about 

15 reducing administrative burden. I get how 

16 everything we've talked about accomplishes that 

17 goal, but I want to ask you to specifically address 

18 whether the goal of this regulation is still to 

19 prevent students from borrowing beyond what 

20 students can repay, and if it is, how you see this 

21 action's effect on that? 

22 Do you really believe that this action 

23 is going to advance that goal? Everything I know 
19

1 about the evidence base from the literature 

2 suggests that that's not true. 

3 There's a lot of evidence that suggests 

4 that disclosures have a limited impact. They tend 

5 to have their impact most when consumers are 

6 relatively sophisticated and well supported in a 

7 way that's unlikely to be true for the low-income 

8 population that populates this sector 

9 predominantly. 

10 So I think it's heroic to believe that 

11 this is going to advance that particular goal, and 

12 so I wonder whether the Department has either 

13 decided that's not an important goal or the kind 

14 of gains in terms of administrative simplicity 

15 really outweigh that.

16 This kind of asymmetry that's been 

17 brought up about the consequences of the rule 

18 falling on one sector relative to the entirety of 

19 higher ed is something that was addressed in the 

20 previous rule, and so I just want to understand 

21 better what the Department believes has changed 

22 about the analysis in that rule for why we shouldn't 

23 be concerned about the GE sector in particular 
20

1 given, again, the weight of the evidence that shows 

2 that the kind of struggles that students experience 

3 in repayment and low earnings are really 

4 disproportionately concentrated in that sector, 

5 and that kind of brings me to date. Greg, you said 

6 you were going to address that, and I just want 

7 to ask you to do so.

8 MR. MARTIN: Like I said, the data, we 

9 are in the final process of clearing it. I believe, 

10 I think I can say without going too far out on the 

11 ledge that we should have that available this 

12 afternoon. Sarah Hay will be back this afternoon 

13 to discuss the data with you.

14 Regarding your other issue, no, I don't 

15 think the Department is backed away from a belief 

16 that students should be protected from 

17 non-performing programs or programs for which there 

18 is no value derived from borrowing. 

19 I want to say up front that, you know, 

20 in no way did my comments take sanctions off the 

21 table. In fact, my comments put sanctions back 

22 on the table. The original proposal that we had 

23 didn't contain any sanctions at all. 
21

1 And I also want to reiterate that we're 

2 back here negotiating. If the intent of the 

3 Department -- if one of the options before us was 

4 to keep the current rule in its current format with 

5 no changes, we wouldn't be here. 

6 I don't mean to sound flip about that, 

7 but it just represents that the views of the 

8 administration are somewhat different than perhaps 

9 previous ones. That's just the fact. That 

10 happens all the time.

11 These rules are not meant -- they're 

12 meant to address some concerns we have, so I don't 

13 think that we're obligated to come back to the table 

14 and say, "Well, you know, we're not going to change 

15 anything from the previous rule," and saying that 

16 we would entertain program sanctions. 

17 I think we're throwing it out to the 

18 group to come up with other types of things that 

19 we could possibly do, and maybe I could have Steve 

20 talk about a few of those, what some of those things 

21 might be that we feel we could do.

22 We are committed to keeping -- you know, 

23 and I think it's a legitimate thing to say that 
22

1 there are programs offered at various different 

2 types of institutions and we're looking at, you 

3 know, I think students should be able to compare 

4 programs and look at what they are, both in the 

5 proprietary sector, other types of sectors, and 

6 I don't -- I'm not --

7 You know, just from a policy standpoint 

8 and a fairness standpoint that, to be here to say 

9 that we're going to start with the premise that 

10 we have started with, that we're going to calculate 

11 these rates for all institutions, all programs, 

12 but that we're willing to introduce, or to consider 

13 talking about anyway, program sanctions, but that 

14 those sanctions are only going to be related to 

15 one group of institutions. 

16 I don't -- we don't accept that all of 

17 the problems in higher education are related to 

18 that type of institution, or that other types of 

19 institutions don't have similar problems, and I 

20 think that's what this regulation represents. 

21 And I think if anything, my comments 

22 this morning have rather broadened what the 

23 position of the Department was as opposed to 
23

1 narrowing them, so I guess it's a matter of how 

2 all of us interpret what I said was, and I'm not 

3 trying to get into your head and convince you that 

4 I'm right. 

5 I'm just saying that that's our 

6 position and I don't think my comments took anything 

7 off the table. I think they put a lot more on the 

8 table.

9 MR. RAMIREZ: All right, so let me do 

10 this because we've gone well beyond the 10 minutes, 

11 and I want to show you all something and ask you 

12 if you think this conversation is productive or 

13 if you want to go with what I was at least initially 

14 thinking.

15 So can we get the -- to the project --

16 all right, so yesterday when we were talking, it 

17 looked like you were breaking down the sanctions 

18 discussions into four different categories. And 

19 whenever we throw up what we call a map here, it's 

20 a mind map, it goes clockwise. 

21 So metrics would be the first one, then 

22 it would go down to applications, corrective 

23 options, notifications, right? So anytime you see 
24

1 a map pop up there, at least the way that we have 

2 them laid out here, they go clockwise, okay?

3 So we were talking about identifying 

4 what were the metrics that we would use in order 

5 to determine whether a school is performing at a 

6 level that it should be, or a program. Then we'd 

7 go down, after we determine that, how would we apply 

8 those metrics? What type of corrective options 

9 would there be, and then how would we notify

10 students, right?

11 So we were hoping to chunk it like that 

12 and walk you through those steps. So if we were 

13 to look at the metrics, and that was some of the 

14 homework that we gave you yesterday was, and we 

15 grabbed some of them off the board. Some of you 

16 emailed me and we captured them and tried to pull 

17 out these pieces here. 

18 And so if I understand correctly --

19 well, let me ask the group here. Which one of these 

20 metrics is the one that determines whether a school 

21 is performing well or not? Which is the one?

22 MR. MATSUDAIRA: None of them, you 

23 know, or all of them.
25

1 MR. RAMIREZ: Right, it's not one, 

2 right?

3 MR. MATSUDAIRA: It's hard to say. 

4 This goes back to the question about what the goal 

5 is here, right? What are we trying -- what

6 information is important to communicate to 

7 students?

8 MR. RAMIREZ: All right, let me finish.

9 MR. MATSUDAIRA: What I was --

10 MR. RAMIREZ: Jordan, let me finish 

11 this point though.

12 MR. MATSUDAIRA: But we can't answer 

13 --

14 MR. RAMIREZ: And I'm coming back, 

15 Jordan.

16 MR. MATSUDAIRA: -- the question 

17 without a Department --

18 MR. RAMIREZ: Jordan?

19 MR. MATSUDAIRA: -- answer to that.

20 MR. RAMIREZ: But I'm coming back to 

21 that. Just let me finish this and then I'll come 

22 back. I just wanted to show you where I wanted 

23 to go, okay?
26

1 So I think it's understood that no one 

2 of these metrics is going to be the silver bullet, 

3 right? And what Greg was talking about was that 

4 whatever metrics you all decide, D/E needs to be 

5 one of them, right? So if you look at the top left 

6 there, composite components, let's put D/E 

7 underneath that one, so debt to earnings needs to 

8 be one of them. 

9 So we would start identifying the ones 

10 that you feel would be the ones that would go 

11 underneath there, right, after we generate more 

12 of these, so that's where I wanted -- I was hoping 

13 that we could work and step you towards.

14 So the question that I have for you, 

15 and it's going back to what Jordan was trying to 

16 say right there, was do you want to continue, put 

17 a few more minutes on the clock to continue the 

18 conversation or do you want to jump into this right 

19 here?

20 PARTICIPANT: Can we put five more 

21 minutes on the clock?

22 MR. RAMIREZ: Let's put five minutes 

23 up on the clock, and if you see, we'll keep this 
27

1 up here. We'll add five minutes up on the clock 

2 over there. You know what? We'll even do five 

3 minutes and -- seven minutes and 20 seconds. How 

4 is that? All right? So we'll just continue down 

5 the list here. So we have Laura, Johnson, Jen, 

6 Daniel, Sandy, and one more. Laura?

7 MS. METUNE: My question was along the 

8 lines of Jordan's question which is what is the 

9 problem we're trying to solve? My understanding 

10 of the history of gainful employment was based in 

11 widespread fraud and abuse that was occurring in 

12 a very specific sector. 

13 In California, we experienced the 

14 closures of Corinthian, and ITT, and countless 

15 other small schools. And so to have an oversight 

16 and accountability structure focused on the area 

17 where there's widespread fraud and abuse occurring 

18 seems to make sense to me. 

19 I am not opposed to giving students 

20 better information to help them make more informed 

21 decisions about where they go to college. I do 

22 think it's a questionable use of government 

23 resources to try and use that kind of a disclosure 
28

1 practice to get at the problems that we've faced 

2 in one specific sector. 

3 And I guess my question is what have 

4 we learned since the closures of Corinthian and 

5 ITT? What have we learned about this sector, and 

6 the problems in this sector, and how are we using 

7 the other tools that have been mentioned to address 

8 those problems?

9 And unless we can answer that question, 

10 I'm just, I'm really at a loss for why going into 

11 a conversation about what good information students 

12 should have to make decisions is useful to what 

13 I thought was the original goal of this work.

14 MR. RAMIREZ: Greg?

15 MR. MARTIN: Just keep going in line.

16 MR. RAMIREZ: Next, Johnson?

17 MR. TYLER: I guess I just need a 

18 clarification here. If we did create a composite 

19 component here and someone failed -- everyone 

20 agrees. We agreed on the composite and everyone 

21 fails that test, and we give them, you know, what 

22 everyone agrees is a reasonable time to fix the 

23 problem and they don't, is no more funding part 
29

1 of what's going to spur them on?

2 And I guess my second question is does 

3 that apply to both sectors, those institutions that 

4 are under gainful employment, or is that only --

5 or as well as the nonprofits, or is it simply to 

6 the ones who are under the gainful employment? 

7 Because I'm a little unclear about that.

8 MR. RAMIREZ: I'll let Steve and Greg 

9 jump in here for a second, but that's what I was 

10 trying to do. I was trying to step you there, 

11 right? So once we determine what those metrics 

12 are, how do we apply them, and then what would be 

13 corrective options?

14 Now, I've heard that one of the 

15 corrective options that the Department is not 

16 interested in is the elimination of the Title IV 

17 funding for that institution, but are there other 

18 corrective options that can be explored? Can we 

19 tap into this brain trust to see what else can be 

20 done short of that? And that's where I'm trying 

21 to get you all to.

22 MR. TYLER: Okay, I may be a little 

23 obtuse here, but I just want to be clear. You're 
30

1 saying you're going to take all this data and then 

2 DOE is going to do whatever enforcement it does 

3 normally? I mean, I just feel like we were creating 

4 a metric that would create some incentive here.

5 MR. FINLEY: So I think we're looking 

6 at ways to identify good programs and bad programs, 

7 and not just make that determination based on a 

8 D/E rate, right? Now, it's -- and then there's 

9 -- the concept I've heard at the table is whether 

10 there's a program that's had an opportunity to try 

11 to improve over time. 

12 So there is a concept there that says 

13 at some point you have a discussion with an 

14 institution that's offering a bad, what looks like 

15 a bad program, and say, "What's been done to improve 

16 it? What's your chart for improving it?" 

17 And there may be a future conversation with 

18 the institution that says, "We don't understand 

19 why you're still offering this program. Maybe you 

20 should stop enrolling students in that program if 

21 it's not improving." And there could 

22 even be a point where the institution is being 

23 recertified and there would be another discussion 
31

1 about what programs aren't included in that 

2 recertification if it's consistently demonstrated 

3 very bad performance. It's a concept. 

4 If it's not just the D/E rates, but it's going 

5 to be that plus other things, and I'm not sure that's 

6 suited to thresholds, but that's what we'd love 

7 to hear discussed at the table, but it's not just 

8 for GE schools. 

9 It would be a good program or a bad 

10 program, whoever is offering it, right? High D/E 

11 rates would be one factor, but certainly not the 

12 only factor, and that's what we're looking at for 

13 this discussion and obviously we want feedback 

14 around the table, and that's why we're here.

15 PARTICIPANT: If I may, there are about 

16 over 5,000 institutions in the United States that 

17 are getting Title IV funding, just a lot of work 

18 if you're going to have to do that sort of hands-on 

19 investigation every time you identify one of the 

20 poorer performing schools with this metric.

21 MR. RAMIREZ: Jennifer?

22 MS. BLUM: I have a process question. 

23 Whitney has now asked twice for a caucus, so what 
32

1 is the process for a caucus?

2 MR. RAMIREZ: I didn't hear the caucus 

3 request.

4 MS. BLUM: Well, last night and this 

5 morning, that was the first thing she said this 

6 morning and it just went right back into the queue, 

7 so I was just curious about --

8 MR. RAMIREZ: I'm sorry. It was my 

9 understanding that you didn't prefer to have a 

10 caucus.

11 MS. BARKLEY-DENNEY: What I didn't 

12 prefer were small group caucuses like we did 

13 yesterday. I think it was helpful and that we 

14 fleshed out our ideas, but now it's time to bring 

15 these ideas to the table as a group instead of being 

16 segmented, and I think that, you know, in having 

17 done --

18 I've done these a couple of times, but

19 sometimes, you know, you can keep certain people 

20 in, but it's mostly closed to the public, but I 

21 think that that would help move the conversation 

22 along in a way that could be more productive for 

23 everyone.
33

1 MR. RAMIREZ: All right, so if I 

2 understand what you're saying, it's a full group 

3 caucus where it is closed off?

4 MS. BARKLEY-DENNEY: I mean, I'll 

5 leave that to the group, but I've heard around the 

6 table that that would make people comfortable and 

7 we definitely want the Department to stay in this, 

8 so we don't want segmented caucuses, you know, where 

9 one group goes off and decides they're going to 

10 do something, the other group decides they're going 

11 to do something, and never the twain shall meet.

12 PARTICIPANT: So we stay in this room? 

13 Where would the group meet? Stay in this room? 

14 The livestream goes --

15 MR. RAMIREZ: Yeah, so let me do this. 

16 We have some people in the queue. Is everyone 

17 okay with suspending the queue for a minute, take 

18 a one function break, and let me see what our options 

19 are? 

20 I just don't know what the logistics 

21 are here as far as, you know, being able to 

22 accommodate that request, so let me explore that 

23 with facilities and see what is doable.
34

1 MS. BARKLEY-DENNEY: Okay, and let me 

2 just say I don't -- you know, we can keep the queue, 

3 and then just once we start talking about the 

4 nitty-gritty, I just think that we might get more 

5 done and people might feel more comfortable putting 

6 things on the table if, you know, there's not so 

7 many eyes on them.

8 MR. RAMIREZ: I see, okay. Yeah, 

9 sorry, I misunderstood that, so I apologize. Then 

10 Daniel is next then.

11 MR. ELKINS: It's Daniel. This is 

12 more of a Braveheart style speech to everyone.

13 MR. RAMIREZ: The short version?

14 MR. ELKINS: Yeah, for sure. We have 

15 a unique opportunity right now around the table, 

16 all of us as experts in higher education to identify 

17 things that are going to protect students and to 

18 reach agreement. I want everyone just to realize 

19 that. 

20 If we come to consensus, it goes on the 

21 books forever. I mean, right, but if we don't reach 

22 consensus, we might not get anything, no sanctions, 

23 sanctions for everyone. We really have an 
35

1 opportunity right now to reach consensus, and I 

2 encourage everyone. 

3 Give a little, just a little in order 

4 that we can protect students from the worst actors, 

5 maybe not get everything that we need, but we have 

6 a unique opportunity right now, and I think that 

7 Whitney's suggestion to have an open conversation 

8 will get us there.

9 MR. RAMIREZ: Thank you, Daniel. And 

10 just a quick reminder on that, right, consensus 

11 is not getting what you, absolutely is your best 

12 desire, right? The lowest level of consensus, if 

13 you remember when we went through the five levels 

14 of consensus, is that you don't necessarily agree, 

15 but you will support, right, for whatever reason. 

16

17 You just understand that, "Look, this 

18 is the best deal that we're going to get. I get 

19 it. I'm not happy with it, but I'm going to support 

20 it." That's the lowest level of consensus and 

21 there's rungs. 

22 You know, there's five total levels of, 

23 well, four because the fifth level of consensus 
36

1 is no agreement, right? So just keep that in mind 

2 as we're having these discussions. It's not always 

3 your best position, okay? So with that, I'll pass 

4 it to Sandy.

5 MS. SARGE: This is Sandy. I agree 

6 with the comment yesterday where somebody said it's 

7 hard when it's 45 minutes later and you got so many 

8 thoughts, but I want to make sure that I understand 

9 something. 

10 So Chris Madaio, on occasions you've 

11 gone back to the definition of gainful employment 

12 in the HEA, and my understanding when all of this 

13 started back, it was because gainful employment 

14 had an interpretation by the Department for years, 

15 and years, and years.

16 Basically it must have been 65 plus 10, 

17 so 45 years there had been an acceptable or at least 

18 theoretically non-contended gainful employment 

19 interpretation that was basically as long as it's 

20 leading to a career and not a hobby of some sort, 

21 or a non-career oriented, that it would be then 

22 seen by the Department as gainfully employed.

23 And that then, as all of this stuff came 
37

1 out -- so with all due respect, to keep going back 

2 to the HEA, the HEA doesn't define it in anything 

3 other than to prepare for gainful employment in 

4 a recognized profession, i.e., not a hobby. 

5 Then we all came through this around 

6 seven, eight, nine years ago, and we began to try 

7 to define how do we measure that? And if we, if 

8 I think about this financially or even just 

9 mathematically, I think that at the end of the day, 

10 it basically says, "Does a program provide the 

11 student with work that enables them to repay the 

12 debt?" 

13 So to me, why wouldn't that be every 

14 program because you're loaning money to somebody 

15 and there isn't -- based on just that financial 

16 relationship of loaning as opposed to giving, or 

17 granting, or scholarship-ing somebody, you're 

18 loaning it. There is an implicit expectation that 

19 repayment will occur. 

20 So if we are -- wouldn't that by default 

21 make every single program in which debt is incurred 

22 to partake in it or to pursue it a gainful employment 

23 definition? So to me, you know, we're talking 
38

1 about debt, not grants. There should be 

2 financially for everybody --

3 And while I really am an advocate for 

4 protecting the extreme cases of students who are 

5 deceived, and I, as many of us have said, we are 

6 for sanctions that find these bad actors that are 

7 certainly extremes that we've all talked about, 

8 but at the end of the day, this is a debt 

9 relationship. 

10 I think where we're failing students 

11 is that we are not educating them on what that really 

12 means the day they enter into school. Promissory 

13 note is arbitrary. They don't know what it means. 

14

15 But in essence, any program in which 

16 you are lending to anybody, there needs to be an 

17 expectation that repayment will happen, and it is 

18 our job as the educators to ensure that that student 

19 understands that and can do it. 

20 If they can't do it because they have 

21 a history degree that I think Todd mentioned 

22 yesterday or a forestry degree, then we need to 

23 question whether debt is the right vehicle to allow 
39

1 them to take that program.

2 MR. RAMIREZ: Okay, thank you. So I 

3 have Tony, Jordan, and then Tim. Whitney, is it 

4 your tent up? Okay, so, Tony?

5 MR. MIRANDO: Thank you, and I'll try 

6 to keep mine very short and sweet. Like many of 

7 my colleagues around the table, I came here to the 

8 first meeting prepared as I was eight, nine years 

9 ago when I joined the group to initiate gainful 

10 employment regs, to again come back to the table 

11 to do the same. 

12 Just like the rest of you, I received 

13 an email with these new issue papers that implicitly 

14 show that it's a new mandate. We have a new job 

15 and it isn't gainful employment. It is now D/E 

16 and it means for everyone. 

17 And I want to give thanks to the 

18 individuals around the table that are getting past 

19 that this is not GE and that we want to come to 

20 the table and fix what is being presented to us 

21 in a more meaningful way to ensure that students 

22 are protected, and I, like a few others, already 

23 say I'm ready to work. Let's get it done and move 
40

1 forward.

2 MR. RAMIREZ: Jordan?

3 MR. MATSUDAIRA: Sorry, I'm going to 

4 keep asking the question of what the goal is here 

5 until we come up with a better answer. The goal 

6 can't be to identify good and bad programs. That's 

7 just too broad a goal. 

8 If we're going to start with something 

9 like that, then I'm going to, you know, we're going 

10 to argue about whether we're really, you know, we 

11 can quantify the amount of time you spend in a forest 

12 clearing getting sunshine on your face and things 

13 like that to try to figure out what a good program 

14 is, you know what I mean? The intent of the 

15 original rule was not to identify good and bad 

16 programs. It was to identify programs that burden 

17 students with debt beyond what they were likely 

18 to repay given their earnings, and I'd suggest that 

19 we come up with a working goal that really reflects 

20 that, and I think doing that really guides metrics, 

21 okay?

22 There are all these things like 

23 completion rates, you know, graduation rates, 
41

1 license or exam pass rates and so on which might 

2 be proxies for kind of struggles with debt and so 

3 on, but they're very noisy proxies for those things. 

4

5 There are very direct measures of how 

6 people perform on their loans and so on, repayment 

7 rates, default rates, debt to earnings and so on 

8 that make a lot more sense in that space, but I 

9 think we have to start with a goal that actually 

10 makes sense here for any of the conversation that 

11 follows to be meaningful.

12 We went through a two-year process 

13 across the White House, the Department of 

14 Education, and other parts of government trying 

15 to decide what was a good program in the college 

16 score card and come up with a whole system of 

17 accountability. 

18 You know, I imagine there are some 

19 people in the back row over there that might break 

20 out in hives just having me talk about it. That 

21 was a process that didn't reach consensus over a 

22 really wide range of people with really deep 

23 exposure to the data and a lot of background across 
42

1 all different sectors working, you know, a lot of 

2 hours every week out of the year for nearly two 

3 years. 

4 So I don't think it's a reasonable goal 

5 for us to even consider over the next six days that 

6 we're going to be meeting, so I'd urge us to kind 

7 of refocus on a goal that makes more sense.

8 PARTICIPANT: Do you have a 

9 suggestion, Jordan, of a goal that makes sense?

10 MR. MATSUDAIRA: Establish a set of 

11 metrics that identify programs where students are 

12 likely to borrow more than they can afford to repay 

13 and establish a set of sanctions that in a 

14 meaningful way kind of prevents students from 

15 entering those kinds of programs. 

16 So, you know, for me, I think having 

17 a sanction structure, a sanction structure meaning 

18 notifications which is in the Department's capacity 

19 to be able to warn students about the outcomes of 

20 students and their likely ability to repay across 

21 the full span of higher ed, but where the Department 

22 is able, I think it also makes sense to take further 

23 action and have sanctions that lead to loss of Title 
43

1 IV.

2 MR. JEROME: Jordan, just one 

3 question, are you comfortable focusing primarily 

4 on student borrowers? Because the focus on 

5 non-borrowers got to looking at programs rather 

6 than borrowers and it's a fundamental difference 

7 the way you approach the regulation.

8 MR. MATSUDAIRA: I think conceptually, 

9 I agree with the point that you've been making, 

10 Marc, that that's a sensible kind of thing to do. 

11 Given the broader constraint about the Department 

12 isn't able to just have a sanction be the loss of 

13 loan eligibility and participating in the loan 

14 program versus all of Title IV, then I think we 

15 have to work within that constraint.

16 MR. JEROME: That's true.

17 MR. MATSUDAIRA: And because of that, 

18 I think it's something that's worth discussing. 

19 I think that's the kind of thing that I think is 

20 a very reasonable criticism of the structure of 

21 the current rule, but I think there are trade-offs 

22 about that that I think we should --

23 MR. JEROME: So I think we will be able 
44

1 to propose some alternative sanctions.

2 MR. RAMIREZ: All right, Tim, and then 

3 we'll see where we go from there. Tim?

4 MR. POWERS: So I'll be as brief as 

5 possible, but, you know, I just, I want to reiterate 

6 that there is a difference between baccalaureate, 

7 master's, and doctoral programs versus certificate 

8 programs. They have fundamentally different 

9 purposes, fundamentally different outcomes. 

10 And while I agree that we should be 

11 providing students with as much transparency and 

12 information as possible, I just, I have to reiterate 

13 that there is just a fundamental difference between 

14 comparing a GE regulatory structure that was 

15 created for short-term certificate programs, which 

16 has extended to baccalaureate and master's programs 

17 at some of the schools based on how they're defined 

18 in the HEA, but which was primarily designed for 

19 certificate programs, and just willy-nilly 

20 applying it to bachelor's, master's, and doctoral 

21 programs in the public and nonprofit sector. We're 

22 just going to have a really hard time getting past 

23 that.
45

1 PARTICIPANT: They're open to that?

2 MR. POWERS: Open to what?

3 PARTICIPANT: Open to the two --

4 MR. RAMIREZ: Wait a second. Put on 

5 your mics and state your names if you're going to 

6 make a comment.

7 PARTICIPANT: Just I'm asking is the 

8 Department open to shrinking the scope of the rule 

9 to certain degree levels? You know, there's been 

10 a lot of discussion on graduate and Tim's bringing 

11 this up, and I think it's a fair discussion.

12 PARTICIPANT: Yeah, the answer to your 

13 question is yes, we would certainly be willing to 

14 consider. I'm not stating that our position would 

15 be firm or we're adopting that if your question 

16 is, "Would we be willing to consider looking at 

17 these as they're applicable to types of programs 

18 at all institutions?" so we wouldn't be saying, 

19 "graduate only programs at not-for-profit public 

20 and privates." 

21 But if, for example, somebody were to 

22 say, "Well, we don't feel this should be applicable

23 to graduate programs," if, and this is just a 
46

1 hypothetical, if that were to be an exception, then 

2 it would have to be applicable to those graduate 

3 programs at all types of institutions. But to 

4 answer your question, yes, we would consider 

5 proposals like that.

6 MR. RAMIREZ: All right.

7 MR. POWERS: I just had one last --

8 MR. RAMIREZ: Very quick.

9 MR. POWERS: -- very quick thing which 

10 is, you know, I just want to again remind folks 

11 that publics and private nonprofits that run 

12 certificate programs still have to report and face 

13 sanctions as it currently stands. 

14 And again I just want to reiterate that, 

15 you know, while we do certainly have issues and 

16 we could probably have a really interesting, and 

17 I'm looking forward to having an in-depth 

18 discussion on what the metrics should be, that I 

19 don't --

20 I just want to remind folks that this 

21 is not only targeting one sector while just sort 

22 of letting the other two sectors slide by, that 

23 the publics and the nonprofits still have to report 
47

1 this stuff and face sanctions currently, and will 

2 likely in any regulatory scheme, at least for 

3 certificate programs offered under this sort of 

4 certificate definition, will continue to face the 

5 same sanctions and outcomes-based -- I don't know 

6 what the word, we're not using penalties or 

7 sanctions -- consequences that the other sectors 

8 would face.

9 So again, I just wanted to remind that 

10 because I think that has just sort of gotten lost 

11 a little bit in the conversation, but really looking 

12 forward to having this metrics conversation because 

13 I think we can really move it forward.

14 MR. RAMIREZ: All right, look, I'm 

15 going to go ahead and stop the conversation here 

16 just because we're going to have opportunities to 

17 discuss, right? But I want to start getting into 

18 the problem solving, right? So what I would 

19 suggest is this, reconvening, and I say reconvening 

20 at 3:00. Let me finish. 

21 So what I'm saying is that we're going 

22 to excuse the public. I'm going to ask the 

23 negotiators and the alternates to stay here and 
48

1 we'll figure out our path, okay? But as far as 

2 -- that incorporates lunch, caucus, work time, but 

3 reconvene so the members of the public can come 

4 back at 3:00 and see where we're at, okay?

5 PARTICIPANT: And I propose we meet 

6 during lunch.

7 MR. RAMIREZ: We're going to discuss 

8 that right now, but, so if I could ask the members 

9 of the public, you're welcome to come back at 3:00 

10 and we're breaking right now. Negotiators, 

11 alternates, maybe take a one function break and 

12 come right back.

13 (Whereupon, the above-entitled matter 

14 went off the record.)

15 MS. BUCK: So while you're still coming 

16 to your seats, I hope you can be thinking about 

17 somebody making a proposal about what time we would 

18 end today.

19 (Off mic comments.)

20 MS. MILLER: So do we have a proposal 

21 for the end time for today?

22 (Off mic comments.)

23 MS. MILLER: So 5 o'clock has been put 
49

1 -- 5 o'clock is what's on the table. Do we have 

2 all the negotiators to take a vote on that?

3 MS. BUCK: So temperature check on the 

4 end time.

5 MS. MILLER: Five p.m.

6 MS. BUCK: Five o'clock is the 

7 proposal.

8 MS. MILLER: So I do not see --

9 MS. BUCK: No down.

10 MS. MILLER: No thumbs down. So we'll 

11 be ending at 5 p.m. today. Okay.

12 MS. BUCK: All right. Well welcome 

13 back, everyone. And just for the general public, 

14 we want to let you know, generally, there were a 

15 few areas of consensus reached, and Crystal's going 

16 to report on that.

17 MS. SMITH: Hi there. This is 

18 Crystal. So first, when it comes to the 

19 application of any sanctions, the group has 

20 determined that they will eliminate all graduate 

21 degrees from consideration of any of these 

22 sanctions.

23 The second item is when it comes to 
50

1 triggering metrics, they have determined that, 

2 tentatively, that a debt-to-earnings ratio plus 

3 the repayment rates are the metrics that they will 

4 be considering.

5 When it comes to corrective actions, 

6 or corrective options, after failing both the D/E 

7 and the repayment rates, the Department must take 

8 some corrective action. And that has been defined 

9 as a number of things, including program reviews, 

10 LS&T, letters of credit, heightened cash 

11 monitoring, provisional credit, all of these up 

12 to and including the loss of Title IV funds. And 

13 that's where they are now.

14 MS. BUCK: And again, those are 

15 recommendations to the Department of Ed. The 

16 Department of Ed isn't saying what it will do. 

17 But these are the recommendations of the group. 

18 Is there any correction to what you thought we had 

19 agreed to?

20 MS. SARGE: To what she just said.

21 MS. MILLER: Okay.

22 MS. SARGE: Not anything --

23 MS. MILLER: Okay, Sandy, really 
51

1 quickly. Then we're going to go round robin, 

2 around the table. Go ahead.

3 MS. SARGE: So when you said the 

4 application part not applied to graduate, these 

5 sanction, you said sanctions, I think everybody 

6 was saying disclosures as in total, right? We're 

7 taking everything off, not just sanctions. So I 

8 just wanted to make sure I heard that right and 

9 everybody agrees. Okay.

10 MS. SMITH: My apologies. That's 

11 correct.

12 MS. MILLER: Okay. So Tony, did you 

13 have a correction?

14 MR. MIRANDO: Yeah. Well, it isn't a 

15 correction but it's an addition. So we talked 

16 about D/E plus repayment as a first-year, but then 

17 we came up with a whole list of secondary tiers 

18 for, as an alternative, as well.

19 (Off mic comments.)

20 MR. MIRANDO: Right. Correct.

21 MS. SMITH: Okay. And that's also 

22 correct. Yes.

23 MR. MIRANDO: Okay.
52

1 MS. MILLER: Thank you.

2 MS. SMITH: That was the first line of 

3 triggering, and then the second tier.

4 MS. MILLER: Jeff, do you have a 

5 correction, or something else?

6 MR. ARTHUR: I have a clarification or 

7 a point on repayment rate.

8 MS. MILLER: Okay. Make it real --

9 okay. Go ahead.

10 MR. ARTHUR: All right. So the -- we 

11 don't want to take for granted the definition of 

12 repayment rate. There's a proposal in the 

13 reauthorization bill already that has a formulated 

14 calculate repayment rate. You've got one you 

15 probably -- that we've used in the past. And there, 

16 it sounds like there's going to be one soon in a 

17 Senate bill.

18 I would suggest that if the House and 

19 Senate bills happen to match, for simplicity 

20 purposes, why not follow their lead on the repayment 

21 rate formula?

22 MS. MILLER: Okay. Thank you. So 

23 next we'll go to Laura. Then we have Ahmad, and 
53

1 then Todd and then Jen.

2 Laura.

3 MS. METUNE: I'm a little bit 

4 uncomfortable with the characterization that 

5 agreement or consensus has been reached. My 

6 understanding is that we were taking temperature 

7 checks. I was out of the room when the decision 

8 was made to close this conversation off to the 

9 public. I probably wouldn't have supported that. 

10 I just think that, you know, a lot of 

11 the areas where I agreed to provide, sort of, that 

12 level was because I wanted to move the conversation 

13 forward and not because I'm not fundamentally 

14 uncomfortable with some of the responses we heard 

15 about what next steps might be or how we might 

16 protect students. I just wanted to go on the record 

17 and say that.

18 MS. BUCK: So the term should be 

19 changed to temperature check as opposed to 

20 consensus. Okay.

21 MR. SHAWWAL: Ahmad here. Just for 

22 everyone's benefit and edification, I just want 

23 to walk through the structure that was just 
54

1 discussed. And please correct me if I'm wrong, 

2 just for my edification, too.

3 So we have D/E and loan repayment rate 

4 as a first step. If an institution passes both, 

5 they're good. If they fail one --- either. 

6 Either, okay. And so if they fail these metrics, 

7 they go into a second step, through which we have 

8 discussed, that the Department ought to implement 

9 one of these tools that they have in the toolbox. 

10 Is that correct?

11 Tier 2, or is Tier 2 a second --

12 MS. BUCK: Tier 2.

13 MR. SHAWWAL: -- set of metrics?

14 MS. BUCK: Yes.

15 MR. SHAWWAL: A way to adjust this 

16 course?

17 MS. BUCK: Exactly.

18 MR. SHAWWAL: Okay. Is that 

19 essentially an appeals process, then?

20 MS. BUCK: Oh, actually, that's an 

21 interesting way of putting it.

22 MR. SHAWWAL: Okay.

23 MS. BUCK: Sort of.
55

1 MR. SHAWWAL: Okay.

2 PARTICIPANT: Administrative 

3 capability does provide an appeals process.

4 MR. SHAWWAL: Okay. Makes sense. 

5 Okay. And if they do not pass this process, then 

6 hand it over to the Department. It's their 

7 responsibility to enact one of these things that 

8 we have -- that they have said that they will do, 

9 which we will see if they will do.

10 Is there anything I'm missing there, 

11 some big points? No? Okay. With that, I think 

12 that's an improvement from where we began these 

13 discussions.

14 My concern, still, is on the time line 

15 of this entire process, because while all this is 

16 going on, and if we have identified a truly bad 

17 actor, students will still be enrolling and being 

18 harmed by these institutions while this entire 

19 process is going on. Do we have any sense of how 

20 long this could take, at all?

21 MS. BUCK: So we didn't get to the time 

22 line, and what I would like to propose is that we 

23 get back to that tomorrow. I've added it to a list 
56

1 here, but that has not been covered yet.

2 MR. SHAWWAL: So one thing I really 

3 want to emphasize is, maybe we should discuss, how 

4 do we make sure that this entire process takes place 

5 in a reasonable time frame so that we can still 

6 protect students, and so that these bad actors are 

7 not continuing to sort of act in an unreasonable 

8 way that would harm students?

9 MS. BUCK: Okay. Good. Thank you.

10 Todd?

11 MR. JONES: I'm just going to offer a 

12 side comment. I've been through more than a few 

13 of these on the Department side, the 

14 sitting-at-the-table side, and I want to offer my 

15 compliments to Whitney, for suggesting we actually 

16 have a closed session.

17 I think a number of what I would call 

18 foolish ideas were dropped, and others which, well 

19 say my ideas, that were not probably supported by 

20 others, were at least vetted and quickly disposed 

21 of in a matter of time --

22 (Laughter.)

23 MR. JONES: -- that would have dragged 
57

1 on if we were out here all having to offer our public 

2 position on, in a rather expeditious manner. And 

3 that is a different than the norm of how these 

4 processes work, but I appreciate -- I want to thank 

5 her for doing that. And I know I'm not the only 

6 one at the table who believes that. Others can 

7 chime in but, you know, our thanks to you for doing 

8 so.

9 MS. BUCK: Jen.

10 MS. DIAMOND: So I just wanted to --

11 well a couple of things. I just wanted to echo 

12 Laura, actually, on the, on where we are, in terms 

13 of, you know, we can either say temperature checks 

14 or sort of a tentative, you know, pathway for a 

15 conversation.

16 And the reason why I share, you know, 

17 Laura's feelings, but also I wanted to add that, 

18 just so the public understands, because I know 

19 you're coming into like, a three-hour conversation, 

20 or a three or four hour conversation, we haven't 

21 reached consensus on the actual, what the 

22 debt-to-earning metric and loan repayment rates 

23 should be.
58

1 So I just want to be clear. I'm saying 

2 this for the benefit of the public. So we still 

3 have a lot of work to do. We set up a framework 

4 and a pathway that I think we should all be, you 

5 know, really proud of, or at least that we've had 

6 this, you know, really good dialogue about it, so.

7 And then, Ahmad, to your point, I do 

8 want to point out something, and we could put this, 

9 I would propose even putting it in. If there's 

10 a school that has multiple programs, and that is 

11 shown -- and again, we'll have to talk about the 

12 end size, but if it's shown two years in a row, 

13 the Department always has the ability, and we should 

14 not preclude through our enforcement, what we're 

15 going to be writing on enforcements, we should never 

16 preclude the ability of the Department to act fast.

17 So if they have, you know, more than 

18 50 percent -- I'm making up the number, and we won't 

19 say a number, because that would be not be 

20 appropriate, but if they -- if there's a -- because 

21 we'll have the benefit of these two metrics, if 

22 there's something that's obviously a theme, you 

23 know, two years in a row or whatever, the Department 
59

1 can always take swift action.

2 And I think, providing some sort of 

3 language that says, of course the Department can 

4 always use its other, you know, fast action tools, 

5 because I -- I said this during the caucus, too. 

6 We want to be careful not to preclude the ability 

7 of the Department to do something else if it feels 

8 the need to more quickly.

9 But what we are dealing with, and again, 

10 I'm saying this for the benefit of the public, what 

11 we are dealing with in real sort of aspects is, 

12 you know, a situation where a university has one 

13 program, that's thematically wrong, over three 

14 years, you know, and needs to be, you know, vetted 

15 out and whatever in a more methodical way, that's 

16 one thing. But if we have a persistent problem 

17 as an institution, that's a different thing.

18 So I do think that that's something that 

19 we could mete out more, you know, over the next 

20 few days and next session.

21 MS. MILLER: Thank you. Steve, did 

22 you want to comment?

23 MR. CHEMA: Yes. There was a concept 
60

1 that Marc Jerome had mentioned earlier, and 

2 probably early in the caucus, or maybe even late 

3 in the time before the caucus, that talked about 

4 looking at the debt-to-earnings ratio at the 

5 institutional level as well. And that seemed to 

6 have gotten some favorable reception around the 

7 table.

8 I just wondered if people had, were also 

9 open to the notion that that would have been part 

10 of the tier 2 things that would be looked at, if 

11 there was this heightened review by the Department. 

12 I don't think anybody had abandoned it. I just 

13 think we didn't come back and talk about it because 

14 it was almost 3 o'clock.

15 MS. MILLER: Daniel.

16 MR. ELKINS: We would be supportive of 

17 adding that into the second tier. And I do think 

18 that it merits a discussion.

19 MS. MILLER: Is that something we want 

20 to discuss this afternoon?

21 MS. BUCK: I do want to say that we do 

22 have a data report coming up. And we might want 

23 to make sure we get that in. What we could do is 
61

1 add that to the list of things we don't want to 

2 forget. Would that be okay?

3 (Off mic comments.)

4 (Laughter.)

5 MS. MILLER: Thank you, Marc Jerome. 

6 So are there any other comments that negotiators 

7 want to make about the caucus before we get into 

8 the much-anticipated data requests? Okay. So I 

9 think -- I don't -- can we break?

10 MS. BUCK: We're ready, I think, for 

11 the data report.

12 MS. MILLER: Let's give Sarah just one 

13 minute.

14 MR. MARTIN: We're going to take a 

15 two-minute, what's it called, single purpose break, 

16 to sit --

17 MS. MILLER: One-function break?

18 MR. MARTIN: One-function break, where 

19 you don't leave your -- you know, here's the thing. 

20 If you leave a desk and get up, then I'll do the 

21 data report as opposed to Sarah. So that might 

22 give you some incentive to stay put.

23 MS. MILLER: Are you ready? Or do you 
62

1 need two more minutes?

2 MR. MARTIN: Two more minutes exactly.

3 MS. MILLER: Okay. Two more minutes.

4 MR. MARTIN: We're starting 

5 distribution of the first of two papers right now.

6 MS. MILLER: Thank you.

7 MR. MARTIN: We had to do just some, 

8 a reprint on the second one, so that's just, takes 

9 some time to get that.

10 MS. BUCK: Okay. Go ahead.

11 MS. HAY: Ready to go?

12 MS. MILLER: Go ahead, Sarah.

13 MS. HAY: Okay. This is Sarah Hay. 

14 Thanks --

15 MS. BUCK: A little bit closer to the 

16 speaker, I think.

17 MS. HAY: Oh sure. Okay. Okay. So 

18 this is Sarah Hay. Thank you very much for your 

19 patience. I really appreciate it. I'm currently 

20 balancing a sick toddler at home and making time 

21 to come in here and answer your questions, so thank 

22 you for your patience with me.

23 The version that we're reprinting had 
63

1 a couple of typos in it, and we want to make sure 

2 we get you the right one. So they're printing it. 

3 They'll bring it back down once they're finished 

4 reprinting it, so I apologize for that little 

5 miscommunication.

6 So before I start jumping in to walking 

7 you through this, I do want to say that these data 

8 are the work of many minds and many hands. And 

9 I really want to thank all of the people at Ed who 

10 helped us get it for you guys. So it's my staff, 

11 it's Federal Student Aid, it's Policy Program Study 

12 Service, it's the Budget Service, and a whole bunch 

13 of other people who looked at it too.

14 So just keep that in mind. It's a lot 

15 of people's work going into getting you these 

16 numbers. Okay.

17 So the first one is Computing 

18 Debt-to-Earnings Using College Scorecard Data. 

19 And before we get too far into it, and there is 

20 sort of a high-level method section in there, but 

21 if you would turn to the second page for me, there's 

22 a couple of important things about the differences 

23 between NSLDS data, or the National Student Loan 
64

1 Data System, and how we compute debt-to-earnings 

2 for GE programs, versus the scorecard data.

3 And so Exhibit 1 sort of walks you 

4 through the differences. So GE programs 

5 debt-to-earnings is at the program level. 

6 Scorecard data is at the institutional level. So 

7 this paper itself does not have anything by Zip 

8 Code. Okay.

9 The GE programs use an exit cohort for 

10 both debt and earnings. The scorecard data uses 

11 an entry cohort, so it's just capturing people at 

12 a slightly different point, okay, and grouping them 

13 based off of when they enter rather than when they 

14 exit. Okay.

15 So it uses an entry cohort for earnings, 

16 but an exit cohort for the debt calculations. 

17 Okay. And sort of how we match up the years is

18 explained further, if you are really truly 

19 interested in getting into the nitty-gritties on 

20 it. Okay.

21 GE debt-to-earnings excludes 

22 non-completers, but the scorecard includes them. 

23 The GE data includes non-borrowers in both the 
65

1 debt and the earnings, but non-borrowers are 

2 excluded in the debt, but included in the earnings. 

3 And this just has to do with the data we have that's 

4 available to us. Okay.

5 So the GE programs, debt is capped at 

6 tuition and fees, but not in the scorecard data. 

7 Debt includes non-federal debt in the GE programs, 

8 but it's excluded in the scorecard data. And 

9 earnings values in the GE programs are based on 

10 a cohort that includes zero-dollar earnings, but 

11 those are excluded in the scorecard data.

12 And then program-level debt and 

13 earnings are calculated for both undergrad and 

14 graduate programs in the GE programs, but only 

15 institutional-level are calculated for 

16 undergraduate students in the scorecard level. 

17 Okay.

18 So there are some important data 

19 differences there. So what I'd like you to 

20 remember is these data are accurate. They're just 

21 grouped a little differently, so don't try and go 

22 and compare the exact number.

23 (Off mic comments.)
66

1 MS. MILLER: Jordan.

2 MR. MATSUDAIRA: Sorry. To compare 

3 the data in this way, you guys know that these data 

4 aren't comparable across the GE space in the College 

5 Scorecard. I just feel like, if the Department 

6 hasn't produced data, that you have the data that 

7 you could actually do this in a comparable way 

8 across the sectors.

9 So why don't we do that? Why don't we 

10 actually have the information that you would 

11 actually use in a rule like this, that would really 

12 inform the situation? I feel like this is not just 

13 a waste of time, it's very misleading. It's going 

14 to give us a misleading impression of the details 

15 across the sectors.

16 MS. HAY: So Jordan, let me finish my 

17 sentence, if you wouldn't mind.

18 These are the data we have. We do not 

19 have program-level data out of NSLDS for every 

20 institution and every program. We don't have it.

21 (Off mic comments.)

22 MS. HAY: So we -- Marc Jerome asked 

23 if we have any. We do have program-level data for 
67

1 the GE programs. And that's the paper that we're 

2 reprinting for you. Okay.

3 So let's talk about the scorecard 

4 paper. That's what we have, across the United 

5 States for all institutions. Okay. It is the data 

6 we have available. We don't have what I think 

7 Jordan thinks we have. So --

8 MS. BUCK: Jordan, Jordan, wait, wait 

9 Jordan.

10 (Off mic comments.)

11 MR. MATSUDAIRA: So --

12 MS. HAY: Why don't you let me walk 

13 through this.

14 MS. MILLER: Okay. Hang on a second. 

15 Let me just, as a facilitator, let's remind of 

16 protocols. One, we are coming from a caucus, so 

17 at this time, if you're speaking, you need to be 

18 in a microphone.

19 Secondly, we've waited a long time for 

20 this information, so let's let Sarah walk through 

21 it, and then we will do questions and other 

22 comments. Thank you.

23 MS. HAY: Okay. So I think, because 
68

1 we have a mix of technical and non-technical people 

2 at the table, I think it will be easier for me to 

3 talk you through it from the pictures rather than 

4 from the tables.

5 But before I do that, I want to show 

6 you that Exhibit 2 does have the numbers. So 

7 there's the number, the mean, the standard 

8 deviation, the minimum, the median and the maximum, 

9 okay, so that you can go and look at the exact, 

10 you know, sort of the numbers themselves. Okay.

11 If you turn to Page 4, this graphs the 

12 distribution of debt-to-earnings. And I want to 

13 point out, it's not both rates that we computed. 

14 It's the annual earnings rate that's shown here, 

15 okay, not the discretionary earnings rate. So the 

16 debt-to-earnings actually has two rates in it.

17 But this picture, on Page 4 in Exhibit 

18 3, shows the annual debt-to-earnings rate, okay, 

19 for the annual earnings. So -- and because this 

20 is being given out to the public, the colors picked 

21 are chosen so that people with different types of 

22 limited color vision can still see these pictures. 

23 And that's why we printed them in color for you. 
69

1 Okay. And I tested them all.

2 So the fact that public is orange is 

3 really just because I needed the contrast. Okay. 

4 So there's no intended meaning behind the colors 

5 other than we're trying to make it visible to 

6 everybody. Okay.

7 So what you can see is that we show, 

8 at the bachelor, associate and certificate level, 

9 the three, sort of, groups of institutions, public, 

10 which is orange, private, which is blue, and 

11 proprietary, which is green.

12 So what you'll see -- and these are 

13 box-and-whisker plots. And so just to talk a 

14 little bit about box-and-whisker plots, in case 

15 you don't remember from your sixth grade math 

16 lessons, okay, so box-and-whisker plots are 

17 essentially, the box contains the middle 50 percent 

18 of the data, where the vertical line inside the 

19 box is the median.

20 So that's the exact middle of the data, 

21 if you put it in order, from zero to a hundred, 

22 okay. The whiskers, or the lines that go outside 

23 of the boxes, are -- and this was done sort of in 
70

1 standard method, so it's 1-1/2 times the amount 

2 here, but it's generally considered that's where 

3 the bulk of the data sit.

4 And if they're not covered by the 

5 whiskers, then statistically they're considered 

6 outliers. Okay. And we -- for those of you who 

7 know stats, we used 1.5 times the inner quartile 

8 range, okay, which is sort of the standard for 

9 box-and-whiskers. Okay.

10 So you can -- notice along the bottom 

11 that this goes from 0 to 15 percent. Okay. That's 

12 the rate. And so you can see where, if one were 

13 to draw in 8 percent, you could see where that would 

14 be. You can tell that since it goes to 15 percent, 

15 20 percent's kind of not on the page. Okay.

16 And some of -- the most interesting 

17 thing that I see when I look at this, is that in 

18 the most part, these data overlap each other. 

19 Okay. There's a lot of overlap. There's a lot 

20 of overlap of the boxes themselves, which is the 

21 middle 50 percent of the data. There's a lot of 

22 overlap of the whiskers.

23 And the one difference that I see when 
71

1 I look at it is that the public, in the associate 

2 and certificate level, are lower. That is, the 

3 box is farther to the left. Okay.

4 Bless you.

5 And that's really the one noticeable 

6 difference that I see there that I would consider 

7 making a decision based on, from sort of a 

8 high-level look at the data. Okay.

9 (Off mic comments.)

10 MS. HAY: Yes. So the pieces in the 

11 picture that are noticeably different from all the 

12 other box-and-whiskers, are the orange ones, which 

13 are the public institutions at the associate and 

14 the certificate level.

15 So you notice, they're shifted farther 

16 left, and the boxes do not overlap. So that means 

17 the middle 50 percent of those data distributions 

18 are different than the private and proprietary 

19 sectors, at those levels. Okay.

20 So those are where -- that's the only 

21 differences that I see that I would feel comfortable 

22 saying, oh yes, maybe we should consider making 

23 a different decision for that group, from a 
72

1 data-driven perspective. Okay. I'm not trying 

2 to tell you what you should or shouldn't do, but 

3 that's what the data show me. Okay.

4 When you turn the page, to Page Number 

5 5, and you look, this is the repayment rates. Okay. 

6 Same data, different calculation. Okay. And so 

7 when you compare them, side by side, they look 

8 different. Yes. So I think that is the 

9 interesting finding here, is that the 

10 debt-to-earnings doesn't look the same as the 

11 repayment rate. And they're not really highly 

12 correlated with each other. Okay.

13 So to me, what that says is that they 

14 probably both have meaningful information there, 

15 but one isn't really replaceable by the other. 

16 Okay. Okay.

17 So again, there are some differences, 

18 but there's a lot more overlap here than in the 

19 debt-to-earnings. And the one where I see the most 

20 difference is at the bachelor's degree, where the 

21 proprietary is the lowest. But in the others, I 

22 would say there's a fair amount of overlap. Okay.

23 So this is a high level of what's in 
73

1 the scorecard data when we compute debt-to-earnings 

2 and repayment rate, using the data that we have 

3 available to us across the United States. Okay. 

4 Are there questions?

5 MS. BUCK: And let's first make sure 

6 that people ask questions, and then we'll get into 

7 comment. So first, let's take questions.

8 MS. MILLER: Okay. So Marc Jerome, 

9 Daniel, and then Jeff.

10 MR. JEROME: So first thing, thank you 

11 for doing it, and I have an appreciation of Jordan's 

12 concerns. Are you able to address this group to 

13 identify either the number or the percentage of 

14 institutions that would be at the lowest end, by 

15 sector, so this group could get a sense of it?

16 MS. HAY: I'm not sure I understand. 

17 Help me understand what you're saying.

18 MR. JEROME: In other words, the -- one 

19 of the purposes of a negotiation is to identify 

20 the worst performing. And because the data shows 

21 higher Ed as a whole, which is a whole lot of 

22 institutions, you see the bulk of institutions. 

23 But the purpose of this negotiation is to get at 
74

1 the lowest performing institutions.

2 And so, you have a sense of it, but you 

3 don't have a sense of, is it a hundred institutions 

4 above eight? Is it, you know --

5 MS. HAY: Okay.

6 MR. JEROME: -- a hundred below 20 in 

7 repayment. And that would be, I think, helpful. 

8 I'll defer to the rest of my negotiators.

9 MS. HAY: Okay. So there are some math 

10 that you can do, based off of the number of 

11 institutions and where the percentiles fall. And 

12 I haven't done it yet, so I'm not going to try and 

13 do it right in front of you, for fear that, you 

14 know, I might make a math error.

15 But I would say that when you look at 

16 -- and we're sort of specifically not calling out 

17 individual institutions, right. But -- and I'd 

18 have to do the math of where this whisker falls, 

19 at 1-1/2 times the inner quartile range, but this 

20 is where 50 percent of your data is.

21 The right-hand side of the box is where 

22 75 percent of your data is. And the stuff farther 

23 to the right, that's in that whisker, is the top 
75

1 25 percent of the data. Okay. So if you multiply 

2 25 percent by the number value in Exhibit 2, you 

3 get a sense for how many institutions are there. 

4 Okay.

5 So let's see. So for example, just, 

6 we'll walk it through one time, and I'll ask the 

7 math people to help me. Okay. So comparing 

8 Exhibit 3, let's look at the bachelor, public, okay. 

9 So there's 531 of them.

10 Ah ha, look at that, technology. And 

11 25 percent of that is 133-ish, establishments, 

12 okay, in sort of the top 25 percent that have the 

13 highest debt-to-earnings.

14 Did I say that right? Yes.

15 (Off mic comments.)

16 MS. HAY: Did I do it wrong?

17 MS. MILLER: So there's a question of 

18 if you're reading from the page or if you --

19 (Off mic comments.)

20 MS. MILLER: You're just doing the 

21 calculations --

22 MS. HAY: Yes.

23 MS. MILLER: -- right here right now? 
76

1 Okay.

2 MS. HAY: So let's see here. 

3 Actually, let me double check.

4 (Off mic discussion.)

5 MS. HAY: I actually don't have all the 

6 percentiles in front of me, and I don't want to 

7 make a mistake. So let me come back to you with 

8 that. Okay. So let me think about it. I just 

9 don't like trying to do it right on the fly. But 

10 usually, it's, what's outside of those whiskers 

11 is really the outliers. Okay.

12 But you wanted to get a sense for how 

13 many institutions were sort of with the really, 

14 really high debt-to-earnings, so where would you 

15 like me to put the cut?

16 PARTICIPANT: Two standard 

17 deviations.

18 MS. HAY: Okay.

19 MS. MILLER: So that was a request for 

20 two standard deviations.

21 MS. HAY: Okay. All right.

22 MS. MILLER: Thank you, Daniel.

23 MS. HAY: Yes.
77

1 MR. ELKINS: This is Daniel. On Page 

2 3, Exhibit 2 --

3 MS. HAY: Yes.

4 MR. ELKINS: In the associate box, 

5 specifically germane to the number within public, 

6 private and proprietary, is that representative 

7 of the aggregate total of all, or were there some 

8 that were left out? If so, if you could explain 

9 how or why some were left out.

10 MS. HAY: Okay. So my understanding 

11 is that, for example, at the associate level, the 

12 number is that there are 622 institutions that are 

13 public, 89 institutions that are private, and 151 

14 that are proprietary. And that is, it gets to an 

15 interesting point in the way we treated the data.

16 So in the scorecard data, institutions 

17 are grouped by what they primarily award. So if 

18 an institution does multiple levels, if they award 

19 25 percent associate degrees and 75 percent 

20 bachelor's degrees, then for the purposes of this 

21 analysis they're considered a bachelor 

22 institution.

23 Does that help you understand? Okay. 
78

1 You picked up a good point for clarification, so 

2 thank you.

3 MR. ELKINS: Absolutely. That 

4 completely answers the question. Thank you. 

5 MS. HAY: Okay. Yes. So these -- under the 

6 associate box, these are the institutions that 

7 award primarily associate degrees. Okay. 

8 Thanks.

9 MS. MILLER: Jeff.

10 MR. ARTHUR: You probably did not 

11 remove any closed institutions, some of the notable 

12 ones from this data. You probably just used what 

13 was there, right?

14 MS. HAY: That's a good question. 

15 I'll double-check. I would suspect that -- oh, 

16 you mean closed now?

17 MR. ARTHUR: Yes. Closed now.

18 MS. HAY: Okay. I don't know the 

19 answer to that, but I will ask. I suspect we used 

20 the data that was available in the years we were 

21 analyzing.

22 MR. ARTHUR: Yes. And then, just an 

23 observation that, so it does appear that there truly 
79

1 are some outliers at the extremes, so, that would 

2 potentially get caught up in whatever the next level 

3 is, level 2 review or --

4 MS. HAY: There are some outliers. I 

5 would agree.

6 MS. MILLER: Chad.

7 MR. MUNTZ: I got the back row seating 

8 here. Exhibit 4, repayment, just so I understand, 

9 since we're dealing with certificates and 

10 associates and bachelors, is it the case that 

11 students did not repay because they're also 

12 continuously enrolled in another level, higher 

13 level certificate?

14 MS. HAY: So our analysis didn't look 

15 at that.

16 MR. MUNTZ: Okay. So it could be 

17 possible that the associate's degree folks are 

18 enrolled at the bachelor's degree level three years 

19 after earning it, and that's why they haven't 

20 entered repayment?

21 MS. HAY: I don't know the answer to 

22 your question.

23 MR. MUNTZ: Great. Thanks.
80

1 MS. HAY: Sorry.

2 MS. MILLER: Jordan.

3 MR. MATSUDAIRA: Thanks.

4 MS. HAY: I'm just taking a few notes 

5 so I can try and get the answers for you. Okay.

6 MR. MATSUDAIRA: Thank you. It's hard 

7 to know where to start with this, but I think the 

8 starting point is that, you know, I consider this 

9 an extremely unresponsible response to the request 

10 for data that would inform the creation of this 

11 rule.

12 There is a lot of variation, as 

13 everybody knows, in the performance of different 

14 programs within an institution. When we're 

15 thinking about how many programs are going to be 

16 identified, how many programs are going to have 

17 extreme outliers, the data that are being presented 

18 here, the qualitative take-aways from this data 

19 about institution-level aggregates is not going 

20 to be informative about what's true at the program 

21 level. It's just not.

22 You can establish that by looking at 

23 the existing gainful employment data and looking 
81

1 at variation in the debt-to-earnings metrics within 

2 a particular institution. You're going to see a 

3 lot more programs outlying. You've kind of laid 

4 out the differences in the measurement in the 

5 scorecard, but I want to highlight a couple that 

6 are important here.

7 Earnings in a College Scorecard is 

8 measured for everyone, not just completers, as it 

9 is for GE. Okay. So first of all, the levels of 

10 earnings are much lower, okay, potentially much 

11 lower.

12 Now there are differences because the 

13 time horizon and so on is different. But, you know, 

14 there's a return to completing from college. Okay. 

15 That return means that earnings are higher for 

16 completers than they are for non-completers at any 

17 given time horizon. They're higher by quite a bit, 

18 10, 25 percent.

19 And that's going to lower, you know, 

20 debt-to-earnings across different sectors. And 

21 it's going to do it in a differential way if you 

22 think that completing a program might have a 

23 different effect on your earnings, depending on 
82

1 the quality of the underlying program.

2 That's exactly what we're here to talk 

3 about, is that different programs might have 

4 different quality. And because of that, looking 

5 at the data in this way is going to mask that 

6 entirely.

7 So, you know, this is just not an 

8 informative way to look at this. The kind of 

9 description of results, the presentation of the 

10 means of different variables and so on is not 

11 relevant to what we're trying to get at, which is 

12 programs that are poor-performing, that are at the 

13 tails of this distribution.

14 We want to know how many programs are 

15 failing some potential standard that we might lay 

16 out. And, you know, even in this space, this data 

17 just is not informative about that issue at the 

18 institution level, let alone at the program level, 

19 which is what we're talking about here.

20 I just want to ask, you know, you might 

21 not have the data to do this, but when we put in 

22 data requests, I want you to explain why, if it's 

23 true, why you don't have the capacity to create 
83

1 the program-level debt-to-earnings measures right 

2 now.

3 So I suspect one reason might be that 

4 if program-level enrollment reporting started in 

5 2014, then not enough time has elapsed for you to 

6 be able to, under the construction of the cohort 

7 period now, get a match to SSA earnings. Is that 

8 the holdup? Or is it simply that that process 

9 hasn't begun yet, you haven't done the match?

10 What exactly is the holdup? Even if 

11 there is a holdup on the SSA end, you ought to be 

12 able to produce the debt measures that we've been 

13 talking about at a program level.

14 There was a vast investment in data 

15 architecture at the Federal Student Aid Office. 

16 There's a Federal Student aid datacenter. It 

17 doesn't run on COBOL. The whole reason for 

18 constructing it was so that the data could be used 

19 in real time to inform policy questions. There 

20 are a lot of smart people over there who did this 

21 kind of work all the time over the last several 

22 years.

23 And so I just want to understand better 
84

1 why the Department isn't producing statistics that 

2 are relevant to the decisions that we have before 

3 us.

4 MS. HAY: I hear your concerns and I 

5 understand them. The Department has earnings data 

6 for GE programs only. The only memorandum of 

7 understanding that we have right now with the Social 

8 Security Administration is for the production work 

9 that's done for GE programs.

10 And so, for every single cohort, when 

11 we send a list of Social Security numbers to the 

12 Social Security Administration for a match, we only 

13 get to do it once, and we can never do it again 

14 for that cohort.

15 And so we have to reserve that 

16 memorandum of understanding for the work that needs 

17 to be done that we are required to do by the existing 

18 regulations for the GE programs. We do not have 

19 an MOU with Social Security Administration right 

20 now that will allow us to match, at the program 

21 level or anything outside of GE, only for the 

22 production work. Okay.

23 MR. MATSUDAIRA: I just want to 
85

1 emphasize that the Department of Education had a 

2 memorandum of understanding with the Office of Tax 

3 Analysis and the Treasury Department to conduct 

4 this kind of background work to inform ongoing 

5 policy analysis, that expired January 20th of last 

6 year. It could be renewed.

7 You know, again, there are ways around. 

8 It's not true that you can only do the match once. 

9 It's true that if you wanted to do the match at 

10 a cohort size of ten people, then that would bind 

11 you in.

12 But in the past, that kind of 

13 restriction around privacy has been negotiated by 

14 giving us some wiggle room, so for example, settling 

15 on producing test data at a higher N size for program 

16 cohorts, so that in subsequent runs, if tweaks were 

17 made to the sample and so on, then it would still 

18 allow for there to be a sufficient number of cases, 

19 differences, between the prior run and the next 

20 run, to handle that sort of problem.

21 So there are fixes. There are --

22 there's proven technology that's been exercised 

23 in the past to get around those problems. So I'd 
86

1 like to see that. I'd like to see that pursued.

2 MS. MILLER: Thank you. Daniel, then 

3 Sandy.

4 MS. HAY: So --

5 MS. MILLER: Oh, sorry.

6 MS. HAY: Just quickly. I hear your 

7 concerns. We don't have the memorandum of 

8 understanding. As you said, it expired. We gave 

9 you the data that we had available to us, that we 

10 were able to get for you, because I figured 

11 something was better than nothing. Okay.

12 And I know that, particularly in this 

13 situation, having information in front of you is 

14 helpful. And so we used what we had, that we could 

15 get to you. And I'm not able to get into an argument 

16 about whether we should or should not have had an 

17 MOU. This is what we have. And we got you what 

18 we had. Okay.

19 MR. MATSUDAIRA: Can I ask you whether 

20 the Department believes that the qualitative 

21 take-aways from this, from the data presented here 

22 would resemble the same qualitative patterns that 

23 would be manifest if the data were to be done as 
87

1 the way -- in the way that is described in the 

2 proposed rule?

3 MS. HAY: So there are differences, but 

4 this is the data we have that's available. Without 

5 having -- without seeing and doing the match, I 

6 can't quantify what the differences would or would 

7 not be, and I'm uncomfortable postulating that. 

8 We gave you what we could.

9 MS. MILLER: Daniel.

10 MR. ELKINS: Thank you so much again 

11 for this data. We're really grateful to have 

12 something. I did have one additional clarifying 

13 question for both exhibits. And I believe you 

14 possibly already explained this, so if it's 

15 redundant, I do apologize.

16 But outside of the ends of the whisker, 

17 any circle that is plotted, is that representative 

18 of a certain number of schools, or is it one school? 

19 And I do apologize if you've already answered that.

20 MS. HAY: No. It's a good question. 

21 Each dot outside of the whiskers should be one 

22 institution.

23 There are cases where -- and the picture 
88

1 is kind of tiny, so if the dot looks kind of like, 

2 oblong, there's probably two dots there. Right. 

3 But each dot is meant to be one institution.

4 MR. ELKINS: That's very helpful. Is 

5 it possible, and it might not be, to get a -- where 

6 they do look oblong, to potentially get a number 

7 associated with the amount of dots that are there?

8 MS. HAY: Sort of the number of 

9 outliers? We'll try and get that for you, but we'll 

10 probably bring it back. I know there's a lot of 

11 information here, and you guys might all 

12 appreciate, sort of, reading it tonight.

13 So I'm trying to capture your 

14 questions. We'll try and get you answers. But 

15 I want to make sure that the answers I give you 

16 are accurate.

17 MR. ELKINS: Thank you.

18 MS. HAY: Okay. Yes.

19 MS. MILLER: Sandy.

20 MS. SARGE: I also have a question. 

21 So Sarah, you're explaining in here -- do you want 

22 to make that note?

23 MS. HAY: Hang on.
89

1 MS. SARGE: Yes. Go for it.

2 MS. HAY: I got a note back from my 

3 analyst, so just a sec.

4 Okay. So the universe of institutions 

5 excludes those not operating as of, is that the 

6 1st of December? Okay, 1st of September, as of 

7 this year. Okay. Does that help answer the 

8 universe question about whether closed 

9 institutions were included? Okay. All right.

10 Sorry. Yes?

11 MS. SARGE: Okay. So you offer, in 

12 your Exhibit 2, the -- excuse me, in Exhibit 1, 

13 the differences in how the data and the methods 

14 are there. In the graphs, did you use consistency, 

15 I guess is the way to think about it, when you 

16 presented each of these?

17 So for example, the scorecard is 

18 available for everybody, right? So did you use 

19 the scorecard for everybody, the data from the 

20 scorecard for everybody? Or did you use the GE 

21 for the GE schools and so on?

22 MS. HAY: That's a good question. We 

23 used scorecard for everybody in this paper. Yes.
90

1 MS. MILLER: Jordan.

2 MR. MATSUDAIRA: I was just going to 

3 ask if you could post the spreadsheet of the data 

4 behind this, since it's all public data.

5 MS. HAY: I will ask that question.

6 MR. MATSUDAIRA: Shy of the 

7 spreadsheet, posting the Stata code that was used 

8 to generate the figures.

9 MS. HAY: I will ask that question.

10 MS. MILLER: Are there any other 

11 questions for Sarah on this paper?

12 Yes, Thelma.

13 MS. ROSS: Well actually, there --

14 Thelma. There seems to be another document that 

15 has been passed around. I think it stopped short, 

16 just now.

17 MR. MARTIN: Yes. Hold it up. It 

18 should be this one that you have there. Did any 

19 -- did everybody get one?

20 MS. ROSS: No.

21 MS. MILLER: If you need one, raise 

22 your hand.

23 MS. HAY: Do we mind sharing until the 
91

1 rest of them show up? I appreciate your patience. 

2 I apologize. I know it's not perfect. Okay.

3 (Pause.)

4 MS. MILLER: Is everyone sharing one, 

5 at least for right now? The negotiators. Okay. 

6 Sandy, can you share with -- yes, okay.

7 (Pause.)

8 MS. MILLER: Okay. Sarah, can you 

9 walk us through this? And then I think we'll take 

10 questions.

11 MS. HAY: Absolutely. So let's see. 

12 I will use my old copy, and I'm not going to steal 

13 out of Greg's binder. Okay. So mine has typos 

14 in it, so I apologize if I refer to the wrong page 

15 number or something. Sometimes changing one word 

16 bumps things over, but I think we should mostly 

17 be okay. Okay. I gave my clean one away. All 

18 right.

19 So, this report that's in front of you, 

20 it looks different than the other one. This one 

21 has a cover page. The other one doesn't. There's 

22 no meaning behind that at all. Okay.

23 This paper is at the program level, but 
92

1 only for GE programs. Okay. So these are data 

2 out of the National Student Loan Data System. They 

3 are matched to Social Security Administration data, 

4 and they are at the CIP level. Okay.

5 So they are really different, right. 

6 The data sources are different. But there are 

7 some interesting things in here. We do, in this 

8 paper, compute both the discretionary income rate, 

9 the annual income rate, and apply the logic rules 

10 to determine which programs were passing, failing, 

11 or zoned. We did not apply the transition rate, 

12 just because there's one year of data here. Okay

13 So we walk through all of that in the 

14 Methods section. It's longer than in the previous 

15 paper. And again, I'm going to cover this at a 

16 high level right now, and I'll answer questions, 

17 and I'll take questions.

18 And I know, this one in particular, it's 

19 a bit more dense. I tried very carefully to lift 

20 the statistical language to a less techie level, 

21 okay, to the best of my ability. So I hope that 

22 it's approachable, and I'll talk you through it 

23 and answer questions now.
93

1 If there are questions, even after you 

2 sort of stew on it and read it tonight, I'll try 

3 and answer those tomorrow, as well. Okay? Okay.

4 So, if we move into the results, that 

5 starts on Page 4, we start with Program and 

6 Enrollment Counts. Okay. And this is all of the 

7 2015 GE programs, regardless of whether 

8 debt-to-earnings was computed. Okay. So this 

9 includes all end values.

10 So it's relatively stable. From 2008 

11 through 2016, we've got 30-ish thousand programs, 

12 ish, right. In Enrollment, we've got 3-1/2 

13 million-ish. And there's variation, right. And 

14 the number of programs, and the count in enrollment 

15 do not peak in the same years.

16 So the program count has a maximum in 

17 2013-14, and enrollment has a maximum in 2010-2011. 

18 Okay. So enrollment started to decline before 

19 program count started to decline. Okay.

20 Turning the page to Number 5, these are 

21 the demographics that are available to us. We 

22 don't really have anything more than this. Okay. 

23 So this is off of the first FAFSA for students 
94

1 that applied for federal financial aid. Okay.

2 And the way demographics are done is, 

3 you're either female or not female. And if you're 

4 not identified as female, you could have left it 

5 blank. So it's -- I mean, it's not a hundred 

6 percent, right. You know, it's possible that there 

7 are women out there who just didn't check the box. 

8 Right.

9 So those are important notes. That's 

10 sort of the way we put the data together, and that's 

11 sort of generally the way we do that. But that's 

12 why we don't say, percent female and percent male. 

13 It's either percent female or something else. 

14 Okay. Just as a clarifier.

15 So when you look at them, there are 

16 columns for the percent Pell, the percent 0 EFC, 

17 the percent married, percent independent, and 

18 percent female.

19 Where percent Pell and percent 0 EFC 

20 could be used as indicators of low socioeconomic 

21 status, percent married and percent independent 

22 could be used as indicators of non-traditional 

23 students, that is, students who go back to school 
95

1 later in life, students who, you know, go back to 

2 school after they're retired, possibly. Right.

3 (Off mic comments.)

4 MS. MILLER: Jeff.

5 MS. BUCK: Could we hold off for 

6 questions until the end? I know that you probably 

7 have some really good ones, but let's just wait 

8 until she gets through and then ask questions.

9 MS. HAY: Okay. So this does show --

10 and remember, these are just GE programs, public, 

11 private, proprietary, at the different credential 

12 levels. Okay.

13 When you turn to Table 5, on Page 6, 

14 this is the percent of programs and the percent 

15 of enrollment by Zip Code. Okay. And so you can 

16 see where the most programs are, where the most 

17 students are. There is privacy suppression 

18 applied here, so apply common sense if there's a 

19 star or a dash, about what that probably means for 

20 enrollment values. Okay.

21 So when you look at it, you see that 

22 there are some that just have higher enrollment 

23 than others, higher percentages of enrollment.
96

1 Oh, sorry. This is actually the 

2 percent of enrollment in programs where 

3 debt-to-earnings was computed relative to all the 

4 programs.

5 So for example, if we look at the first 

6 one, CIP 01, Agriculture, Agriculture Operations 

7 and Related Sciences, it would be that when you 

8 look at the percent of programs, that 0 percent 

9 of those programs had a debt-to-earnings measure 

10 calculated for it. Okay.

11 And we rounded, so there probably are 

12 some programs there. Right. But it has been 

13 rounded to the nearest whole number. Okay. And 

14 you have to apply some common sense, because if 

15 there's enrollees for which we computed a 

16 debt-to-earnings measure, and that would be that 

17 20 percent of enrollees were in programs that had 

18 a debt-to-earnings measure computed, relative to 

19 all of the students who were enrolled in all of 

20 the agriculture CIP programs.

21 So this really is an assessment of what 

22 proportion of programs and what proportion of 

23 students at programs were at programs that had 
97

1 debt-to-earnings calculated. I know that's a 

2 little bit tricky to wrap your brains around, okay, 

3 so I'll repeat, because not everyone in here is 

4 a data scientist, right? Okay.

5 So when we did the math, the numerator 

6 was the number of programs that got 

7 debt-to-earnings. The denominator was all of the 

8 programs in that year. Okay. All right. Same 

9 for enrollment. The numerator was the number of 

10 students enrolled at the programs that had 

11 debt-to-earnings measures calculated, and the 

12 denominator was all of the students enrolled at 

13 the programs for that particular CIP. Okay.

14 So then when you look at it, you'll

15 notice that there are some interesting things. 

16 So when you go down, say for example, we'll just 

17 talk about CIP 23, 20 percent of its programs had 

18 debt-to-earnings calculated, but that covered 70 

19 percent of the students enrolled in that CIP. 

20 Okay? Okay.

21 I'll let you think about that, and if 

22 you have further questions on sort of what this 

23 means, and there may be certain CIPs that are more 
98

1 interesting to some people than to others. Okay. 

2 But we can talk through it a little bit more 

3 tomorrow, at a more granular level.

4 When you turn to Page 7 -- am I doing 

5 this right? Yes. Okay. Making sure I don't skip 

6 anything for you guys. Okay. So on Page 7, these 

7 are the count of institutions, or I should say the 

8 count of programs for which we computed 

9 debt-to-earnings, by whether they passed or were 

10 in zone or were in fail.

11 And that does mean we computed both 

12 debt-to-earnings rates, and we applied the logic 

13 rules to decide whether they were pass, zone or 

14 fail. Okay.

15 So the columns are, the number of 

16 programs, the number of passing programs, the 

17 number of zone programs, the number of failing 

18 programs. And then we do the same thing for 

19 enrollment, the number of enrollees, the number 

20 enrolled in passing programs, the number enrolled 

21 in zone programs, and the number enrolled in failing 

22 programs.

23 The rows are by public, private and 
99

1 proprietary, also by credential level. There's 

2 a lot of information in these tables. Okay.

3 We did apply privacy protection, 

4 particularly to the enrollment values, okay, that 

5 represent individual people. Okay.

6 I'll pause. Are there questions at 

7 this point?

8 MS. BUCK: So you mean questions of 

9 anything up until now. Is that what you're saying?

10 MS. HAY: Right.

11 MS. BUCK: So do you have questions 

12 about anything in the report up until this point?

13 MS. MILLER: Jennifer, Jordan and then 

14 Thelma.

15 MS. DIAMOND: So, I just had a quick 

16 question -- hold on. Sorry. I have to find the 

17 chart again. The one that had the Pell.

18 PARTICIPANT: It's on Page 5.

19 MS. DIAMOND: Thank you. So on Page 

20 5, where you had the EFC and the Pell. Pell's not 

21 -- students aren't eligible for Pell at the graduate 

22 level, so I just wasn't sure why it said percent 

23 Pell. Does that mean that the student, at some 
100

1 point, received Pell when they were an undergrad?

2 MS. HAY: So the demographic 

3 information was taken from the first time they 

4 filled out the FAFSA. And so it's meant to be an 

5 indicator of low socioeconomic status from sort 

6 of a family level. Does that make some sense? 

7 But yes, that was --

8 MS. DIAMOND: Oh yes. It just means 

9 that when you get to the doctoral level, the data 

10 is outdated, from a -- I mean, you know, we are 

11 talking many years later for that first FAFSA.

12 MS. HAY: I understand that.

13 MS. DIAMOND: Okay. Okay.

14 MS. MILLER: Jordan.

15 MR. MATSUDAIRA: Just to clarify. 

16 These data are tabulations of the 2015 

17 informational rates that have been -- or the final 

18 2015 rates that were -- that are posted on the 

19 Department's website?

20 MS. HAY: So these are tabulations.

21 MR. MATSUDAIRA: Or is this -- is there 

22 any new data here that hasn't been publicly released 

23 already?
101

1 MS. HAY: They may be parsed at a more 

2 granular level. I haven't done a comparison across 

3 the two.

4 MR. MATSUDAIRA: But that is the source 

5 of the data that's being used here. Is that 

6 correct?

7 MS. HAY: The data were taken out of 

8 the National Student Loan Data System, out of the 

9 GE tables contained within the database itself, 

10 not off of the public website.

11 MR. MATSUDAIRA: But still, for the 

12 last official run you just did that over again. 

13 Is that correct?

14 MS. HAY: So we had data at different 

15 levels, and we wanted to be able to compare 

16 information from programs that were not included 

17 in the debt-to-earnings. So that's that first 

18 chunk of the paper, where we compare counts of 

19 institutions and programs and students, for which 

20 there was a debt-to-earnings rate computed versus 

21 those for which debt-to-earnings was not computed.

22 So there are data included in here that 

23 go beyond the debt-to-earnings calculations that 
102

1 are available on the public website.

2 MS. BUCK: Does that answer your 

3 question, Jordan?

4 MR. MATSUDAIRA: It might. I'm sorry. 

5 I don't understand, but --

6 MS. BUCK: Thelma, you had your card 

7 up.

8 MS. ROSS: It was asked and answered. 

9 Thank you.

10 MR. JEROME: Thanks. Marc Jerome. 

11 So on Page 7, 3.2, did you note anywhere the number 

12 of actual programs by sector, so that we could see 

13 the number of programs for what -- and the 

14 percentage for which GE did not apply, and 

15 debt-to-earnings was not run?

16 So for example, under public 

17 undergraduate certificate, my understanding is, 

18 there might have been 40,000 in existence, but data 

19 was run on 2,000. That data would be very 

20 interesting for all sectors.

21 MS. HAY: Okay. I see what you mean. 

22 We did not parse it that way in the first chunk. 

23 The second part of the paper is subset just to 
103

1 those programs for which we computed 

2 debt-to-earnings. Okay.

3 But it does bring up a good question. 

4 For those of you who are looking for much larger 

5 tables, the appendices do include extra data, but 

6 they are not broken down by type and control. Okay. 

7 They just got kind of large to be including in 

8 the main paper. Okay. But I don't want you to 

9 miss that they're there. Okay.

10 MS. MILLER: Chad.

11 MR. MUNTZ: Couple of questions. 

12 Table 4, is it fair to interpret this data that 

13 the current GE programs are serving low income 

14 females that are not married?

15 MS. HAY: So I would say that the GE 

16 programs were serving, at a high percentage, 

17 students who at one point qualified for Pell grants. 

18 And at about 20 to 30-ish percent were married. 

19 So if you do the reverse of that, for the most 

20 part they were -- 70-ish, 50 to 70-ish percent were 

21 unmarried the first time they filled out their 

22 FAFSA.

23 MR. MUNTZ: And they're independents, 
104

1 right? Meaning that they're, been emancipated or 

2 they're no longer dependent, so they're probably 

3 older?

4 MS. HAY: Correct.

5 MR. MUNTZ: Okay.

6 MS. HAY: Yes. And -- yes. I would 

7 say a high proportion were female.

8 MR. MUNTZ: Okay. And then on Table 

9 6, there is only one program in the public sector 

10 that failed. Is that correct?

11 MS. HAY: Yes.

12 MR. MUNTZ: Okay. And looking at 

13 Table 3 -- sorry, I went back one -- you rounded 

14 to the nearest 10 percent, correct?

15 MS. HAY: Sorry. Table 3?

16 MR. MUNTZ: Yes. The -- I'm sorry, 

17 table, Page 6. I just got off here.

18 PARTICIPANT: Table 5.

19 MR. MUNTZ: Table 5. Sorry.

20 MS. HAY: Okay. Yes.

21 MR. MUNTZ: Okay. And with that, why 

22 was the rounding to 10, the nearest 10 percent 

23 instead of 5 percent?
105

1 MS. HAY: That was the request made for 

2 privacy protection.

3 MR. MUNTZ: Okay. Thank you.

4 MS. HAY: Yes.

5 MR. JEROME: Just one comment back. 

6 On the earnings, it looks like you used nine- to 

7 ten-year earnings. So I'm just giving some 

8 caution, because I'll give an example. The data 

9 shows no institution in certain areas above, let's 

10 say 10 percent -- okay, let's go --

11 So if you go to the chart on Page 4 of 

12 the small document, you -- it essentially shows 

13 only one proprietary college with debt-to-earnings 

14 based on scorecard data above 10 percent.

15 MS. MILLER: Are you back on the first 

16 document, Marc Jerome?

17 MR. JEROME: I'm back on the first.

18 MS. MILLER: Okay.

19 MR. JEROME: Because this is back to 

20 Chad's comment. So Page 4, Sarah said, if I'm 

21 correct, a dot represents one institution. So I'm 

22 just giving caution. On -- I've looked at this 

23 using six-year earnings, which is, I feel, more 
106

1 closely akin to the gainful employment.

2 And I just pulled an institution out, 

3 regardless of sector, $43,000 of debt, $25,000 of 

4 earnings. In my mind, that's closer to a 17 percent 

5 under the current GE. So I just ask you to look 

6 at that, and I guess -- I'm cautioning the group 

7 on it.

8 It's back to Jordan's point. The data, 

9 you have to -- it's a little -- you have to be a 

10 little careful how we look at the data.

11 MS. HAY: Yes. And I think you said 

12 we used a nine-year out earnings. We didn't. So 

13 we did have to make some assumptions. So with 

14 scorecard data, you build the cohort at entry, 

15 right. And so then for primarily associates, we 

16 assumed two years. Okay. And then we took 

17 earnings from three years after that.

18 Does that make sense? Right. And 

19 then similarly, for bachelors, the cohort was built 

20 when they started, when they entered. You add four 

21 years, and then you add three years. Does that 

22 help clarify?

23 But it's, it was done so that we grasped 
107

1 earnings at the point three years that we would 

2 assume past completion, okay, to try and be as 

3 comparable, given the data that we had available, 

4 to be as comparable as we could be to the way we 

5 compute GE debt-to-earnings. Okay.

6 MS. MILLER: Jennifer.

7 MS. DIAMOND: So I had a different 

8 question, but just to stay on that for a second. 

9 So you assumed a normal time to completion for 

10 bachelors of four years?

11 MS. HAY: So --

12 MS. DIAMOND: I'm just saying.

13 MS. HAY: That was the assumption that 

14 was made, because we had to make an assumption for 

15 the purposes of producing a computation.

16 MS. DIAMOND: Okay. So I think, to 

17 Marc and everybody else's points, we just need to 

18 be careful about the data, because national reality 

19 is national reality on bachelor's degrees.

20 But can I ask my second question, and 

21 it's just to help me fully understand. So going 

22 back to the bigger paper, on Page 6, I just want 

23 to use one example to -- I just want you to, if 
108

1 everybody doesn't mind, just to make sure that I 

2 understand it.

3 CIP Number -- and I know you did this 

4 on 23, but education sort of struck, because there 

5 were --

6 MS. HAY: Which CIP?

7 MS. DIAMOND: CIP 13.

8 MS. HAY: Okay. Thank you.

9 MS. DIAMOND: And of course, you're at 

10 the two -- so you're at the two-digit CIP, right, 

11 just as, so you're rolling --

12 MS. HAY: Correct.

13 MS. DIAMOND: -- a bunch into -- right. 

14 So that's -- I don't know if that point was made, 

15 but obviously that's a big roll-up. But anyway, 

16 education as a whole, can you just walk me through 

17 what that, what it says in terms of the students? 

18 Because that's only 10 percent.

19 MS. HAY: Exactly. So what that means 

20 is that for 10 percent of the programs, we computed 

21 debt-to-earnings in that CIP. So that means that 

22 90 percent of the programs had less than 30 

23 completers.
109

1 We -- that 10 percent of programs 

2 equated to 70 percent of enrollment in that CIP. 

3 So 70 percent of the students enrolled in an 

4 education CIP were at a program for which 

5 debt-to-earnings was computed at N 30.

6 MS. DIAMOND: Got it.

7 MS. HAY: Okay. Thanks.

8 MS. MILLER: Johnson.

9 MR. TYLER: Just a follow-up. That 

10 means they're -- for the education CIP, there are 

11 relatively few programs that educate a lot of 

12 people, that were evaluated?

13 MS. HAY: Yes.

14 MR. TYLER: Yes. Okay. So on the 

15 gainful employment, there is a question there. 

16 I think they're like 30 -- if we look at Table 3 

17 on Page 4, there are 32,000 programs. But when 

18 we look at, on Page 7, Table 6, only 8,600 programs 

19 are reviewed. Is that because the N number is below 

20 30 on all of these programs?

21 MS. HAY: Sorry. Say it for me one 

22 more time.

23 MR. TYLER: Sure.
110

1 MS. HAY: You're comparing Table 3 to 

2 --

3 MR. TYLER: Just look at the number of 

4 programs, the universe on Table 3. For 2016, there 

5 are 32,000 programs. And then when we go to Table 

6 6, which is scoring --

7 MS. HAY: Right.

8 MR. TYLER: -- everyone, there are only 

9 8,650 programs that were evaluated. So that means 

10 that all those --

11 MS. HAY: Correct.

12 MR. TYLER: -- all those programs have 

13 less than 30 students in them.

14 MS. HAY: Correct.

15 MR. TYLER: Okay. The ones that were 

16 excluded?

17 MS. HAY: That's correct. Yes, that 

18 had fewer than 30 completers.

19 MR. TYLER: Completers. I'm sorry. 

20 Less than 30 completers.

21 MS. HAY: I knew what you meant.

22 MR. TYLER: Okay. I didn't know what 

23 I meant, but I appreciate that.
111

1 MS. HAY: Got you. Okay.

2 MS. MILLER: Okay. Are you done, 

3 Johnson?

4 MR. TYLER: Yes.

5 MS. MILLER: Okay. Thank you. So we 

6 have Sandy, Jordan, Whitney, Jeff. Chad, are you?

7 (Off mic response.)

8 MS. MILLER: Okay. So Sandy, Jordan, 

9 Whitney, Jeff, Ahmad and then Jennifer.

10 So Sandy.

11 MS. SARGE: Okay, thank you.

12 MS. BUCK: So we want to move on as 

13 quickly as we can, because we do want to get through 

14 the rest of the --

15 MS. HAY: Report?

16 MS. BUCK: -- the report. Yes.

17 MS. SARGE: This is Sandy. On Table 

18 Number 5, at the bottom it says, Total, and then 

19 it says 20 percent and 70 percent. Is that really

20 -- can you explain what that really means?

21 MS. HAY: Sure. so that's for all 

22 CIPs.

23 MS. SARGE: So all CIPs, the whole 
112

1 universe, only 20 percent of them, which kind of 

2 correlates to what he was just saying, 8,000 out 

3 of 32,000, 25 percent, so roughly 20 percent of 

4 the programs received a D/E calculation, and 70 

5 percent of total enrolled were in those programs 

6 for which a D/E calculation was computed?

7 MS. HAY: Correct. Okay.

8 MS. MILLER: Jordan.

9 MR. MATSUDAIRA: I have a couple of 

10 questions. So the first one is, during the last 

11 Neg Reg session and in following up afterwards, 

12 I submitted a list of seven different data requests, 

13 none of which are responded to in the data that 

14 you've given us here, so I want to ask whether any 

15 of that information is going to be forthcoming.

16 Now a number of these requests don't 

17 involve any kind of link to earnings data but are 

18 still germane. For each one of the questions that 

19 I submitted, I included an explanation of the 

20 question before the committee here, that I thought 

21 the data would be relevant in making a choice.

22 A multi-part question follows. First 

23 of all, can we expect any more data to be coming 
113

1 from the Department?

2 MS. HAY: So Jordan, let me just 

3 interrupt you. Let's finish the paper we're doing. 

4 And if you have questions about your other data 

5 requests, let me -- I can address those after we 

6 finish this paper. Is that okay?

7 MR. MATSUDAIRA: Next question is 

8 about the data that we're looking at.

9 MS. HAY: Okay.

10 MR. MATSUDAIRA: Can the Department 

11 answer the question of how you would like us --

12 what question before the committee do you think 

13 each one of the tables is relevant for answering, 

14 and kind of what your takeaway from the data is?

15 And then I'm going to go ahead and 

16 insert the last question, which is, I believe the 

17 Department has an MOU for the College Scorecard 

18 with the Department of Treasury, which could be 

19 used to obtain program-level earnings. And that, 

20 together with the debt data that you have would 

21 allow you to create program-level debt-to-earnings 

22 across the full higher ed space.

23 MS. HAY: So I will ask, Jordan, but 
114

1 it's my understanding that the MOU for the College 

2 Scorecard is at the institution level, not at the 

3 program level, and therefore would require 

4 renegotiation of that memorandum of understanding.

5 Relative to which specific question 

6 each of these tables is meant to help you answer, 

7 the first fundamental question I heard really 

8 clearly last time was that you wanted to know what

9 data we had relative to GE programs and what 

10 debt-to-earnings looked like across the United 

11 States. And so this is our intention of trying 

12 to get you that information.

13 And it may help you answer a lot of 

14 different questions, but I did not -- we did not 

15 intend for -- you know, to produce a table for each 

16 single question that we wanted you to answer.

17 This is meant to help you understand 

18 what's going on in the United States and what's 

19 going on in different GE programs relative to the 

20 type of institution, or the CIP, or the level, so 

21 that you can, for yourselves, try and understand 

22 what that means, and help inform your decisions 

23 across all the different items that we're asking 
115

1 you to make decisions on. Okay.

2 MR. MATSUDAIRA: I just want to 

3 underscore, you know, with respect about the 

4 difficulty of getting these kinds of things done, 

5 you know, you knew this was coming. You can put 

6 in place the MOUs to be able to do the data runs, 

7 to help create a rule that's rational.

8 And so I just, you know, would like to 

9 see that work under way. I'd like an update on 

10 the progress. If the agreement needs to be 

11 renegotiated, it seems like the kind of thing that 

12 would be worth doing if we're really to impose a 

13 rule that covers all of higher ed.

14 MS. HAY: So Jordan, that's heard and 

15 noted. I call ELMO on that topic. Let's move on.

16 MS. MILLER: Okay.

17 MS. HAY: Other questions --

18 MS. MILLER: Yes.

19 MS. HAY: -- up to this point on the 

20 paper, relative to the paper and the data that I 

21 can help you answer from a math perspective?

22 MS. MILLER: Okay. So I'm going to --

23 before I call the next name, I do want to say that 
116

1 it is 4:36. So just keep that I mind, negotiators. 

2 Whitney, then Jeff.

3 MS. BUCK: And I think we really should 

4 take only about five minutes more for questions 

5 so we can hear the rest of the report.

6 MS. BARKLEY-DENNEY: Okay. I just 

7 have a quick question for Marc. With the data that 

8 you've referred to, do you have that in any form 

9 that could be brought to the committee so that we 

10 can see it when you're talking about it? Is that 

11 something you can pass out to the group?

12 MS. MILLER: Jeff.

13 MR. ARTHUR: Yes, just real quick. I 

14 know the first year's data that was released was 

15 based on 2014's income. And I'm sitting here 

16 looking at this and I'm wondering, is this our first 

17 look at the second year's data? Is it -- since 

18 you ran if from NSLDS in the 2015 DMYR, is it 

19 possible this is actually some data compiled for 

20 the first time we've seen it?

21 MS. HAY: That's a good question. I 

22 -- to be totally straight with you, I did not run 

23 these numbers myself, but I will double-check and 
117

1 get that answer for you.

2 MR. ARTHUR: I think the first year's 

3 might have been referred to as DMYR 14, so this 

4 may be the first time we've ever seen data on the 

5 second year's debt-to-earnings --

6 MS. HAY: Okay.

7 MR. ARTHUR: -- release. Just FYI.

8 MS. HAY: This is DMYR 15. Okay.

9 MS. MILLER: Ahmad.

10 MR. SHAWWAL: Ahmad here. Table 6, 

11 there's a column that describes failing programs. 

12 It's my understanding that 97 percent of the 

13 failing programs are at proprietary institutions. 

14 Is that correct?

15 MS. HAY: All right. Repeat your 

16 question. Sorry.

17 MR. SHAWWAL: Am I correct to 

18 understand that 97 percent of the failing programs 

19 are at proprietary institutions, based on Table 

20 6?

21 MS. HAY: Okay. Well I haven't done 

22 the math, but that would be the failing -- oh, I 

23 see what you mean. So to compute that, it would 
118

1 be 878 divided by 903.

2 Mr. Calculator Man, can you do that for 

3 me, please?

4 PARTICIPANT: I need to find my 

5 calculator on the iPhone.

6 PARTICIPANT: 97.23.

7 MR. SHAWWAL: 97.2?

8 MS. HAY: 97.23.

9 MR. SHAWWAL: Okay. Thank you.

10 MS. HAY: Okay.

11 MR. SHAWWAL: And Sarah, what would you 

12 say are the take-aways from this report for us?

13 MS. HAY: That there are some 

14 differences by CIP, and that the findings at the 

15 program level are slightly different than they are 

16 when you look at it at enrollment count, and that 

17 I would say, you know, there are, there's 

18 interesting information across the different 

19 sectors. And I want you to think about it before 

20 I come up with any conclusions, to sort of pick 

21 out one or the other.

22 So take some time to look at it. But 

23 there are some specific CIPs that do have more data 
119

1 behind them than others. Okay. I know that's not 

2 a real answer, but I want you to think about it 

3 overnight, and we can talk about it again tomorrow.

4 MR. SHAWWAL: All right.

5 MS. HAY: Okay.

6 MS. MILLER: Okay.

7 MS. HAY: All right. So I do have an 

8 answer on the 2015. They're not new data. 2015 

9 DMYR is published in 2017. That's what my analyst 

10 said.

11 MS. BUCK: Are you anticipating that 

12 there would be questions that would carry over until 

13 tomorrow?

14 MS. HAY: I think there may be. And 

15 I think, as people read it tonight, they may have 

16 other questions.

17 MS. BUCK: Then I'm wondering if you 

18 might want to just quickly cover the --

19 MS. HAY: Sure.

20 MS. BUCK: -- rest of the report.

21 MS. HAY: Absolutely.

22 MS. BUCK: People hold their questions 

23 and bring them back tomorrow, since they're going 
120

1 to be bringing additional ones, so that we do have 

2 time for public comment, which we do have to have.

3 MS. HAY: I think that's a good idea. 

4 Okay. So remind me where I left off? Were we 

5 on Page 7? Okay. So we start on Page 8, or did 

6 we do Page 8?

7 (Off mic comment.)

8 MS. HAY: Start on Page 8? Okay. 

9 Thank you. It's the sleep deprivation. I was up 

10 all night with a sick toddler, so thank you. I 

11 appreciate it. Okay.

12 So Table 7 compares mean annual 

13 earnings and annual loan payments by 

14 debt-to-earnings results for pass, zone and fail 

15 programs. So you can see that in the passing 

16 category, the mean program level annual earnings 

17 were about 8,000 more than in zone or fail, and 

18 that the annual loan payment was 1,000 to 2,000 

19 less than in zone or fail.

20 Okay, so this is in GE programs. There 

21 are differences there, as we would assume, right. 

22 That's kind of the way the metric is set up. Okay.

23 So then Figure 1 shows you distribution 
121

1 of earnings for those programs for which we computed 

2 debt-to-earnings rates. And you can see that there 

3 is still a lot of overlap. There's a lot of 

4 variation there. Just like we talked about in the 

5 other box-and-whisker plots, the boxes overlap, 

6 and the zone and the fail look a lot alike in 

7 earnings.

8 The box, the sort of the middle 50 

9 percent is larger in the passing programs, but you 

10 can see that the range of those whiskers, it covers 

11 both the zone and the fail programs. Okay.

12 Now I want to note they're just 

13 different runs of the data and the outliers were 

14 not printed on this graph. I assume there are some. 

15 Okay. Sometimes math people do that so that it 

16 fits on a page, and you can see the bulk of the 

17 data. Sometimes the outliers are really, really, 

18 really far out, and then your graph looks like this 

19 big and the outlier's up here or something. Okay.

20 Okay. Turning the page to Page Number 

21 9, Figure 2, is the same information but the 

22 distribution of annual loan payment. And you'll 

23 see that there are some much more meaningful 
122

1 differences here. But there's a lot more 

2 difference in annual loan payment than there was 

3 in earnings.

4 So the annual loan payment, there's not 

5 a whole lot of overlap in the middle 50 percent 

6 of the data, where the passing programs are the 

7 lowest annual loan payments, the zone are above 

8 that, and the failing programs are above that. 

9 There's a slight overlap there between the zone 

10 and the fail, in those, sort of the main boxes of 

11 the middle 50 percent of the data. Okay.

12 So that, to me, is a take-away from this 

13 paper, that the difference you see in the data is 

14 really more meaningfully noticeable in the annual 

15 payment than it is in the earnings. Okay.

16 Table Number 8 breaks down the passing 

17 programs by the two different debt-to-earnings 

18 rates. Okay. Because remember, there's logic 

19 that's applied to those two rates in order to 

20 categorize them. Okay.

21 So for one example, all of these 

22 programs passed, they were put in a passing 

23 category, but 5 percent of them, for example, were 
123

1 in the zone for the discretionary income rate, but 

2 passed the annual earnings rate. Twenty-eight 

3 percent of the programs that passed failed the 

4 discretionary income rate. Okay. So there are 

5 some differences between what those two rates do.

6 You'll notice that the reverse is not 

7 as true, that for the passing programs, 0 percent 

8 failed the discretionary income rate -- sorry, 0 

9 percent of them failed the annual earning rate, 

10 of those that passed the discretionary income rate. 

11 So when you think about it -- let me see if I can 

12 explain this a little bit better.

13 When you look at comparing the 

14 discretionary income rate to the annual earnings 

15 rate, or the annual -- yes, annual earnings rate, 

16 the passing programs were more likely to pass the 

17 annual earnings rate than they were to pass the 

18 discretionary income rate. So that is, the 

19 discretionary income rate was the tougher rate to 

20 pass.

21 Does that help you think about it a 

22 little bit better? Okay. So this table is just 

23 the passing programs. Okay.
124

1 When you turn the page, Table Number 

2 9 is the zone programs, so it's a different group 

3 of programs. Okay. But the finding is still the 

4 same. They were in the zone, but they're in the 

5 zone --

6 (Off mic comments.)

7 MS. HAY: Right. So they made the zone 

8 because they passed the annual earnings rate, but 

9 78 percent of them failed the discretionary income 

10 rate. Okay.

11 So those -- that's a walk-through of 

12 the data. I do, I hear the mediators and them 

13 saying we need to move on. So write down your 

14 questions tonight. I invite you to read through 

15 it. I will try and get answers to the questions 

16 that were already asked today. And I will try and 

17 make myself available tomorrow, but I know Greg 

18 has loads and loads and loads to do with you, right. 

19 And I don't want our math discussion to take over 

20 the primary work that you're here to do. Okay.

21 MS. MILLER: Okay. Thank you, Sarah. 

22 So to that point, we have a lot of work to do 

23 tomorrow. There are still tents up. Are they 
125

1 questions for Sarah or clarifying points that she's 

2 already made? Okay. So let me get Jeff, then 

3 Sandy and then Todd.

4 MR. ARTHUR: Just real fast. It's not 

5 that surprising that these differences between 

6 discretionary and debt-to-income. I mean, as we 

7 talk about, if your income's below 30,000, it's 

8 the debt-to-earnings. If it's over 30,000, it's 

9 the discretionary. It's just the way the math 

10 works. It's nothing so amazing.

11 MS. HAY: No, that's true. Not 

12 everyone understands equations as intuitively 

13 though, as everybody else.

14 MS. MILLER: Sandy.

15 MS. SARGE: Yes. Ahmad, I'd like to 

16 point something out about the numbers that you were 

17 saying. You're correct in, if you go up the column. 

18 So 878 out of 903 fail. However, if you look 

19 across the rates, and you compare those categories 

20 that are similar, so for example, undergraduate 

21 certificates, under proprietary, there was a total 

22 of 3,260, of which 196 failed, or 6 percent. I 

23 did the math.
126

1 And then you look at the privates, there 

2 was 21 out of 405, the same certificate program. 

3 So that was 5.2 percent of those within the 

4 category. And what I want to make sure that we 

5 understand, if you go up, it's obviously going to 

6 be the higher amount in the proprietary because 

7 that's the majority of programs that have to report.

8 So when you're looking at your totals, 

9 it does not surprise me that 97 percent were 

10 failing. But you really have to look at 

11 comparatively the same categories within the 

12 schools if you want to see if there's an issue. 

13 So I just wanted to make sure we looked at it. 

14 I'd be glad to go over it with you again, privately, 

15 if you want to talk about it. Thank you.

16 MS. MILLER: So Todd is going to be our 

17 last negotiator to comment up to this point, because 

18 we will have time, we will bring Sarah back. She 

19 is not leaving.

20 But I do want to -- before we do that, 

21 how many public comments do we have? One, two. 

22 MS. BUCK: So we do need to allow time 

23 for that.
127

1 MS. MILLER: Okay. So Todd, take us 

2 home on this point.

3 MR. JONES: It's a short question. 

4 Going back to Page 6, Table 5, what we're saying 

5 is, in essence, that currently of, that 70 percent 

6 of the students who are enrolled in the programs 

7 are within 20 percent of the programs?

8 MS. HAY: Exactly.

9 MR. JONES: And that the effort to 

10 decrease cell or data size from 30 to 10 is an 

11 attempt to massively increase the number from 20, 

12 to squeeze out some percentage of that last 30 

13 percent. Is that a fair analysis?

14 MS. HAY: I would say that that would 

15 be the impact of changing the N size.

16 MR. JONES: Okay. Thank you.

17 MS. BUCK: I think we really do need 

18 to allow time now for the public comment, because 

19 it is 10 of 5, and we said we would end at 5. Are 

20 there any final process points by anyone around 

21 -- by any of the negotiators before we ask the public 

22 for public comment?

23 MS. MILLER: What's the deal with 
128

1 tomorrow as far as weather and things like that? 

2 Any update? And then also, is there any thought 

3 about whether tomorrow should be a similar caucus 

4 day like we had this afternoon? Only because, to 

5 be respectful of people in the public to come all 

6 the way down here from various places only to be 

7 told to go away may not be fair. And if we make 

8 a thought about that, would it be helpful to the 

9 team?

10 MS. BUCK: Well first, let's answer the 

11 first question.

12 Greg, is there anything you want to say 

13 about the start time?

14 MR. MARTIN: Well, I don't think the 

15 weather's supposed to be a problem tomorrow. I 

16 haven't heard that it will be. So I think we can 

17 start at the proper time.

18 As far as what we have to cover, we have, 

19 as by my estimation here, 5, 6, 7 and 8. I don't 

20 know how much time we need with all of them. 

21 Obviously, with appeals, I think we would all agree 

22 -- I'm not trying to come to consensus for 

23 everybody, but I think we would all agree that if 
129

1 we reintroduce a sanction aspect, that appeals is 

2 definitely back on the table. So there might be 

3 some discussion around that, more than would have 

4 been had we just looked at that.

5 But I think if -- to finish these 

6 tomorrow, we're going to have to lay some probably 

7 strict guidelines as to the time we're going to 

8 spend on each, because I do want to cover all of 

9 them before you leave. And I don't think anybody 

10 wants to be here past when their flights leave, 

11 right. So --

12 MS. BUCK: So it doesn't sound to me 

13 like -- well, okay. Go ahead. Do you have a --

14 MS. MILLER: Well first, can I have the 

15 public comment people line up, so that we can have 

16 them ready to go, so that when it's time for them 

17 to go?

18 So those who are making public comment, 

19 your microphone is here. If you could come and 

20 line up right here.

21 MS. BUCK: Whitney, did you have a 

22 proposal?

23 MS. BARKLEY-DENNEY: Yes. I was just 
130

1 going to say, I think we should keep it open 

2 tomorrow. I think that we, you know, tried the 

3 closed thing. We got what we could out of it. 

4 And now, in the interest of public, you know, 

5 knowing what's going on, I'd like to keep it open.

6 MS. BUCK: I saw some nods around the 

7 table with regard to that.

8 MS. MILLER: Can we do a thumbs up?

9 MS. BUCK: Would you like to do thumbs 

10 up? Our thumb sides, or thumbs down for having 

11 it public tomorrow, open to the public.

12 MS. DIAMOND: I mean, just to clarify, 

13 we can call a small caucus whenever we feel like 

14 that.

15 MS. BUCK: Certainly. Certainly. 

16 You still have that option. But we'll start --

17 you're proposing to start out this way. Okay.

18 MR. MATSUDAIRA: Can I put --

19 MS. BUCK: It seems like there are no 

20 thumbs down that I can see. Okay.

21 MR. MATSUDAIRA: I want to make just 

22 a small request. So I just want to follow up about 

23 the data request, and just ask whether data are 
131

1 forthcoming or not, Greg.

2 MR. MARTIN: Jordan, I'll say this to 

3 you. You gave us data requests. We made every 

4 effort we could to get back to you the data that 

5 we had available. We have made no efforts to hide 

6 any data, or obfuscate anything. We had a lot of 

7 good people spend a lot of time putting data 

8 together.

9 I understand and concede that it wasn't 

10 everything you wanted. And I'm, you know -- but 

11 we've done everything we can. I had our data expert 

12 come up here and explain what we've done. You know, 

13 I don't think it's in the interest of the committee 

14 for me to sit here and go into --

15 MR. MATSUDAIRA: I understand that, 

16 but --

17 MR. MARTIN: -- extreme details as to 

18 why I couldn't get every little thing.

19 MR. MATSUDAIRA: It wasn't -- to be 

20 clear, it wasn't every little thing. It was only 

21 information that was already publicly available.

22 MR. MARTIN: But I'm sensing that 

23 you're characterizing it as an effort on our part 
132

1 to somehow keep something from you. And I'm a 

2 little bothered by that characterization --

3 MR. MATSUDAIRA: I don't --

4 MR. MARTIN: -- because we have not 

5 done that.

6 MS. BUCK: So there is a difference of 

7 perspective here on the data. That's pretty clear. 

8 But I'm not sure --

9 MR. MATSUDAIRA: Can -- let me just 

10 emphasize --

11 MS. BUCK: I'm not sure continuing --

12 MR. MATSUDAIRA: Please let me just 

13 emphasize why it's important. The data in the 

14 Gainful Employment Data Analysis suggests 95 

15 percent of failing programs are in the proprietary 

16 sector. That's why I, in particular, feel like 

17 the asymmetric treatment of sanctions in the rule 

18 is appropriate, because the problem seems so 

19 concentrated there.

20 Marc and others have been saying if we 

21 had data on other programs we'd see that there were 

22 actually a lot of problems in that sector as well. 

23 And if the data showed that, I'd be a lot more 
133

1 sympathetic to that position overall. And so 

2 that's the, what I see as one of the main issues 

3 of the rule.

4 And I believe the Department has the 

5 ability to create that data. So I, you know, I'd 

6 just like to, again, ask that every effort be made 

7 to produce that. I don't mean to ascribe motives 

8 to anybody. That's not my point. I just --

9 MR. MARTIN: Again, and I will 

10 reiterate that you are welcome to make any data 

11 request you want to. And I can say on behalf of 

12 the Department that we will make every effort to 

13 comply with it. If there's something else you 

14 want, or something you want to reiterate, or make 

15 a case for, for us to reconsider, whether we can 

16 do it, I'm more than willing to entertain that.

17 I can't promise anything, and I don't 

18 mean to belittle the importance of data in any way. 

19 I think if we had more data, it certainly would 

20 inform our decisions more. But I can't create it

21 out of thin air, and I can't make possible something 

22 that isn't possible.

23 MS. MILLER: Okay.
134

1 MR. MARTIN: But certainly, if you have 

2 more requests, feel free to send those to Scott.

3 MS. MILLER: Okay. Thank you, Greg.

4 So now at this time, we will have --

5 open the floor to public comment. The time is now 

6 4:55. So if we could have our first public speaker 

7 come up, say your name, and then give your comment. 

8 Thank you. 

9 MS. LUBYA: Hello. My name is Lubya 

10 (phonetic). I'm a student in higher ed, and I'm 

11 just here to read a former student's story, because 

12 I feel like it's very important for this 

13 conversation.

14 "My name is Jensy Morales (phonetic). 

15 I am 39 years old and I work two jobs to take care 

16 of my mom and myself. One job is as a greeter at 

17 Home Depot and the other is as a salesperson/cashier 

18 at The Gap. I have worked since I was 16 to help 

19 support my mom who received only a seventh grade 

20 education in the Dominican Republic before 

21 immigrating to the U.S., where I was born.

22 "Most of my work has been in stores 

23 where clothes are sold, such as Gap, where I only 
135

1 earn -- where I earn only a little more than the 

2 minimum wage. In 2004, I received in the mail a 

3 brochure that promised top pay of $35,000 to $65,000 

4 and up, and even a $100,000 annually as a medical 

5 biller.

6 "I had no idea what a medical biller 

7 did, but I was intrigued by the pay, which was much 

8 higher than what I had ever earned. So I enrolled 

9 in a eight-month certificate course at the Career 

10 Institute of Health and Technology. I paid the 

11 $6,600 tuition with Pell Grant and federal loans.

12 "Halfway through the course, my teacher 

13 left, leaving us with a clueless substitute. My 

14 course work included a 120-hour unpaid internship 

15 at a small medical coding office which contracted 

16 with outpatient doctors. My job was to help the 

17 outpatient doctor get paid for claims that were 

18 rejected by insurance companies.

19 "My medical coding internship involved 

20 little more than telephone work. I called Cigna 

21 or Blue Cross, et cetera." Sorry. "Asked why --

22 and asked them why the claim was rejected, asked 

23 for correct billing or treatment code and 
136

1 resubmitted the claim.

2 "When school ended, I could not find 

3 a job that paid well. Eventually, I went back to 

4 my boss at the internship, who hired me for $9 an 

5 hour, $18,000 annually, about the same as what I 

6 was making in retail. After 18 months of 

7 employment without better pay despite such 

8 promises, I left in search for a high-paying medical 

9 billing job that supposedly was out there.

10 "Perspective employers were 

11 unimpressed by my skills, which admittedly were 

12 minimal, given the poor quality of my instructors. 

13 Unable to find a better paying medical biller job, 

14 I returned to retail work. I also ignored my 

15 student loan debt as I felt I'd been ripped off.

16 "In 2011, the U.S. Department of 

17 Education began garnishing my Gap wages for my loans 

18 that had now doubled in size. Unbeknownst to me, 

19 during the 2000s, dissatisfied students at my 

20 school regularly sent complaints to New York 

21 regulators about the school.

22 "These complaints included false 

23 promises of hands-on instruction, bogus 
137

1 internships, large class sizes, unprepared 

2 instructors and fee gouging. In 2011, New York 

3 had received so many complaints that it closed the 

4 school. Nevertheless, the U.S. Department of 

5 Education continued to garnish my wages, and more 

6 recently, my tax returns.

7 "In 2017, I filed a Defense Against 

8 Repayment Discharge application with the 

9 Department of Education that is still pending. 

10 While I am hopeful that my discharge application 

11 will be granted and the money taken to repay the 

12 loan returned, I remain upset. I am 39 years of 

13 age and support my ailing mom with whom I live. 

14 "I tried to better my life when I was 

15 25 by studying hard for a job that I was told would 

16 increase my earnings by 75 percent or more. That 

17 sales pitch was a total lie. I didn't know that 

18 then. In fact, the median wage of a medical biller 

19 a few years after school completion is $20,000, 

20 according to the U.S. Department of Education's 

21 own data.

22 "Had I known, I would have gone to a 

23 different school and studied something else. 
138

1 Perhaps by making the right choice on school, I 

2 would be in a better financial place than I am today.

3 "I have recently learned that the U.S. 

4 Department of Education has created measurements 

5 that identify trade school programs that are shams. 

6 More particularly, the Department of Education 

7 identified in its 2017 Gainful Employment Data, 

8 75 percent medical billing coding programs that 

9 are so expensive that the borrowers will have great 

10 difficulty repaying their loans given their modest 

11 wages.

12 "But the Department of Education is 

13 planning on doing little with this data other than 

14 ask the failing school to warn their students and 

15 prospective students that the school needs 

16 improvement.

17 "New York State received almost 600 

18 pages of student complaints about my school, The 

19 Career, Inc. New York then acted on this 

20 information and closed the school.

21 "The Department of Education must do 

22 the same with gainful employment data that 

23 identifies failing programs. Otherwise, tens of 
139

1 thousands of men and women from humble origins will 

2 find themselves in my own situation, behind a cash

3 register, or in some other minimum wage, with 

4 nothing to show for their schooling but debt."

5 Thank you.

6 MS. MILLER: Thank you. Next.

7 MS. GARCIA: Good afternoon. Buenas 

8 tardes, everyone. My name is Joseline Garcia. 

9 I serve as the president of the United States 

10 Student Association. I'm also the primary student 

11 negotiator for the borrower defense rule, so I will 

12 be in your shoes next week. Fun.

13 So I am here to express my concerns over 

14 the closed session that happened earlier. As 

15 someone who is familiar with this process, I think 

16 and strongly believe that that was very 

17 inappropriate. It was disrespectful, and 

18 completely goes against the process of negotiated 

19 rulemaking.

20 There are students, in addition to 

21 other public members inside this room. This 

22 process is a public process that is what everyone 

23 who is a negotiator in gainful employment and 
140

1 borrower defense signed up for. Having a closed 

2 session completely eliminates that.

3 A lot of the injustices that take place 

4 in our society often happen, or the decisions that 

5 lead to those injustices happen behind closed 

6 doors, when the people who are most impacted are 

7 not in the room. And although we do have student 

8 negotiators at the table, it is not okay that we 

9 did not have the public in the room.

10 So I hope that tomorrow, and also for 

11 the final session, you all comply to what you all 

12 signed up for, and that is having a public meeting.

13 This room is already very inaccessible, 

14 and I appreciate that you all have allowed for 

15 livestream, which has made it more accessible, not 

16 only for students but other members of the public 

17 to understand what is going on.

18 But when we closed those doors to the 

19 public, I got numerous phone calls and messages 

20 from students saying, what is going on here? My 

21 life is going to be greatly impacted by the 

22 decisions that take place at this table and I have 

23 no idea what is going on.
141

1 So please, as negotiators and also as 

2 people who I hope to be professional and 

3 responsible, do not agree to go through that process 

4 again. Thank you.

5 MS. MILLER: Thank you, Joseline. 

6 Thank you to our public commenters. You've given 

7 us something to think about. And with that, I'll 

8 see you all in the morning. Thank you.

9 MS. BUCK: So we will see you at 9 

10 o'clock. We have a lot to do tomorrow.

11 (Whereupon, the above-entitled matter 

12 went off the record at 5:00 p.m.)

13

14

15

16