U.S. GOVERNMENT PUBLISHING OFFICE

30–540 WASHINGTON : 2018

CBO OVERSIGHT: THE ROLE OF BEHAVIORAL 

MODELING IN SCORING AND BASELINE CONSTRUCTION 

HEARING 

BEFORE THE

COMMITTEE ON THE BUDGET 

HOUSE OF REPRESENTATIVES 

ONE HUNDRED FIFTEENTH CONGRESS 

SECOND SESSION 

HEARING HELD IN WASHINGTON, D.C., FEBRUARY 27, 2018 

Serial No. 115–10 

Printed for the use of the Committee on the Budget 

( 

Available on the Internet: 

www.govinfo.gov 


BU00-A363290 with DISTILLER
(II) 

COMMITTEE ON THE BUDGET 

STEVE WOMACK, Arkansas, Chairman 

TODD ROKITA, Indiana, Vice Chairman 

DIANE BLACK, Tennessee 

MARIO DIAZ-BALART, Florida 

TOM COLE, Oklahoma 

TOM MCCLINTOCK, California 

ROB WOODALL, Georgia 

MARK SANFORD, South Carolina 

DAVE BRAT, Virginia 

GLENN GROTHMAN, Wisconsin 

GARY J. PALMER, Alabama 

BRUCE WESTERMAN, Arkansas 

JAMES B. RENACCI, Ohio 

BILL JOHNSON, Ohio 

JASON SMITH, Missouri 

JASON LEWIS, Minnesota 

JACK BERGMAN, Michigan 

JOHN J. FASO, New York 

LLOYD SMUCKER, Pennsylvania 

MATT GAETZ, Florida 

JODEY C. ARRINGTON, Texas 

A. DREW FERGUSON IV, Georgia 

JOHN A. YARMUTH, Kentucky, 

Ranking Minority Member 

BARBARA LEE, California 

MICHELLE LUJAN GRISHAM, New Mexico 

SETH MOULTON, Massachusetts 

HAKEEM S. JEFFRIES, New York 

BRIAN HIGGINS, New York 

SUZAN K. DELBENE, Washington 

DEBBIE WASSERMAN SCHULTZ, Florida 

BRENDAN F. BOYLE, Pennsylvania 

RO KHANNA, California 

PRAMILA JAYAPAL, Washington, 

Vice Ranking Minority Member 

SALUD O. CARBAJAL, California 

SHEILA JACKSON LEE, Texas 

JANICE D. SCHAKOWSKY, Illinois 

PROFESSIONAL STAFF

DAN KENIRY, Staff Director 

ELLEN BALIS, Minority Staff Director 


BU00-A363290 with DISTILLER
(III) 

CONTENTS 

Page 

Hearing held in Washington, D.C., February 27, 2018 ........................................ 1 

Hon. Steve Womack, Chairman, Committee on the Budget ......................... 1 

Prepared statement of ............................................................................... 3 

Hon. John A. Yarmuth, Ranking Member, Committee on the Budget ........ 5 

Prepared statement of ............................................................................... 6 

Keith Hall, Director, Congressional Budget Office ........................................ 8 

Prepared statement of ............................................................................... 12 

Hon. Steve Womack, Chairman, Committee on the Budget, questions 

submitted for the record ............................................................................... 50 

Answers to questions submitted for the record .............................................. 51 


BU00-A363290 with DISTILLER

BU00-A363290 with DISTILLER
(1) 

CBO OVERSIGHT: THE ROLE OF BEHAVIORAL 

MODELING IN SCORING AND BASELINE 

CONSTRUCTION 

TUESDAY, FEBRUARY 27, 2018 

HOUSE OF REPRESENTATIVES, COMMITTEE ON THE BUDGET, Washington, D.C. 

The Committee met, pursuant to call, at 10:00 a.m., in Room 

1334, Longworth House Office Building, Hon. Steven Womack 

[Chairman of the Committee] presiding. 

Present: Representatives Womack, Renacci, Johnson, Faso, 

Smucker, Lewis, Ferguson, Smith, Sanford, Bergman, Woodall, 

Arrington, Grothman, Yarmuth, Jeffries, DelBene, Schakowsky, 

Jayapal, and Khanna. 

Chairman WOMACK. This hearing will come to order. Welcome to 

the Committee on the Budget’s hearing on the Congressional Budget Office’s Role of Behavioral Modeling in Scoring and Baseline 

Construction. This is the third of five hearings that the Committee 

plans to hold this spring on oversight of the Congressional Budget 

Office. 

Today’s hearing, as we said, marks the third in our five-part series. As I have mentioned before, we are conducting these hearings 

to learn more about how CBO carries out its nonpartisan mandate 

and to consider potential areas for improvement. 

Since its formation in 1974, CBO has been a vital support agency 

to Congress providing nonpartisan budgetary analysis and directly 

assisting the House and Senate Budget Committees. While CBO 

was established by the Congressional Budget Impoundment Act of 

1974, the agency has not undergone a comprehensive review since 

then—more than 40 years ago. The demands on the agency have 

undoubtedly changed. 

CBO’s mission of supporting the Congressional budget process remains the same, but, as we have already learned, expectations of 

the agency have evolved over time. So, like our last two hearings, 

we are here to make sure the agency has everything it needs to effectively and efficiently fulfill its mission in the 21st century. 

We open the oversight series with CBO director Dr. Keith Hall 

to broadly discuss the agency’s organizational and operational 

structure including its staffing, assumptions processes, and work 

products. 

In our last hearing we began to explore some of the more technical aspects of how CBO actually crafts the impartial work products Congress relies on to make informed legislative decisions. And 


BU00-A363290 with DISTILLER
2 

today, we will focus on how CBO conducts its modeling and scores 

legislative proposals including the assumptions made and processes 

used by the agencies analysts. 

CBO uses models as a tool for analysts to estimate the cost of 

legislation, project baseline impacts, and produce long-term economic analysis. In order to estimate the effects of legislative proposals, CBO must make assumptions about the likely responses to 

new policies. There is no doubt a challenge, and I know we have 

a lot to learn today, from the experts. With that, I want to introduce our witnesses each coming with knowledge and understanding 

of CBO’s current modeling framework. 

At the witness stand today is the CBO director Dr. Keith Hall. 

A little later on Dr. Jessica Banthin, Deputy Assistant Director for 

health retirement long-term analysis. Dr. Banthin is here to share 

her expertise on how the agency practically applies its health insurance simulation model, as well as other models, in the course 

of cost estimating. And we will be joined by Dr. Jeffrey Kling Associate Director for Economic Analysis. He is here in his capacity as 

the Associate Director responsible for maintaining and enhancing 

the quality and transparency of CBO’s work. 

Before we start our discussions today, I want to remind everyone 

that our goal here is to learn and identify potential areas for improvement at CBO. CBO is vitally important to the congressional 

budgeting process. So, these discussions are not meant to be one 

sided or accusatory. Throughout this hearing series, CBO has welcomed the opportunity to discuss its work and undergo oversight. 

However, it is certainly fair to acknowledge that CBO’s scoring 

of legislative proposals has led to those scores being questioned— 

particularly related to healthcare policy. But, when it comes to recent efforts to repeal and replace Obamacare, the failure of Congress to do so is certainly not CBO’s fault. However, it is a legitimate reaction of Congress to investigate how CBO, as a congressional support agency, comes to any of its conclusions and scores 

legislative proposals, including those related to healthcare policy. 

Much of the interest in and debate surrounding CBO has been 

centered on transparency, and these hearings have already enhanced communication between the agency and Congress. They 

have also started to clarify CBO’s role in the congressional budgeting process and begun to improve understanding of how CBO 

does its work. Without question, there is genuine interest on both 

sides of the aisle to fix the broken budget process, and I want to 

remind my colleagues that CBO is a critical piece to ensure Congress has a successful in fully functioning budget process. 

Today, I look forward to hearing from our witnesses and engaging in more productive conversations with CBO. And with that I 

would like to yield to the Ranking Member, Mr. Yarmuth from the 

Commonwealth of Kentucky. 

[The prepared statement of Chairman Womack follows:] 


BU00-A363290 with DISTILLER
3 


BU00-A363290 with DISTILLERInsert offset folio 1 here 30540.001

WOMACK OPENING STATEMENT 

CBO OVERSIGHT: THE ROLE OF BEHAVIORAL MODELING IN SCORING AND 

BASELINE CONSTRUCTION 

Washington, D.C., Tuesday, February 27, 2018 

Remarks as prepared for delivery: 

Good morning, and thank you for joining us as we continue the House Budget Committee's series 

of oversight hearings with the Congressional Budget Office. 

Today's hearing marks the third in our five-part series with CEO. 

As I've mentioned before, we are conducting these hearings to learn more about how CEO carries 

out its nonpartisan mandate and to consider potential areas for improvement. 

Since its formation in 1974, CEO has been a vital support agency to Congress, providing 

nonpartisan budgetary analysis and directly assisting the House and Senate Budget Committees. 

While CEO was established by the Congressional Budget and Impoundment Act of 1974, the 

agency has not undergone a comprehensive review since then more than 40 years ago. The 

demands on the agency have undoubtedly changed. 

CEO's mission of supporting the congressional budget process remains the same, but as we've 

already learned, expectations of the agency have evolved over time. So like our last two hearings, 

we are here to make sure the agency has everything it needs to effectively and efficiently fulfill its 

mission in the 21st century. 

We opened this oversight series with CEO's director, Dr. Keith Hall, and broadly discussed the 

agency's organizational and operational structure-including its staffing, assumptions, processes, 

and work products. 

In our last hearing, we began to explore some of the more technical aspects of how CEO actually 

crafts the impartial work-products Congress relies on to make informed legislative decisions. And 

today, we'll focus in on how CBO conducts its modeling and scores legislative proposals, including 

the assumptions made and processes used by the agency's analysts. 

CEO uses models as a tool for analysts to estimate the cost of legislation, project baseline impacts, 

and produce long-term economic analysis. In order to estimate the effects of legislative proposals, 

CEO must make assumptions about the likely responses to new policies. This is no doubt a 

challenge, and I know we have a lot to learn today from the experts. With that, I want to 

introduce our witnesses, each coming with knowledge and understanding of CEO's current 

modeling framework. 

First, joining us again today is CEO's director, Dr. Keith Hall. Also with us is Dr. Jessica Banthin, 

Deputy Assistant Director for Health, Retirement, and Long-Term Analysis. Dr. Banthin is here to 

share her expertise on how the agency practically applies its Health Insurance Simulation model, 

as well as other models, in the course of cost estimating. 
4 


BU00-A363290 with DISTILLERInsert offset folio 2 here 30540.002

And finally, we welcome Dr. Jeffrey Kling, Associate Director for Economic Analysis. Dr. Kling is 

here in his capacity as the associate director responsible for maintaining and enhancing the 

quality and transparency of CBO's work. 

Before we start our discussions today. I want to remind everyone that our goal here is to learn and 

identify potential areas for improvement at CBO. CBO is vitally important to the congressional 

budgeting process, so these discussions are not meant to be one-sided or accusatory. Throughout 

this hearing series, CBO has welcomed the opportunity to discuss its work and undergo oversight. 

However, it's certainly fair to acknowledge that CBO's scoring of legislative proposals has led to 

those scores being questioned, particularly related to healthcare policy. But when it comes to 

recent efforts to repeal and replace Obamacare, the failure of Congress to do so is certainly not 

CBO's fault. 

However, it is a legitimate reaction of Congress to investigate how CBO, as a congressional 

support agency, comes to any of its conclusions and scores legislative proposals-including those 

related to healthcare policy. 

Much of the interest in and debate surrounding CBO has been centered on transparency, and 

these hearings have already enhanced communication between the agency and Congress. They 

have also started to clarify CEO's role in the congressional budgeting process and begun to 

improve understanding of how CBO does its work. 

Without question, there is genuine interest on both sides of the aisle to fix the broken budget 

process. And I want to remind my colleagues that CBO is a critical piece to ensure Congress has a 

successful and fully functioning budget process. 

Today, I look forward to hearing from our witnesses and engaging in more productive 

conversations with CBO. Thank you, and with that, I yield to the Ranking Member, Mr. Yarmuth. 
5 

Mr. YARMUTH. Thank you very much, Mr. Chairman. I would like 

to join you in welcoming our three CBO witnesses today: Director 

Hall, Dr. Banthin, and Dr. Kling. We appreciate you coming before 

the Committee to testify and answer questions about how CBO incorporates behavioral responses into its analysis. 

I anticipate that a lot of today’s discussion will center on 

healthcare, and I am glad the Committee is looking into it. I am 

not necessarily being critical of the timing of this hearing, but I do 

think it is important to point out that it would have made more 

sense to hold this hearing before voting on the majority’s bill to repeal the Affordable Care Act and gut Medicaid or before voting on 

their tax bill which repealed a key component of the ACA or before 

voting on a budget resolution that cuts $1 trillion from Medicaid 

and ends the Medicare guarantee. Nevertheless, I am pleased that 

you were here Dr. Hall, Dr. Banthin, and Dr. Kling. 

A year ago, the President said nobody knew healthcare could be 

so complicated. I am quite certain that all three of you have long 

known just how complicated it is. That is why Congress relies on 

your objective nonpartisan expertise to research model and estimate the effects of large-scale changes to our nation’s healthcare 

system. When you consider that our system represents fully onesixth of the U.S. economy, there is absolutely no way we can responsibly legislate on behalf of the American people without knowing the entire impact of our healthcare proposals. 

During the debate on the Affordable Care Act, long ago as that 

was, I called healthcare policy a giant Rubik’s Cube. You may 

think you are changing one small part of one side but that change 

unexpectedly impacts a different side or many different sides. 

Healthcare has a lot more sides then a Rubik’s Cube, which is why 

we have to rely on impartial analysis and input from unbiased experts, or we will find ourselves neck deep in unintended consequences. 

And when it comes to access to healthcare, these unintended consequences can translate into severe consequences for American 

families. For example, after rigorous modeling, CBO ultimately estimated the House-passed bill to repeal the ACA would reduce the 

number of people with health insurance by 23 million in 2026. Now 

it is still unclear to me if that was the goal of my Republican colleagues or an unintended consequence, but either way, it is critical 

information for us to have. And we must have it before we vote. 

That said, Director Hall, I am interested in hearing more about 

this estimate as well as about how CBO is updating the baseline 

to reflect actions taken by Republicans in Congress and the administration to deter ACA enrollment, destabilize the insurance markets, and prevent the law from being implemented as designed. I 

am also interested in hearing about how CBO is re-estimating proposals in the President’s 2019 budget request. 

My Democratic colleagues and I have grave concerns about many 

of his policies including massive cuts to healthcare, antipoverty 

programs, and investments in economic growth. 

Again, I thank our witnesses for coming, and I look forward to 

their testimony. And I yield back. 

[The prepared statement of Mr. Yarmuth follows:] 


BU00-A363290 with DISTILLER
6 


BU00-A363290 with DISTILLERInsert offset folio 3 here 30540.003

YARMUTH OPI<~NING STATEMENT 

CBO OVERSIGHT: THE ROLE OF BEHAVIORAL MODELING IN SCORING AND 

BASELINE CONSTRUCTION 

Washington, D.C., Tuesday, February 27, 2018 

Remarks as prepared for delivery: 

Thank you, Mr. Chairman. I would also like to join you in welcoming our three CBO witnesses 

today: Director Keith Hall; Deputy Assistant Director for Health, Retirement, and Long-Term 

Analysis Jessica Banthin; and Associate Director for Economic Analysis, Jeffrey Kling. 

We appreciate you coming before the committee to testify and answer questions about how CBO 

incorporates behavioral responses into its analyses. I anticipate that a Jot of to day's discussion will 

center on health care, and I'm glad the committee is looking into it. 

I cannot blame our new Chairman for the timing of this hearing, but I do think it's important to 

point out that it would have made more sense to hold this hearing before voting on the Majority's 

bill to repeal the Affordable Care Act and gut Medicaid. Or before voting on their tax bill which 

repealed a key component of the A CA. Or before voting on a budget resolution that cuts $1 trillion 

from Medicaid and ends the Medicare guarantee. 

Nevertheless I am a pleased you are here, Dr. Hall, Dr. Banthin, and Dr. Kling. A year ago, the 

President said, "Nobody knew health care could he so complicated." I am quite certain all three of 

you have known just how complicated it is. 

That is why Congress relies on your objective, non-partisan expertise to research, model, and 

estimate the effects oflarge-scale changes to our nation's health care system. When you consider 

that our system represents fully one sixth of the US economy, there is absolutely no way we can 

responsibly legislate on behalf of the American people without knowing the entire impact of our 

health care proposals. During the debate on the Affordable Care Act, I called health care policy a 

giant Rubik's cube. You may think you are changing one small part on one side, but that change 

unexpectedly impacts a different side or many different sides. Health care has a lot more sides 

than a Rubik's cube, which is why we have to rely on impartial analysis and inpnt from unbiased 

experts, or we will find ourselves neck deep in unintended consequences. When it comes to access 

to health care, these unintended consequences can translate into severe consequences for 

American families. 

For example, after rigorous modeling CBO ultimately estimated the House-passed bill to repeal 

the ACA would reduce the number of people with hc,alth insurance by 23 million in 2026. Now, it 

is still unclear to me if that was the goal of my Republican colleagues or an unintended 

consequence, bnt either way it's critical information for us to have- and we must have it before we 

vote. 

That said, Director Hall, I am interested in hearing more about this estimate, as well as about how 

CBO is updating the baseline to reflect aetions taken by Republicans in Congress and the 

Administration to deter ACA enrollment, destabilize the insurance markets, and prevent the law 

from being implemented as designed. 
7 


BU00-A363290 with DISTILLERInsert offset folio 4 here 30540.004

l am also interested in hearing about how CBO is re-estimating proposals in the President's 2019 

budget request. My Democratic colleagues and I have grave concerns about many of his policies 

including massive cuts to health care, anti-poverty programs, and investments in economic 

growth. 

Again, I thank our witnesses for coming and I look forward to their testimony. 
8 

Chairman WOMACK. Thank you, Mr. Yarmuth. In the interest of 

time if any other members have opening statements I would like 

to ask for unanimous consent that members submit them for the 

record. Without objection. 

Chairman WOMACK. We would now like to recognize the Director 

of the Congressional Budget Office back to the chamber here, Dr. 

Keith Hall. Dr. Hall, thank you for your time again today. Committee has received your written statement it will be made part of 

the formal hearing record. You will have 10 minutes to deliver your 

oral remarks, and that time begins right now. Sir, you are recognized. 

STATEMENT OF KEITH HALL, DIRECTOR, CONGRESSIONAL 

BUDGET OFFICE 

Mr. HALL. Chairman Womack, Ranking Member Yarmuth, and 

Members of the Committee, thank you for inviting me to testify 

this morning about how CBO does its analysis. I would like to 

make seven points. 

First, our goal is to provide budget estimates in the middle of the 

distribution of possible outcomes. All budget estimates are uncertain, but the budget process relies on estimates of specific dollar 

amounts. So those are what we produce in our baseline budget projections and in cost estimates for legislative proposals. To the extent feasible, we describe the uncertainty associated with those estimates, and we regularly compare our estimates with actual outcomes when available and to improve our estimating methods. 

Second, our estimates are derived from data and research. Baseline projections and analysis of policy proposals rely on various 

types of information depending on the program involved in the specifics of any particular proposal. When applicable, we use historical 

information about spending, revenues, and other factors affecting 

budgetary outcomes. 

We draw upon research from the professional community and 

utilize information about outcomes in analogous circumstances. We 

consult with congressional staff, staff members at relevant Federal 

agencies, and other experts with diverse views including experts 

from state governments, industry think tanks, and universities. 

And we receive input from our panel of economic advisors and 

our panel of health advisors whose members represent a variety of 

perspectives. We write about the data and research that informed 

our analysis in our description of the basis of an estimate. 

Third, our estimates incorporate behavioral reactions to proposed 

policy changes. Estimating how individuals, firms, government entities would react to a policy is a fundamental part of analyzing its 

effects. For example, our estimates account for changes in the production of various crops that would result from new farm policies, 

changes in people’s likelihood of claiming government benefits if 

policies pertaining to those benefits are altered, and changes in the 

quantity of health services that would be provided if Medicare’s 

payment rates to certain providers were adjusted. 

Fourth, for major legislation and when practicable, our cost estimates reflect additional behavior changes that would affect total 

output in the economy. Those macroeconomic changes, including 

changes in the labor supply or private investment, are incorporated 


BU00-A363290 with DISTILLER
9 

into certain cost estimates using what is sometimes termed dynamic scoring. 

Because such macroeconomic analyses are complex and time-consuming they are produced for only a small number of proposals. 

Usually when the gross budgetary effect is at least one quarter of 

1 percent of gross domestic product in any year over the next 10 

years, and when there is sufficient time to conduct the analysis. 

Fifth, our analyses cover a broad range of topics. At the end of 

2017 we had 40 people working on issues related to Medicare, Medicaid, the Affordable Care Act, prescription drugs, and other 

health-related topics; 29 working on education immigration, income 

security, labor, and retirement issues; 23 on national security 

issues; 17 on tax issues; 16 on energy and natural resource issues; 

15 on macroeconomics; nine on the overall budget outlook; and 

eight on finance and housing issues. 

Those people have specialized training to work on those topics 

and develop analyses specific to issues at hand. The analysis involved regularly used hundreds of models and other estimating 

techniques. 

For example, in its analysis of our proposal to increase the counseling people received before obtaining a mortgage, CBO used evidence about how such counseling reduces the volume of loans and 

default rates among borrowers to estimate how the proposal would 

affect the cost of loan guarantees made by the Federal Housing Administration. 

As another example, the agency estimated the effects of the ACA 

on labor supply in its economic projections. Mainly by calculating 

the effects of the law and marginal average tax rates and drawing 

upon research about changes in the labor supply resulting from 

changes in tax rates. 

Six, their estimates are produced by a team of people not by 

models. Although our analysis often use models in preparing cost 

estimates, they also use information obtained from experts data 

and research to each other in which models or other inputs to use: 

how to distill the proposed changes in law, into inputs, into those 

models, and how to combine the results of the models with their 

other available information to produce a final estimate. The general process for preparing cost estimates is followed in our analysis 

of major proposals that would affect health insurance coverage for 

people under the age of 65. 

For such proposals an especially large number of analysts and 

modeling efforts are usually involved because of the complexity of 

health insurance decisions. In those analyses CBO focuses on estimating the effects on coverage, premiums, and Federal spending, 

and the staff on the Joint Committee on Taxation estimates the tax 

related budgetary effects. 

The analyses have three main steps. First, develop an analytical 

strategy. We first review the proposal and identify the key effects 

that it would have. With an estimate issue surrounding its implementation, such as how eligibility for a subsidy would be verified, 

assessing the probable timing of the effects. 

As part of that process, we consult with outside experts and review existing evidence. Second, we model the effects of the proposal. We use several models including our health insurance simVerDate Mar 15 2010 15:15 Aug 13, 2018 Jkt 000000 PO 00000 Frm 00013 Fmt 6633 Sfmt 6633 F:\HEARINGS 2017, 2018\2.27.18 CBO ROLE OF BEHAVIORAL MODELING IN SCORING 

BU00-A363290 with DISTILLER
10 

ulation model, models of Medicaid, and JCT’s individual tax model 

to analyze the proposals effects on health insurance coverage in the 

Federal budget. 

We translate the features that proposal into changes relative to 

current law and the price and generosity of health plans and other 

in factors that would affect the decisions of all parties involved— 

states, employers, insurers, individuals, and others—and use those 

changes as inputs into modeling the proposal’s effects on health insurance coverage and premiums. We then use the results from 

those analyses as building blocks to project the proposals budgetary 

effects including effects on the cost of the Medicaid program and 

on receipts of individual income taxes. 

Third, we review and write about the estimate. At several points, 

we thoroughly review the projections for objectivity and analytical 

soundness. That rigorous process involves multiple people at different levels in CBO and JCT. When an estimate of the proposals 

total budgetary effect is nearly complete we write up their results 

along with a detailed explanation of how we arrived at them for the 

Congress. We followed these steps to prepare our November 2017 

estimate of the effects of repealing the ACA’s mandates for individual to have health insurance. 

In developing an analytical strategy, we examine data and research that might shed light on the effects, we discussed the issues 

at a meeting of CBO’s panel of health advisors and consulted other 

experts. In modeling the effects of the proposal, we estimated the 

number of people who would no longer enroll in health insurance 

as a result of the repeal, the effects and premiums, and the speed 

at which those changes would occur. And we projected how those 

changes in coverage and premiums would affect tax revenues, Medicaid spending, and other spending. 

In writing about the estimate, we also indicated that we were 

undertaking considerable work to revise our methodology and that 

the preliminary results indicated that the estimated effects on 

health insurance coverage would be smaller than previously reported. In addition to other technical and economic changes, our 

next baseline projections will incorporate estimates of those effects 

because they repeal the penalty for not having insurance is now 

part of current law. 

Seventh, we strive to make our analysis transparent, and we recently reallocated resources to make it still more so. We have released new publications this year describing our processes for producing economic forecast, budget baselines, and cost estimates. Key 

staff are making presentations to congressional staff about those 

processes. 

In the coming months, some of our specific efforts to explain how 

our models have contributed to our estimates will include the following: exploring ways to make more supporting documentation for 

the methods used in baseline projections and costly estimates publicly available; publishing detailed information about key aspects of 

our updated model for stimulating health insurance coverage, including computer code, in how an analyst used the model in preparing estimates; developing a version of our model for projecting 

spending on discretionary programs to facilitate replication of 

roughly 40 percent of CBO’s formal cost estimates. 


BU00-A363290 with DISTILLER
11 

Releasing technical documentation and computer code explaining 

how key parts of our long-term budget model work and how they 

contribute to our analyses; providing information online that enables users to examine how a large variety of changes in baseline 

economic projections can affect projections of the Federal budget; 

publishing revised estimates of how certain changes to laws governing medical malpractice would affect medical spending; explaining the reasons behind the revisions to the methodology used; documenting the model used to project how those changes to laws would 

affect medical cost; and making computer code for that model available. 

Posting a tool for examining the cost of different military force 

structures on our website; and providing computer code that generates results discussed in our working paper about our model of 

the maximum sustainable output of the economy. 

More broadly, we plan to increase the public documentation of 

our modeling efforts by publishing more slide decks, working papers, appendices, supplemental data, related spreadsheets, and 

other technical material. We look forward to getting feedback on 

the usefulness of these transparency efforts. 

I appreciate the opportunity to explain how we do our work, and 

I am happy to answer your questions. And thank you for your support and guidance as we carry out our mission to provide information to the Congress as you grapple with the difficult issues facing 

the Nation. 

[The prepared statement of Keith Hall follows:] 


BU00-A363290 with DISTILLER
12 


BU00-A363290 with DISTILLERInsert offset folio 5 here 30540.005

Testimony 

Seven Things to Know About CBO's Analyses 

Keith Hall 

Director 

Before the 

Committee on the Budget 

U.S. House of Representatives 

February 27, 2018 
13 


BU00-A363290 with DISTILLERInsert offset folio 6 here 30540.006

Chairman Womack, Ranking Member Yarmuth, and 

Members of the Committee, thank you f()r inviting me 

to testify this morning about how CBO does its analysis.1 I would like to make seven points. 

First, our goal is to provide budget estimates in the 

middle of the distribution of potential outcomes. All 

budget estimates are uncertain, but the budget process 

relies on estimates of specific dollar ~unounrs, so those arc 

what we produce in our baseline budget projections and 

in cost estimates for legislative proposals. 'TO the extent 

fca.~ihle, we describe the uncertainty associated with 

those estimates. And we regularly compare our estimates 

with acrual outcomes, when available, to improve our 

estimating methods. 2 

Second, our estimates are derived from data and 

research. Baseline projections and analyses of policy 

proposals rely on various types of information, depending on the program involved and the specifics of any 

particular proposal. When applicable, we usc historical 

information about spending, revenues, and mher fJ.ctors 

affecting budgetary outcomes; we draw upon research 

from the professional community and utilize information about outcomes in analogous circumstances; 

we consult with Congressional sraff, staff members at 

relevant federal agencies, and mher experts with diverse 

views, including experts from state governments, industry, think tanks, and universities; and we receive input 

from our Panel of Economic Advisers and our Panel of 

Health Advisers, whose members represent a variety of 

perspectives.3 We write about the data and research that 

informed our analysis in our description of rhe basis of 

an estimate. 

Third, our estimates incorporate behavioral reactions to 

proposed policy changes. Estimating how individuals, 

firms, and governmental entities would react to a policy 

is a fundamental parr of analyzing its effects. For example, our estimates account for in the production 

of various crops that would result new farm policies, changes in people's likelihood 

ernmenr benefits if policie.~ pertaining to those 

were altered, and changes in the 

services that would be provided 

rates to certain providers were adjusted. 

Fourth, for major legislation, and when practicable, 

our cost estimates reflect additional behavioral changes 

that would affect total output in the economy. 'Those 

macroeconomic changes-including changes in the labor 

supply or privare investment-are incorporated into 

certain cost estimates using what is sometimes termed 

dynamic scoring. 4 Because such macroeconomic analyses 

are complex and time-consuming, they are produced 

for only a small number of proposals-usually when the 

gross budgetary ctrect (excluding any efFect arising from 

macroeconomic changes) is at least one-quarter of 1 percent of gross domestic product in any year over the nexr 

10 years and when there is sufficient time to conduct rhe 

analysis. 

Fifth, our analyses cover a broad range of topics. At 

the end of 20 17, we had 40 people 

related to Medicare, Medicaid, the Care Act 

(ACA), prescription drugs, and other healrh-relatcd 

topics; 29 working on education, immigration, income 

security, labor, and retirement issues; 23 on national 

security issues; 17 on tax issues; 16 on energy and natural 

resource issues; 15 on macroeconomics; 9 on the overall 

4. Sec Congressional Budget Office, "Dynamk Analysis," 

and ''Does CBO Do 

'Dynamic 2018), 

www.cbO.f!.,OV/faq.<.#dynamic. 
14 


BU00-A363290 with DISTILLERInsert offset folio 7 here 30540.007

2 SEVE:\ THJ:\<;S TO K'\0\VABOlT CBO'S i\~AIYSES 

budget outlook; and 8 on finance and housing- issues.s 

Those people have specialized training to work on those 

topics and develop analyses spcdfit: to the issues at hand. 

'fl1e analyses involve usc of hundreds of models 

and other estimating For in its 

analysis of a proposal ro increase the people 

receive before obtaining a mortgage, CBO evidence 

about how such counseling reduces the volume of loans 

anJ JcEwlt rates among borrowers to estimate how the 

proposal woulJ affect the costs of loan guarantees made 

by the Federal Housing Administration. 6 As another 

example, the agency estimated the effects of the ACA on 

the labor supply in its cconom.ic projections mainly by 

calculating the efFects of the law on marginal and average 

tax rates and drawing upon research about in the 

labor supply resulting from changes in tax 

Sixth, our estimates are produced by a team of people, 

not by models. Although onr analysts often use models 

in preparing cost estimates, they also usc information 

obtained from experts, data, and research to determine 

which models or other inputs to use, how to disrill the 

proposed changes in law into inputs to those models, 

and how to combine rhe results of the models with other 

available information to produce a final estimate. 

That general process for preparing cost estimates is 

followed in our analyses of major proposals that would 

aHCct health insurance for people under age 65. 

For such proposals, an large number of analysts 

and modeling efforrs arc involved because of 

the complexity of health insurance decisions.x In those 

analyses, CBO focuses on estimating the effects on 

premiums, and federal spending, and the staff 

Committee on Taxation (JCT) estimates 

the tax-related budgetary effects. The analyses have three 

main steps: 

7 .See Edward Harri'i and Shannon Mok, How CEO I;Stimatn the 

R. 

FEBRI::\HY 2'"~. 20JR 

We first review the 

effects it would 

We then examine issues 

implementation, such as how eligibility 

would be verified, and assess the probable 

of efftcrs. As part of that process, we consult 

outside experts and review existing evidence. 

of the proposal. \\'le usc several 

mc>ctels--mcludcrtgour health insurance simulation 

and JCT's individual 

effects on 

federal budget. We 

proposal into changes, 

relative to current law, in rhe price and generosity of 

health plans and in other factors that would affect the 

decisions of all parties involvcd~srares, employers, 

insurers, individuals, and others-and usc thos(' 

changes as inputs in modeling the proposal's effects 

on health insurance coverage and pn.·miums. We then 

use the results from those analyses as building blocks 

to project the proposal's budgetary cffCcrs~including 

effects on the costs of the Medicaid program and on 

receipts of individual income raxes. 

• RelJiew and write about the esth-nate. At several points, 

we thoroughly review the for objectivity 

and analytical soundness. rigorous 

involves multiple people at ditferent 

and JCT. When an estimate of the proposal's total 

budgetary effect is nearly complete, we write up the 

results, along with a detailed explanation of how we 

arrived at them, for the Congress. 

We followed those steps to prepare our Novemhcr 2017 

estimate of the effects of repealing the ACA's mandate 

for individuals to have health insurance.'! In 

an analytical strategy, we examined data and 

that might shed light on the effects, and we discussed the 

issues at a meeting of CBO's Panel off Icalth Advisers 

and consulted orher experts. 

In modeling the effects of the proposal, we estimated the 

number of people who would no longer enroll in health 

insurance as a result of the the effects on prcmi~ 

ums, and the speed ar which changes would occur. 

And we projected how those changes in coverage and 

9. 
15 


BU00-A363290 with DISTILLERInsert offset folio 8 here 30540.009

TESTIMO:'IIY 

premiums would affect tax revenues, Medicaid spending, 

and other spending. 

In writing about that estimate, we also indicated that 

we were undertaking considerable work to revise our 

methodology and that preliminary results indicated 

that the estimated eHCcts on health insurance coverage 

would be smaller than reported previously. In addition to 

other technical and economic changes, our next baseline 

will incorporate estimates of those effects 

the repeal of the penalty for not having insurance (beginning in 2019) is now pan of current law. 

Seventh, we strive to make our analysis transparent, and 

we have recently reallocated resources to make it stilt 

more so. We have released new publications this year 

describing our for producing economic forecasts, budget and cost estimates. w Key staff are 

making presentations to Congressional staff about those 

processes. 1 1 In the coming months, some of our specific 

dFons to explain how our models have contributed to 

our cstimarcs will include the following: 

• Exploring ways co make more supporting 

documentation for rhe methods used in baseline 

projections and cost estimates publicly available; 

• Publishing detailed information about key aspects of 

our updated model for simulating health insurance 

coverage~including computer code-and about how 

analysts use the model in preparing estimates; 

SEVE\ TIII:-IGS TO K:'IIOW ,\BO\:T CBO"S A:\.\LYSES 3 

• Developing a version of our model for 

spending on discretionary programs to 

replication of roughly 40 percenr of CEO's formal 

cost estimates; 

• Releasing technical documentation and computer 

code explaining how key of our long-term 

budget model work and they contrihure to our 

analyses; 

• Providing informarion online that enables users to 

examine how a of changes in baseline 

projections of the 

• Publishing revised estimates of how certain changes 

to laws governing medical malpractice would afFect 

medical spending, explaining the reasons behind 

revisions to the methodology used, documenting 

the model used to project how those changes to laws 

would affect medical costs, and making computer 

code for that model available; 

• Posting a tool for examining the costs of different 

military force structures on our website; and 

computer code that 

in a working paper 

maximum sustainable output of the econon1y. 12 

.tvlore broadly, we plan to increase the public documentation of our modeling efforts by publishing more slide 

decks, papers, appendixes, supplemental data, 

related and other technical materiaL 

We look forward to getting feedback on the usefulness 

of these transparency eH~)rtS. I the opportunity to explain how we do our and [am happy to 

answer your questions. Thank you for your support and 

guidance as we carry om our mission to provide information to the Congress and as you grappk with the difficult 

issues facing the nation. 

12 
16 


BU00-A363290 with DISTILLERInsert offset folio 9 here 30540.008

4 SE\'El\ Tlii.\GS TO K.\0\\' ABOtT CBO'S M\ALYSP.S FEBHL:ARY 27, 2018 
17 

Chairman WOMACK. With 3 seconds to go, that is very good, Dr. 

Hall. 

Mr. HALL. I have had some practice lately. 

Chairman WOMACK. I am impressed. We appreciate your testimony this morning, and as we begin the Q-and-A, I would like to 

welcome two additional witnesses to the table. They are Jessica 

Banthin, Deputy Assistant Director for health retirement long-term 

analysis at the CBO, and Jeffery Kling, Associate Director for economic analysis, at the CBO as well. 

Chairman WOMACK. All three of our witnesses are available to 

answer the questions. 

It should be noted that the Ranking Member and I have conferred prior to the meeting, and we have both agreed to defer our 

opening questions to the end out of respect for the colleagues of 

ours on both sides of the aisle who have full schedules and have 

other things going during the course of this particular hearing. So, 

with that, I am going to recognize first up my good friend Mr. 

Renacci from Ohio. The floor is yours. 

Mr. RENACCI. Thank you, Mr. Chairman, for holding the hearing, 

and I want to thank Director Hall and his associates for the participation work at the CBO. You were tasked with the incredibly 

difficult job of forecasting outcomes of complex legislation. Oftentimes having to use various assumptions to predict the implications 

of policies that have never been tested before. 

I want to touch upon a process the CBO uses to make assumptions on human behavior in response to the enactment of new policy. Last fall, S&P rating analysis estimated that repealing the individual mandate penalty for uninsured individuals will lead to 3 

to 5 million Americans no longer having healthcare coverage. Compare this to CBO’s analysis that estimated that 13 million Americans would no longer have coverage in a report issued in November 

2017. This is quite a dramatic difference in analysis and is based 

on assumptions made by each organization. 

Dr. Hall can you first please explain the difference in assumptions made by the CBO and S&P which led to such discrepancies, 

and why you think your assumptions are dramatically different 

from institutions like the S&P? 

Mr. HALL. Well, let me start with just a broad characterization 

on the mandate. You know, our previous estimates of the effects of 

the mandate were in some respects anticipating. They were sort of 

projecting the likely impact. 

So what has happened with the ACA is we have had a significant 

increase in coverage. So the question is kind of an empirical question. That big increase in coverage, how much of it was due to the 

mandate? How much was it due to just publicity? How much of it 

was due to other things? 

So we are trying to parse out what the effects of the mandate 

were from the effects of other things, and we are beginning to get 

some data now, but we do not have very much data. And that has 

been our big challenge. It is because we have a little bit of data. 

We probably do not have enough data to come up with a solid 

statistical estimate on those numbers. That is where we have this 

sort of notion that we are probably a little high. We were probably 

a bit high in our mandate, but we were not very confident, we now 


BU00-A363290 with DISTILLER
18 

know how much to lower it over time. Dr. Banthin can probably fill 

in a little bit more if that is okay? 

Ms. BANTHIN. Good morning, Mr. Renacci. So, we published a 

slide deck in September 2017 talking about some of the issues Dr. 

Hall just mentioned. Both the financial effects of the penalty associated with the mandate as well as some of the nonfinancial effects. 

For example, people’s desire to comply with the law, possible 

changes in social norms regarding insurance coverage. Those nonfinancial effects are much harder to tease out. We are in the process of revising our methodology, and we will be discussing that in 

our forthcoming baseline report. 

Mr. RENACCI. It is an important question because, as you can 

imagine, the number is always used, especially when it comes from 

CBO. Well, you are going to take 13 million Americans off of 

healthcare coverage where another organization said three to five. 

So, it is important that we figure out where your 13 million came 

from. Especially, when many times the other side uses those numbers against the legislation. 

So, I think it is important. And I am not condemning you. I am 

just trying to make sure we understand how these differences come 

about because clearly when it is for me it has been real key. As 

a businessman for almost 30 years, I rely on information, and if 

your information is different from other organizations then, again, 

we have to figure out what the differences are. So I am hoping that 

in the future that can become more open. 

I want to shift gears to assistance that is provided to the CBO 

through the CBO’s panel of health advisors. Dr. Hall, can you talk 

about the composition of the panel of healthcare advisors and what 

role they play in assisting the CBO in producing its reports? 

Mr. HALL. Sure. Well, the idea of the panel is to have a diversity 

of views and have people on there who can offer us advice on our 

work on healthcare. In particular, they meet once a year and we 

talk about topics of interest. So this is part of our way of sort of 

magnifying our expertise. We have a number of experts on the topics but getting some outside experts on there as well is really helpful. 

We most recently use them on exactly the issue that you mentioned, on the mandate. We were thinking about changing our 

methodology, and we made it a major topic at our health panel 

meeting and got input from our health panel members. And they 

agreed that they thought our estimate was a little too high. But it 

did wind up being that helpful for us in actually figuring out how 

to change our methodology and how much to lower the estimate. 

Mr. RENACCI. Thank you. I yield back, Mr. Chairman. 

Chairman WOMACK. Mr. Jeffries, New York. 

Mr. JEFFRIES. Thank you, Mr. Chairman, and I thank the distinguished Ranking Member as well. Director Hall, the Congressional 

Budget Act created the CBO to give Congress direct access to expertise is that correct? 

Mr. HALL. Yes. 

Mr. JEFFRIES. And that was done in 1974, is that right? 

Mr. HALL. That is right. 


BU00-A363290 with DISTILLER
19 

Mr. JEFFRIES. And prior to the 1974 Act, Congress largely relied 

on the executive branch for most of its budgetary information. Is 

that right? 

Mr. HALL. Yes. 

Mr. JEFFRIES. And is it fair to say that relying on the executive 

branch to make congressional decisions largely put Congress at an 

informational disadvantage since the executive branch estimates 

were designed to enhance support for White House and administration policy? 

Mr. HALL. Actually, no, that was one of the reasons that Congress decided to create us. 

Mr. JEFFRIES. Now, for the last 40-plus years CBO has provided 

high quality information to Congress under, I believe, nine different directors appointed by the House Speaker and the Senate 

President pro tempore. Is that right? 

Mr. HALL. That is correct. 

Mr. JEFFRIES. And the CBO appoints staff based on their qualifications to do their job without any deference to ideology or political party. Is that right? 

Mr. HALL. That is correct. 

Mr. JEFFRIES. Now, the Trump administration has been very 

critical recently of the Congressional Budget Office. Is that right? 

Mr. HALL. I have heard some, yes. 

Mr. JEFFRIES. So, for example, the OMB Director, Mick 

Mulvaney, asked rhetorically, ‘‘Has the day of the CBO come and 

gone?’’ Are you familiar with that statement? 

Mr. HALL. I am. 

Mr. JEFFRIES. The Trump administration attacks have sought to 

discredit the CBO’s analysis of Republican efforts with respect to 

healthcare. Is that right? 

Mr. HALL. Yes. 

Mr. JEFFRIES. So, in July of 2017, the White House tweeted a 

video claiming that the CBO inaccurately estimates healthcare covers. Is that right? 

Mr. HALL. Yes. 

Mr. JEFFRIES. Now, before unceremoniously resigning last September, HHS Secretary Tom Price claimed, ‘‘The CBO was wrong 

when they analyzed Obamacare’s effect on cost and coverage.’’ Are 

you familiar with that statement? 

Mr. HALL. I am. 

Mr. JEFFRIES. But as House Budget Chair, prior to his ascension, 

Tom Price recommended you for your current position. Is that 

right? 

Mr. HALL. That is correct. 

Mr. JEFFRIES. And in fact, when Tom Price recommended you he 

praised you for your vast understanding of economic policies. Is 

that right? 

Mr. HALL. I believe so. 

Mr. JEFFRIES. And you were appointed as Director of the CBO 

by John Boehner, the Republican Speaker of the House of Representatives. Is that true? 

Mr. HALL. Yes. 

Mr. JEFFRIES. Now the CBO’s estimates, with respect to the 

ACA, have largely been accurate. Is that fair to say? 


BU00-A363290 with DISTILLER
20 

Mr. HALL. I think so. I think that is right. We are not perfect, 

but I believe we have done pretty well. 

Mr. JEFFRIES. So, for instance, in July of 2012, the CBO estimated that 30 million people would be uninsured in 2016. Is that 

right? 

Mr. HALL. Yes. 

Mr. JEFFRIES. And the actual number turned out to be a close 

approximation about 27 million. Is that right? 

Mr. HALL. That is correct. 

Mr. JEFFRIES. And the CBO estimated that 7 million people 

would be enrolled in the Obamacare marketplaces and receiving 

subsidies in 2014. Is that right? 

Mr. HALL. I think that is right. 

Mr. JEFFRIES. I think the actual number turned out to be about 

5 million. Is that correct? 

Mr. HALL. I think that is right. 

Mr. JEFFRIES. Now, were those estimates not more accurate than 

the estimates from many other entities including the RAND Corporation. Is that true? 

Mr. HALL. I think generally we have done very well. 

Mr. JEFFRIES. It was more accurate than an estimate from an estimate from the Lewin Group. Is that correct? 

Mr. HALL. Yes. 

Mr. JEFFRIES. It was more accurate than estimates from the Department of Health and Human Services Centers for Medicare and 

Medicaid. Is that right? 

Mr. HALL. That one is about the same. 

Mr. JEFFRIES. Okay. So, you have exceeded the estimations of almost every other entity that has analyzed the future of the Affordable Care Act. Is that correct? 

Mr. HALL. I believe we have done pretty well, yes. 

Mr. JEFFRIES. Now, is it fair to say that recent attacks on the 

CBO are part of a pattern by the Trump administration of undermining expertise in our democratic institutions? 

Mr. HALL. I would not want to comment on that, but we are certainly not above criticism and critiques on our work. 

Mr. JEFFRIES. Okay. I appreciate that. Republicans have attacked the FBI, attacked Special Counsel Bob Mueller, who has 

had a distinguished career and is a life-long Republican, attacked 

the intelligence community, attacked the Article III judicial branch 

as embedded in the United States Constitution, attacked the free 

press, so, I would just suggest Mr. Director, that you are in good 

company, and, as Jay-Z from Bedford-Stuyvesant, would say, 

‘‘Brush your shoulders off.’’ I yield back. 

Chairman WOMACK. Mr. Johnson from Ohio. 

Mr. JOHNSON. I thought this was a hearing on CBO, but I guess 

I missed something, Mr. Chairman. And I thank you for your diligence and having this series of hearings and helping us understand 

how we can, first of all, reauthorize and then reform the budget 

process and CBO’s role and responsibility in that. We have had 

some discussions, Mr. Director, with you and your staff. 

So, I have got some sort of basic questions here. Since CBO was 

founded on the principles of objectivity and transparency, why or 


BU00-A363290 with DISTILLER
21 

how would relying on outside data impact CBO’s core duties and 

functions in serving the effort to provide Congress with scoring? 

Mr. HALL. Well, we do in fact rely on outside data a fair amount. 

We look for data almost anywhere where we can find it. With keeping our eyes open, of course, of the data source and the level of objectivity and that sort of thing. 

Mr. JOHNSON. Okay, well why is there so much tension then between CBO’s access to private data and data transparency. 

Mr. HALL. Well, lots of times we buy private data from companies, and, even if we do not buy the data they are kind enough to 

let us use it, we are careful not to disclose it freely because they 

are in the process of selling the data to other people. And if they 

notice I have it, and we let it go freely then that is a problem for 

them. So, we have to worry about confidentiality. 

Mr. JOHNSON. Do you get pushback from data sources that say 

we do not want to release this data? 

Mr. HALL. Oh, absolutely. 

Mr. JOHNSON. How do you get past that hurdle when you think 

you need that data to do an accurate score? 

Mr. HALL. Well, we do not always have to disclose the details of 

the data. We do try to detail where we got the data, on the nature 

of the data. We just feel like we often cannot actually release the 

data itself. 

Mr. JOHNSON. Okay, how does CBO verify information provided 

by the various agencies through its own research? How do you 

verify that what they are telling you is accurate? 

Mr. HALL. Well, we do rely on information from the different 

agencies. We do make an effort to look at other data as well to see 

if it is consistent. Though, actually, lots of times where agencies 

will perform an analysis and, rather than take the analysis on, we 

will ask for the data and do our own analysis. So, we do make an 

effort to make sure that the data we are getting from agencies is 

accurate, and, I say, by comparing it to other sources of data sometimes. 

Mr. JOHNSON. Okay. All right. Moving on to another subject I 

know that CBO makes assumptions, has to from time to time in 

doing your scoring work. So, how does CBO make assumptions 

when the existing evidence and literature are unclear on the effects 

and the impacts of a particular legislative proposal? 

Mr. HALL. Well, the first thing we do is we try to keep in mind 

that there is uncertainty in anything that we do and when we 

make an estimate we are trying to be in the middle of the range. 

We actually, literally sometimes, will sit down and come up with 

our best estimate, and then go through the exercise of saying, 

‘‘Well, what are the chances the real number is higher? What are 

the chances it is lower? Are we in the middle of that?’’ So, we really 

try to make an effort to deal how we can with the uncertainty. 

And, you know, sometimes we can look at data that is sort of related, maybe not exact data on the issue. Maybe how individuals 

make choices on different types of healthcare. How that informs 

their choices, for example, on Medicaid or something like that. 

Mr. JOHNSON. Okay. All right. Shifting gears just a little bit, and 

I think we talked about this the last time you were before our 

Committee here, about the workload and the challenges of how 


BU00-A363290 with DISTILLER
22 

many different scores you have on your plate or scoring requests 

at any particular time. So how does a CBO analysts manage the 

challenges of making assumptions with limited time or access to information? You are trying to get through this process quickly. How 

do you manage that? 

Mr. HALL. Well, we do what we can to support them. Actually, 

we have supervisors who help them and help them work things 

out. We will host some additional resources sometimes when data 

is harder to come by. But that is a continuing challenge for us, the 

turnaround times. Sometimes they are very quick. One of the 

things managers spent a lot of time on, are sort of urging committees to let us look at legislation early so we can start putting together the data that we are going to need once the legislation becomes more formal. 

Mr. JOHNSON. Got you. Okay, Mr. Chairman, I yield back. 

Chairman WOMACK. Ms. DelBene from Washington, you are recognized. 

Ms. DELBENE. Thank you, Mr. Chairman, and thanks to all our 

witnesses for being with us today. Since the Trump administration 

took office in 2017 there have been a number of significant policy 

changes to the Affordable Care Act, including the elimination of the 

individual mandate. 

A study released by the Urban Institute just yesterday found 

that the elimination of the individual mandate penalties and the 

other policy changes: such as the withdrawal of cost-sharing reduction payments and diminished Federal investments in advertising 

and enrollment assistance during 2017 that affected the 2018 open 

enrollment period, that those together will lead to an additional 6.4 

million people uninsured in 2019 compared with prior law. 

So, the advertising budget alone for open enrollment was cut by 

90 percent. How will CBO account for these types of actions in its 

upcoming baseline projections? 

Mr. HALL. Sure. Do you want to go? 

Ms. BANTHIN. So we are watching closely all information regarding open enrollment advertising and all actions of the government 

that influence that, and we will be incorporating that into our baseline. We are also following and analyzing recently proposed regulations regarding association health plans and short-term plans. So 

we will speak to that when we publish our forthcoming baseline. 

Ms. DELBENE. So, but how do you incorporate that? You brought 

up short-term health insurance plans proposed rule to expand 

short-term health insurance plan that the administration has released. So when you look at that rule, changes and advertising 

budget, how does that affect coverage and premiums in the marketplace? How are you going to address that specifically? Those are 

two specific examples but how are you going to approach those? 

Ms. BANTHIN. So, we look at all of the changes that we believe 

will affect enrollment in the marketplace, and we discuss what impact we think they will have, we seek evidence and information 

that may help us analyze those effects, maybe through historical 

evidence. 

We talk to outside experts, and we compile all that information 

and then use our model to assess the impact on enrollment. When 

a regulation is proposed but not yet final, we follow a budget rule 


BU00-A363290 with DISTILLER
23 

where we incorporate just 50 percent of the effect until it is final 

and then we incorporate the full effect. 

Ms. DELBENE. Is that a standard that you used before, the 50 

percent of the effect? 

Ms. BANTHIN. Yes. 

Ms. DELBENE. Okay. How did that come about? 

Mr. HALL. It is just a tough way of dealing with a proposed rule 

that may not become a final rule. So, there is some probability it 

will become final, and so if you do not take it into account at all 

that is a zero chance. We do not want take 100 percent because 

they do not all turn into final rules. So, we have we generally 

viewed this as a 50/50 chance that the rule will become final and 

incorporate that in our baseline. And then, of course, once it becomes final then we incorporate the rest of it. 

Ms. DELBENE. I understand. Okay, thank you very much; I yield 

back, Mr. Chairman. 

Chairman WOMACK. Next we go to Mr. Faso of New York. 

Mr. FASO. Thank you, Mr. Chairman, and I thank you and the 

Ranking Member for letting us go ahead of you. That is very indulgent. So, I appreciate. 

Chairman WOMACK. Trying to be respectful of your time. 

Mr. FASO. Well, thank you sir. On the issue of the individual 

mandate that has been routed about throughout the Nation over 

the last number of years: is it correct that when CBO did its forecast it counted as those people who would be losing insurance those 

who were currently paying the penalty to avoid buying insurance? 

Ms. BANTHIN. So, we publish estimates that represent average 

annual coverage numbers, and we realize that in the real-world 

people may be uncovered for half a year. So, in real life some of 

the people who pay the penalty may pay a prorated penalty due to 

certain number of months uninsured, but they have been covered 

for the other months of the year. So, our estimates try to account 

for that. 

Mr. FASO. So, when you did that how many people were paying 

the penalty in the most recent year? 

Ms. BANTHIN. I do not know the number off the top of my head. 

Mr. FASO. So, in other words when you did your analysis of loss 

of insurance, you did not actually count all of the people who were 

paying the penalty as losing insurance? 

Ms. BANTHIN. That is correct. 

Mr. FASO. If the fine or penalty is repealed, you counted only a 

portion of them? 

Ms. BANTHIN. That is true, because we are presenting average 

annual statistics. So, for example, the real data from the IRS may 

show lots of people paying a small portion of the penalty. Let’s say 

a whole lot of people paid half the penalty. We would then cut that 

number in half. 

Mr. FASO. I think this was an example where frankly what came 

from CBO was a bit of a muddle. It was not really understood, I 

think, by either side in this political debate and certainly not in the 

media and among the general public. And that might be an area 

where you might seek more clarity—just a point. 

The other thing, I am aware that the budget office in Canada 

publishes models that citizens can look at. Netherlands invites 


BU00-A363290 with DISTILLER
24 

international and domestic experts to come in and critique their 

CBO equivalent office. Have you considered doing such a thing, Director Hall, as well? In terms of trying to modernize and improve 

the forecasting abilities of your office. 

Mr. HALL. Yes, actually if you sort of look at our transparency 

plan we have a few proposals that are doing just that. 

Mr. FASO. So, what specifically are you going to be doing? 

Mr. HALL. Yeah, well we have what we are calling our waterfall 

table. Which is looking at discretionary spending by category, and 

we are going to put that up so you can sort of see how our projection of discretionary spending is going to change from year to year. 

We are putting up a model on a military force structure that is sort 

of similar to that. 

I think we are also going to work towards, what I call, a rule of 

thumb interactive model. Which will be based on our budget outlook forecast where people can then change some things like productivity or labor force participation and see the effects on the 

budget once you sort of vary those things. So, we have a few things 

like those that are designed to be interactive. 

Mr. FASO. And what about in terms of outside experts to come 

in and help review and critique performance? 

Mr. HALL. Well, we certainly bring them in all the time to look 

at our models. Especially when we are developing new models, we 

go to our experts, and we certainly use our panels to talk about our 

performance and get feedback from them on how we are doing in 

various aspects, especially on the economic forecast. 

Mr. FASO. And is that something where you report to the committees in the House and Senate in terms of that process. 

Mr. HALL. I do not think we officially report, but we certainly 

could do that if there is interest. 

Mr. FASO. I think it might be useful to at least advise the committees as to who you have brought in, what the discussion was, 

and what decisions or determinations you made as a result of those 

evaluations that outside parties assist you with. 

Mr. KLING. So, there is a process for that, you know, panel economic advisors, for example. Where at the meeting there is a number of the Committee staff who are attending the panel meeting 

itself. So, they can hear in real time what the panel members are 

telling us, what the discussion is about, the economic forecast, and 

then there is also a follow-up meeting with Budget Committee staff 

where they give us information about their takeaways from the 

meeting. And we talk about what we think we learned. So, that 

process is happening at a staff level at this time. 

Mr. FASO. Great. Thank you, Mr. Chairman, I yield back. 

Chairman WOMACK. Ms. Jayapal, Washington. 

Ms. JAYAPAL. Thank you, Mr. Chairman, and thank you all so 

much for being with us. I wanted to go to the healthcare law and 

how you have incorporated changes that were made by the Republicans last year into this analysis, including rules that were just 

applied. 

So a new analysis by the Urban Institute indicates that continuing unraveling of the ACA will increase premiums by doubledigit percentages in most states next year, an estimated 18 percent 

on average. And that includes the elimination of the individual 


BU00-A363290 with DISTILLER
25 

mandate along with withdrawal of cost-sharing reduction payments 

and slashing of budgets to get the word out about the ACA and encourage enrollment. 

On top of this, pulling Americans out of the ACA and into shortterm limited duration policies would raise the uninsured in this 

country to 36.9 million next year. How are you incorporating these 

effects of the changes in both policy and law into your baseline estimates? 

Mr. HALL. Sure. This is something we routinely do with our baseline. We have a specific healthcare baseline. So, we will do exactly 

this. We take on board the changes in law and et cetera. The rules 

are a little bit trickier, as we sort of discussed you know, because 

rules are not always finalized. And it is sometimes hard to see 

that, but we will be doing our own analysis with this. We will probably look at the Urban work, but we will do our own independent 

work to see what the effects will likely to be. 

Ms. JAYAPAL. And, Director Hall, is part of the reason that these 

rules are difficult to analyze, or even some of the early projections 

around the ACA, when you assumed for example that states would 

welcome Medicaid expansion because it would help their populations in their states. That would be a logical assumption to make, 

I believe. I do not know if you agree with that or not. You certainly 

made it in your initial estimates. 

Mr. HALL. That is right. That was the issue the Supreme Court 

overruled, the part that made it a requirement for states to expand. 

Ms. JAYAPAL. So then you revised your budget estimates and you 

actually had to take into account the fact that certain states would 

not be expanding Medicaid. So, there would be fewer people in 

those states, but some of your projections underestimated the deep 

need across the country for health insurance. So, actually more 

people are enrolled in Medicaid and are pushing the projections up 

a little. Is that correct? 

Mr. HALL. Well, that is right; we did underestimate the Medicaid 

numbers a little bit. 

Ms. JAYAPAL. So, you know, what I am trying to understand is 

how much you look at behavior, the finality of a rule. So, for example, let us move to immigration in the budget and the estimates in 

the budget. DACA repeal and deportation of about 800,000 DACA 

recipients would reduce economic growth by an estimated $280 billion and cost more than $460 billion in economic output over the 

next decade. 

Is the update to the baseline that CBO is currently working on 

taking into account this economic impact if the President and the 

Republicans refused to extend DACA permanent solution for DACA 

recipients? How do you take something like that into consideration? 

Mr. HALL. Well, this is part of our annual update of the baseline 

where we, in fact, do try to take those on board. It is a little tricky 

for us when it happens off our schedule. So, we try not to make 

too many adjustments off our regular schedule for updating the 

baseline, but the changes that happen in time for us to put them 

into the baseline we definitely will take them into account. 

Ms. JAYAPAL. So what would you do in a case like that. Would 

you estimate that DACA recipients are not going to have a permaVerDate Mar 15 2010 15:15 Aug 13, 2018 Jkt 000000 PO 00000 Frm 00029 Fmt 6633 Sfmt 6633 F:\HEARINGS 2017, 2018\2.27.18 CBO ROLE OF BEHAVIORAL MODELING IN SCORING 

BU00-A363290 with DISTILLER
26 

nent solution? The deadline is March 5th. So, obviously the Supreme Court has ruled, and so that may, you know, provide us 

with a little bit more time. But the reality is at some point this 

year, those recipients are going to be deportable if there is not legislation passed. So, how do you take something like that into consideration? 

Mr. HALL. I do not know how we are going to do it exactly this 

time. Obviously, we will be finishing our baseline over the next 

month or so, and so we will have to have to look at these things 

and decide. 

Ms. JAYAPAL. Let me ask you about another extended complication or result of any proposals to deport or to limit legal migration 

to this country. At the end of 2012 the Social Security earnings 

suspense file—one of my favorite accounts because nobody knows 

anything about it, but I have studied it for years—contained $1.2 

trillion in uncredited earnings. 

So these are earnings from undocumented immigrants using Social Security numbers that are not legitimate Social Security numbers. They are still paying taxes into the Federal budget that supports Social Security for all Americans. 

If you were to deport undocumented immigrants, or if you were 

to limit legal migration, how do you estimate the enormous impact 

that that would have on the economics of our budget? And I see 

that I am out of time. So can I have 5 seconds for him to answer? 

Chairman WOMACK. I will give you 10. 

Ms. JAYAPAL. Oh, you are wonderful. Thank you. 

Mr. HALL. Well one of the best things maybe to look at is we fairly recently did an estimate of the DACA, and we took a lot of these 

things right into account explicitly in their write-up of the estimate. It will give you an idea of how we think about this, and how 

we may deal with this if it comes up. 

Ms. JAYAPAL. I will work with you afterwards just to see exactly 

that. 

Mr. HALL. Yeah, we would be happy to follow up. 

Ms. JAYAPAL. Thank you very much. I yield back. 

Chairman WOMACK. Mr. Smucker, Pennsylvania. 

Mr. SMUCKER. Thank you, Mr. Chairman. Director Hall, my congressional district, 16th district in Pennsylvania, leads the State in 

agricultural production. Local agriculture sector supports more 

than 96,000 jobs, and farmers producers in my district rely in 

many cases on a specific provisions within the farm bill to, during 

tough times, keep their operations open. 

And today it is tough times. Net farm income is that at 12-year 

low. Farmers in my area are in crisis and in some situations on the 

brink of bankruptcy. So, it is critical that Congress gets this upcoming farm bill reauthorization right, and, as you know, CBO’s 

cost estimate is an integral part of that reauthorization process. 

So I would like to understand a little bit about the assumptions 

that you make in regard to the farm bill. And particularly my question is, with ag being one of the most volatile markets that that 

we have, how does CBO formulate its models to anticipate potential catastrophic risks, whether it is weather-related or in other 

areas? And how does it anticipate how that applies to farming 

practices? 


BU00-A363290 with DISTILLER
27 

Mr. HALL. I will give a sort of a general answer and then pass 

it on. But this is one of the reasons why we have experts in so 

many different areas. We have experts in ag. We have some ag 

economists on board who spent a lot of time with these issues, and 

you talk with ag experts at the Department of Agriculture and 

elsewhere. 

So they have an understanding of these particular issues for agriculture. So, of course, our effort is to be as professional and use 

our expertise as much as possible and have these models accurately 

represent what would happen. 

Mr. KLING. So you can see on our website, with every baseline 

in the spring there is a set of tables, 20 or so, that show for different kinds of crops what we think the yields are going to be and 

the prices are going to be on average. And then, the key thing that, 

I think, you are interested in, is that sometimes it is not the average. Sometimes it is higher. Sometimes it is lower. 

So we have developed a set of modeling tools to look at history 

and see how much volatility has there been, and if the future is 

like the past then we can use that as a way to figure out what are 

the chances that prices are going to be below some threshold that 

might trigger some kind of payment. So, we are trying to keep 

track of those things through the model. 

Mr. SMUCKER. Do you have different models for different titles of 

the farm bill? 

Mr. KLING. Yes. 

Mr. SMUCKER. Do you work with any of the other Federal agencies such as the USDA or private organizations to help better assess the uncertainty in the ag markets? 

Mr. KLING. Yeah. So, we get a lot of information from them, and 

we also have an annual meeting where we have a bunch of people 

who study the agricultural sector—both private organizations, 

thinks tanks, universities, people from the executive branch—come 

in and talk about what is our outlook for the market and what are 

the key sources of uncertainty. And we try to take that information 

all on board. 

Mr. SMUCKER. Thank you. I have a little time so I am going to 

switch gears. My fellow member from New York has already asked 

some questions in regards to other agencies in other countries 

internationally that we could potentially learn from. I know, as a 

business owner, some of my best ideas were when we share best 

practices with a similar organization. So I would like to hear just 

a little bit more about whether you have spent any significant 

amount of time looking at operations in other areas. 

Mr. HALL. Sure. Well, first of all, we get lots of visits actually 

from Parliamentary Budget Offices of other countries and members 

to come and talk with us. And we have some discussions back and 

forth that way. The most formal thing we do is we participate in 

the OECD: Organization for Economic Cooperation and Development. They have actually a group of parliamentary budget or budget offices. They get together once a year and talk about common 

problems and such. So we do that. And then, there have been some 

pretty good examples, you know, Canada and the U.K. where. 

Mr. SMUCKER. Okay, and I am almost out of time, but that is one 

of the most interesting ideas in information the Committee has 


BU00-A363290 with DISTILLER
28 

provided for us about online tools that Canada has made available 

so that legislators, or even members of the public who might be interested, could actually look at specific impacts of various policy 

proposals on their own time as they are developing those policies. 

So, and I know there may not be time to answer this, but are you 

looking to improve what we have in regard to that? 

Mr. HALL. Yeah, absolutely. And I say we have a pilot on a number of things, and we are looking for feedback. When we try something, whether you find it helpful, that is great. Then we will put 

more resources into doing that kind of thing. One of our challenges 

is there is so many different ways of being transparent, and it 

takes resources to do all of them. We have to sort of make up, what 

I call business decisions, about which ones you find most useful 

that we should spend our time on and which ones probably are not 

worth our time. 

Chairman WOMACK. Mr. Khanna, California 

Mr. KHANNA. Thank you, Mr. Chairman. Director Hall, I just 

wanted to go over your background so people understand some of 

your qualifications. Tyler Cowen, who I disagree with but is one of 

the most brilliant economists in the world, said when you were appointed that you were an outstanding pick. Is that correct? 

Mr. HALL. I hope so. I am not sure I heard that. 

Mr. KHANNA. And you served President George W. Bush as a 

White House Chief Economist in his administration, correct? 

Mr. HALL. That is correct. 

Mr. KHANNA. And you were appointed by Speaker Boehner, correct? 

Mr. HALL. Yes. 

Mr. KHANNA. And before you were appointed, you wrote op-eds 

criticizing the increase in the minimum wage and philosophically 

criticizing the Affordable Care Act. Correct? 

Mr. HALL. I would not use the word criticizing, but I was pointing out some effects that I thought were not being taken into account. 

Mr. KHANNA. It would be fair on the spectrum of economists to 

say, before you took this role, you were more conservative in the 

conservative economists, correct? 

Mr. HALL. Yes. 

Mr. KHANNA. So, how do you feel when you hear the President 

or the Speaker attack the credibility of the CBO? I mean, do you 

do you think President George W. Bush would ever have done that? 

Mr. HALL. No, I found the CBO to be a just a really great place 

with very competent people who do really great objective work. So, 

I would I would disagree with those criticisms. 

Mr. KHANNA. I mean, you served as the Economic Advisor to 

President George W. Bush. Do you think he or his administration 

ever contemplated those kinds of attacks on the CBO? 

Mr. HALL. I do not know. 

Mr. KHANNA. Do you think that the attacks and the rhetoric 

against the CBO is undermining an essential part of our democracy 

in a way that for 40 years we have never seen before? 

Mr. HALL. I can say I am not happy about it. But on the positive 

side, I have been very pleased with the people who have come to 


BU00-A363290 with DISTILLER
29 

our defense. Very capable people who really understand what we 

do. So, in that respect it is not been all negative for us. 

Mr. KHANNA. I mean, because if you do come with a slightly conservative perspective, as there have been people who have come 

with a liberal perspective, would you say that you can vouch that 

the analysis of the CBO does not have a liberal bias? 

Mr. HALL. Well, I think that is correct, and let me be clear that 

we are not above criticism. If people have complaints, if they think 

we are doing something wrong and want to give us feedback that 

we can take on board, that is actually helpful. So, I do not disagree 

that we should not be criticized. 

Mr. KHANNA. But, sir, do you think there is any reason to doubt 

because of bias or ideological bias when you come out with estimates of how many people would lose insurance or gain insurance? 

Is there any reason to think that there is ideological bias behind 

that? 

Mr. HALL. No, we work we work pretty hard to keep bias out of 

it. 

Mr. KHANNA. And what would you be your recommendation, 

then, to those of us in Congress in terms of our rhetoric in attacking the CBO? 

Mr. HALL. Well, I would I would hope that that people who are 

unhappy or unclear about what we have done would let us talk to 

them. You know, we have done a lot of outreach. I think we probably need to do more outreach to where you can explain what we 

have done and why we have done things. It would probably at least 

get some of the mystery out of what we have done. So, people can 

see that there is, in fact, real analysis behind our work. 

Mr. KHANNA. Well, those are the only questions. I just want to 

make this comment that, you know, I feel for you. Because I think 

I disagree with a lot of your work before you were appointed in this 

position. I mean, I disagreed with the op-ed you wrote on minimum 

wage, the op-ed expressing skepticism of the Affordable Care Act. 

But I think it is outrageous for people to question your integrity 

or the CBO’s integrity when you have conservative economists 

questioning, invalidating your credibility—some of the leading 

economists—when you probably understand economics more than 

95 percent of the people sitting in this body. To question the integrity, your integrity, or the integrity of the body I think is an ultimate disservice to our democracy. And I really appreciate your 

withstanding those political attacks and continuing to do your job. 

Chairman WOMACK. Let’s go now to Minnesota, and the gentleman from Minnesota is recognized, Jason Lewis. 

Mr. LEWIS. Thank you, Mr. Chairman. I have to say, I do get 

somewhat of a kick out of my colleagues on the other side bemoaning attacks while immediately attacking anyone who disagrees 

with them. But be that as it may, I am not going to attack anybody 

today. 

I do want to talk a little bit about your estimates on baseline, 

though, on baseline budgeting and some of the more volatile variables that you have to assume there. For instance, as I understand 

it, when you develop a baseline budget, you assume that temporary 

provisions will continue, or the discretionary appropriations passed 

each year will continue, as is emergency funds will continue; trust 


BU00-A363290 with DISTILLER
30 

fund depletion will not be a factor in spending; mandatory programs continue unabated. 

What has been your experience when doing the baseline? What 

have been the most volatile variables that you think, in an honest, 

objective way, you have probably been wrong on? 

Mr. HALL. Well, let me first go to the first part of your question. 

All those rules that you are talking about are rules that are set by 

either law or the Budget Committee or with consultation with the 

Budget Committee. So, we are actually not really making those decisions about how to do those. Those decisions are coming from the 

Budget Committees. 

Mr. LEWIS. And we need to evaluate whether those decisions are 

prudent. And so, I am asking what variables have been the most 

problematic in assuming a baseline budget, say, for 10 years? 

Mr. KLING. I would say the most problematic variable has actually come from the economic forecast. It has been our forecast on 

interest rates. Our projections of net interest have been farther off 

than any other budget category on a percentage basis, and we are 

not at all alone in that we forecast that interest rates would return 

to closer to the historical levels. Like just about every private sector forecaster, we have been very close to the blue chip consensus 

on our interest rate forecast, but nevertheless, everybody has been 

wrong, and it has been bad for our baseline budget because our net 

interest payments have been—— 

Mr. LEWIS. Emergency spending; you assume that continues? 

Has that been an issue? 

Mr. KLING. So we do project forward emergency spending, and so 

that is a rule that, you know, Congress changes that from time to 

time. Sometimes it is higher; sometimes it is lower. That has not 

been what I would call the biggest category. 

Mr. LEWIS. Let’s talk a little bit about dynamic scoring. Again, 

our friends on the other side of the aisle think you do too much of 

it; some of us on this side think you do too little of it. When we 

are factoring that into the costs estimates, do you have a rule? How 

much do you use, and what is the rule? 

Mr. HALL. Sure. Well, the biggest thing is the legislation needs 

to be big enough to have a dynamic effect to actually affect that 

the GDP or the labor supply or something like that. So, the rule 

we use is it has got to be greater than one quarter of 1 percent of 

GDP, so that puts it at, right now, probably $40 billion or something like that. 

And then, the second thing is, to be honest, whether or not Congress has the patience to wait for us to take a couple extra weeks 

to do the dynamic aspect of legislation. That has not always happened. Because it does take some time and effort to do that. Let 

me just say, just generally, about the dynamic: dynamic analysis 

is always part of our baseline. It is always part of our analysis of 

the President’s budget. 

So dynamic elements just have always been part of what worked 

for a long time, and a big part of what we do. Dynamic scoring is 

just now introducing it to individual pieces of legislation. 

Mr. KLING. One important thing to add there is that we are following the guidelines that are set forth in the budget resolution, 

and so like the quarter percent of GDP, it is not something where 


BU00-A363290 with DISTILLER
31 

we have made a decision about that. It is that you collectively have 

made that decision, and we are implementing it. 

Mr. LEWIS. True. But, I mean, calculating GDP is also a little bit 

like calculating interest rates. If you could predict either one, we 

would probably all be doing something else for a living. I mean, I 

understand how difficult that is. 

I do want to take a microcosm of the dynamic effect, however. 

In my home state of Minnesota the Department of Health just released another report that 116,000 more Minnesotans lost their 

health insurance in the last two years. I happen to think the exchange from the ACA is not working too well. That is 349,000 now 

without health insurance. 

So let me just ask you a very simple question, Director Hall. And 

that is, in your opinion, would more or fewer people buy health insurance if, indeed, the price of, say, a catastrophic policy, what my 

parents used to call major medical only—without the mandates, 

without the wellness benefits required from the ACA—if that price 

were to drop precipitously, would more or less people buy health 

insurance? 

Mr. HALL. Well, more, yes. On the catastrophic? Yes. 

Mr. LEWIS. Yes. I mean, so those effects of the ACA had an effect 

on fewer people buying health insurance? 

Mr. HALL. Right. Now, one of the challenges we have got, right? 

Is how do you define coverage. Is it substantial enough to be coverage, or is it too small? 

Mr. LEWIS. My time is up. I thank you for your indulgence and 

the Chairman’s indulgence. 

Chairman WOMACK. Mr. Ferguson from Georgia. 

Mr. FERGUSON. Thank you, Mr. Chairman. And, Dr. Hall, I want 

to thank you again for spending time in our office and having some 

really good conversations. As I sit here, and I kind of think about 

it—I listen to the comments from my colleague from California, and 

talk about, you know, the attacks on the CBO office—you all probably feel a lot like we do sitting over here. You are involved in a 

process that has too many acceptable pathways to failure, because 

that is the way that it has been designed, and all you are doing 

is you are operating within the framework that we tell you to. 

Okay? 

So, you know, what I am more interested in is more of the processes, and I have been trying to focus on the accuracy of the CBO 

scores going out into further years. And I think before, when we 

had the discussion, we talked about how the fact that you have got 

an accuracy rate—I think you said 2 or 3 percent at the 6-year 

mark. Okay? From what you scored on day one, going to 6 years 

out, I think you are at about 3 percent, right? 

Mr. HALL. That is on the outlays. That is right. 

Mr. FERGUSON. Yes, on the outlays. So, what I want to look at 

and talk about a little bit today is to get your thoughts about, well, 

how do we get better numbers for years 7, 8, 9, and 10? Okay? Because what we are doing is we are making decisions on 10-year 

numbers, and we have absolutely no idea what the accuracy of that 

10-year number is. I mean, I think we feel like a lot of times, and 

what we are learning, is that we are, you know, closing the door, 

cutting the lights out, and throwing a dart at those out years. 


BU00-A363290 with DISTILLER
32 

So, when we say we are going to make this budget assumption, 

and it is going to score for this many, whether it is positive or negative, what kind of tools do you need? What do we need to think 

about? What conversations do we need to have about understanding what the accuracy of year 7 through 10 is? 

If we cannot do that, do we need to clarify in the CBO scoring 

that the accuracy is really good in years one and two; it is this 

number in years three and four; and it goes out. And so, by the 

end—by the time you get to years 9 and 10, you know, you can say, 

‘‘Well, we are dealing with about a 50 percent accuracy rate.’’ So, 

I will let you all take it and roll with that. 

Mr. KLING. So, we have been doing 10-year projections starting 

in 1996. So, 1996, we could look ahead to 2007, and in 1997, we 

could look ahead to 2008. Those are the big years—the recession, 

for example—and so a lot of our 10-year projections have been at 

this time where the economy was in a place where I think people 

really did not expect it to be there, and we certainly did not expect 

that in the mid-1990s. And so, using that very short history, those 

11 observations where we have the 10-year projections, to get a 

sense of what we expect our accuracy going forward is not really 

very helpful, because those conditions were very unusual. 

It is the case that over a longer period of time, if we look at our 

4-year, 5-year, 6-year projections, that they tend to be fairly stable, 

and so we do not have any reason to think that years 7, 8, or 9 

particularly worse than 4, 5, and 6 in that regard. 

Mr. FERGUSON. So, I mean, that is an interesting point that you 

can make that, but you cannot prove that those outlying years are 

just as stable. And just as you point out, it does not matter what 

the reason is; we were way the heck off in those 3 years that you 

just described. The point I am trying to make here is that we put 

so much value on that—either side—whether you like the score, or 

you hate the score, you go, and you put a lot of value on that 10- 

year number. Okay? And the point is we need to know what the 

probability of that number being right is, and you just pointed out 

a great example. 

You had no idea in 1996 and 1997 what 2007 and 2008 were 

going to look like. Okay? So, I just think we need to be more honest 

with ourselves. Can we and do we need to say, ‘‘We can give you 

really good numbers for these 5 years, but beyond that, there are 

too many variables, and we just need to make decisions based on 

that on that uncertainty in the outlying years?’’ And you know, we 

are putting that 10-year window in there because we need that 

length of time to be able to balance, to get to balance. 

So, my point is that I want you all to think about how we communicate the accuracy of those outlying years so that we can make 

better informed decisions as a Committee. 

Chairman WOMACK. Mr. Smith from Missouri. 

Mr. SMITH. Thank you, Mr. Chairman. Director, I want to make 

sure these numbers are correct. But in 2013, CBO projected 24 million people would be enrolled in the healthcare exchanges, is that 

correct? 

Mr. HALL. Yes. 

Mr. SMITH. And in fact, how many in 2017 were enrolled? 

Mr. HALL. Do you have this? 


BU00-A363290 with DISTILLER
33 

Mr. SMITH. More like 10. So, that is a big miscalculation from 24 

million to around 10 million. Do you know why there was such a 

miscalculation there? 

Ms. BANTHIN. As we stated earlier, we did pretty well in forecasting 2014 and 2015 subsidies. We overestimated subsidies in 

2016, and we have since revised down our estimates. Some of the 

reasons that we overestimated subsidies in 2016 have to do 

with—— 

Mr. SMITH. We are talking 2017. 

Ms. BANTHIN. Right. 

Mr. SMITH. So, you are only talking about 2016. 

Ms. BANTHIN. Okay, so, 2017, but we are referring to our earlier 

estimate, say, from 2013. We did not anticipate the fact that employers would continue to offer as much coverage as they have. 

They are continuing to offer—— 

Mr. SMITH. Why did you not anticipate that number? 

Ms. BANTHIN. We assumed employers would either stop offering 

or new firms might decide not to offer coverage so that their employees could take advantage of subsidies in the marketplace. 

Mr. SMITH. But why did you assume that? 

Ms. BANTHIN. Because the subsidies were available to low-income 

employees. 

Mr. SMITH. And so, why did people not take advantage of that? 

Ms. BANTHIN. We believe that, given the uncertainty surrounding 

the marketplace and also the nature of the coverage in the marketplace, employers made the decision to continue to offer typical employer policies to their workers. 

Mr. SMITH. So, did you not consider that as a factor in making 

these evaluations early on in 2013? 

Ms. BANTHIN. We have learned from that experience, and—— 

Mr. SMITH. But that was a reasonable expectation, and I am 

wondering why, when this report was done in 2013, that that reasonable outcome was not part of the factor? 

Mr. HALL. So, in 2012 we published a report about employer response, and at the time there was a great deal of uncertainty, and 

there was some evidence that it could be very small, as it turned 

out to be. But there was also some evidence that pointed to it 

might be very, very large. And so, we were trying to be in the middle of the distribution of those potential outcomes, and we had a 

kind of moderate employer response. 

It did turn out to be smaller, but at the time in 2012, looking 

at the debate at that point, there was also reason to be concerned 

that it could have been very large, and so that was one of those 

cases where we did—— 

Mr. SMITH. So, you erred on the side of extremely large. 

Mr. HALL.—the kind of analysis that we talked about before, 

which was trying to—— 

Mr. SMITH. I have got a minute-45. I want to ask some more 

questions. 

Mr. HALL. Sure. 

Mr. SMITH. Director, when Mr. Lewis asked you a question, you 

made a statement that caught my interest, and you said, ‘‘The definition of coverage is what matters,’’ and maybe that is why you did 


BU00-A363290 with DISTILLER
34 

not go into the definition of how many people would have ‘‘coverage’’ under catastrophic plans. 

In regards to evaluating the healthcare bill that was passed out 

of the House and we discussed, would coverage have been defined 

under CBO if we would have put in the language of the bill that 

anyone with some kind of healthcare coverage—if we defined it, 

then would you have considered HSA involvement or catastrophic 

plans as coverage? If we would have defined it in the law? 

Ms. BANTHIN. Yes. 

Mr. HALL. Yes. 

Mr. SMITH. And so, if we would have defined it in the law, under 

your review and expectations we probably would have had an increase of people having access to healthcare rather than the decrease because of the individual mandate. 

Ms. BANTHIN. I am not sure that would have been the outcome—— 

Mr. SMITH. But you have never figured that. 

Ms. BANTHIN. Our definition of coverage is very broad. 

Mr. SMITH. Have you ever figured that? So, what is the definition 

of coverage that you use? 

Ms. BANTHIN. It is just insurance that covers serious illness and 

a range of services. 

Mr. SMITH. So a catastrophic plan is not considered coverage? 

Ms. BANTHIN. Yes, it is. 

Mr. SMITH. So what was the number that you felt, under the 

healthcare bill that was passed, would increase coverage under a 

catastrophic plan? 

Ms. BANTHIN. Catastrophic plans under the proposal would 

not—— 

Mr. SMITH. The healthcare bill that you evaluated. 

Ms. BANTHIN. The EHB would still have been in effect, so those 

plans—— 

Mr. SMITH. So it would not have been considered coverage? 

Ms. BANTHIN. They would not have been available for sale. 

Mr. SMITH. So it would not have been considered coverage? 

Ms. BANTHIN. There would be no policies like that to purchase. 

Mr. SMITH. But there were policies within the healthcare bill 

that you evaluated and gave a CBO score that we passed out of the 

house. And you did not consider coverage for that? 

Ms. BANTHIN. We did not have to make that determination. People were either covered through current law or not. 

Mr. SMITH. Okay. I wish I had more time. But thank you. 

Mr. HALL. Well, let me just say, the short answer is if it had included catastrophic coverage, that would have counted. We are just 

saying that the legislation we evaluated did not have policies with 

just catastrophic coverage. And we can follow it up but—— 

Mr. SMITH. Yeah, we will follow it up, because it did. 

Chairman WOMACK. Welcome back, Ms. Schakowsky. You are 

recognized for 5 minutes. 

Ms. SCHAKOWSKY. Thank you, Mr. Chairman. In the March 13, 

2017 score for American Healthcare Act, the CBO estimated the effects of defunding Planned Parenthood. ‘‘To the extent that there 

would be reduction in access to care under the legislation, they 

would affect services that help women avert pregnancies. The peoVerDate Mar 15 2010 15:15 Aug 13, 2018 Jkt 000000 PO 00000 Frm 00038 Fmt 6633 Sfmt 6633 F:\HEARINGS 2017, 2018\2.27.18 CBO ROLE OF BEHAVIORAL MODELING IN SCORING 

BU00-A363290 with DISTILLER
35 

ple most likely to experience reduced access to care would probably 

reside in areas without other healthcare clinics or medical practitioners who serve low-income populations. CBO projects that about 

15 percent of those people would lose access to coverage.’’ So, on 

what basis did the CBO estimate the 15 percent of those people 

would lose coverage? Dr. Kling? 

Mr. KLING. So, that estimate is based on CBO’s analysis of data 

on the locations and capacity of providers that specialize in offering 

family planning services, as compiled by the Health Resources and 

Services Administration. 

So some areas of the country would lack other providers who 

serve low-income populations; other areas of the country would 

have those providers, but they would have limited capacity to serve 

all of Planned Parenthood’s clients, and a change in providers 

would affect access in instances where those providers do not offer 

the same breadth of family planning services as those offered by 

providers who specialize in family planning services, such long-acting and reversible contraceptives, same-day appointments, on-site 

refills, and confidentiality for minors. 

Ms. SCHAKOWSKY. Thank you. CBO also looked at the effect that 

loss of access to reproductive health care would have on births. 

‘‘CBO estimates that the additional births stemming from the reduced access under the legislation would add to Federal spending 

for Medicaid. In addition, some of those children would themselves 

qualify for Medicaid and possibly for other Federal programs. 

‘‘By CBO’s estimate, in the 1-year period in which Federal funds 

for Planned Parenthood would be prohibited under the legislation, 

the number of births in the Medicaid program would increase by 

several thousand, increasing direct spending from Medicaid by to 

$21 million in 2017 and by $77 million over the 2017 to 2026 period.’’ And on what basis did CBO determine that kind of reduction 

would mean more births? 

Mr. KLING. So, CBO used data on the family planning services 

available from different types of health care providers and relied on 

research identifying the relative effectiveness of those services in 

preventing an unintended pregnancy. We projected that less effective services would be available under the proposal on average, resulting in additional births that would increase Medicaid spending. 

Ms. SCHAKOWSKY. Thank you. Did you want to say something? 

Mr. HALL. No. 

Ms. SCHAKOWSKY. So, the former Chairman of this Committee, 

Diane Black, contested the CBO estimate during our budget markup last year. She said, ‘‘It is unreasonable for CBO to assume that 

women are incapable of finding another provider. Women are 

smarter than this.’’ So, when calculating the estimate, did the CBO 

assume that women are not smart enough to find another provider, 

or was its estimate based on some evidence that the closures of 

health providers in medically underserved areas could reduce access to birth control and thus lead to more births? 

Mr. KLING. It was based on our analysis of the locations and the 

capacity of providers that we talked about in the answer to your 

first question. 

Ms. SCHAKOWSKY. Thank you. One source of uncertainty in CBO 

estimates is that your office cannot always anticipate how a law 


BU00-A363290 with DISTILLER
36 

will be implemented, but I understand that the CBO does adjust 

its baseline and cost estimate as it receives new data and observes 

changes in implementation. The Trump administration cut the outreach budget for the insurance marketplace under the Affordable 

Care Act by 9 percent. Is that the sort of change that the CBO 

would look at? 

Mr. HALL. Yes, it would, absolutely. We will be taking a look at 

that. We have not made any conclusions yet. 

Ms. SCHAKOWSKY. The administration wants to let states take 

Medicaid healthcare benefits away from families who cannot provide employment. Is that the sort of change in implementation 

CBO would consider as it updates its model? 

Mr. HALL. Yes. 

Ms. SCHAKOWSKY. And after years of decline, the uninsured rate 

went up—is that the last year? Okay. Is that the sort of new data 

the CBO would incorporate in its baseline future estimate? 

Ms. BANTHIN. I can answer that. We take account of all survey 

results that you are referring to, the Gallup poll. However, we 

place a little bit more weight on the National Health Interview 

Survey because of its larger sample and much higher response 

rate. That was released last week. 

Ms. SCHAKOWSKY. And who does that survey? 

Ms. BANTHIN. The Center for Health Statistics, CDC, HHS. It is 

a large, federally sponsored survey. That showed a tiny uptick that 

was not statistically significant in the number of uninsured. 

Mr. HALL. But that only covered the first three quarters of 2017, 

so we have some more data to get. 

Ms. SCHAKOWSKY. Good. Thank you. I yield back. 

Chairman WOMACK. Governor Sanford, South Carolina. 

Mr. SANFORD. Thank you, Chairman, and I thank each of you all 

for being here. I come from the coast of South Carolina, and there 

the tides gently come in, and they gently go out each day. When 

I look at your forecasts, here is my concern. Is it optimistic? Are 

we lulling ourselves to sleep based on forecasts that are relatively 

benign in what may prove not to be the case? 

And, specifically, I want to get at the notion of asset values and 

the behavior that comes with that in terms of consumer spending 

and other. I will go back to what Mr. Smith was just getting at. 

If you look at your projections in 2012 on the ACA, it was not margin of error. It was just plain error. 

And I am not faulting you for the different things you thought, 

but, I mean, the numbers; he focused in on a single year. We are 

talking from sort of a high point of 67 percent projected to, as the 

trend ran out of the window, down to 27 percent are projected. I 

mean, again, not margin of error, but just outright error. And I will 

go back to what you said, Dr. Kling, just a moment ago, which was, 

in fact, interest rates were really the larger area of error in forecasts. 

So my question is this: if that is the case, and we were off on 

our interest rate numbers, given interest rate as a driver of the 

Federal budget going forward over the next 10 years, we could put 

ourselves into one heck of a financial conundrum. And so, I look 

at some of the wealth numbers; we are in uncharted waters with 

regard to Federal and central banks with zero interest rate policy 


BU00-A363290 with DISTILLER
37 

around the world. We have a global debt number that has run up 

to the highest ever. 

I pulled the Shiller PE ratio. Our numbers are now higher than 

on Black Tuesday in terms of the Shiller PE index. I look at household net worth to disposable income; we have never before in the 

history of our country been at as high a level. I look at household 

net worth to GDP; again, same phenomenon. We had blips, both 

prior to the tech bubble and the crash of roughly 2008, but never 

before at the levels that we are seeing right now. 

So maybe we are living in a bubble, and we do not even know 

it, and if that was the case, what does that do in terms of your projections? Because there is a wealth effect in the way that people 

spend money that would be real and have, I think, very grave consequences, not just in terms of the Federal budget but in terms of, 

ultimately, people’s wealth in this country. Give me a little bit in 

terms of the degree to which you are incorporating that into your 

budget forecast. 

Mr. HALL. Sure. Actually, the risk, I think, is a little difficult to 

talk about, difficult to summarize for folks. You know, when we do 

our economic forecaster budget forecasts, we give our best estimate. 

We try to give an idea of what the ranges are, and then we can 

say things like when the debt gets up to really high levels, the risk 

is higher, but it is hard to communicate exactly how high. 

Mr. SANFORD. I am not faulting you, but I am just pointing out 

that there is a substantial contingent liability built into whether 

our budget—in fact, if the CBO budget is, from my standpoint, generous, then the administration budget is wildly optimistic. Would 

that be fair? 

Mr. HALL. Well, we have not looked at the President’s budget 

yet, so we know their take on that. 

Mr. SANFORD. I understand the political distance, but, I mean, if 

yours is optimistic, then that would be wildly optimistic by comparative standard, correct? 

Mr. HALL. Yeah, we are usually pretty close to private forecasters. We are pretty close to the consensus, because we are all 

looking at the same data. 

Mr. SANFORD. I have got 45 seconds. Dig in just a little bit on 

the thought you were getting at, though, which is what Rumsfeld 

talked about, the known unknown, or the unknown that we do not 

know about. Where are we? I mean, to what degree is that built 

or not built into what you are forecasting? 

Mr. KLING. So, one of the things that you might be interested in 

is in our long-term budget outlook we have analysis of budget uncertainty, where we look at some different factors of ‘‘Well, if productivity growth and economy is higher or lower than we project, 

then what are the effects on the budget? What are the effects of 

higher or lower healthcare spending?’’ 

And so if you take a number of factors into account, you can see, 

like, the path of debt is either much higher than our central forecast, or it seems to be relatively flat. So, it would be unlikely for 

it to turn out that debt would be a lot lower than current levels. 

We project that it is going to be somewhat higher; it could be considerably higher, or it could remain around the same. But that cone 


BU00-A363290 with DISTILLER
38 

is definitely mostly up, and so that, I think, is something that 

would be of interest to you. 

Mr. SANFORD. Thank you. 

Chairman WOMACK. General Bergman, Michigan. 

Mr. BERGMAN. Thank you, Mr. Chairman, and thank you all for 

being here, because this is always an educational experience, I 

think, for all of us who are listening. Sometimes I think it, may not 

be impossible, but if you are talking, you are not listening. So, I 

am always appreciative of the opportunity to sit here and listen to 

people who are deep in the details of what you do on a daily basis. 

We are all, in our own way, deep in the details of what we do, 

but unless we share and collaborate across any arbitrary lines, artificial lines, then none of us are really doing our job on behalf of 

the American people. 

My district, the first district of Michigan, and a couple of important details. Michigan has 14 districts, congressional districts. My 

district has 46 percent of the landmass. Okay? Why is that important? I get a chance to drive a lot of beautiful places, but I also 

get a chance to meet a lot of people. 

For over a hundred years, the Upper Peninsula of Michigan has 

been on its economic back because of closure of mines, decreases 

in logging, all sorts of different things that go. So, we, just in 

Michigan alone, as the Upper Peninsula, feel left out of the decisions made in Lansing at times, our state capital. So there is a certain sensitivity, but a certain sense of realism, about what it means 

to be able to live your life, raise your family in the Upper Peninsula of Michigan. 

So we know that on the bell curve we use, the UP fall out on 

the outside the statistical norm. Okay? When you look at assumptions, and you look at then validation of those assumptions, can 

you say they are in that statistical norm of the 80 percent across 

the board, for the most part? 

Mr. HALL. Yeah, that is a good question, because so much of 

what we deal with is not based on statistical analysis. You know, 

if we had a nice statistical model, we could make an estimate, and 

we would have some idea of the uncertainty. So much of the uncertainty is unknown. These are the unknown unknowns. And so, we 

are—— 

Mr. BERGMAN. But if you take a list of—because I know time 

goes fast. When you are listing or creating your assumptions, that 

assumption, if you put it right back into a bell curve, does it fit in 

the middle, or do you ever look at ‘‘Where does my assumption fit 

into a norm?’’ Or is it just kind of pie-in-the-sky? How do you develop your list of assumptions? 

But also, more importantly, do you share those lists of assumptions going in with folks like us who know our districts like the 

back of our hand? And when we look at something, I can say that 

there is no fit in my district for that assumption, because it is not 

a valid assumption. That is why I am curious as to how you do 

that. 

Ms. BANTHIN. So, one example is that we do—in thinking about 

marketplaces and how well those marketplaces are functioning, we 

consider insurer participation, and we realize that in more rural 

areas there is less competition among insurers. We try to work that 


BU00-A363290 with DISTILLER
39 

uncertainty and to discuss it in our reports and work it into our 

estimates. That is just one example I can provide. 

Mr. BERGMAN. Okay. Does the CBO communicate to Congress if 

you have a low level of confidence in any of the, you know, underlying assumptions, you know, of a cost estimate? Do you let us 

know ‘‘Hey, I feel really good about this,’’ or ‘‘It is a little bit 

shaky’’? 

Mr. HALL. We try to make that as clear as we can in the writeup of the cost estimate. For example, with the ACA repeal proposals from last year, we had a section called uncertainty. So we 

discussed the uncertainty in some detail, because we thought there 

was a high level of uncertainty. 

Mr. BERGMAN. Okay. And also, does the CBO ever collaborate 

with the private sector or foreign budget agencies when attempting 

to develop cost estimates in any particular area that you are looking at? Do you look at outside, you know, models and others that 

you say, ‘‘Hey, maybe we can apply this,’’ or ‘‘This does not work 

for us because—’’ what is your process for doing that? 

Mr. HALL. Well, absolutely. It is one of the reasons why we have 

expertise in-house. They spend a lot of time not only developing 

their own models, but looking to see other models that are available, whether it is the private sector or somewhere else, to see if 

that captures something that we are capturing. We will occasionally use outside models if it is something that we do not have on 

board. 

Mr. BERGMAN. Thank you. Mr. Chairman, I see my time is out. 

Thank you very much. 

Chairman WOMACK. Mr. Woodall, Georgia. 

Mr. WOODALL. Thank you, Mr. Chairman. Thank you for having 

this hearing and thank you all for being here. I wanted to follow 

up on something that Dr. Banthin said, that CBO was surprised 

by how many employers went above and beyond to protect their 

employees from the uncertainty of the marketplace. 

Continuing to provide that insurance might not have been there 

in their immediate financial interest, but certainly, in the longterm financial interest, taking care of your employees is good for 

business. How will that new understanding of how employers operate be reflected in the baseline going forward? 

Ms. BANTHIN. Well, so we talk about this in our report on the 

process we follow in developing a cost estimate for any major proposal, but it is true for our baseline as well. We try to consider the 

timing of responses by affected parties. So employers are one of 

those important groups. 

Mr. WOODALL. I might not be asking the right question, and so 

I apologize for interrupting. But I guess what I am saying is I 

thought what I heard you say is you expected employers to dump 

their employees. They did not, and they did not because they were 

protecting their employees from uncertainty. 

Ms. BANTHIN. Exactly. 

Mr. WOODALL. So you were surprised by the effort that employers went to protect their employees. My question is not the timing 

of that protection but how does the baseline going forward reflect 

that employers are protecting their employees more than you expected? 


BU00-A363290 with DISTILLER
40 

I am reading again about bonuses coming out from the tax bill. 

I do not think that was predicted behavior by JCT, but it is, in fact, 

behavior that is happening. So, how do we incorporate this new 

and better understanding of how employers are trying harder than 

you expected to take care of their employees? 

Mr. HALL. Let me just say we will in fact take that into account 

in our model going forward and we have. This take on board that 

companies are going to be more likely to continue offering coverage. 

We can and do. That is one of those behavioral assumptions which 

we will change now, because what happened was different than 

what we expected. 

Mr. WOODALL. Changing, Director Hall, just with respect to 

healthcare coverage, or change it with respect to dental coverage 

and child care and salary assumptions? Is it a behavioral change, 

that we thought employers were out to feather their own nest, but 

it turns out that to run a good business you have to take care of 

your employees? 

And so, it is across the spectrum, and we will see it reflected in 

almost every employer-related bill going forward, or have you restricted that surprise to just healthcare? And employers are doing 

this unexpected behavior only with healthcare, but they are going 

to do exactly what we expected them to do with child care and 

transportation assistance and salaries and everything else. 

Mr. HALL. Well, I mean, certainly, we have changed it. We will 

change it with healthcare. Yeah, I think we will probably think 

about other things, see that this experience helps inform our assumptions of behavior, behavioral assumptions in other things like 

child care and some of those things. 

Mr. WOODALL. I see the headline over and over again; 23 million 

Americans expected to lose healthcare. It is not your fault; the 

media gets to report what the media wants to report. I have not 

seen—and thank goodness you all were here today—I had not seen 

that observation that employers take better care of their people to 

protect them from uncertainty than models had previously assumed. I understand that was reported, but it was not highlighted. 

Does CBO go out of its way to highlight for us? You know, folks 

have been here for 10, 20 years; they expect the CBO to do what 

the CBO does. When CBO uncovers new behavioral information, is 

that highlighted for folks somewhere? ‘‘These are the changes we 

are assuming going forward from 2016; these are the changes we 

are assuming from 2017.’’ 

Mr. HALL. Well, we certainly do that in our budget outlook. We 

actually have sections on there about how we are changing things. 

We do it also in what we call an annual analysis of actuals, where 

we look at all the budget categories once a year; see how we are 

doing, how we did; see if we need to adjust our forecast, adjust our 

models. We have done that in private so far. 

One of the things that we are proposing to do is start releasing 

that. It would give you information on areas where we think our 

forecasting was not as accurate as it could be and where we have 

made adjustments. It would give you a feel for when we are making those changes. 

Mr. WOODALL. We cannot sort this next one out in the 26 seconds 

that are remaining, but I noticed your reluctance to respond to Mr. 


BU00-A363290 with DISTILLER
41 

Sanford’s ‘‘Are the President’s expectations wildly optimistic for the 

economy?’’ When I go back to 1972 and the debate around the Act 

of 1974, the goal was to have CBO be the advocate for Article I. 

I know that Article II will not have any problem talking about how 

we have done our math wrong, and I wonder what the wisdom is 

of us not that being equally strong in defense of our work here. A 

different issue for a different hearing. Mr. Chairman, I thank you. 

Chairman WOMACK. All right, thank you. The dais is now clear 

of the members seeking to ask questions. I am going to now yield 

to the Ranking Member, Mr. Yarmuth. 

Mr. YARMUTH. Thank you, Mr. Chairman, and thanks to our witnesses for their responses. Just segueing off of Mr. Woodall’s question, I am curious; maybe when you have analyzed or were analyzing the tax bill that recently was enacted and trying to model 

behavior, did you have a sense of how much—or the Joint Tax— 

have a sense of what the behavior was going to be with corporations? Because Mr. Woodall mentioned the bonuses. 

And Morgan Stanley just did an estimate, according to a New 

York Times editorial, that 43 percent of corporate tax savings 

would go to buybacks and dividends; nearly 90 percent would help 

pay for mergers and acquisitions. Just 17 percent would be used for 

capital investment, and even a smaller share, 13 percent, would go 

toward bonuses and raises. Do you know whether those numbers 

correspond to expectations that maybe Joint Tax used or that you 

would use in your analysis of the tax bill? 

Mr. HALL. Yeah, I do not know offhand. I do know that is going 

to be an important part of our budget update, our analysis of the 

tax bill, how exactly we view that. We will try to detail it once we 

sort of iron it all out. I kind of do not want to give away pieces 

of what we are still working on. 

Mr. KLING. The outlook that we are preparing for you to publish 

this spring does have the advantage of this additional information 

that has come out in the last few months that the staff of the Joint 

Committee on Taxation did not have when they were doing their 

estimates in December. And so, we do have more information, and 

our estimates will take that on board. 

Mr. YARMUTH. Great. I appreciate that. Thanks very much. We 

have talked a lot about healthcare today; I do not really want to 

talk too much about it anymore, so I want to move on to a different 

subject that may or may not enter the national dialogue pretty 

soon. That is the question of infrastructure. And, you know, the 

President has a plan that relies on basically a 20 percent investment by the Federal Government in projects, matched by an 80 

percent investment by either state, local, or private investors. 

And I assume that different types of infrastructure would yield 

different returns to the economy, would have different types of impact, and I am curious as to what factors might determine the impact of an infrastructure project. I just read something that I had 

never really thought of but makes good sense, that there are two 

different types of infrastructure spending; one is maintenance, and 

one is essentially new construction. And I am just curious as to 

how you would evaluate all of those different factors. 

Mr. HALL. Shall we just say, just in general, it is pretty clear 

that specific infrastructure projects vary quite a lot in their return, 


BU00-A363290 with DISTILLER
42 

but it is really hard to characterize types of infrastructure that 

way, because there is so much variation in individual things. With 

respect to something like transportation, you know, we have done 

some work on that; you know, for example, shifting resources from 

rural areas to congested urban areas. That does increase the productivity of Federal spending. 

I think what you are probably referring to is repairing and rehabilitating facilities like highways. Probably has a higher return 

then adding new highways, that sort of analysis we have found. 

Mr. YARMUTH. Yes, that is interesting. And you mentioned rural 

and urban differences and how they interact. I was just reading 

something the other day that talked about—and I do not want to 

pit rural America against urban America—but that there is a possibility, depending on how you allocate resources, that you would 

be spending a disproportionate amount of money—disproportionate 

to the population—that might end up in a less than optimum return. 

If you were spending 40 percent of the budget on rural areas as 

areas which have either a much smaller percentage of the population or a much smaller portion of GDP, is that something that 

should be considered as part of any infrastructure plan? 

Mr. HALL. I think our point really kind of goes more toward the 

idea that there are urban areas that are very congested in terms 

of highways, and spending more in those areas has a bigger payoff 

than urban areas. It is not because they are rural areas; it is because they may not be as congested as urban areas. 

Mr. YARMUTH. Got you. This is a question that is obviously timely, and I throw it out there because I have no idea. Has the CBO 

ever done any economic impact of gun violence? Did the CBO score 

the assault weapons ban in 1994? Are you aware? 

Mr. HALL. We will have to back to you. I do not know. 

Mr. YARMUTH. If there was assault weapons ban legislation introduced, what would CBO consider doing a score of assault weapons ban? 

Mr. HALL. Well, I suppose if it came out of a Committee to the 

floor, we would have been required to do it, to do an estimate of 

that. So, we could check. 

Mr. YARMUTH. But as far as you know, you do not have any 

sense now of what factors you would consider if you were actually 

considering doing a score of that. It would be new territory. 

Mr. KLING. Unfortunately, you have the wrong people the table 

for that. We will have to get back to you. 

Mr. YARMUTH. That is good. Going back to healthcare for just a 

second, I just want to get something into the record, when we were 

considering the Affordable Care Act in 2009 and 2010, CBO did 

score that legislation. 

And I raise this question because, just during the last debate, a 

lot of our friends on the other side of the aisle tried to make a parallel that, yes, we used reconciliation to pass the Affordable Care 

Act in 2010, and they used reconciliation to pass the tax bill. But, 

of course, the difference was that the ACA had a positive score at 

that point over both the initial 10-year window and the second 10- 

year window. Is that not correct? 

Mr. HALL. Yeah, that is right. 


BU00-A363290 with DISTILLER
43 

Mr. YARMUTH. And the tax bill obviously had a proposed deficit 

of $1.5 trillion. So, it is a considerable difference in the budgetary 

impact of those two. One other question about healthcare. One of 

the things that we know we have to do, and we continue to talk 

about it and work on it, is reducing costs. And it seems to me that 

CBO has an incredible amount of knowledge and expertise and resources that it has devoted to analyzing certain cost impacts of certain policies. 

And I guess my question is, is there any way Congress could 

avail itself of that expertise without regard to an individual proposal? Because it seems to me that you all have probably a better 

handle on what actually might drive cost down in the healthcare 

delivery system than anybody else in this body. And is there any 

way to get, essentially, a generic analysis of what possible steps we 

might consider? Essentially, asking you to propose solutions. 

Ms. BANTHIN. Well, so we do not make policy recommendations, 

but—— 

Mr. HALL. But if you throw some at us, we can tell you tell you 

what we think. 

Ms. BANTHIN. Or rank them somehow. 

Mr. YARMUTH. Okay, that is fair enough. One other thing, just 

in the minute I have left. And I have talked about this many times 

before, and that is the pace of change in society and what a difficult time that gives you or anybody trying to make projections of 

more than a year or two. 

I read an interview recently with the managing director of Mercedes-Benz. And he talked about a wide range of things, but one 

of the things he talked about which he says could happen essentially any day now is that the impact of artificial intelligence, and 

specifically IBM’s Watson, on careers like basic legal profession, accounting, radiologists and so forth, that essentially IBM’s Watson 

has proven itself to be more accurate than humans in many of 

these areas and that people could start losing jobs in these areas 

almost immediately. Is that something that you think about or 

have any way of incorporating into projections? 

Mr. HALL. No, that is really hard to project that sort of thing. 

The sort of thing that we can and do is do these sorts of sensitivity 

analysis. What if productivity was much higher than we expect at 

the moment? And get some idea of what effect that would have— 

or much lower. So, we get some idea of the possible range of outcomes. But it is really hard to obviously predict something like 

that. 

Mr. YARMUTH. Absolutely. Well, thank you very much for your 

work and for your participation today. I yield back. 

Chairman WOMACK. Mr. Arrington, are you ready? 

Mr. ARRINGTON. I am, Mr. Chairman. 

Chairman WOMACK. Let’s go to Texas. 

Mr. ARRINGTON. Thank you, Mr. Chairman. And I was Chairing 

a Subcommittee roundtable, and I apologize for being late. And Mr. 

Hall and your colleagues, I appreciate your time and input here. 

I have got several questions but let me start with just an observation stemming from the Affordable Care Act implementation and 

your analysis around that. 


BU00-A363290 with DISTILLER
44 

The Obamacare, as we refer to it sometimes, expanded Medicaid 

beyond the 100 percent or 120 percent poverty. In some cases, because the Federal Government put that carrot out that they would 

pay 100 percent of it, it expanded upwards of 400 percent poverty 

in some states. When it did that, it captured able-bodied people 

who are able to work and were not working and are not working 

but receive government assistance. 

And so, in that sort of vein of trying to get on behalf of the taxpayers and recipients of welfare, because we think it is good for everyone to have an honest, hard day’s work and the value of that, 

along with we have a shortage of labor, along with, as fiduciaries 

of taxpayers, this is a really important thing for the American people. 

Your analysis, though, suggested, I think, early on—and rightfully so, but maybe off the mark—that when President Obama and 

the Congress at that time passed the Affordable Care Act, that 

work would actually decline as a result based on the dynamic I just 

mentioned. And then you came back, I think, at a later time—I 

think maybe the first report was 2010, and then in 2014—revised 

those numbers. 

So, now we are entering into this opportunity for the American 

people, whether it is with food stamps or Medicaid or whatever the 

government assistance program, compassionately and respectfully 

and responsibly, though, asking people who are able to work to 

work if they receive these benefits. Tell me about your behavioral 

modeling in that instance and why it changed, and have you perfected it and—— 

Mr. HALL. Sure. Taking into account the labor supply effective 

policies is an important behavioral component, I think, of what we 

do, and we do try to do that. I will give you, for example, I think 

it was 2016 we did an analysis of not a repeal and replace but just 

a full repeal of the ACA, and we did our dynamic portion of that, 

where we actually estimated the effect on the labor supply of eliminating the ACA. So you get this sort of labor supply impact. 

We did that, and then we followed up with a piece, a research 

paper on how we came up with our labor supply effects. You know, 

what research we used et cetera. In fact, I think I referenced it in 

my statement as well. That is the sort of thing which may help inform your thinking, but that is certainly something we would want 

to take into account. 

Mr. ARRINGTON. Were your findings that less people would work 

as a result of expanding Medicaid? 

Mr. HALL. Yes. 

Mr. KLING. The Medicaid effect is relatively small, so our analysis of the effects of the Affordable Care Act on labor supply mostly 

stems from the taxes in the act and from the phase-out of the premium tax credits, and very little from Medicaid itself. 

Mr. ARRINGTON. But expanding Medicaid and raising taxes in 

that combination or some—and maybe there are other factors— 

caused less people to participate in the workforce. Is that yes or 

no? 

Mr. KLING. Yes. 


BU00-A363290 with DISTILLER
45 

Mr. ARRINGTON. Were the numbers higher than you predicted in 

terms of people not participating in the workforce as a result of 

this Federal policy and legislation enacted or less? 

Mr. KLING. So it is pretty hard to know exactly what those effects have been, because we only observe what happened, and we 

did not observe what would have happened if the law had not been 

enacted. 

Mr. ARRINGTON. My only point is you came out with ‘‘Here is 

what will happen: fewer people will be in the workforce; less people 

will be incentivized to work as a result of the factors we just discussed.’’ Were you off the mark because there were more people 

that did not participate and pulled back from the workforce, or 

were there less when you revised in—— 

Mr. KLING. The research that has come out since our original estimate, effects have been smaller in the literature, and that has 

caused us to revise down a little bit but not very much. 

Mr. ARRINGTON. Okay. I am out of time. I yield back, Mr. Chairman. 

Chairman WOMACK. Mr. Grothman, Wisconsin. 

Mr. GROTHMAN. Thank you much, Mr. Chairman. In your view, 

what are the greatest challenges that you guys face in modeling for 

cost estimates that require extensive behavioral assumptions? Can 

you give me some examples of what you go through and [inaudible]? 

Mr. HALL. Sure. Well, we are literally always fighting the clock, 

first of all. You know, lots of times the Congress moves at paces 

that are just much faster than we can move, so we do our best to 

sort of try to, you know, put out models and understand some of 

the behavioral issues ahead of time before that. And we kind of do 

the best we can with our current resources, frankly. We work flatout on things. 

Mr. GROTHMAN. Okay. Could you republish your confidence intervals for cost estimates produced using behavioral modeling? 

Mr. HALL. Sometimes it might be possible. Most of the time we 

can try to give you some idea of the uncertainty. For us to produce 

actual ranges like that, it is kind of a decision, I would think, from 

the Budget Committee whether you want to see that or not. I think 

there are some issues with that, because you all want a point estimate, but when they see a range, they can pick the number that 

they like, so they will pick the low one or the high one. So, that 

can cause some confusion, but that is sort of above our paygrade 

as to whether you want us to produce those. 

Mr. GROTHMAN. Can you walk us through the review process at 

to how you arrive at those figures? 

Mr. HALL. Sure, sure. Well, just generally, you know, we have 

analysts who spend a lot of time on a piece, and they go through; 

they look at prior estimates to see what informed things. They do 

a lot of talking with folks to see how things will be implemented. 

We have sort of a process like that, but then there is a supervisor 

involved helping with that, helping with that, with the assumptions, with the modeling, and et cetera. 

And then we have review at various levels. It is not just within 

one division. The work, especially for the important pieces, the review comes up to review at different levels within the organization. 


BU00-A363290 with DISTILLER
46 

And then everything goes out of the director’s office, so we get a 

chance to look at everything that eventually comes out where we 

are. 

Mr. GROTHMAN. You are frequently at a rush for time. How does 

that affect your outcomes, do you think? 

Mr. HALL. It probably adds to the uncertainty. It increases the 

chance that we make a mistake, to be honest. You know, I would 

say it is like anything else: If you have more time, you have a better chance to be more accurate and more comfortable with things. 

That is always a balance for us to feel comfortable enough with an 

estimate that we are ready to let it go. 

Mr. GROTHMAN. Okay. I will give you another question that always concerns me. Obviously, when you are under of the clock, you 

to a certain extent have to rely on information provided to you by 

the agencies, and sometimes the agencies may have an agenda. 

How often do you review the agency’s assumptions, or do you do 

independent checking on what you are getting from them? 

Mr. HALL. Well, we generally try to do that, in fact. We really 

do try to do that. We try to get data rather than analysis from 

agencies. If we have experience with an agency, and we found that 

some of the information we are getting is unreliable, we will go to 

a different source other than the agency sometimes, to be honest. 

That is actually one of the delicate things for us, because we find 

people within agencies that have the information and we think are 

reliable, and they do not always want to be identified. 

Mr. GROTHMAN. Thank you. I will [inaudible]. 

Chairman WOMACK. All right, thank you. We have cleared the 

dais again, which is a good thing, and I will get to my questions. 

Dr. Hall, Dr. Banthin, Dr. Kling, are you all baseball fans? 

Mr. HALL. Yes. 

Ms. BANTHIN. Of course. 

Chairman WOMACK. Follow a little bit of baseball? You know, we 

are 30 days away from opening day. You guys are at the plate; you 

are the hitters. Life; the economy; climate; all kinds of conditions 

are the pitcher. And if you follow baseball, you know that the pitcher does not just throw fastballs. 

If the CBO was just up there to hit fastballs—fastballs meaning 

facts, easy-to-apply data in your calculations—life would be really 

easy for you. But the fact is there are a lot of curve balls out there. 

What are the breaking pitches that come at you that are the hardest to hit? 

Mr. HALL. It is the things that are very new, where we have no 

experience, where we have not done an estimate that is at all like 

it before, where there is not necessarily much data, there is not 

much research on a topic. That becomes very, very hard for us. And 

then also in cases that are really complex. 

Sometimes we even get estimates where, in terms of behavioral 

responses, one portion of the estimate pushes behavior one way; 

another pushes the estimate, pushes it exactly the opposite direction, so we wind up having this judgment as to which one is pushing harder. Those are the sort of things that are difficult. But it 

is the lack of information, lack of basis for making our assumptions 

that is the most difficult for us. 


BU00-A363290 with DISTILLER
47 

Chairman WOMACK. Hitters like to find the sweet spot. Dr. 

Banthin, is it hard to find the sweet spot in your calculations? 

Ms. BANTHIN. Sometimes it is. When we are estimating a major 

health proposal that changes the entire healthcare system, that is 

when we have to think very carefully, review all available evidence, 

and then go beyond that and reach out to a range of experts. That 

will take us a lot of time, because if a new world is going to be created, and there is no real evidence we then have to talk to key 

stakeholders—insurance companies; state commissioners—and we 

cannot just talk to one or two. We have to talk to three, four, five, 

or six, and really think carefully about how these stakeholders will 

respond to the new policy. 

Chairman WOMACK. Yeah. If you follow baseball long enough, 

you have hitters that just cannot hit a curve ball. They just cannot. 

Dr. Kling, is it important for CBO to be capable of adjusting on the 

fly to the unpredictability of the whole plethora of things that you 

have to consider when you are performing calculations for Congress? 

Mr. KLING. I mean, it is absolutely important to try to pay attention to all the things that are happening. And so, as Dr. Banthin 

mentioned, in a case that is very uncertain, one of the ways we try 

to cope with that is to reach out to as many different experts with 

a variety of perspectives in order to take advantage of the wisdom 

of crowds in that circumstance. 

Chairman WOMACK. Dr. Hall, are there any areas in your business that you just lack the adequate tools with which to be able 

to adjust to the changing environment where you have to deal 

every day on the basis? And the fact that you are on the clock, having to do it was some kind of speed and a level of accuracy that 

can be redeeming to the agency and at the same time provide all 

of us actionable information that is accurate enough that we can 

make well-informed decisions. 

Mr. HALL. I would say the biggest challenge probably is how 

nimble we are, because when we get new topics we did not anticipate, we can acquire some expertise, we can acquire models and et 

cetera, but sometimes it takes time. And so, it is this sort of 

nimbleness that is probably our biggest challenge, because we have 

a really broad range of expertise throughout the place, but it is not 

so broad that that we have experts in everything that we could possibly get asked about, unfortunately. 

Chairman WOMACK. Are the needs of the CBO reflected in your 

budget request? 

Mr. HALL. Yes. And I have to say, ever since I have been in the 

job, we have been asking for more resources for healthcare and dynamic scoring, and we are hoping to get some more. And I think 

we actually have a plan going forward where we would like to over 

the next 2 or 3 years add about 20 people, just generally, so we can 

deal better with the transparency that that you all—seems like 

would want to have an increased level transparency from us, and 

a quicker turnaround. 

We have real issues with peak load issues. Right? Because we 

cover a really broad waterfront of areas, when get a lot of action 

in one area, we do not always have a lot of people, so we are trying 


BU00-A363290 with DISTILLER
48 

to figure out how to be more deeper or have a deeper bench going 

forward. So, we are asking for some more resources for that. 

Chairman WOMACK. I want to follow up on a question that Mr. 

Grothman asked just a minute ago, and that is the input information you get from other bureaucratic agencies. It has been asserted 

more than once that these agencies can provide input to you that 

can be way out of bounds, terribly inaccurate, a swag, if you will. 

How often do you go back and verify, and is there a failsafe way 

to be able to take information from one agency that is critical for 

you to make a calculation and know within a certain amount of 

reason that that particular piece of information is really nothing 

but garbage to you? Garbage in; garbage out. So, how often do you 

do it, and are there mechanisms that you have in place or need to 

have in place to ensure that we can troubleshoot—maybe not necessarily second-guess, but it sounds like that is where I am going 

with this—these agencies? 

Mr. HALL. Well, one of the things we do is when we get legislation, the very first thing we do is we will ask the Committee staff 

work in legislation, ‘‘Who should we talk to? What experts do you 

think we should be talking to? What data do you think we should 

look at?’’ So, we start from the same place that Committee staff are 

starting from, and then we go out on our independent way to sort 

of look at other data sources and other things. So, we try not to 

just rely on agency data; we try to do our best to get other data 

and confirm it and be sure that we touch all the bases. 

Ms. BANTHIN. Yeah. In healthcare, we have enough expertise 

that I do not believe we fall into any trap like that. We get data 

from the executive branch agencies: Medicare claims data; Medicaid claims data; Household Survey data. And so, we analyze those 

data ourselves independently. 

Chairman WOMACK. So, quantifiable data I understand, but if it 

is just seeking information from an agency, does it ever happen 

where the analyst says, ‘‘That does not look right. There is no way 

that that is accurate,’’ and go back and make them show their work 

as to why that data is being provided? Is there ever a circumstance 

like that? 

Mr. HALL. There are, actually, and there are probably agencies 

who historically have not been very forthcoming with us we have 

struggled with. I would have to ask around for some other examples, but that certainly is true that we do have that happen sometimes. 

Chairman WOMACK. You have described the process as a little bit 

of art and a little bit of science. You know, that sounds like a recipe 

at home where, based on experience, you get all of those components just right because you have done this over and over again, 

and you know exactly whether it needs a pinch of that or a dash 

of something else. How capable are you of making sure that the 

outcome that we are getting is based on accurate measurements 

and less of a particular bias that one or others may have toward 

the given subject? 

Mr. HALL. We work in that under almost every level. When we 

hire people, we hire them based on their expertise, and we actually 

vet them, to the degree we can, about their ability to do objective 

work and not bring their personal views in. And we train them 


BU00-A363290 with DISTILLER
49 

very carefully about what we want them to do, how we want them 

to approach issues, how we want them trained. Then we have a review process of how people are doing. 

Are they doing a good job? Are they being accurate? Are they following the right procedures to be objective? Are they touching all 

the bases? Products that come up; they get reviewed at a number 

of different levels as well. We just really try everything we can to 

be sure, because everybody has opinions, and we need to have people who can take their personal views out and work just purely on 

their professional views, and I think CBO does a really good job of 

that. I have been in a lot of places. 

Chairman WOMACK. I want to thank you, Dr. Hall, and the two 

other witnesses that are with us today—Dr. Kling, Dr. Banthin— 

for your testimony today. I want to advise members that they can 

submit written questions to be answered later in writing. Those 

questions and your answers will be made part of the formal hearing record. Any members who wish to submit questions or any extraneous material for the record may do so within 7 days. And with 

that, we wish you a very pleasant rest of the day, and this committee hearing stands adjourned. 

[Whereupon, at 12:09 p.m., the Committee was adjourned.] 


BU00-A363290 with DISTILLER
50 


BU00-A363290 with DISTILLERInsert offset folio 48 here 30540.010

WOMACK Questions for the Record 

Why is confidentially a prerequisite to getting agency inf(mmttion? 

• Are there other ways to get the same infonnation that do not require confidentiality'~ 

• Does CBO use commercial models or are all ofCBO's models developed in-house'' 

• Has CBO considered using commercial models instead of generating its own? Why or 

why not? 

• A model clearly cannot account tor all the factors associated with a particular policy 

change. As such, analysts likely need to estimate some effects that are then input into the 

estimate itself How do analysts include these types of estimates- where the model is 

unable to account fully tor some of the effects? 

• Why does CBO not publish all of the assumptions that its analysts make in estimating the 

cost of a bill? 

• What are the safeguards in place to ensure that hehavioralmodeling follows rigid 

standards, but at the same time remains flexible enough for analysts to score unique 

proposals') 

• In addition, Office tor Budget Responsibility [OBR] began publishing in December 2014 

a systematic approach to assessing the uncertainty of policy decisions. It divides the 

particular measure into three separate categories: data uncertainty, modeling uncertainty, 

and behavioral uncertainty. In each category, it assigns a value to the uncertainty, 

describes in its opinion which factor is most impactful, and provides an overall 

uncertainty rating. Has CBO ever considered developing a systematized approach to 

categorizing the level of uncertainty of its estimates'! 
51 


BU00-A363290 with DISTILLERInsert offset folio 49 here 30540.011

Answers to Questions for the Record Following a Hearing 

Conducted by the House Committ(~ on the Budget on CBO Oversight: 

The Role of Behavioral Modeling in Scoring and Baseline Construction 

Question. \~Chy is confidentiality a prerequisite to getting agency information? 

Answer. Confidentiality is sometimes required to make 

CBO\ estimates. l~or CBCYs baseline of rcvcnacs are 

paid by and individuals, "l11c datJ. contain 

ioi(Hmation and cannot he madt: publicly available. Confidentiality may also be 

information informally from staff of other govemment who 

ro share informmion or judgments that not in public 

domain. 

Question. Are there other ways to get the same information rbat do not require 

confidentiality? 

Answer. confidential data is usually a sk)\v ,1nd costly pro~.,_css, so CBO 

ally rdics on coJWclcnnaJ data only -..vhen the information is superior to noJKonrJ<JC11\!,ll 

Question. Does CBO me commercial models or are aU of CBO's models devdoped 

in-hou~{.'? 

Answer. CBO uses some commercial models as complements to its O\..vn. For example, CB<J 

licenses nLJCroc~..-onomic forecasting models from t\VO finns and uses them to int(Hm the 

dc-vdopmcnt of its own fl)rcclsting modeL To rake another example. CBO has 

access to models developed by actuarial firms rn assist "virh some estimates 

in.smancc, such as the csrimatc.s ofdcductibtes that would occur under certain proposals. 

2\JIB 
52 


BU00-A363290 with DISTILLERInsert offset folio 50 here 30540.012

Question. Has CR() considered u~ing commercial modds imtead of generating its ovvn? 

\Xlhy or why nm? 

Answer. In general, commercial modds dn not have the detail about federal and 

that CBO needs to perform its work, nor do they h;Jve 

ro address in certain ;tspects of a program that mighr be afFected legislation. 

"1herd0re, rnadc more u:.e of commercial models, it would need to mod if)· 

them. 

Question. A model dearly cannot accounr fi.)r all rht.' tacrors a:;sociat-cd wi rh. a pJ.rticular 

change. As sud1, ana!ysr.s likely need to estimate son1e eff(·ct:. that tht'n into 

estimate itself Hnw do analysts includt· these types of rstimatcs----\vhcrc the 

unable to accounr ful1y for some of the eHCct<;? 

bur multipk modds c:m. Some mock·ls built around 

health insurance simulation model irs usc of survey data on 

CBO's models of Medicaid enrollment .md costs in usc of 

the Joint Committee on "E1.xation's (JCT's) individual tax model in 

its usc of tax returns. In CBO and JCT's for preparing cost cstimatt.'S, f(Jr cxamplc, 

output from one or more of those is ofi::cn input to another one of them. 

im.porram hctors nor directly addressed by a of models, then 

usc historical evidenn· from similar 

from cxpar.'. on the and their own analytical dSscssn1cms to 

to the !1lodds or to adjust the mod~:!ing resuhs. For example, in 

state~' decisions about for vvaivers to federal ruks gon~rning 

used evidence ahont similar decisions by states to make projections to he 

as inpms to models.: 

Question. \'\lhy Jocs CBO not publish Jtl of the assumptions that its analyst:.; in 

estimating th{' cost of a bill? 

Answer. CBO strives to be transpan·nt. providing clear, concise cxpbn::uions and 

the key compnncnrs of' the basis of each cstirnatc. The agency also endeavors to mJkc 

mation ,1\."CCssiblc by context and tcchnic.tl terms. "D1c sundard 

for CBC)'s formal t.~stinutcs ;t ''Ibs!.'t 

action generally does nor ~-dlow t{)r dcraileJ 

key dements of an estimate. 

consuming to produce, and the 

doc\tmentation of an estimate. Director l Iall mentioned in his at the 

rhc agency is 

Jn ba:;elinc 

rncthod:;-

Question. What arc the saf{:guards in place to ensure that behavioral modeling fdlovvs 

rigid srandJrds, hut at the same time remains Acxib!e enough for analyst:. to unique 

propos;lls? 

2. 
53 


BU00-A363290 with DISTILLERInsert offset folio 51 here 30540.013

Answer. 1l1c review process for estimates is rhc that CHO ensures thar the modding used in its estimarcs is of high abo is ad:lptcd to the pa1ricular fCatun.'s of a 

Every formal cost estimate is tO extensive internal review to ensure that 

and 

rcvievvs, all estimates undergo further revic\v intt:'mal!y by numerous people at different levels 

of the organization. 1 

Responsibility (OBR) hegan in 

to assessing the unccrt~!inty decisions. Tr 

divides the nwasurc into three separate categories: data uncertainty, modeling 

uncertainty, and behavioral uncertainty. In each a value to the uncertainty, 

describes in its which f:KtGf is IllOSt all overall uncertainty 

Has approach to categorizing the 

Answer. Robert Chotc, chairman of the committee 

DJTSer>tatiorJS to CBO's staff about to unceranalysis. CBO is examining how to systematically expand how 

office's 

u"'P'''"'""" Budget Of:lice, How CEO Prcj~tl1'PJ' Cost Ei·tirmtti!J· (February 

gnv/pnb]/,'cci<<n/j l'