Daniel K Tarullo: Stress testing after five years
Speech by Mr Daniel K Tarullo, Member of the Board of Governors of the Federal Reserve
System, at the Federal Reserve Third Annual Stress Test Modeling Symposium, Boston,
Massachusetts, 25 June 2014.
* * *
Born of necessity during the depths of the financial crisis as part of an effort to restore
confidence in the U.S. financial system, supervisory stress testing has in the intervening five
years become a cornerstone of a new approach to regulation and supervision of the nation's
largest financial institutions. First, of course, it is a means for assuring that large, complex
financial institutions have sufficient capital to allow them to remain viable intermediaries even
under highly stressful conditions. More broadly, supervisory stress testing and the associated
review of capital planning processes have provided a platform for building out a regulatory
framework that is more dynamic, more macroprudential, and more data-driven than pre-crisis
practice.
Each year we have refined elements of both the substance and process of the stress tests.
These changes have been informed not only by our own experience, but also by critiques
and suggestions offered by others. This annual symposium hosted by the Federal Reserve
Bank of Boston has become an important channel for eliciting reactions and advice from
outside experts and the banks subject to the annual Comprehensive Capital Analysis and
Review (CCAR). Although the major elements of our approach have now been successfully
established, I anticipate that we will continue to make enhancements. If supervisory stress
testing is to give regulators, banks, and the public a dynamic view of the capital positions of
large financial firms, it must itself respond to changes in the economy, the financial system,
and risk-management capabilities.
This morning I will give a retrospective on the first five years of supervisory stress testing,
highlighting some of the accomplishments and identifying some areas in which we may
consider changes in the future. Then I would like to turn to a topic that has gained increasing
attention in the past couple of years - the qualitative assessment of firms' capital planning
processes that we conduct in parallel with our quantitative assessment of firms' capital
positions.
From SCAP to CCAR
The potential value of comprehensive stress testing had been much discussed among
academics, analysts, and regulators in the years preceding the financial crisis, but only
during the crisis was this supervisory tool first used at the same time across the largest firms.
The Supervisory Capital Assessment Program (SCAP) demonstrated in practice, not just in
theory, the value of simultaneous, forward-looking projections of potential losses and
revenues based on each bank's portfolio and circumstances. The forward-looking feature
overcame the limitations of static capital ratios. The simultaneity, along with stress test
features such as the use of common macroeconomic scenarios, introduced a critical
macroprudential dimension that offered insights into the condition of the entire financial
system, including whether banks were sufficiently resilient to continue to lend to households
and businesses under such adverse conditions.
The Federal Reserve's basic approach to stress testing has not changed materially since the
SCAP. We continue to take a multidisciplinary approach, drawing on a wide range of staff
expertise. We create hypothetical macroeconomic scenarios that incorporate an assumed
sharp deterioration in economic and financial conditions. Supervisors estimate each bank's
expected losses and revenues and use these estimates to project post-stress capital levels
and ratios under the hypothetical scenarios. The estimated capital ratios are then compared
with regulatory benchmarks. We use common scenarios for all firms; for the firms with the
largest trading activities, we add a market-shock scenario that incorporates market
turbulence of severity similar to that of the latter half of 2008.
While the basic approach has remained consistent, much else has changed. In the first
place, of course, the requirement for stress testing has become statutory, as Congress drew
on the lessons of SCAP in crafting prudential standards for large financial firms in the
Dodd-Frank Wall Street Reform and Consumer Protection Act (Dodd-Frank Act).1 Second,
the annual supervisory stress test has been incorporated into CCAR, a broader program that
requires large banking organizations to submit a capital plan annually. The CCAR process
gives supervisors an opportunity to evaluate plans for capital distributions against the
backdrop of the firm's overall capital position under both baseline and hypothesized stressed
conditions. It also provides a regular, structured, and comparative way to assess the capacity
of these firms to manage their capital positions and, by implication, more generally to
manage their risks. I will return to this feature of CCAR a bit later.
Third, there have been substantial enhancements to the supervisory stress test. Perhaps the
most important change has been the development of independent supervisory models.
Because the original SCAP was developed on the fly and under considerable time pressure,
supervisors necessarily had to rely on firms' own estimates of losses and revenues as a
starting point for analysis, although they evaluated the firms' estimates and made significant
adjustments. In each stress test that has followed, and building upon the considerable
progress in data collection, we have made completely independent estimates of a
progressively greater proportion of potential net income or losses. These improvements in
data and models have increased our ability to distinguish risks within portfolios.
We have also refined the formulation of the hypothetical scenarios that form the basis of the
stress tests. The severely adverse scenario is designed to reflect, at a minimum, the
economic and financial conditions typical of a severe post-World War II U.S. recession.2 In
devising recession scenarios, we draw on many of the same macroeconomic modeling tools
used in making monetary policy. Because not all significant risks facing banks are tied to the
business cycle, our scenarios now incorporate other adverse developments such as an
exceptionally large decline in house prices, sharp drops in the value of stocks and other
financial assets, or a worsening of global economic conditions more severe than might
normally be expected to accompany a deep recession in the United States. We have
implemented the Dodd-Frank requirement for an "adverse," as well as a "severely adverse,"
scenario not by simply hypothesizing a milder recession, but by testing for somewhat
different risks. The past two years we have used the adverse scenario to test the impact of a
sudden, significant increase in interest rates.
More discrete changes of note include the assumption of default by each firm's largest
counterparty and the incorporation of salient risks beyond those in the overall scenarios.3 The
former obviously serves a microprudential purpose, but it also could promote systemic
stability objectives if it were to identify a single exceptionally large exposure for the entire
financial system. The incorporation of salient risks helps to use stress tests to "lean against
the wind," not just build a buffer for future losses. For example, the 2011 and 2012 scenarios
incorporated possible severe stress in Europe. Inclusion of this factor may have led to
greater awareness and better risk management of U.S. firms' European exposures.
Finally, I might mention a change made this year that actually reverted to a feature of the
SCAP. We did not allow banks to assume their balance sheets would shrink in the stress
scenarios as a way of meeting the minimum capital charge, something that many banks had
done in the stress tests of the past few years.4 Foreclosing this assumption serves the
macroprudential goal of helping to ensure that the major financial firms remain sufficiently
capitalized to support lending in a severe downturn.
As I have already suggested, we are likely to continue to hone the supervisory stress test. In
particular, I expect that we will devote more attention to developing the macroprudential
elements of the stress tests. For example, we might sharpen our focus on the risks to the
financial system of significant common exposures among firms. We have already adjusted
the market shock applied to the trading books of the six largest firms to ensure that firms are
not incentivized to hold significant amounts of certain assets simply because they performed
well in the second half of 2008.5 But there is more that could be done. One idea would be to
test whether individual firms that are revealed by the tests to be vulnerable to serious stress
might engage in asset fire sales that could produce knock-on damage to other firms. Another
would be to incorporate more assumptions pertaining to the increased cost of, or reduced
access to, funding in stressed environments, when lots of credit lines may be drawn
simultaneously.
In short, we do not regard the supervisory stress test and CCAR as finished products. In fact,
we should never regard them as finished products, since to do so would be to overlook
changes in the real economy, financial innovations, and shifts in asset correlations across
firms. But I think it fair to say that supervisory stress testing and the CCAR have already
made important contributions to financial stability and, in the process, have led the way in
transforming supervision of the nation's largest financial firms.
First, they have played a key role in strengthening dramatically the capital position of the
industry. The firms participating in CCAR have more than doubled their tier 1 common capital
since 2009, an increase of $500 billion of additional, high-quality capital in the U.S. financial
system. It is noteworthy that supervisors in other countries have themselves been moving
toward greater use of stress tests as a centerpiece of efforts to build strong capital positions
for their banks. Most members of the Basel Committee on Banking Supervision now use
some form of stress testing.6 The European Union and the United Kingdom are currently
conducting supervisory stress tests whose results will be publicly released.
Second, they have been the leading edge of a movement toward greater supervisory
transparency. The Federal Reserve's decision in spring 2009 to disclose the results of the
SCAP on a firm-specific basis proved to be an important step in re-establishing market and
public confidence in the U.S. financial system. In the Dodd-Frank Act, Congress endorsed
the practice of disclosing both supervisory and firm stress test results in normal times, as well
as in crisis periods. During the past five years we have progressively increased disclosures
of firm-specific information, the methodologies we use, and the details of the stress test
scenarios. From the outset, we have published the broad framework and methodology used
in the supervisory stress test, including information about the types of models we use.7 We
solicited comments on our design framework for scenarios in 2012, and incorporated into the
final document some of the ideas we received - such as providing more information about
the economic and financial conditions represented by paths of the variables in the
scenarios.8 And this past spring, we published the additional set of stress test results that
described how the firms would perform under the adverse scenario, in addition to the results
of the severely adverse scenario, which we had previously released.9
Because bank portfolios are often quite opaque and thus difficult for outsiders to value, this
information should allow investors, counterparties, analysts, and markets more generally to
make more informed judgments on the condition of U.S. banking institutions. Coupled with
other regulatory measures, this transparency should in turn increase market discipline. This
level of transparency also subjects us to greater outside scrutiny and analysis, a process that
increases our accountability as regulators and helps us improve our assumptions and
methodology over time. These choices for greater transparency in the supervisory stress
tests and CCAR have prompted healthy discussion on the merits of disclosure in other
supervisory areas.
As I think everyone in this audience knows, we have not disclosed the supervisory models
themselves. We do not want firms simply to copy our modeling in their own assessment of
risks and capital needs. And we certainly do not want them to construct their portfolios in an
effort to game the model - a kind of analogue to teaching to a test. But even though we do
not publicly release the models, we have put systems in place to ensure oversight and
accountability. The models are evaluated by a special model validation group made up of
experts within the Federal Reserve who do not work on the stress tests. We have also
created a Model Validation Council made up of outside experts to provide independent views
and advice.10
The third important effect of the supervisory stress tests and CCAR has been to pave the
way for other horizontal, simultaneous supervisory exercises. We created the Large
Institution Supervision Coordinating Committee (LISCC) in 2010 to implement an approach
to supervision of systemically important firms that was better coordinated, more data-driven,
and more focused on the largest institutions as a group.11 While CCAR includes a number of
firms that are not in the LISCC portfolio, the stress tests and CCAR have been the proving
ground for LISCC, and as the committee has evolved to administer these programs more
efficiently, these exercises have shown the way to other horizontal supervisory exercises or
assessments. Some, like the newer Comprehensive Liquidity Analysis and Review, which
focuses on assessing liquidity sufficiency and banks' internal liquidity risk-management
practices, are intended to be recurring. Others are ad hoc efforts responsive to more episodic
or transitory concerns.
Fourth, and in some sense an extension of the prior point, CCAR in particular has defined an
approach toward developing and maintaining better risk management within the banking
organizations subject to these exercises. Let me turn now address this topic in somewhat
greater detail.
Ensuring strong risk management
Here again, the origins of our current program lie in our experience with the SCAP in 2009,
during which we learned a great deal beyond just post-stress loss and capital numbers. The
horizontal, cross-firm nature of the exercise also allowed us to evaluate and compare the
effectiveness of the firms' processes for determining and addressing their own capital needs.
We discovered that, at the time, many of the firms had critical deficiencies in the fundamental
risk-measurement and risk-management practices necessary to assess their capital needs.
Many firms did not have a systematic program for assessing their capital needs and even
lacked the basic data on firm-wide risks and exposures needed to begin such a program. Not
surprisingly, there appeared to be a correlation between firms found to be insufficiently
capitalized to withstand further financial and economic stress and firms lacking effective
processes for assessing their risks and vulnerabilities.
In response to these deficiencies, the Federal Reserve initiated the annual CCAR process for
assessing the capital adequacy and internal capital planning processes of large, complex
bank holding companies (BHCs). CCAR is premised on the belief that thorough, rigorous risk
management must underpin all the activities of all such firms. Risks that are identified and
understood can be evaluated and, where appropriate, assumed with proper safeguards and
capital planning. Risks that are overlooked, misunderstood, or taken outside of a
wellconsidered and comprehensive risk-management framework plant the seeds for serious
trouble for individual firms and potentially for the financial system. Through CCAR, we have
sought to ensure not only that all large BHCs have strong capital positions as determined
through our supervisory stress test, but also that they have strong capital planning practices
that are appropriately focused on the capital needed to withstand possible losses from the
specific risks in each firm's business model. These processes, grounded in strong
quantitative and qualitative risk management, also give supervisors a clearer window into
each firm's activities and thus increase the effectiveness of regular supervision.
Since the first CCAR there has been notable improvement in the firms' capital planning
processes. Many firms have made meaningful investments in their risk-measurement
processes, including significant enhancements to their internal data and management
information systems. Many firms have adopted a formal framework to inform their capital
planning through an analysis of their vulnerabilities and capital adequacy under a range of
potential adverse scenarios. They have also taken steps to enhance the integrity of their risk
measures, analysis, and the decision-making around their capital levels and distributions.
Despite these advances, there is continuing need for improvement in the firms' capital
planning processes. Some firms still lack reliable information about their businesses and
exposures. Firms also are sometimes unable to measure or understand how stressful
conditions can change the performance of their material business lines. In particular, the
capacity to assess the impact of tail risks remains in need of further development at many
firms. These deficiencies are, in some instances, compounded by weak oversight by senior
management and boards of directors, and lack of effective checks and accountability in the
process.
The qualitative assessment in CCAR
The importance we attach to these risk-management and capital planning processes is
reflected in the component of CCAR known as the qualitative assessment. This assessment
covers a range of topics, including the extent to which the design of a firm's internal scenario
captures the specific risks from the firm's activities, the firm's methods for projecting losses
under stress scenarios, and how the firm identifies appropriate capital levels and plans for
distributions. As detailed in our CCAR regulation, where these processes are found to be
inadequate, or to raise safety and soundness concerns, they may form the basis for an
objection by the Board of Governors to the capital distribution plans of a firm.12 If a firm's
internal processes are unreliable, supervisors will necessarily be concerned that a firm may
not properly have assessed risks clearly and, where needed, assured adequate capital for
the many risks that cannot be fully captured by standardized stress tests.
As the process and expectations for the quantitative test have become better understood,
attention has shifted a bit to the qualitative assessment. To place that part of the CCAR in
context, let me make several additional points.
First, as I have already noted, many firms had a long way to go to meet high standards of
capital planning backed by strong risk management when we began CCAR. Given that initial
gap, we have allowed time for firms to work toward full achievement of those standards.
Thus, what may be perceived as a raising of the bar every year is better understood as our
effort to provide a demanding, but still realistic, glide path for firms to reach that goal. We do,
however, expect firms to continue to make steady progress each year.
Second, the horizontal nature of the qualitative assessment does not mean that every year
one or more firms must receive an objection on qualitative grounds. The comparative
approach helps ensure that firms are evaluated consistently and fairly. And it does allow us
to see where specific firms may be outliers from good practices followed in the rest of the
industry. But this is not a PGA tournament - there is no foreordained cut that some
participants will miss.
Third, while there are minimum standards for all CCAR firms, the standards are more
stringent for firms of greater systemic importance.13 It is not enough for the largest and most
complex banking organizations to meet only the minimum standards in CCAR. We expect
the more systemically important firms to establish and maintain the most sophisticated
riskmanagement and capital planning practices. Their risk management and capital adequacy
should be sufficiently strong to help ensure their resiliency to a range of unexpected stress
events, since their distress could pose a threat to the financial system and to the broader
economy.
Fourth, precisely because of its importance to our supervisory program, the qualitative
assessment will continue to be progressively more integrated into year-round supervision of
the CCAR firms. While some important features of capital planning are observable only
during the formal CCAR process, most of the risk-management and capital planning
standards incorporated in CCAR are operative and observable by supervisors throughout the
year. These should be an important focus of ongoing supervisory oversight and of
discussions between firms and supervisors. Only in unusual circumstances should
supervisors learn for the first time during CCAR of significant problems in the quality of the
capital planning processes, and only in unusual circumstances should firms be surprised at
the outcome of the qualitative assessment.
12
See 12 CFR 225.8(e)(2)(ii).
13
It is also worth emphasizing that both elements of CCAR - the qualitative and the quantitative assessments -
only apply to BHCs with $50 billion or more in total consolidated assets. The Dodd-Frank Act requires
companies with between $10 and 50 billion in assets to conduct an annual company-run stress test. The
Federal Reserve does not conduct a CCAR-like exercise to evaluate these company-run stress tests, although
we do review the firms' company-run stress tests and capital planning processes in the normal course of
supervision. This review is tailored to the smaller size, reduced complexity, and limited systemic risk of these
firms. See footnote 5, p. 14160, of Board of Governors of the Federal Reserve System, Federal Deposit
Insurance Corporation, and Office of the Comptroller of the Currency (2014), "Supervisory Guidance on
Implementing Dodd-Frank Act Company-Run Stress Tests for Banking Organizations with Total Consolidated
Assets of More Than $10 Billion but Less Than $50 Billion," final supervisory guidance (Docket No. OCC-
2013-0013), Federal Register, vol. 79 (March 13) (PDF). For other, smaller banking organizations - those
with $10 billion or less in total consolidated assets - there is no supervisory expectation or regulatory
requirement that these firms will conduct capital stress testing.
We have already taken steps to further this integration of CCAR and regular supervision. At
the end of each CCAR process our supervisors send to each firm a letter detailing their
conclusions concerning the qualitative assessment. To the extent weaknesses or areas for
improvement are identified, those letters provide a basis for regular stocktaking by both firms
and supervisors. More generally, last year we released a paper on our expectations for all
aspects of capital planning, providing greater detail than what is included in the annual
CCAR instructions.14
I anticipate that we will take additional steps to integrate ongoing supervisory assessments of
risk-management and other internal control processes with the annual CCAR exercise, and
to assure that communications in both directions are heard. One such step has just recently
begun: The committee chaired by senior Board staff that is responsible for the oversight of
CCAR, supported by the relevant horizontal assessment teams, will directly engage with
firms during the course of the year to evaluate progress in remediating weaknesses or other
issues identified in the post-CCAR letters. Increasingly, our regular supervisory work on
topics such as risk-identification and internal audit will focus on processes that are critical to
risk management and capital planning at the firms, areas of focus for CCAR. The aim of
these and additional measures is to make CCAR more the culmination of year-round
supervision of risk-management and capital planning processes than a discrete exercise that
takes place at the same time as the supervisory stress tests.
Finally, I would note that, to provide investors, counterparties, analysts, and the public with
better information on the meaning of an objection on qualitative grounds to a capital
distribution plan, we now release our decisions on each capital plan and, for firms whose
capital plans were objected to, provide a summary of the specific reasons for those
objections.15
Conclusion
Although strong capital regulation is critical to ensuring the safety and soundness of our
largest financial institutions, it is not a panacea, as indeed no single regulatory device can
ever be. Similarly, supervisory stress testing and CCAR, while central to ensuring strong
capital positions for large firms, are not the only important elements of our supervisory
program. Having said that, however, I hope you will take at least these three points away
from my remarks today.
First, supervisory stress testing has fundamentally changed the way we think about capital
adequacy. The need to specify scenarios, loss estimates, and revenue assumptions - and to
apply these specifications on a dynamic basis - has immeasurably advanced the regulation
of capital adequacy and, thus, the safety and soundness of our financial system. The
opportunities it provides to incorporate macroprudential elements make it, in my judgment,
the single most important advance in prudential regulation since the crisis.
Second, supervisory stress testing and CCAR have provided the first significant form of
supervision conducted in a horizontal, coordinated fashion, affording a single view of an
entire portfolio of institutions, as well as more data-rich insight into each institution
individually. As such, these programs have opened the way for similar supervisory activities
and continue to teach us how to organize our supervisory efforts in order most effectively to
safeguard firm soundness and financial stability.
Finally, supervisory stress testing and CCAR are the exemplary cases of how supervision
that aspires to keep up with the dynamism of financial firms and financial markets must itself
be composed of adaptive tools. If regulators are to make the necessary adaptations, they
must be open to the comments, critiques, and suggestions of those outside the regulatory
community. For this reason, transparency around the aims, assumptions, and methodologies
of stress testing and our review of capital plans must be preserved and extended.
With that point, I end where I began - by emphasizing the importance of forums such as this
one, and thanking you for your participation.