Simple Allocation Rules and Optimal Portfolio
Choice Over the Lifecycle*

Victor Duarte

Julia Fonseca

Aaron Goodman

University of Illinois

University of Illinois

MIT

Jonathan A. Parker
MIT and NBER
Preliminary
July 14, 2021

Abstract
We develop a machine-learning solution algorithm to solve for optimal portfolio choice ina detailed and quantitatively-accurate lifecycle model that includes many features of reality
modelled only separately in previous work. We use the quantitative model to evaluate theconsumption-equivalent welfare losses of using simple rules for portfolio allocation acrossstocks, bonds, and liquid accounts instead of the optimal portfolio choices. We find thatthe consumption-equivalent losses from using an age-dependent rule as embedded in current
target-date/lifecycle funds (TDFs) are substantial, around three percent of consumption, despite
the fact that TDF rules mimic average optimal behavior by age closely (until retirement). The
TDF portfolio does not improve on investing a constant 2/3 share in equity. Finally, optimal
equity shares have substantial heterogeneity, particularly by wealth level and dividend-priceratio, implying substantial gains to further customization of advice or TDFs, particularly inthese dimensions.
JEL codes:
Keywords: Portfolio allocation* For useful discussions on model components we thank Matthias Doepke, Eric French, Fatih Guvenen,
Rory McGee, Maarten Meeuwis, Mariacristina De Nardi, Lawrence Schmidt, Michele Tertilt, and Motohiro
Yogo. Duarte: Gies College of Business, 1206 South 6th Street, Champaign IL 61820, vduarte@illinois.edu;
Fonseca: Gies College of Business, 1206 South 6th Street, Champaign IL 61820, juliaf@illinois.edu; Goodman:
Department of Economics, 50 Memorial Drive, Cambridge MA 02142, agoodm@mit.edu; Parker: Sloan
School of Management, MIT, 100 Main Street, Cambridge, MA 02142, JAParker@mit.edu.

How should people allocate their portfolios? The robust proscription from frictionlessequilibrium models is that investors should invest in a weighted combination of themarket portfolio and a risk-free asset, where the weight is determined by the investor’srisk aversion relative the population. However, this advice is difficult to implement. The
market portfolio is hard to determine and construct. Prices and returns may be alteredby governments, institutions, and biased beliefs. Markets for insuring many individual
rises are incomplete. As a result, most investment advice is based on partial-equilibrium,
lifecycle models of saving and portfolio choice that take as given available investment
opportunities, as pioneered in Samuelson (1969) and Merton (1969, 1971). This approachis also amenable to the incorporation of individual preference changes and idiosyncraticrisks such as from non-traded labor income, home ownership and mortgages, healthand mortality risks, pension income, family dynamics, and taxes, each of which can bequantitatively important for optimal investor behavior.
In this paper, we develop a machine learning method to solve for optimal consumption
and portfolio allocation rules in a lifecycle model with a relatively complex economic
environment that includes all of the above factors as well as other realistic features of theinvestment problem facing the typical US household. In addition to age, the model has 22
state variables (some of which only directly matter during retirement or during workinglife). We model a household that consists of a husband and wife who consume bothhousing and non-housing consumption and are each endowed with with a gender-specificearnings profile that has stochastic, left-skewed, serially-correlated shocks. The household
allocates its financial wealth among a stock index, a bond index, and a money market
account, all with returns that are both serially-correlated and correlated with labor income.
Households can choose to rent or own with a mortgage or own outright, and pays a cost tosell or refinance. can in housing and borrow using mortgages, and can allocate some of
their financial investments into retirement accounts with limited employer matching. The
household faces a simple tax and benefit system and gets utility from bequests. During
retirement, each individual receives a pension that is a function of lifetime labor income,and faces mortality risk and stochastic medical expenses.
We use the model to study optimal lifecycle portfolio choice and to measure how well
simple portfolio rules, like those embedded in existing (low-fee, index) Target Date Funds(TDFs), do at approximating optimal-allocations and delivering similar levels of welfare forthe two-earner couples that we study.1 TDFs have become increasingly important for the
1 As such, our model applies to a relatively well-off subset of the US population that accumulates investable1

typical retirement investor, rising from less than $8 billion dollars in 2000 to more than $2.3
trillion dollars in 2019. Including balanced funds, which most TDFs turn into a few years
after the target date, implies that these funds manage more than $4 trillion, or roughly 5%of the US mutual fund market.2 We have three main substantive results.
First, our model suggests that on average, the share of financial wealth that the households we study should invest in equity is hump-shaped over the working life, peaking
around age 40 and 80% and declining thereafter to just under 60% at retirement. However,for retirement wealth, TDFs do a good job of matching the average optimal behavior. Whileon average, households optimally hold almost all their retirement wealth in equity when
they first start working, this share declines to 80% at age 45. A typical TDF maintains an
80-90% share in equity over this age range. After age 45, the average optimal share inequity declines linearly to about 50%, after which it is roughly constant. Equity shares in
TDFs typically continue to decline after retirement to 30-40%, but other than this difference,
TDFs are quite good at mimicking average optimal behavior in retirement accounts.
Second, the average optimal behavior masks substantial variation. The 90th percentileof optimal equity share is roughly 100% after age 45 for all financial wealth and is very
close to 100% through the entire working life for wealth in retirement accounts. The 10th
percentile of the distribution of optimal equity shares is much lower than that average
optimal share; the tenth percentile declines across ages from almost 100% at age 25 to only
20% at retirement. As a result, the portfolios delivered by portfolio rules that condition
only on age, like TDFs, are inappropriate for many investors, and increasingly so as
investors age, so that the differences are substantial in the period of life when people haveaccumulated the most retirement wealth.
Our third set of results quantify the loss, relative to optimal behavior, of investing agiven age-specific share of a household’s investable wealth in equity as in current TDFs.3
In our model, a household that is invests its retirement wealth following the portfolio of thetypical (index, low-fee) TDF loses the equivalent of 2.6% of consumption on average in any
wealth throughout life and does not become extremely wealthy either from inter-generational transfers or
own business income.
2 See Parker, Schoar, and Sun (2021b). The $2.3 trillion in TDFs includes $0.9 trillion in target date collective
investment trusts (CITs) which invest like TDFs but have lower fees than the equivalent mutual funds andare primarily used by large employers. Dollar amounts are from Investment Company Institute (2020),
figures 2.2 and 8.20, and Morningstar (2020).
3 While few investors currently hold all their retirement wealth in TDFs, the share invested in TDFs has
been steadily rising and is on average around two thirds for young workers in the late 1910’s (Parker, Schoar,
Cole, and Simester, 2021a). The TDF equity shares are also stable in that investors in TDFs do not re-allocate
into and out of TDFs following market returns (Parker et al., 2021b; Mitchell and Utkus, 2021).

2

given age (and state) when they can re-optimize all other behaviors. This loss rises to 3.4%of consumption if we do not allow the household to re-optimize their other behaviors.4
These welfare losses are similar to those calculated by Dahlquist, Setty, and Vestman (2018)
in a simpler model moving from age-based to completely optimal rules. While TDFs closely
mimic optimal age-contingent allocations, these losses are similar to those of a simple rulethat imposes a constant 2/3 equity share across all states and dates. We conclude that there
is substantial room for improving investor behavior further by conditioning on more statevariables, and from our analysis, dividend price ratios and wealth levels appear to be thestate variables that could add the most value. This conclusion quantifies the set of strengths
and weaknesses of TDFs discussed in Campbell (2016) (Section 5.1) as well as echoing theconclusion of Gomes, Michaelides, and Zhang (2021) that there are large welfare gains to
TDFs that take advantage of return predictability.
Our findings are made possible by our methodological contribution. Building on
Duarte (2019), we develop a method that uses deep learning to solve dynamic stochastic
models like this lifecycle model – models with many states and controls, and highly nonlinear policy functions – using a policy gradient algorithm (Sutton, McAllester, Singh, andMansour, 2000). In brief, we parameterize the policy functions over the high-dimensional
state space using using fully connected feedforward neural networks as cells for tworecurrent neural networks (one for working life and one for retirement). We use a stochastic
gradient descent algorithm to solve for the parameters of the networks that maximize theexpected lifetime utility over a large number of simulated sample paths. In contrast, thetraditional numerical dynamic programming (NDP) approach would first characterize
the solution to the household’s problem by a set of optimality conditions and budget
constraints, then construct grids on which choices can be characterized by matrices, andthen finally use an optimization algorithm and numerical integration to solve for optimal
behavior recursively. There are three advantages of our approach relative to traditional
NDP.
First, and foremost, our method is much faster than traditional NDP in complex applications and so can accommodate a large number of state variables and shocks, highly
non-linear policy functions, and both discrete and continuous actions. The gains in speed
come from our use of new tools from the field of deep reinforcement learning, the fieldof machine learning that studies sequential decision making. For example, optimizing
4 Both of these losses are smaller – 0.74% and 0.81% respectively – from the perspective of the householdat the start of life due to discounting.

3

expected utility using simulated sample paths avoid computationally-slow numericalintegration. Second, our method is far easier to use and program (and so less prone toerror) than traditional numerical dynamic programming methods. For example, there isno need to specify the density and scale of grids over which policy functions can then be
defined as matrices. Finally, this method arguably captures how investors, practitioners,
or data scientists actually determine optimal behavior. Optimal behavior is determinedby simulating and learning from the outcomes of a large number people living their lives
according to different portfolio allocation and consumption rules, that is, from watching
and learning from the lives of others. We describe our method in detail in Section 3.
While we apply our method to lifecycle consumption and portfolio choice, our method
may readily be applied to a wide range of partial-equilibrium dynamic problems faced bymanagers, employers, small open economies, etc.
Our quantitative model of lifecycle portfolio choice is also advances the large literatureon portfolio choice (see the surveys Curcuru, Heaton, Lucas, and Moore, 2010; Wachter,2010) and provides a baseline for future quantitatively studies of optimal portfolio choice.
While our model omits some features of reality, and while each individual ingredient is notnew, the model contains the features of the lifecycle problem that we judge to be both most
relevant and most important for lifecycle portfolio choice. Further, our model is calibratedusing state-of-the art estimates of the deterministic and stochastic processes facing theinvestor (e.g. DeNardi, French, and Jones, 2010; Wachter, 2010; Guvenen, Ozkan, and Song,2014)). In the conclusion we discuss missing elements and weaknesses that present avenues
for future improvements.
While our focus in this paper is on a particular subset of the population and theevaluation of sub-optimal decision rules, our paper contributes to a long line of quantitative
analyses of optimal portfolio behavior in lifecycle models. These analyses can be grouped
into two different types: those that focuses on inferring features and parameters of themodel from observed behavior, and those that offer proscriptive analysis of how investors
should allocate their portfolios. Bodie, Merton, and Samuelson (1991), Gakidis (1998),
Campbell and Viceira (2002) (Chapter 6), Gomes and Michaelides (2003), Storesletten,
Telmer, and Yaron (2004), Cocco, Gomes, and Maenhout (2005), Davis, Kubler, and Willen(2006), Catherine (2021) all pioneered analysis of lifecycle portfolio choice in the presence
income risk that cannot be fully hedged. Gomes, Michaelides, and Polkovnichenko (2006)
and Dammon, Spatt, and Zhang (2004) considered the role of taxes and tax deferred
accounts, Cocco (2005), Hu (2005), and Yao and Zhang (2005) incorporate housing, and4

Yogo (2016) focuses on the role of medical expenses during retirement.5
We cite the main papers that provide the ingredients for our model – both the mathematical structure and the calibrated parameters – as we describe each element of the model.
Similarly, we place our method in the machine learning literature when we describe ourmethod in section section 3.1.

1

The Lifecycle Model
Time is discrete and measures age in years. At each age, the household chooses how

much to save and how to much to consume of non-housing goods and housing. Households
receive labor income and earn investment returns that follow known stochastic processes.
Optimal consumption and portfolio choices are a set of decision rules for every age thatmaximize the household’s (mathematically) expected present discounted value of utility
flows from consumption given knowledge of the structure and parameters of the problem
and the history of realizations of stochastic processes up to that age.
Overview The complexity and the realism of the model come from the budget constraint
which has the following features. The income process for each spouse consists of a deterministic lifecycle profile subject to negatively skewed persistent and transitory shocks
during working life. During retirement, individuals receive pension income based on
lifetime earnings and face medical expense shocks. Households face an approximation of
the progressive US income tax and benefit system including a consumption floor during
retirement. Households can allocate their non-housing wealth among 3 financial assets
representing stocks, long-term bonds, and liquid saving. Each returns processes consists ofan idiosyncratic shock and a loading on a common autoregressive process based on the
dynamics of the dividend-price ratio of the aggregate stock market. Correlation amonglabor income and asset returns is captured by having both the dividend price ratio and thelabor income process depend on a stochastic business cycle state. Households can saveinto liquid financial accounts or retirement accounts, and the model contains a detailed
representation of the legal and institutional structure of retirement saving. The household
can rent or own housing, and there are adjustment costs associated with changing the
5 More recently Calvet, Campbell, Gomes, and Sodini (2021) include both housing and returns althoughperfectly correlated. Also of note, a subsection of Cocco et al. (2005) considers the role of medical expenses
while Koijen, Nieuwerburgh, and Yogo (2016) considers allocations across annuity and insurance products
rather than stocks and bonds.

5

size of an owned home or a mortgage. The household faces constraints on borrowing, on
short selling, on consumption from a liquid-wealth-in-advance requirement, and on theloan to value ratio of its mortgage when purchasing or refinancing. Finally, the household
experiences mortality shocks during retirement, and values leaving a bequest at death.
The most notable omissions from our model is that we study only a traditional family
consisting of a man and a woman with fixed ages of retirement and who remain married
until death do them part.
We set out the model structure and budget constraints below, starting with the household’s working life and then describing retirement. We conclude with the objective function
and statement of the complete problem. The calibration of the model is presented in Section 2.

1.1

Working Life

The household consists of two individuals, a man and a woman, indexed by i ∈ {1, 2},
who both work from age t = T0 = 25 until the exogenous retirement age t = TR = 65 andwho both survive until age TR with probability one. Households are also differentiated by
the deterministic component of each spouses labor income which depends on their rank in
the initial distribution of permanent income, which we denote by q10 and q20 . Varying the
choice of cohort and initial income quantile allows us to determine how optimal portfolio
choice decisions differ for individuals with differently-shaped income profiles and atdifferent places in the lifetime income distribution. Since this is a permanent characteristic,
we generally omit q10 and q20 as state variables for notational simplicity.
All random variables sub-scripted by t are realized before the households makes any
time-t decisions.
1.1.1

Common Risks

There are two ‘aggregate’ risks in our partial-equilibrium framework which create
correlations among stochastic processes. A stochastic process representing the businesscycle generates correlation among labor income and returns on different assets. Second,a stochastic process representing aggregate effective risk aversion generates correlation
among asset returns. These two aggregate processes are correlated.
First, the economy can either be in a recession or expansion, with the state variable
et = 1 indicating a recession and et = 0 indicating an expansion. The economy’s state et6evolves according to the 2x2 transition matrix Pe .
Second, we capture fluctuations in expected returns from serially-correlated fluctuationsin the log of the aggregate dividend yield (divided price ratio) of the stock market, vt .
vt = θv1 + θv2 ∆et+ + θv3 ∆et− + θv4 vt−1 + ev,t ,(1)

ev,t ∼ N (0, σv ),(2)where ∆et+ = max{0, et − et−1 } and ∆et− = max{0, et−1 − et }. The dividend yield follows
an AR(1) process, with an intercept that shifts when recessions start or end.
1.1.2

Labor Income

To model the household’s labor income, we follow the work done with Social Security
earnings records by Guvenen et al. (2014) (GOS) and Guvenen, Kaplan, Song, and Weidner(2017) (GKSW). Each individual’s log labor income, yit = ln(Yti ), depends on two factors:
a deterministic age profile ȳit , and a stochastic process of shocks around this profile that
follows GOS’s main parametric model. The deterministic profile is a function of gender i,age, and the initial income quantile for each individual, q0i , so that ȳit = ȳ(t; i, q0i ). To ease
notation, we suppress the dependence of labor income on q0i for the rest of Section 1.1.
In each period, log labor income for individual i is equal to the age profile, plus apersistent shock, xti , and a transitory shock, eti .
yit = ȳit + xti + eti ,(3)xti = ρ x xti −1 + ηti ,(4)eti ∼ N (0, σe ),(5)η i ∼ N (µη ,e , ση ) with prob. p
1
1 t
1
1,t
i
ηt =
η i ∼ N (µ, σ ) with prob. 1 − p .
2,tη2 ,etη2(6)

1

The innovation ηti in the AR(1) process for xti follows a mixture distribution, which allows
for the non-normalities (in particular, negative skewness and excess kurtosis) that Guvenen,
Ozkan, and Song document in earnings growth rates. Cyclicality (in particular, countercyclical negative skewness) enters the income process through the dependence of the mean
AR(1) innovations, µ1,et and µ2,et , on the economy’s state et .
To model taxes and convert the household’s gross income to net income, we use the7parsimonious tax function introduced by Heathcote, Storesletten, and Violante (2017)(HSV). The household’s tax liability as a function of combined gross income, Yt = Yt1 + Yt2
is given by

L(Yt ) = min{Yt − λYt1−κ , 0.5Yt }(7)where the parameter κ controls the progressivity of the tax schedule and the parameter λshifts the tax function to determine the economy’s average tax burden. Heathcote et al.
(2017) show that this simple tax function provides a very good approximation to the US
tax system (including federal, state, and payroll taxes), and we use the parameter estimatesreported in their paper. We cap the tax rate at 50% which occurs for our parameterization
just above $500,000.
1.1.3

Financial Assets

The household’s financial wealth is composed of assets held in liquid accounts, a L ,or a (relatively) illiquid retirement account, a I . The illiquid retirement accounts capture401(k) and other tax-deferred retirement saving accounts that receive special tax treatment.
Contributions to these tax-deferred accounts reduce the household’s current tax burden
and accumulate tax-free, but are taxed as income when withdrawn and are subject to a 10%
penalty if withdrawn before retirement.6 Households start life with no assets.
In each account, the household can save and invest in J = 3 financial assets: short-term
government bills (j=1), long-term corporate bonds, and equities.7 To incorporate realistic
return predictability in a tractable way, we follow Wachter (2010) and make each returncorrelated with the stock market’s aggregate log dividend yield, vt (equations (1) and (2)).
The log of the gross return on asset j at (the beginning of) time t, denoted by R j,t , is givenby:log( R j,t ) = θ 1j + θ 2j ∆et+ + θ 3j ∆et− + θ 4j vt−1 + e j,t ,(8)

e j,t ∼ N (0, σr,j ),(9)

6 We model traditional accounts only for now, and not the increasingly-common Roth-type accounts.

Contributions to Roth accounts do not reduce current taxes, but withdrawals during retirement are tax free.
7 Thus we do not consider diversification within stocks or bonds, an issue that can amplify risk (Fagereng,
Gottlieb, and Guiso, 2017).

8

where again ∆et+ = max{0, et − et−1 } and ∆et− = max{0, et−1 − et }. Each asset’s return
depends on an intercept term, which is allowed to differ at the beginning and end ofrecessions; the aggregate log dividend yield state variable; and a transitory shock.
We denote the wealth in asset class J in each account type, L and I, after returnsI
are realized at (the start of) age t but before age-t saving or withdrawals by a Lj,t and a j,trespectively so that:J

atL = ∑ a Lj,t

(10)

j =1
J

I
atI = ∑ a j,t

(11)

j =1

Define s xj,t to be the household’s net contribution to its holdings of asset j in account typex during period t (i.e., contributions net of withdrawals, after time t returns and incomesare realized and at the same time as consumption and other decisions are being made),with stx = ∑ jJ=1 s xj,t . We can then write the state-evolution equation for financial assets asJ

atL+1 = ∑ R j,t+1 ( a Lj,t + s Lj,t ),

(12)

j =1

J

I
I
atI+1 = ∑ R j,t+1 ( a j,t+ Γ(s j,t
)),

(13)

j =1

where Γ(·) is a function allowing for matching employer 401(k) contributions, with match
rate k and limit l until households reach retirement:Γ( x ) =



x




if x ≤ 0 or t ≥ TR

(1 + 2k ) x x + k lY2

if 0 < x ≤ lYt(14)

if x > lYt .

t

We divide the match rate k by 2 to save on state variables and yet to capture the idea that
not all jobs match contributions. Annual contributions to tax-deferred retirement accounts
are capped as in US law:I
stI ≤ smax,t
.

9

(15)

I
The contribution limit smax,tis constant during the working life, then becomes negative(and age-dependent) beginning at age 70 in order to reflect the IRS schedule of required
minimum distributions.8
Finally, households cannot short any asset (other than with a mortgage, as described
subsequently), so holdings in each asset class and account type must be nonnegative:a Lj,t ≥ 0,

(16)

I
a j,t
≥ 0.

(17)

Now define Ba,t to be the net contribution to the household’s time t budget constraintof all decisions regarding portfolio choice and saving in financial assets. We have:J

Ba,t = −stL − stI − L0 (Yt ) ∑j =1

R j,t − 1
R j,t!
a Lj,t − ξ (stI , t) + L0 (Yt )max{0, stI }(18)

where L0 (Yt ) is the marginal tax rate, the first derivative of the tax function,9L0 (Yt ) = 1 − λ(1 − κ )Yt−κ .

(19)

and where ξ ≥ 0 is a function describing the penalty associated with early withdrawal(s I < 0) of retirement assets and the taxes paid pre-retirement:.
−( L0 (Yt ) + 0.1)s Iξ (s I , t) = − L0 (Yt )s I



0

if s I < 0 and t < 60
if s I < 0 and 60 ≤ t(20)

if s I ≥ 0.

8 We maintain the assumption that households behave optimally other than the constraints we explicitly

introduce, so that the primitives of the problem contain no references to default retirement saving rates or
portfolios, features that are relevant for actual behavior and welfare (Beshears, Choi, Laibson, and Madrian,
2008; Choukhmane, 2021).
9 Note that for simplicity, and to match the reality that households effectively face constant marginal tax
rates with the piecewise linearity of the US tax schedule, we take the marginal tax rate on withdrawals
during working life to be the derivative of the tax function evaluated at the household’s current labor income(and do not account for second-order effects where withdrawals change income and thereby change themarginal tax rate). Note also that we simply tax realized returns in taxable accounts at the marginal income
tax rate rather than including separate dividends or capital gains rates. Since withdrawals from the illiquid
retirement account are likely to be a substantial share of income during retirement, we treat withdrawals
during retirement equivalently to other income which accounts for second-order effects during retirement(see equation (41)).

10

Net contributions to asset holdings (i.e., savings) enter negatively into the budget constraint,
as do taxes incurred on the realized returns on taxable asset holdings and any taxes or
penalties incurred by withdrawing from retirement accounts. The cost of contributing totax-deferred retirement accounts is partially offset by the reduction in current tax liability,
which enters positively into the budget constraint. Note that ξ includes the 10% penalty
associated with withdrawals from the illiquid account prior to age 60, but does not include
the regular taxation of withdrawals post-retirement since (as described in Section 1.2.1) wetreat these withdrawals as equivalent to labor income for tax purposes.
Finally, we impose a cash-in-advance constraint that reflects the fact that people holdliquidity for transactions related to consumption. We do this in order to capture thetransactions demand for liquidity, and to prevent the household from (unrealistically)financing normal consumption expenditures by making repeated withdrawals from its
illiquid retirement savings accounts. In particular, as long as the household has any wealthin its illiquid retirement account, it cannot consume more than a fixed multiple of its liquid
wealth held in short-term debt:L
ct ≤ Ma1,t
if atI > 0.

(21)

where ct denotes non-housing consumption.
1.1.4

Housing

Our model of housing choices is an expanded version of the structure of Berger, Guerrieri, Lorenzoni, and Vavra (2018) (BGLV). Households consume non-housing consumption
ct and housing ht . At any point in time the household can choose to either rent or own ahome, with ot = 0 indicating renting and ot = 1 indicating ownership. Households startlife as renters. The stock of housing owned by the household is given by h̃t , with h̃t = 0
when the household is a renter. Thus ht = h̃t if ot = 1 and it will turn out that households
choose ht > h̃t = 0 if ot = 0. Households who own housing stock may take on mortgage
debt, dt , and they also receive an additional utility benefit from any level ht of housing.10
We define by Bh,t (ot−1 ) the net contribution to the household’s time t budget constraint of
all housing decisions which depends on whether the household entered the period as arenter or owner.
If the household is a renter and consumes housing ht , it pays a rental cost of φpt ht
10 This utility benefit is not in BGLV and is discussed subsequently as part of the objective function.

11

where φ is the constant rent-to-house-price ratio and pt is the price of housing (relative tothe numeraire non-housing consumption good ct ). Following Berger et al. (2018), pt is ageometric random walk with drift:
pt+1 = νt pt ,(22)

log(νt ) ∼ N (µν , σν ).

(23)

Renters who become homeowners make down payments equal to the cost of the houseless the amount of the new mortgage which is subject to a loan-to-value constraint. They
must also pay for maintenance on their new house, δh̃t , to cover depreciation.
Thus, for households that enter the period as renters (ot−1 = 0), we have:
Bh,t (0) = −(1 − ot ) [φpt ht ] − ot pt h̃t − dt+1 + δpt h̃t
= 0
if ot = 0
d t +1≤ (1 − ι) p h if o = 1.
t t
t

(24)
(25)

The mortgage, dt+1 , must stay at zero if the household remains a renter. If the household
becomes a homeowner, its mortgage choice is restricted by a loan-to-value constraint:
households buying a new house (or refinancing, below) must satisfy dt+1 ≤ (1 − ι) pt ht ,so that the loan-to-value ratio is at most 1 − ι. Note that, in order to match the real-world
housing market, the loan-to-value constraint is only imposed when a house is purchased
or a mortgage refinanced. An existing homeowner whose house declines in value is neverforced to refinance or accelerate payment on its mortgage.
If the household enters the period as an owner and either adjusts its housing stock or
becomes a renter, it pays a transaction cost of f h pt ht−1 (i.e., a fraction f h of the housing
stock it inherited from last period and is now selling, valued at the current house price).
Households that remain homeowners must pay maintenance costs. Continuing homeowners must also make mortgage payments if they do not refinance. The amortization
schedule of mortgages is described by the parameter χ: a household with current mortgage
debt dt that does not refinance must service its debt by making a payment of at least χdt .
Alternatively, continuing homeowners can adjust their mortgage debt up (refinance) bypaying a transaction cost of f d pt ht (a fraction f d of their house’s current value).

12Thus, for households entering the period as homeowners, ot−1 = 1, we have:Bh,t (1) = −ot [δpt h̃t + (1 − L0 (Yt ))( R1,t − 1 + ∆rm )dt+1{h̃t =h̃t−1 , dt+1 >dt } f d pt h̃t + 1{h̃t 6=h̃t−1 } pt (h̃t − (1 − f h )h̃t−1 )]d t +1 =+(1 − ot )[(1 − f h ) pt h̃t−1 − φpt ht ] − (dt − dt+1 )=0
if ot = 0
≤ (1 − ι ) p t h t



≤ χd
t

if ot = 1 and (h̃t 6= h̃t−1 or dt+1 > χdt )

(26)

(27)

otherwise.

Depreciation costs enter negatively into the budget constraint, as does tax-deductible
mortgage interest (paid at the short-term interest rate plus a mortgage spread of ∆rm )and principal repayment. If the household adjusts its owned housing stock h̃t (either bychoosing a new nonzero value or choosing h̃t = 0 and becoming a renter) it pays the priceof its new house (if any) and receives the selling price of its old house net of transaction
costs. Finally, if it maintains its current housing stock but refinances its mortgage, it pays
refinancing costs and gains the additional funds borrowed in liquid wealth (since the lastterm, dt − dt+1 , is negative). Equation (27) states that if the household becomes a renter
then next period’s mortgage must be zero. If the household adjusts its housing stock orrefinances, then the household can choose any mortgage amount allowable under the
loan-to-value constraint. Otherwise, the household can choose any mortgage amount
below that required by the amortization rule.
1.1.5

Working-life Objective Function and Unified Flow Budget Constraint

We can now write down a complete statement of the household’s working life problem.
To account for the fact that consumption needs change with family size, consumption
enters utility in per-effective-householder form, where the effective size of the household
is wt , the square root of the average family size at each age. As described in Section 2, thisprofile is such that household size declines to 2 at retirement. Total consumption at eachage is a Cobb-Douglas aggregate of non-housing consumption ct and housing consumption
ht with share parameter α, where housing consumption is the service flow from housing,φht .11 We assume constant relative risk aversion utility of total consumption in each year,with risk aversion coefficient γ.
11 φ plays no role whatsoever so we set it to the rental rate.

13

Define the discount factor as β and the retirement value function, which returns the
maximized expected discounted utility during retirement, as VR (·). Additionally, as a statevariable summarizing the household’s liquid financial resources, define “cash on hand” Qt
as the liquid wealth available to the household at the beginning of time t, net of taxes andmortgage interest payments:J

Qt = atL + Yt − L(Yt ) − L0 (Yt ) ∑j =1

R j,t − 1
R j,t!
a Lj,t − (1 − L0 (Yt ))( R1,t − 1 + ∆rm )dt(28)

Policy functionsnare functions of the complete set of state variables at theobeginning of
L , aI , p , oeach age t: Ξt = et , vt , { xti }2i=1 , Yt , {∑tt0 =T0 yit0 }2i=1 , Qt , a1,tt t−1 , ht−1 , dt . The sum ofteach household member’s income up to time t is a state variable because several stochastic
processes during retirement, discussed below in Section 1.2, depend on individuals’ lifetime
labor earnings. The cash-in-advance constraint in (21) makes the liquid holdings of shortterm debt (in addition to cash on hand) a state variable.
The household chooses age-specific policynfunctions for consumption, portfolio allocao

JI
tions, home ownership, and mortgage debt, ct , ot , ht , { a Lj,t+1 } jJ=1 , { a j,t+1 } j =1 , d t +1

as afunction of the state variables at each age, Ξt , to maximize the expected discounted sum oftime-separable flow utility. Defining the consumption aggregate as
Ct = cαt (φht )1−α ,the household maximizes:
"
E

TR −1

∑ β

t= T0

γ −1 1− γCt

t− T0 wt

1−γ

(29)

#

+ β TR −T0 VR (Ξ TR ) Ξ T0(30)

subject to the comprehensive flow budget constraint
Yt − L(Yt ) − ct + Ba,t + Bh,t (ot−1 ) = 0,(31)

the constraints (16)-(15) and (21), and where the income process is defined in (3)-(6), the tax
function is defined in (7), Ba,t defined in (18)-(20), and Bh,t (ot−1 ) is defined in (24)-(27).

14

1.2

Retirement

When the household reaches retirement, it continues to makes consumption, saving,and portfolio-choice decisions to maximize expected discounted utility over the remainderof its life. However, following DeNardi et al. (2010) (DFJ), we make several changes toreflect the changing financial risks that households confront as they age: income becomes
deterministic pension payouts rather than stochastic labor earnings, stochastic health costs
become a significant household expenditure, and individuals face mortality risk. Let νtiequal 1 if individual i is still alive at time t and 0 otherwise (the exact mortality process forνti is described below).
1.2.1

Pension Income, Medical Expenditures, and Mortality

Instead of labor income during retirement, each individual receives non-stochastic
income representing Social Security and other pension income and faces stochastic expenditure shocks representing medical expenses. Following DFJ, each individual’s pension,
health, medical expenditure, and mortality processes depend on the individual’s permanent income rank. We denote this rank by qiR and calculate it as the quantile into which theindividual’s realized lifetime labor earnings, ∑ TT0R −1 yit , fall in the distribution of life-timelabor earnings for the distribution including all q0i (as described in subsection 2.5).
Pension income is given by
Yti = νti Y (t; qiR ),(32)

so that income Yti is a deterministic function of age during retirement, although stochastic
from the perspective of working life since Yti depends on lifetime earnings. We postpone
the treatment of taxes until section 1.2.2 because withdrawals from retirement accounts aretaxed as regular income. We set pension income Yti to zero when individual i is no longer
alive.
Health shocks and stochastic medical expenditures follow DFJ. We introduce new statevariables, gti , representing individual i’s current health status: gti = 0 indicates “healthy,”
while gti = 1 indicates “sick.” Health status transitions depend on previous health status,
gender, and age, withPr( gti = 1) = Pg ( gti −1 , i, t; qiR ).

15

(33)

Each individual’s out-of-pocket medical expenditures mit (i.e., net of Medicare and otherinsurance) depends on their health status gti . Family medical expenses are the sum of theindividual expenses but is capped at m̄. The process for medical expenses is given by:
mt = min[m1t + m2t , m̄](34)

log(mit ) = νti m( gti , i, t; qiR ) + νti σ ( gti , i, t; qiR )ψti ,(35)i
ψti = ζ ti + eψ,t
,

(36)

i
ζ ti = ρζ ζ ti −1 + eζ,t
,

(37)

i
eψ,t
∼ N (0, σψ ),

(38)

i
eζ,t
∼ N (0, σζ ).

(39)

Log medical expenditures are the sum of a health, sex, and age specific mean cost m( gti , i, t)
and a shock term ψti , magnified by a health, sex, and age specific volatility term σ( gti , i, t).
The medical expenditure state variable ζ ti follows an AR(1) process. Medical expendituresare zero once an individual dies, mit = 0 if νti = 0.
Turning to mortality, each person’s probability of death depends on their age, sex,
income rank at retirement, and health status. If individual i is alive, vit = 1, else vit = 0.
Conditional on having survived through age t, each individual i survives, vit+1 = vit , withthe health-, sex-, and age-specific probability Pd ( gti , i, t; qiR ). Following DFJ, anyone still
alive at age Tmax = 102 dies with probability 1.
To reflect means-tested government programs such as Medicaid that support the elderly,
the household is guaranteed a minimum level of consumption when necessary expenditures like realized medical expenses and mortgage payments are sufficiently high. This
government transfer rule is described by0 if a L + a I > 0 or ot 6= ot−1 or ht+1 6= ht or dt+1 6= χdt
t +1
n t +1ont =cmax 0, + m + τ − Y + L(Y ) − B − B (o )otherwise,
t
t
t
t
a,t
t
h,t
wt

(40)

where nt is the government transfer and c is the consumption floor (adjusted by theeffective consumption shifter). The household is not allowed to receive government transfer
payments if it carries any financial wealth to the next period (atL+1 + atI+1 > 0), adjusts
its housing consumption (ht+1 6= ht ), changes its home ownership status (ot 6= ot−1 ),or refinances or makes a larger-than-necessary payment on its mortgage (dt+1 6= χdt ).

16

Otherwise, when the household’s budget constraint is tight enough to push consumption
below c, the government transfer makes up the difference.
Finally, we continue to assume that the household pays taxes during the year as during
working life, but also makes an additional tax payment or receives a refund based on theactual tax due calculated from a more accurate version of the true non-linear tax system.
As during working life, taxes paid on income in a given year are a non-linear function of Y
plus the marginal tax rate (based only on Y) applied to non-retirement capital income, any
retirement saving withdrawals, and (as a benefit) mortgage interest payments. However,during retirement, withdrawals of retirement saving may be substantial and they also
affect the tax rate on Social Security income. Thus we calculate the complete tax bill that
results from including withdrawals from illiquid accounts in the nonlinear tax function.
The household must then pay as taxes (or receive as a refund) in the following year any
difference between the linear approximation that it pays in t and the actual tax owed basedon the non-linear calculation.
The (nonlinear) tax on income and withdrawals from retirement accounts in retirement,L̃, is given by:
L max{0, −stI }




I
L̃(Yt , st ) = L 0.5Yt + max{0, −stI } L 0.85Y + max{0, −s I }tt

if 0.5Yt + max{0, −stI } < I0.5
if I0.5 ≤ 0.5Yt + max{0, −stI } < I0.85(41)

if 0.5Yt + max{0, −stI } ≥ I0.85

which reflects the tax treatment of Social Security benefits: either 0%, 50%, or 85% of Social
Security income is taxable, depending on the level of “combined income” (half of Social
Security income plus illiquid asset withdrawals).12 The difference between this and what
is paid during the year is the tax bill (or refund) that the household must pay in t + 1:
τt+1 = L̃(Yt , stI ) − L(Yt ) − L0 (Yt )s I .

(42)

12 This is only approximately correct because non-asset income, Y , from DFJ includes not only SocialtSecurity benefits but also defined-benefit pensions and annuities which are actually taxed as ordinary income(in the same way as illiquid asset withdrawals −stI in our model). This approximation is close however
because Social Security is all or most of Y for households over 65, averaging 3.5 times the income from private
pensions and annuities (Social Security Administration, 2016, pages 222-223).

17

1.2.2Financial Assets

During retirement the household does not face withdrawal penalties on its illiquid
assets nor does it receive employer matching on retirement contributions (as stated fort ≥ TR in Equations 14 and 20). However, it must pay income taxes on withdrawals fromthe retirement account (as given in equation 20 and is subject to the IRS’s rules regarding
required minimum distributions from retirement accounts after age 70 (equation 15). Thus,
Ba,t , as defined in Equation 18, summarizes the contribution of financial assets to the budget
constraint.
1.2.3

Housing

During retirement, the part of the budget constraint relating to housing and mortgages,
Bh,t (ot−1 ), as well as the constraints on these choices are exactly the same as during working
life.
1.2.4

Objective Function and Unified Flow Budget Constraint

We can now write down a complete statement of the household’s problem during
retirement. Following DFJ, we define utility over bequests, bt , asub (bt ) = b̃( bt + b ) 1 − γb,
1 − γb(43)

which the household receives when the longer-lived spouse dies at age t and where b̃
captures the intensity of the bequest motive, b shifts the curvature of utility function andallows bequests to be a luxury good, and γb is risk aversion over bequests. Mortality isrealized at the beginning of the period, so the bequest amount is given by
bt = atL + atI − L
where atL + atI − L

 I
at
5

 I
at
5

− τt + max{(1 − δ − f h ) pt h̃t−1 − dt , 0},(44)gives financial assets at the beginning of the period net of taxes

paid by the recipient on inherited pre-tax wealth,13 τt gives the carry-forward tax bill
determined in the previous period, and the final term gives the current value of the
13 Under current US tax law, inherited 401(k) or IRA wealth must be withdrawn over 10 years and is taxed

as income when withdrawn. Because generally the inheritor will also have labor income, we suppose a tax
bill as if the inheritor withdrew over only 5 years.

18

owned housing stock h̃t−1 carried from the previous period (net of the beginning-ofperiod mortgage debt, depreciation, and the transaction costs associated with selling thehouse). The maximum operator around the last term means that underwater mortgages(i.e., (1 − δ − f h ) pt h̃t−1 − dt < 0) cannot cause the household’s bequest to be less than its
financial wealth net of taxes. Additionally, if the carry-forward tax bill is large enough tomake the bequest value in (44) negative, we set it to zero.
As in the pre-retirement period, consumption needs depend on household size. When√both members of the household are still alive (v1t = v2t = 1), wt = 2. When the shorterlived spouse dies, wt = 1, reflecting the one-member
household’s diminished consumptionq

v1t + v2t = 0. We also allow consumption

needs. When the second spouse dies, wt =utility to depend on the health status of household members. In particular, to reflect
the difficulty of maintaining a home and the increased likelihood of moving to assisted
living environments when in poor health, the utility benefit of homeownership relative torenting decreases if the household’s members are in the “sick” state. We implement thisutility reduction by multiplying homeowners’ housing consumption ht by a scaling factorũ < 1, raised to the number of household members who are sick so that we redefine theconsumption aggregate as1

2

Ct = cαt (ũot ( gt + gt ) φht )1−α .

(45)This equation collapses to (29) during working because we assume everyone is healthy sothat gt1 + gt2 = 0.
The set of state variables during retirement is different from that during the working
life in two ways. First, in place of the state variables for forecasting labor income, xti ,and pension income, ∑tt0 yit , are state variables for summarizing pension income andforecasting medical expenses, qiR and ζ ti . Second, the cash-on-hand state variable mustaccount for medical expenses mt and the previous year’s owed taxes τt . Denoting themodified retirement cash-on-hand variable as QtR , this gives
QtR = Qt − mt − τt ,(46)

where Qt is defined in (28). Finally, the health status (gti ) and survival (νti ) states of thehousehold’s members are now state variables.14
14 Thus,duringretirement

the

state

at

19

age

t

is

given

by

Ξt

=

Given Ξ TR at retirement, the household chooses the set of policy functions for consumption, portfolio allocations, home ownership, and mortgage debt as a function of Ξt at every
age to maximize:
"
E

Tmax

∑ βt−T0t= TR

γ 1− γw C( bt + b ) 1 − γ
1 { w t >0} t t+ 1{wt =0, wt−1 >0} b̃
1−γ
1−γ

!

#
Ξ TR

(47)

subject to a modified version of the budget constraint during the working life (31) which accounts for the fact that the household faces medical expenditures, may receive governmenttransfers, and pays any additional tax liabilities from the previous year:
Yt − L(Yt ) − ct − mt + nt − τt + Ba,t + Bh,t (ot−1 ) = 0.

(48)

The household is subject to the contribution and withdrawal constraints (16)-(15) and(21). Income is given by (32), the tax function is defined in (41), the process for medical
expenditures is defined in (34)-(39), the benefit nt is defined by (40), b is given by (44),Ba,t and Bh,t (ot−1 ) are defined as during working life by (18)-(20) and (24)-(27), and finallyconsumption is given by (45).
The retirement value function referenced in the working-life objective (30), VR (Ξ TR ), is
given by the maximized value of (47).

2Model Parameters
The complexity of our setup creates a large number of parameters that we must calibrateor estimate before proceeding to solve the model. Wherever possible, we take parametervalues directly from the same literature that we use to construct the model itself.15 We
discuss our parameterization decisions here and provide a complete list of parametervalues in Table I. All dollar amounts in the paper are given in 2013 dollars and are inflationadjusted using the PCE deflator (following GKSW).
We parameterize the model to match a married couple both born in 1924 (the youngest
cohort included in DFJ’s data) and their working lives run from age 25 (in 1949) to age 65 (in1989). We adjust all dollar amounts to 2013 dollars. Households begin life as renters withno
L , a I , { g i }2 , { q i }2 , { ζ i }2 , { ν i }2 , p , oet , vt , Yt , QtR , a1,ttt i =1t i =1t i =1 t t −1 , h t −1 , d t .
R i =115 The online appendix contains the inputs that cannot be neatly listed in Table I because they are vectors or
matrices.

20

no financial assets, no housing wealth, no mortgage debt, and the persistent component oflabor income variance xiT0 set to 0.

2.1

Common Risks

We use official NBER recession-dating betweeen 1915 and 2015 to estimate the recession
transition matrix Pe . To estimate the dividend yield process in (1) we use data on U.S.
dividend yields during the 1915-2015 period from Jorda, Kroll, Kuvshinov, Schularick, andTaylor (2019).

2.2

Labor Income and Taxes

We take all parameter values for the labor income process in (3)-(6) directly from GOS.
To specify the deterministic age profile, we use the summary statistics that GKSW reportin their appendix. In particular, GKSW compute a series of quantiles (the 25th, 50th, 75th,
and 90th) of the log income distribution, by gender, age (between 25 and 55) and cohort(those that turned 25 between 1957 and 1983). After specifying a gender i, an initial income
quantile q0i , and a cohort k, we simply read off the relevant 31-observation time series fromGKSW and use it as our age profile, so that ȳit = ȳ(t; i, q0i , k ).16
Because the oldest cohort for which GKSW report income summary statistics turned
25 in 1957, and because GKSW only report summary statistics through age 55, we must
perform some interpolation to populate our deterministic income profile for the 25-32 and
56-65 age ranges. For the 25-32 age range, we simply use the data that GKSW report for the1957 cohort, adjusted by the average annual rate of real wage growth during the 1950s.17
For the 56-65 age range, we use the predicted values obtained from regressing the GKSW
income data between ages 25 and 55 on a second-order polynomial of age.
The tax progressivity parameter κ comes directly from HSV, and we set the shift parameter λ so that tax liability becomes positive at the same real income level as in HSV.
16 In their baseline sample, GKSW impose minimum-participation requirements and include only individu-

als that earn sufficiently high income in a sufficient number of years. We use the male income profile derived
from GKSW’s baseline sample. However, given that our targeted cohorts worked during a period of risingfemale labor force participation, we prefer not to assume that all of our simulated households have a second
earner working full-time during the entire working life. We therefore use the female income profile derived
from an expanded sample in GKSW that does not impose minimum-participation requirements. This allows
us to capture empirical patterns of female labor-force participation during the period of study.
17 We obtain real wage growth rates from Peake and Vandenbroucke (2020).

21

Table I: Parameter Values

Parameter
T0
TR
Tmax
Pe
θv1 − θv4
θ 1j − θ 4j
ρxµη1 ,0µη1 ,1µη2 ,0µη2 ,1
ση1
ση2
σe
p1
λ
κ
k
l
I
smax,t
M
δ
φ
fh
fd
χ
ι
µν
σν
∆rm
ρζ
σζ
σψ
m̄
c
γ
b̃
b
ũ
m
I0.5
m
I0.85
s
I0.5
s
I0.85
α
γ
β
wt

Value
25
65
102
see appendix
see appendix
see appendix
0.979
0.119
-0.102-0.026
0.094
0.325
0.001
0.186
0.490
5.5
0.181
0.5
0.06
see appendix
12
0.022
0.060
0.050
0.012
0.969
0.100
0.012
0.039
0.03
0.922
0.224
0.815
1,000,000$3,9573.84
23.6
369,000
0.90$32,000$44,000$25,000$34,000
0.888
3.84
0.96
see appendix

Description
beginning of working life
retirement age
maximum attainable age
business cycle transition matrix
dividend yield processreturn processes for j ∈ {long bond, short bond, equity}
labor income persistence
first labor income innovation mean, expansion
first labor income innovation mean, recession
second labor income innovation mean, expansion
second labor income innovation mean, recession
first labor income innovation std. dev.
second labor income innovation std. dev.
labor income transitory shock std. dev.
labor income mixture probability
income tax level
income tax progressivity
employer 401(k) match rate
employer 401(k) match limit
401(k) contribution limit & req. min. distributions
cash-in-advance constraint on liquid wealthhousing stock depreciation rate
rent-to-house-price ratio
housing stock adjustment transaction costmortgage refinancing transaction costmortgage amortization speed
mortgage down payment requirement
mean house price innovation
house price innovation std. dev.
mortgage interest rate spread
medical expenditure shock persistence
medical expenditure persistent shock std. dev.
medical expenditure transitory shock std. dev.
family medical expenditure cap
retirement consumption floor
risk aversion
bequest utility intensity
bequest utility intercept
homeownership scaling factor for poor healthSocial Security 50% tax cutoff for married households
Social Security 85% tax cutoff for married households
Social Security 50% tax cutoff for singles
Social Security 85% tax cutoff for singles
Cobb-Douglas utility share of non-housing consumption
relative risk aversion
discount factor
effective family size scaling factors

Sourcecalibrated
calibrated
DFJ
estimated
estimated
estimated
GOS
GOS
GOS
GOS
GOS
GOS
GOS
GOS
GOS
HSV
HSV
calibrated
calibrated
US tax code
calibrated
BGLV
BGLV
BGLV
BGLV
BGLV
BGLV
BGLV
BGLV
calibrated
DFJ
DFJ
DFJ
calibrated
DFJ
DFJ
DFJ (2016)
DFJ
calibrated
US tax code
US tax code
US tax code
US tax code
BGLV
DFJ
calibrated
calibrated

Notes: The following process from DFJ are presented in the appendix: deterministic retirement pension income profiles, Y (t; qiR ); health status transition probabilities, Pg ( gti −1 , i, t; qiR ); mean medical expenditure profiles, m( gti , i, t; qiR ); medical expenditure volatility profiles, σ ( gti , i, t; qiR ); survival probabilities, π ( gti , i, t; qiR ).
Additional parameter values are as listed in the text of Section 2.

2.3

Financial Assets

We estimate the asset return processes in (8) with U.S. data over the 1915-2015 period.
We take data on short-term government bills and equities from Jorda et al. (2019) and
use the Dow Jones Total Corporate Bond index to estimate the long-term corporate bondprocess.
We set k = 0.5 and l = 0.06 in the employer-matching rule (14) to reflect the common
employer policy of matching 50% of contributions up to 6% of income. As reflected in(14), we divide k by 2 on the assumption that only one spouse has access to employer
matches at any given time, yielding an effective match rate of 0.25. We set the 401(k)Icontribution limit smax,t
for t < TR in equation (15) to $20, 603 reflecting the highest IRS

limit that our target cohort experienced during their working lives (in 1989), adjusted for
inflation and multiplied by 2 to account for the two-member household.18 We calibrate the
cash-in-advance parameter M to 12 so that the household must carry the equivalent of 1
months’ worth of consumption in safe, liquid wealth.

2.4

Housing

We take all parameter values for the housing part of the model directly from BGLV,
except for two small changes. First, we estimate the µν and σν parameters governing thehouse price process using the housing capital gain series in Jorda et al. (2019) since 1985.
Second, we add the mortgage interest spread ∆rm and set it to 0.03.

2.5

Retirement

We take all parameter values for the pension income, health status, medical expenditures, and mortality processes directly from DFJ. The tax liability parameters I0.5 and I0.85
I
in (41) and the required minimum distributions (smax,t
for t ≥ TR in equation (15)) are set

based on IRS regulations.
We determine qiR as follows. For each initial quantile q0i , we simulate paths for income
during the working life from the earnings process (described by equations (3)-(6)). We thus
obtain a simulated distribution of working-life incomes. When an individual in the model
reaches retirement, we sum their total labor income and let qiR be their rank within thissimulated distribution of cumulative income. We then base Y (t; qiR ), and the other DFJ
18 In reality, the maximum contribution rises each year but we have a real not a nominal model.

23

processes described below, on qiR : specifically, we use the DFJ processes estimated at thepermanent income quantile closest to qiR .19

2.6

Objective Functions

Following DFJ, we set both coefficients of relative risk aversion (γ for the consumption
utility function and γb for the bequest utility function) to 3.84. We also use DFJ’s estimatedvalues for the bequest utility parameters b̃ and b (adjusting the latter for inflation).20 Finally,
we set the discount factor β to 0.96.
To construct our age profile of family size, we follow Salcedo, Schoellman, and Tertilt(2012), who use decennial census data to compute average family sizes for 5-year age
buckets (see their Figure 3). We set our family size equal to 2 at age 65, then apply theaverage growth rates implied by the estimates in Salcedo et al. (2012) to fill in our profileback to age 25. wt is the square root of this average family size.
We set the homeownership scaling factor ũ to 0.9, so that effective housing consumption
is reduced by 10% if the household is a homeowner and one member is sick, and by 19% if
the household is a homeowner and both members are sick.

3

Solution Method
To solve the investor’s optimization problem, we build on the framework of Duarte(2019) which develops a method to use policy gradient algorithms to solve high-dimensional
problems in economics and finance. In general, policy gradient methods work by parameterizing the agent’s decisions with flexible functional forms and optimizing the expectedvalue of a stochastic reward (see Sutton and Barto, 2018) . In our setting, the stochastic
reward is the household’s lifetime utility.
We parameterize the investors’ policy functions as fully connected feedforward neuralnetworks and update the networks’ parameters with stochastic gradient descent. Thismachine learning approach greatly improves the efficiency and speed of the algorithm
relative to a more traditional numeric dynamic programming approach of defining policy
19 DFJ estimate their processes at each fifth percentile, so we choose the DFJ quantile in {0, .05, .1, ..., .95, 1}

that is closest to qiR .
20 De Nardi, French, and Jones (2016a) find somewhat different parameter values for the bequest function
for risk aversion closer to 3. DeNardi, French, and Jones (2016b) discuss how the parameters of the bequest
function are not well identified due to precautionary saving motives, at least when identification comesprimarily from the structural model and data on saving choices.

24

functions over grids of state variables and performing numerical integration to compute
expectations. These efficiency gains make it feasible to solve models as rich and complex
as the one described in Section 1.
There are however two key challenges that arise in applying machine learning tools
to solve our complex dynamic stochastic problem (and also many other problems ineconomics and finance).
The first challenge is the presence of both discrete and continuous choices in a modelwith large state and action spaces. With only continuous choices, it is possible to compute
the gradient of the objective function with respect to the parameters of the neural network
representing policy functions using automatic differentiation, an algorithm used in mostmachine learning applications. With these gradients in hand, one could then adjust network
parameters in the direction of the gradient, which is the direction that (locally) increases theobjection function the fastest.21 However, with discrete choices, the objective function is notdifferentiable with respect to network parameters and we cannot make use of automaticdifferentiation.
We overcome this first challenge by parameterizing the probability that a household
makes a certain discrete choice, instead of parameterizing the discrete choice itself. Thismeans that, while training our networks, discrete choices are stochastic, but the policy
functions in our solution are deterministic and map states into the discrete choices thatmaximize the objective function. We describe this approach in detail in Section 3.1.
The second challenge is the combination of the long length of life and the variance andpersistence of shocks (as well as the significant number of state variables). These featurespose a problem because stochastic gradient descent uses simulated data (lifetimes of shocks)
to compute approximations of the gradient of the objective function with respect to network
parameters. And from these approximations, determine how to optimally adjust these
network parameters. In an environment with a large finite horizon and high-varianceshocks, the variance of these gradients is very large, making it hard to approximate themthrough simulations.
We address this second challenge by including time as an input of our network, insteadof having a separate network for each time t, which is equivalent to using a recurrent
neural network. With this structure, changing a parameter of the network affects actions
across all time periods, and thus increases gradients without increasing their variance,
making them easier to estimate. We discuss this further in Section 3.2.
21 For a more detailed description of this approach, see Duarte (2019).

25

Further, because the set of control variables and the set of state variables both change atretirement, we use separate networks for the pre-retirement and post-retirement periods oflife.
In the remainder of this section, we expand on this solution algorithm, describing
how we simulate the model, choose network parameters to maximize lifetime utility
over simulated sample paths, and deal with discrete choices. Finally, we present thearchitecture of our neural networks, which are chosen due to their computational efficiency,
and describe how we implement this solution algorithm.

3.1

Algorithm

Let π be a policy function representing the choices of a household. The expected lifetime
utility associated with policy π is
"
Eε∼D

#

TD

∑ βt ut (πt (Ξt ), Ξt ) + βTD B(TD , ΞTD ) ,

(49)

t= T0

where ut is the flow utility at time t, B is the bequest function, TD is the stochastic age at
which the longest surviving member of the household dies, Ξt is the set of state variables
at time t, and ε is the collection of all shocks. Shocks and initial states are sampled from the
distribution D .
Our goal is to find policy functions that maximize lifetime utility. To do so, weparametrize a given policy π using a neural network
π t ( Ξ t ) ≡ π ( Ξ t ; Θ π ).

(50)

For simplicity, we refer to the collection of parameters for all networks as Θ and to the sumof discounted utilities given a collection of parameters and shocks as R(Θ, ε). Given thisparametrization, we define the loss function as

L(Θ) = −Eε∼D [ R(Θ, ε)] .

(51)

The set of parameters Θ is chosen to minimize the loss function. To find this minimum,
we use stochastic gradient descent and proceed iteratively. For a set of initial parameters Θ,
assuming that the loss function is differentiable with respect to Θ, one step of standard (non-26

stochastic) gradient descent adjusts the parameters according to the following equation∆Θ = −α∇Θ L(Θ),(52)

where α denotes the learning rate and ∇Θ denotes the gradient with respect to Θ. The
idea behind gradient descent is to move Θ in the direction that (locally) reduces the loss
function the fastest. However, since some of choices represented by π are discrete, the loss
function given by Equation (51) is not differentiable with respect to Θ.
We circumvent this issue by relying on Parameter-Exploring Policy Gradient methods(Sehnke, Osendorfer, Rückstieß, Graves, Peters, and Schmidhuber, 2010). Instead of
searching for a single vector of parameters Θ, these methods consist of searching for a
distribution of parameters. Specifically, we assume
Θ ∼ N (µ, σ2 I ),(53)

where µ is a vector with the same dimensions as Θ, σ is a scalar, and I is the identity matrix.
We chose µ so as to minimize the loss function, which we can now define as

L(µ) = −EΘ∼ N (µ,σ),ε∼D [ R(Θ, ε)] .

(54)

As in Salimans, Ho, Chen, Sidor, and Sutskever (2017), we treat σ as a fixed hyperparameter, initially setting σ to 0.001 and annealing it to zero during training. Importantly,as σ approaches zero, our algorithm settles on a deterministic vector of parameters Θ andpolicy functions map a set of state variables into deterministic actions. Sehnke et al. (2010)
show that the loss function given by Equation (54) is differentiable with respect to µ evenwhen the original loss function given by Equation (51) is not, and is given by∇µ L(µ) = −EΘ∼ N (µ,σ),ε∼D ∇µ log ϕ(Θ) R(Θ, ε) ,(55)

where ϕ is the probability density function of Θ.
It is infeasible to compute the expectation in the equation above for our model. Instead,the insight of stochastic gradient descent is to approximate this expectation with a small
i.i.d. sample of simulated paths. Accordingly, we adjust µ at each iteration by∆µ ≈ α1 N∇µ log ϕ(Θi ) R(Θi , ε i )
N i∑
=1

27

(56)

and proceed by adjusting parameters according to this equation at each iteration until asuitable convergence criteria is met.

3.2

Network Architecture

We parameterize policy functions using two neural networks, one for the working age
period and one for the retirement period. Every possible action the agent takes is modeled
as a sequence of two hidden layers and one output layer. We illustrate this structure forthe portfolio choice of an agent during the working age period in Figure I. As mentioned
in Section 3.1, this network takes as inputs (represented as green nodes in Figure I) time(t) and the set of state variables. It produces as outputs (represented as red nodes) the
portfolio shares of the three assets.
Figure I: Network Architecture

Input
layer

Hiddenlayer 1

Hiddenlayer 2

Output
layer

t
et
vt
..
.
pt

α1,t
..
.

..
.

α2,t
α3,t

h t −1dt

Note: This figure illustrates the architecture of a neural network representing the portfolio choice at time t ofan agent during working life.

Note that this network has two hidden layers (represented by blue nodes in Figure I).
Each node in the first hidden layer is a non-linear transformation of a linear combination28

of the inputs. We normalize inputs using a slow-moving average of inputs over the first
10,000 iterations.22 Each node in the second hidden layer is a non-linear transformation ofa linear combination of the nodes of the first hidden layer. Finally, each output (or node)of the output layer is a non-linear transformation of a linear combination of the nodes ofthe second hidden layer. These non-linear transformations are commonly referred to asactivation functions.
The three portfolio shares illustrated in Figure I sum to one due to our choice ofsoftmax as an activation function for the output layer.23 The softmax function is a function
σ : Rn → Rn such thatσ ( x )i =e xi

x , for i ∈ {1, . . . , n }∑nj=1 e j

(57)

This function normalizes a vector x into a vector of non-negative fractions that sum toone. The usefulness of this transformation is immediate in our portfolio choice example, inwhich the agent must choose how much of his financial wealth to allocate across the three
assets and the no-borrowing constraints of Equations 16 and 17 imply that portfolio sharesmust be non-negative.
We also use a softmax activation function when representing a household’s decisionto split his disposable income between consumption, financial investment, housing, anddebt repayment. This implies that each of these variables will be a fraction of disposable
income and, consequently, the budget constraint will always be satisfied. The activation
function for the output layer associated with discrete choices is also a softmax, and theagent chooses the discrete case associated with the highest output.
Finally, as illustrated in Figure I, time is an input of our network and we do not havea separate network for each time t. The intuition for this structure is similar to what is
known as parameter sharing (Caruana, 1993), recognizing that an output at time t will bevery similar to an output at time t + 1. This reduces the number of network parameters
and increases efficiency (Tsitsiklis and Van Roy, 2001). In particular, this increases themagnitude of gradients without increasing their variance, since changes to a parameter
will affect actions across all time periods. This allows us to better approximate gradientswith a small i.i.d. sample of simulated paths.
22 As in any application using neural networks, it is important for our algorithm that inputs have similar
magnitudes. In our setting, some inputs are endogenous, so we do not know ex ante their respective meansand variances. We thus use a slow-moving average of simulated states in this normalization.
23 For the hidden layers, we use a hyperbolic tangent (tanh) activation function.

29

3.3

Solution

We run code to implement this solution in Google’s TensorFlow Research Cloud, acloud service intended for researchers.24 At each iteration, we sample 2,048 vectors ofparameters according to the distribution given by Equation 53.25 As we describe above, avector of parameters determines the policy functions of an investor and, for each vector ofparameters, we simulate the lives of 256 investors who make decisions according to thosepolicy functions. This amounts to simulating the lives of 524,288 investors (256×2,048) per
iteration.
As we describe in Section 2.2, the deterministic age profile in our labor income process
is conditional on an initial permanent income percentile q ∈ {25, 50, 75, 90}. We solve for
optimal behavior separately for each permanent income percentile assuming that both
members earn labor income according to the percentile q process.26 The code that solves
for optimal behavior given a permanent income percentile takes approximately 24 hours to
run on Google’s TensorFlow Research Cloud.

4

Lifecycle portfolio behavior
This section characterizes the optimal portfolio choices over the life implied by themodel and shows two main results. First, the average share of financial assets optimallyinvested in equity starts quite low but rises rapidly early in life, peaking around age 45 anddeclining thereafter. For retirement wealth, the average optimal share invested in equity
starts high, and overall looks a lot like the glide path of a typical TDF. Second, however
optimal portfolios differ significantly depending on other state variables, most importantly
wealth level and the ‘aggregate’ variables that affect expected stock and bond returns.
As just described in Section 3, our algorithm involves simulating the shocks, statevariables, and choices of a large number of households. We use the final simulations at the
optimal parameter values (optimal policy functions) to characterize optimal behavior andoutcomes. We approximate the population of interest by combining the lifecycles of statevariables and actions of households from each of the four percentiles (q) of average income
24 The TensorFlow Research Cloud can be accessed at https://www.tensorflow.org/tfrc.
25 We follow Google’s recommendations for optimal performance and define the size of all arrays

as multiples of 128. More information on the architecture of Google’s cloud system is available athttps://cloud.google.com/tpu/docs/system-architecture.
26 This amounts to an assumption that assortative marriage matching causes all individuals to have thesame permanent earnings capacity as their spouse.

30profiles (ȳi ). We take a random subset of households from the q = 25 group to represent
the lowest 37.5% of the permanent income distribution, and appropriately lower numberof households for each other percentile so that the q = 50 represents the next 25% of thepermanent income distribution (between the 37.5th and 62.5th percentiles), the q = 75
group represents the 20% of the distribution between the 62.5th and 82.5th percentiles, andthe q = 90 group represents the top 17.5% of earners.
To interpret dollar amounts, we note again that our model applies to a married, twoearner, couple born in 1924 and amounts are 2013 dollars. We plot choices and outcomesfrom age 27 (so that initial conditions to not drive some figures) to the end of working life
or until 85 (a household can survive until age 102 with very low probability).

4.1

Optimal Behavior

Figure II.a plots the average level of non-housing consumption (c), total consumption
expenditures (c plus mortgage and homebuying transaction costs, medical expenses, andrent or rental equivalent (φh)), and the consumption aggregate per effective family mem1

2

ber, cαt (ũot ( gt + gt ) φht )/wt .27 Because returns are high relative to households impatience,
because of precautionary saving, and because of matching of retirement saving, the firsttwo measures rise rapidly until the mid-40’s and all three measures rise until retirement.
Non-housing consumption, c, falls after retirement while average total consumption expenditures is roughly flat and the difference between the two series is occasional large
medical expenses (which become more likely as people become older. Average per capita
consumption rises steadily but more slowly throughout working life and then accelerates
in old age largely in response to death of one spouse. Total consumption expenditures
average about $30, 000 at the start of life, rise to just over $90, 000 during mid-life, andare around $110, 000 including medical expenses at age 85. Again these figure pertain topeople retiring in 1989 and are in 2013 dollars.
Figure II.b shows the 10th, 50th and 90th percentiles in the distribution of consumption
expenditures by each age. There is substantial increase in dispersion until around age 45, at
which time consumption inequality remains roughly constant until the retirement period
when it is steadily increasing due to medical expenses and death of one spouse. In mid life
27 We exclude maintenance costs from consumption expenditures since they are part of the rental equiva-

lence and when someone hits the consumption floor their consumption expenditures are c

31

Figure II: Average Consumption, Saving Rates, and Wealth by Age(a) Consumption and Consumption Expenditures(b) Distribution of Consumption Expenditures

Consumption Expenditures

200,000

Amount90,00060,00030,00025

3545

Non−Housing Cons.

55
Age

65

75

Cons. Expend.

150,000

100,000

50,00085

25Cons. Agg./Capita

35

45

Mean

10th Percentile(c) Saving Rates

55
Age

65

75

85

90th Percentile(d) Wealth

0.10
2,000,000

1,500,000
AmountSaving Rate

0.05

0.00

−0.05

1,000,000

500,000

−0.10

0
25

30

35

40

45
Age

Liquid Savings

50

55

60

65

Illiquid Savings

25

35

Retirement Wealth

45

55
Age

Liquid Wealth

65

75

Home Equity

85

Mortgage

Notes: Consumption expenditures are defined as: non-housing consumption, health expenditures, house andmortgage transaction costs, and rent or rental equivalence (φh). The consumption aggregate per capita is theargument of the CRRA period utility function. Saving rates are averages of x L /Y and x I /Y dropping familylabor incomes below $10,000 and are plotted from 25 to 64.

the 10th percentile of expenditures is only about $45,000 in a year while the 90th percentile
is about $180,000.
Turning to saving rates, Figure II.c shows that in their first few years of working life,
young households build a little liquid wealth but do not save much on average in liquid
wealth until age 40. Until late in working life, households primarily save in retirement
accounts, with retirement saving rates rising from 2.5% at age 35 to peak at 5% at age
50. The average saving rate in retirement accounts turns negative as people approachretirement because those agents having saved the most in retirement accounts, experiencing
32

Figure III: Financial Wealth by Age(a) Distribution of Financial Wealth(b) Retirement Share of Financial Wealth
Retirement Share of Financial Wealth

1.0

Total Financial Wealth4,000,000

3,000,000

2,000,000

1,000,000

0

0.8

0.6

0.4

0.2

0.0
25

35

45

55
Age

Mean

10th Percentile

65

75

85

90th Percentile

25

Mean

35

45

Ratio of Means

55
Age

65

10th Percentile

75

85

90th Percentile

Notes: Financial wealth is liquid wealth plus retirement wealth.

the highest returns, and getting low labor income start withdrawing substantial amounts
which lowers the average despite most households still contributing to their retirement
account prior to retirement.
Figure II.d shows the average wealth accumulated at each age. On average, households
build liquid and illiquid wealth steadily and rapidly during their working lives, and at a
slower ate during retirement. The share of households that are homeowners remains very
low until around age 35, at which age the home ownership rate rises over the next 20 years
so that by (and during) retirement just over 60% of households own homes. Mortgage debt
peaks about age 50, but overall remains small relative to average wealth (partly becauseof the low homeownership rate, partly because wealthy households hold little mortgagedebt). At retirement, average wealth is just above $2 million, with less than half of that inretirement accounts. Because II.d averages each type of wealth, Figure III does not capture
the experience of the typical family (it is a wealth-weighted average share).
More importantly, underlying these averages are substantial differences in financial
wealth across households at every age. Figure III.a shows that, during retirement, the 90th
percentile of the wealth distribution is double the average, above $4 million, and the 10th
percentile is half the average, only around $500,000. Further, wealth inequality rises during
retirement as the wealthy earn high returns and save for bequests while the medical costs
matter more for relatively low-wealth households who deccumulate wealth as they age.
While the average household holds most of its financial wealth in retirement accounts from33

Figure IV: Distribution of Equity Shares of Wealth(a) Equity Share of Financial Wealth(b) Equity Share of Retirement Wealth
1.0
Retirement Wealth Equity ShareTotal Financial Wealth Equity Share

1.0

0.8

0.6

0.4

0.2

0.0

0.8

0.6

0.4

0.2

0.0
25

Mean

35

45

Ratio of Means

55
Age

65

10th Percentile

75

85

25

90th Percentile

Mean

35

45

Ratio of Means

55
Age

65

10th Percentile

75

85

90th Percentile

age 30 to 60, III.b shows that both early in working life and during retirement households
hold most of their wealth in liquid assets. As agents accumulate financial wealth, they
accumulate more in retirement accounts than in liquid wealth so that the average (andmedian) household maintains more than 70% of its financial wealth in retirement accounts
between ages 40 and 55. However, the top 10 percent of households hold nearly all theirwealth in retirement accounts at all ages above 35, while the bottom ten percent at each age
only have about half their wealth in retirement accounts in their early 40’s and have muchless at most other ages.
Turning to our main focus, portfolios, the average optimal equity share in total financial
wealth is hump-shaped over the lifecycle but the average optimal share of equity inretirement accounts is quite similar to that provided by TDFs. Figure IV.a shows that theshare of financial wealth at the beginning of life is below 30%, a low level driven entirely by
non-retirement wealth and due primarily to the need for liquidity. But the average equity
share rises rapidly with age to over $80% by the early 40’s, then declines to roughly 60% at
retirement and remains steady thereafter. Figure IV.b shows that the optimal equity sharein retirement accounts looks quite different, and is strikingly similar to that proscribed bytarget date funds. For young households, the average equity share in retirement accounts
is 100%. That optimal share declines to reach 80% at age 45, and thereafter declines more
rapidly to just over 50% at retirement with only a slight decline therafter.
Importantly however, there is a lot of variation in the optimal equity share across people
at each age, variation that is far from the homogeneous allocations provided by current

34

Figure V: Optimal Equity Shares of Retirement Wealth Over the Business Cycle(a) Equity Share by State of the Cycle(b) Equity Share by Dividend-Price Ratio
1.0
Equity Share, Retirement Wealth

Equity Share, Retirement Wealth

1.0

0.8

0.6

0.4

0.2

0.0

0.8

0.6

0.4

0.2

0.0
25

35

45

55
Age

Expansion

65

75

85

25

Recession

35

45

Quartile 1 (lowest)

55
Age

Quartile 2

65

75

Quartile 3

85

Quartile 4

TDFs. Figure IV.a shows that the 90th percentile of the optimal equity share in financial
wealth rises rapidly early in life to reach 95% in equity at the start of the households 40’s,
and remains between 90% and 100% for the remainder of life. For retirement wealth, the90th percentile of the optimal equity share remains at 100% until just prior to retirement
when it declines slightly to 90%. In contrast, the 10th percentiles of optimal equity share of
retirement wealth in equity falls from 100% to below 40% before age 50, and reaches 20%
at retirement, declining only slightly more thereafter.
The differences in optimal portfolio share arise from differences in the economic circumstances. We now turn to the reasons why optimal equity shares vary across people.

4.2

Optimal Equity Shares of Retirement Wealth by State

This subsection shows how the substantial variation in optimal equity share at each age
shown in Figure IV relates to differences in economic circumstances. We focus on equity
shares in retirement accounts, but Appendix Figure A.1 shows that the differences in equity
shares across values of state variables for financial wealth are similar to those for retirement
wealth shown in this subsection, just with average lifecycle profiles reflecting Figure IV.a
instead of Figure IV.b.
The largest cause of differences in optimal portfolio at each age are differences in theaggregate state variables that determine expected returns. Figure V.a shows that equity
shares are about 10% higher in a recession than an expansion. Differences in response
to variation in the dividend price ratio are even larger. Figure V.b shows that when the35

(a) Equity Share by Transitory Income Shock(b) Equity Share by Expected Income Profile

1.0

1.0
Equity Share, Retirement Wealth

Equity Share, Retirement Wealth

Figure VI: Optimal Equity Shares of Retirement Wealth

0.8

0.6

0.4

0.2

0.0

0.8

0.6

0.4

0.2

0.0
25

30

35

40

Quartile 1 (lowest)

45
Age

50

Quartile 2

55

60

Quartile 3

65

25

Quartile 4

Percentile 25(c) Equity Share by Total Financial Wealth

45

55
Age

Percentile 50

6575Percentile 75

85

Percentile 90(d) Equity Share by Housing Leverage
1.0
Equity Share, Retirement Wealth

1.0
Equity Share, Retirement Wealth

35

0.8

0.6

0.4

0.2

0.0

0.8

0.6

0.4

0.2

0.0
25

35

45

Quartile 1 (lowest)

55
Age

Quartile 2

65

Quartile 3

75

85

25

Quartile 4

35

45Quartile 1 (lowest)

55
Age

Quartile 2

65

Quartile 3

75

85

Quartile 4

dividend price ratio is in the bottom quarter of its distribution – so expected stock returns
are high – the share of retirement wealth invested in equity should be below 40% on
average for households over 45 and as low as 20% for households over 70. When thedividend-price ratio is in the top quarter of its distribution, the equity share should be
100% for these households.
In contrast, there are only small variations in the optimal portfolio of equity whenhouseholds receive good or bad transitory income shocks (Figure VI.a). There are also only
small differences across percentiles of the income distribution which are associated withdifferent lifecycle profiles of average income (Figure VI.b).
Figure VI.c shows more significant differences in portfolio allocations associated withdifferent wealth levels, with wealthier households maintaining higher equity shares until
around age 40 and lower equity shares thereafter, relative to high wealth households. There36

are two reasons that high wealth households hold less of their retirement wealth in equity.
First, the risk aversion in the bequest function is higher (because of the intercept) than in
the flow utility function, so high wealth households are more risk aversion once likely
to leave bequests. Second, high wealth households have less bond-like pension income
relative to their accumulated assets, so they hold more of their accumulated in bonds.
Figure VI.d shows that at retirement, homeowners in the top quartile of housing
leverage hold about 10% more of their retirement portfolios in equity than households inthe bottom quartile. Finally, as shown in the appendix, optimal equity shares of retirement
wealth and of financial wealth do not vary significantly by health or mortality status during
retirement (see Appendix Figure A.2).28

5

The Welfare Costs and Benefits of Simple Portfolio Rules
Suppose that the household were either not sophisticated enough to make reasonable

portfolio choices or not interested in spending the time and effort to make them. How
much would the household lose by instead following simply portfolio rules? Such simple
rules are embedded in Target Date Funds (TDFs) which now serve as default investment
options in most employer-sponsored defined-contribution retirement plans.
We begin by considering how much worse the household would do if their retirementsaving were allocated to stocks and bonds following the mix proscribed by current TDFs.
Specifically, following closely the design of the Vanguard Target Retirement Fund series,
we impose that up to age 40 retirement wealth is invested 90% in stocks and 10% in bonds.
From age 40, the equity share declines 1.5% per year to reach 60% at age 60, then declines
by 2.5% per year to reach 49.5% at age 65, and then declines by 3% per year to reach 32% at
age 70. The equity share remains at 30% from age 71 onward.29
Panel A of Table II shows how much a household would hypothetically pay in percentof consumption (in every state at every age) to be able to (costlessly) follow the optimal
portfolio rules rather than the TDF rules in retirement wealth. Columns 1 – 5 show thesefigures from the perspective of the household at age 25, both for the average household
28 Also, between age 35 and 55, as households transition from renting to home ownership, renters should

consistently hold very close to 100% of their retirement wealth in equity while the optimal equity share forhomeowners declines linearly from 93% to 78% over these ages on average.
29 TDFs do not all follow the same glide path, and do not all deliver the same returns, so that different TDFs
with the same target date give different returns (Balduzzi and Reuter, 2019) and can have quite different
post-fee performance (Shoven and Walton, 2020; Brown and Davies, 2020).

37

Table II: Consumption certainty equivalent loss of imposing simple rules on retirement portfolio

(1)
(2)
(3)
(4)
(5)
Age 25 certainty equivalent
Other behaviors
Income percentile
not re-optimized All hhs 25th 50th 75th 90th

0.60
0.63
0.63

(6)
(7)
(8)
(9)
(10)
Avg. flow utility certainty equivalent
Income percentile
All hhs 25th 50th 75th
90th

Panel A: Impose TDF on retirement portfolio
0.74 1.02 0.74
2.61
2.33 2.70 2.97
0.80 1.12 0.83
3.19
2.85 3.23 3.84
0.80 1.14 0.84
3.43
3.10 3.55 4.64

2.65
3.11
3.24

None
Liq. portfolio
All

0.74
0.81
0.81

None
Liq. portfolio
All

Panel B: Impose constant 2/3 equity share on retirement portfolio
0.79
0.59 0.81 1.10 0.81
2.64
2.14 2.71 3.38
2.78
0.60 0.83 1.16 0.87
2.73
2.27 2.94 3.18
2.92
0.82
0.82
0.60 0.84 1.18 0.87
3.06
2.40 3.36 4.03
2.91

Notes: Average family is a weighted average of the given percentiles as described in the text (the 25th percentile
represents 32.5% of the population, . . . the 90th percentile represents 17.5%.) The certainty equivalent at age
25 is the percent reduction in consumption at all ages and possible outcomes in the original problem thatdelivers the same expected present discounted value of utility at age 25. The certainty equivalent for average
flow utility is the same calculation with β = 1.

and for households at different points in the distribution of deterministic income profiles.
This perspective discounts future flow utility at the discount rate β = 0.96 per year and soputs more weight on flow utility when young than when old. Columns 6 – 10 report theresults of the same calculations but for an equally-weighted average of flow utility acrossthe household’s life, as if β = 1, so that the flow utility of the household when old receives
equal weight to the flow utility of the household when young.30
The utility loss from following a TDF’s proscriptions rather than the optimal statedependent portfolio rule would cost the average household 0.8 percent of all their consumption in present discounted value or from the perspective of age 25, and between 2.6
and 3.4 percent of consumption in any year chosen at random. We reach this conclusionfrom several different calculations.
First, we impose the TDF portfolio on retirement wealth and allow the household to
re-optimize all other behaviors. The first row of Panel A shows that the average householdloses the equivalent of only 0.74 percent of consumption from the perspective of age 25.
But this under-weights the utility of the household when they are old which is when theeffects of an incorrect portfolio choice are the largest. That is, from a welfare perspective,
30 In this calculation, we simply use the objective function and include an equal percent loss in bequests.

38this measures the cost to a 25 year old rather than considering equally the perspectives ofthe family at other ages. Column 6 shows that when weighting flow utility at different
ages equally, the consumption equivalent is a much larger 2.61 percent of consumption at
all ages and in all state of the world.
However, this first experiment is in some ways inconsistent. We are implicitly assumingthat the household is somehow unable or unwilling to optimize its retirement wealthportfolio, but at the same time we allow the household to optimize its portfolio choice for
non-retirement wealth conditional on holding a TDF in its retirement account. The TDF
might be inferred to do little because the household in the model to some extent ‘unwinds’
the portfolio imposed by the TDF with their portfolio choice in their non-retirement, liquid
account. Thus we run a second experiment where we impose the TDF investment choices
also on liquid wealth. As shown in row 2 of Panel A in Table II, in practice this increases
the welfare loss from the TDF, by 7 basis points of consumption from the perspective ofage 25, and by just under 0.6 percent from the equally-weighted lifetime perspective. The
reduction is significant, but not as large as imposing the investment behavior only in theretirement account because most households accumulate the majority of their financial
wealth in their retirement accounts and because the cash in advance constraint keeps low
wealth households from taking too much risk.
An alternative way to impose that the household does not unwind the TDF allocations
using liquid wealth is to impose that the household simply uses its the old decision rules
for all choices other than its portfolio of retirement wealth. That is, we impose the TDF
shares in retirement wealth and keep the remaining behaviors of the household for allother decision variables – optimized for the own choices of retirement portfolio – the same
(conditional on state variables). As shown in the third row of Panel A of Table II, this leads
to a loss equivalent to roughly 0.81% of consumption from the perspective of household atage 25 and to a larger 3.43% of consumption for the average household simply averaged
across all years of life.
Panel A of Table II shows that all of these losses tend to be largest for households inthe upper middle class. Because high net worth individuals tend to use personal financial
advisers and because low income households have less access to 401k plans (or are offeredplans with lower match rates than we have modelled), this is a segment of the populationthat has leaned most heavily on TDFs. Finally, Panel B of Table II shows that the glide
path of the TDF does not actually improve outcomes for the average investor relative tothe pre-existing advice to maintain an age-independent share of wealth in equity, advice
39still embedded in balanced funds. Panel B of Table II shows that the consumption-based
welfare losses of a constant equity share are sometimes slightly larger than the equity
shares in the TDF, but the constant equity share for some households is better than theequity shares embedded in the TDF.
To summarize our findings, while TDFs may lead household to avoid worse mistakes,
because they impose the same portfolio on everyone of the same age, there is scopefor substantial improvement from more individualized financial advice or from morecustomized TDFs.

6

Concluding Remarks
We see four important areas for future work, all of which we view as omissions thatare potentially quantitatively significant. First, it would be interesting to also consider
the choice between traditional and Roth IRA/401k’s. We currently assume that Roth-type
investments are not available to the investor, but instead that all retirement saving is in‘traditional’ accounts where contributions are made pre-tax and withdrawals are taxed asincome.
Second, we do not consider different exposures of labor income to aggregate returns, norlong-run correlation between dividends and labor income. The labor income of different
occupations, industries, and skill sets appear to be differentially exposed to fluctuations inboth the dividend price ratio and dividends. Our current analysis contains this differential
exposure (through variances and serial correlations) but assumes that investors are unawareof their specific covariances (and do not learn about them). With respect to the long-term
relationship between dividends and labor income, our current analysis completely omits it.
We hypothesized that this relationship would have little effect on portfolio choice becauseof the small correlation between individual income risk and aggregate returns.
Third, we consider a canonical time-separable flow utility function with constant relativerisk aversion. It would be interesting to see how robust our results are to other realistic
assumptions about utility that have been studied in lifecycle models of portfolio choice,such as habits (Gomes and Michaelides, 2003), luxury goods (Wachter and Yogo, 2010),
hyperbolic discounting (Angeletos, Laibson, Repetto, Tobacman, and Weinberg, 2001; Love
and Phelan, 2015), flow utility from information (Pagel, 2018), risk aversion that declines
with age or wealth (Meeuwis, 2019), or incorporating flow disutility from anxiety about
future uncertainty as in Epstein-Zin preferences as many papers do.
40

The final issue for our analysis is that we have assumed that consumption and saving
behavior are optimal, and the deviations from this assumption that we analyze are minimal.
Presumably if investors make significantly sub-optimal choices in saving, then thesechoices would affect portfolio choices. While our optimal proscriptive rules can condition
on current asset levels, they are optimal under the assumption that future saving behavior
is optimal. An important question is then whether – either through learning from advice
or the experiences of others – people will arrive at close-to-optimal rates of saving andconsumption, or whether our model should also be used proscriptively in providing
savings advice to investors.

41

References
A NGELETOS , G.-M., D. L AIBSON , A. R EPETTO , J. T OBACMAN , AND S. W EINBERG (2001):“The Hyperbolic Consumption Model: Calibration, Simulation, and Empirical Evaluation,” Journal of Economic Perspectives, 15, 47–68.
B ALDUZZI , P. AND J. R EUTER (2019): “Heterogeneity in target date funds: strategic risktaking or risk matching?” The Review of Financial Studies, 32, 300–337.
B ERGER , D., V. G UERRIERI , G. L ORENZONI , AND J. VAVRA (2018): “House Prices andConsumer Spending,” Review of Economic Studies, 85, 1502–1542.
B ESHEARS , J., J. C HOI , D. L AIBSON , AND B. M ADRIAN (2008): “The Importance of Default
Options for Retirement Saving Outcomes: Evidence from the United States,” in Lessons
from Pension Reform in the Americas, Oxford University Press, 59–87.
B ODIE , Z., R. C. M ERTON , AND W. F. S AMUELSON (1991): “Labor Supply Flexibility andPortfolio Choice in a Life Cycle Model,” Journal of Economic Dynamics and Control, 16,
427–49.
B ROWN , D. C. AND S. D AVIES (2020): “Off target: On the underperformance of target-datefunds,” Available at SSRN 3707755.
C ALVET, L. E., J. Y. C AMPBELL , F. G OMES , AND P. S ODINI (2021): “The Cross-Section ofHousehold Preferences,” .
C AMPBELL , J. AND L. V ICEIRA (2002): Strategic Asset Allocation: Portfolio Choice for LongTerm Investors, Oxford University Press.
C AMPBELL , J. Y. (2016): “Restoring Rational Choice: The Challenge of Consumer Financial
Regulation,” American Economic Review: Papers & Proceedings, 106, 1–30.
C ARUANA , R. (1993): “Multitask Learning: A Knowledge-Based Source of Inductive Bias,”
in Proceedings of the Tenth International Conference on Machine Learning, Morgan Kaufmann,
41–48.
C ATHERINE , S. (2021): “Countercyclical Labor Income Risk and Portfolio Choices over theLife-Cycle,” .
C HOUKHMANE , T. (2021): “Default Options and Retirement Saving Dynamics,” .
C OCCO , J. F. (2005): “Portfolio Choice in the Presence of Housing,” Review of Financial
Studies, 18, 535–567.
C OCCO , J. F., F. J. G OMES , AND P. J. M AENHOUT (2005): “Consumption and Portfolio
Choice over the Life Cycle,” Review of Financial Studies, 18, 1502–1542.

42

C URCURU , S., J. H EATON , D. L UCAS , AND D. M OORE (2010): “Heterogeneity and Portfolio
Choice: Theory and Evidence,” in Handbook of Financial Econometrics: Tools and Techniques,
ed. by Y. Ait-Sahalia and L. P. Hansen, London: North-Holland, chap. 6.
D AHLQUIST, M., O. S ETTY, AND R. V ESTMAN (2018): “On the Asset Allocation of a Default
Pension Fund,” The Journal of Finance, 73, 1893–1936.
D AMMON , R., C. S. S PATT, AND H. H. Z HANG (2004): “Optimal Asset Location and
Allocation with Taxable and Tax-Deferred Investing,” Journal of Finance, LIX, 999–1037.
D AVIS , S. J., F. K UBLER , AND P. W ILLEN (2006): “Borrowing Costs and the Demand for
Equity over the Life Cycle,” The Review of Economics and Statistics, 88, 348–362.
D E N ARDI , M., E. F RENCH , AND J. J ONES (2010): “Why Do the Elderly Save? The Role ofMedical Expenses,” Journal of Political Economy, 118, 39–75.
——— (2016a): “Medicaid Insurance in Old Age,” American Economic Review, 106, 3480–
3520.
——— (2016b): “Savings After Retirement: A Survey,” Annual Review of Economics, 8,
177–204.
D UARTE , V. (2019): “Machine Learning for Continuous-Time Finance,” Working paper.
FAGERENG , A., C. G OTTLIEB , AND L. G UISO (2017): “Asset Market Participation andPortfolio Choice over the Life-Cycle,” Journal of Finance, 72, 705–750.
G AKIDIS , H. E. (1998): “Stocks for the Old? Earnings Uncertainty and Life-Cycle Portfolio
Choice,” Working Paper, M.I.T.
G OMES , F. AND A. M ICHAELIDES (2003): “Portfolio Choice With Internal Habit Formation:
A Life-Cycle Model With Uninsurable Labor Income Risk,” Review of Economic Dynamics,
6, 729–766.
G OMES , F., A. M ICHAELIDES , AND V. P OLKOVNICHENKO (2006): “Optimal Savings with
Taxable and Tax-Deferred Accounts,” Review of Economic Dynamics, 12, 718–735.
G OMES , F. J., A. M ICHAELIDES , AND Y. Z HANG (2021): “Tactical Target Date Funds,”
Management Science, forthcoming.
G UVENEN , F., G. K APLAN , J. S ONG , AND J. W EIDNER (2017): “Lifetime Incomes in the
United States Over Six Decades,” NBER Working Paper No. w23371.
G UVENEN , F., S. O ZKAN , AND J. S ONG (2014): “The Nature of Countercyclical Income
Risk,” Journal of Political Economy, 122, 621–660.
H EATHCOTE , J., K. S TORESLETTEN , AND G. V IOLANTE (2017): “Optimal Tax Progressivity:
An Analytical Framework,” Quarterly Journal of Economics, 132, 1693–1754.

43

H U , X. (2005): “Portfolio Choices for Homeowners,” Journal of Urban Economics, 58, 114–136.
I NVESTMENT C OMPANY I NSTITUTE (2020): Investment Company Fact Book, Investment
Company Institute, Washington, DC.
J ORDA , O., K. K ROLL , D. K UVSHINOV, M. S CHULARICK , AND A. M. TAYLOR (2019): “TheRate of Return on Everything, 1870-2015,” Quarterly Journal of Economics, 134, 1225–1298.
K OIJEN , R. S., S. N IEUWERBURGH , AND M. Y OGO (2016): “Health and Mortality Delta:
Assessing the Welfare Cost of Household Insurance Choice,” Journal of Finance, 71, 957–
1010.
L OVE , D. A. AND G. P HELAN (2015): “Hyperbolic discounting and life-cycle portfolio
choice*,” Journal of Pension Economics and Finance, 14, 492 – 524.
M EEUWIS , M. (2019): “Wealth Fluctuations and Risk Preferences: Evidence from U.S.
Investor Portfolios,” MIT Sloan Working Paper.
M ERTON , R. C. (1969): “Lifetime Portfolio Selection under Uncertainty: The ContinuousTime Case,” Review of Economics and Statistics, 51, 247–57.
——— (1971): “Optimum Consumption and Portfolio Rules in a Continuous-Time Model,”Journal of Economic Theory, 3, 373–413.
M ITCHELL , O. S. AND S. U TKUS (2021): “Target date funds and portfolio choice in 401(k)plans,” Journal of Pension Economics and Finance, 1–18.
M ORNINGSTAR (2020): “2020 Target-date strategy landscape,” .
PAGEL , M. (2018): “A News-Utility Theory for Inattention and Delegation in Portfolio
Choice,” Econometrica, 86, 491–522.
PARKER , J. A., A. S CHOAR , A. C OLE , AND D. S IMESTER (2021a): “Household Portfolios
and Retirement Saving over the Life Cycle,” .
PARKER , J. A., A. S CHOAR , AND Y. S UN (2021b): “Retail Financial Innovation and Stock
Market Dynamics: The Case of Target Date Funds,” .
P EAKE , M. AND G. VANDENBROUCKE (2020): “Worker Diversity and Wage Growth Since
1940,” Federal Reserve Bank of St. Louis Review, 102, 1–18.
S ALCEDO , A., T. S CHOELLMAN , AND M. T ERTILT (2012): “Families as Roommates:
Changes in US Household Size from 1850 to 2000,” Quantitative Economics, 3, 133–175.
S ALIMANS , T., J. H O , X. C HEN , S. S IDOR , AND I. S UTSKEVER (2017): “Evolution Strategies
as a Scalable Alternative to Reinforcement Learning,” .
S AMUELSON , P. A. (1969): “Lifetime Portfolio Selection by Dynamic Stochastic Programming,” Review of Economics and Statistics, 51, 239–246.
44

S EHNKE , F., C. O SENDORFER , T. R ÜCKSTIESS , A. G RAVES , J. P ETERS , AND J. S CHMIDHU BER (2010): “Parameter-exploring policy gradients,” Neural Networks, 23, 551–559, the18th International Conference on Artificial Neural Networks, ICANN 2008.
S HOVEN , J. B. AND D. B. WALTON (2020): “An Analysis of the Performance of Target Date
Funds,” NBER Workingpaper, W27971.
S OCIAL S ECURITY A DMINISTRATION (2016): “Income of the Population 55 or Older, 2014,”
Social Security Administration Report: Report Link.
S TORESLETTEN , K., C. I. T ELMER , AND A. YARON (2004): “Consumption and Risk Sharing
Over the Life Cycle,” Journal of Monetary Economics, 51, 609–633.
S UTTON , R. S. AND A. G. B ARTO (2018): Reinforcement Learning: An Introduction, The MIT
Press, second ed.
S UTTON , R. S., D. A. M C A LLESTER , S. P. S INGH , AND Y. M ANSOUR (2000): “Policy Gradient Methods for Reinforcement Learning with Function Approximation,” in Advances in
Neural Information Processing Systems 12, ed. by S. A. Solla, T. K. Leen, and K. Müller, MIT
Press, 1057–1063.
T SITSIKLIS , J. N. AND B. VAN R OY (2001): “Regression methods for pricing complex
American-style options,” IEEE Transactions on Neural Networks, 12, 694–703.
WACHTER , J. (2010): “Asset Allocation,” Annual Reviews of Financial Economics, 2, 175–206.
WACHTER , J. A. AND M. Y OGO (2010): “Why Do Household Portfolio Shares Rise in
Wealth?” Review of Financial Studies, 23, 3929–3965.
YAO , R. AND H. H. Z HANG (2005): “Optimal Consumption and Portfolio Choices with
Risky Housing and Borrowing Constraints,” Review of Financial Studies, 18, 197–239.
Y OGO , M. (2016): “Portfolio Choice in Retirement: Health risk and the demand for annuities, housing, and risky assets,” Journal of Monetary Economics, 80, 17–34.

45

Appendix: Proofs and derivations
Detailed notation for retirement objective:
max u(ct , ht ; 1)"+E

Tmax − TR

∑

β

f

f =1

f −2

∏

π ( gti +k , i, t + k; qiR )



k=−1



∗ π ( gti + f , i, t + f ; qiR )u(ct+ f , ht+ f ; 1) + 1 − π ( gti + f , i, t + f ; qiR ) z(bt+ f )


#
,

(58)
where Tmax is the maximum attainable age (i.e., π (·, ·, Tmax − 1; ·) = 0). In the more
complicated case where both individuals are alive at time t, the objective is:max u(ct , ht ; 2)
"


+E

Tmax − TR

∑

β

f


Pr(one individual alive at time t + f − 1)

f =1



∗


π ( gti + f , i, t + f ; qiR )u(ct+ f , ht+ f ; 1) +1 − π ( gti + f , i, t + f ; qiR ) z(bt+ f )+ Pr(both individuals alive at time t + f − 1)
∗ π ( gt1+ f , 1, t + f ; qiR )π ( gt2+ f , 2, t + f ; qiR )u(ct+ f , ht+ f ; 2)
+ 1 − π ( gt1+ f , 1, t + f ; qiR ) π ( gt2+ f , 2, t + f ; qiR )u(ct+ f , ht+ f ; 1)
+ π ( gt1+ f , 1, t + f ; qiR ) 1 − π ( gt2+ f , 2, t + f ; qiR ) u(ct+ f , ht+ f ; 1)



1
i
2
i
+ 1 − π ( gt+ f , 1, t + f ; q R ) 1 − π ( gt+ f , 2, t + f ; q R ) z(bt+ f ) ,
(59)
where
Pr(one individual alive at time t + f − 1) =
f −2
f −2


1
i
2
i
∏ π ( gt+k , 1, t + k; qR ) 1 − ∏ π ( gt+k , 2, t + k; qR ) +
k=−1



k=−1

f −2

1− ∏

π ( gt1+k , 1, t + k ) f −2∏ π ( gt2+k , 2, t + k)

k=−1

k=−1

46

(60)

and
f −2

Pr(both individuals alive at time t + f − 1) = ∏

k=−1

f −2

π ( gt1+k , 1, t + k; qiR )∏ π ( gt2+k , 2, t + k; qiR ).

k=−1

(61)

47

Unpublished Appendix for

Optimal Portfolio Choice Over the Lifecycle
byVictor Duarte

Julia Fonseca

Aaron Goodman

July 14, 2021

Jonathan A. Parker

A

Additional Figures
Figure A.1: Optimal Equity Shares of Financial Wealth(b) Equity Share by Dividend-Price Ratio(a) Equity Share by State of the Cycle
1.0
Equity Share, Total Financial Wealth

Equity Share, Total Financial Wealth

1.0

0.8

0.6

0.4

0.2

0.0

0.8

0.6

0.4

0.2

0.0
25

35

45

55
Age

Expansion

65

75

85

25

Recession

55
Age

Quartile 2

65

75

Quartile 3

85

Quartile 4(d) Equity Share by Permanent Income
1.0
Equity Share, Total Financial Wealth

1.0
Equity Share, Total Financial Wealth45

Quartile 1 (lowest)(c) Equity Share by Transitory Income

0.8

0.6

0.4

0.2

0.0

0.8

0.6

0.4

0.2

0.0
25

30

35

40

Quartile 1 (lowest)

45
Age

Quartile 2

50

55

60

Quartile 3

65

25

Quartile 4

35

Percentile 25(e) Equity Share by Total Financial Wealth

45

55
Age

Percentile 50

6575Percentile 75

85

Percentile 90(f) Equity Share by Housing Leverage
1.0
Equity Share, Total Financial Wealth

1.0
Equity Share, Total Financial Wealth

35

0.8

0.6

0.4

0.2

0.0

0.8

0.6

0.4

0.2

0.0
25

35

45

Quartile 1 (lowest)

55
Age

Quartile 2

65

Quartile 3

75

85

25

Quartile 4

35

45Quartile 1 (lowest)

1

55
Age

Quartile 2

65

Quartile 3

75

85

Quartile 4

Figure A.2: Optimal Equity Shares of Wealth by Health, Mortality(a) By Health and Mortality; Retirement Wealth(b) By Health and Mortality; Financial Wealth
1.0
Equity Share, Total Financial Wealth

Equity Share, Retirement Wealth

1.0

0.8

0.6

0.4

0.2

0.0

0.8

0.6

0.4

0.2

0.0
25

2 Alive, Neither Sick

35

45

2 Alive, 1 Sick

55
Age

65

2 Alive, Both Sick75

85

1 Alive, Healthy25

2 Alive, Neither Sick

35

45

2 Alive, 1 Sick

55
Age

65

2 Alive, Both Sick75

85

1 Alive, Healthy

Figure A.3: CAPTIONThis figure illustrates the architecture of a neural network representing the portfolio choice at time t of anagent during working life.

@tf.function
def train():with tf.GradientTape() as tape:V, other_outputs = simulate(training=True)
negative_mean_V = -tf.reduce_mean(V)grads = tape.gradient(negative_mean_V, parameters)
optimizer.apply_gradients(zip(grads, parameters))
return -negative_mean_V, other_outputs

https://www.overleaf.com/project/5e3dbd63f0631c0001bff755

2

Figure A.4: CAPTIONThis figure illustrates the architecture of a neural network representing the portfolio choice at time t of anagent during working life.

def brain(n_state_vars):state = Input(n_state_vars)
t = Input(1)shared_features = Sequential([BatchNormalization(), Dense(256), Activation('relu'),
Dense(128), BatchNormalization(), Activation('relu'),
Dense64), BatchNormalization(), Activation('relu'),])(state)
inputs = tf.concat([t, shared_features], 1)# output layer dimensionsclasses = [4, 2, 4, 4, 5, 3, 4, 6, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3]
n_classes = len(classes)
activations = [gumbel_softmax] * 2 + [tf.nn.softmax] * (n_classes - 2)latent = tf.split(Dense(sum(classes))(inputs), classes, axis=1)
actions = [activations[i](latent[i]) for i in range(n_classes)]return Model([t, state], actions)3