数理解析研究所講究録 1215 巻 2001 年 53-63

53

Toward Sophisticated Learning
Yuichi Noguchi*
Hitotsubashi University
May 10, 2001

Abstract

We summarize arecent development of the study of sophisticatedlearning with emphasis on learning to optimize.

1Introduction
In the last decade, the theory of learning in games has intensively beenstudied. Many researchers have investigated learning dynamics including fictitious play and replicator dynamic. However, those learning rules are naivein the sense that those cannot adjust to even simple patterns, so that someresearchers recently consider sophisticated learning as an important issue tostudy. It is natural to think that Bayesian learning is acandidate of sophisticated learning. In fact, Kalai and Lehrer (1993) study Bayesian learning in
infinitely repeated games and show that the grain of truth condition implies
Bayesian learners eventually play aNash equilibrium path. This approach,however, has two problems. First, it is unclear how the grain of truth condition or other similar conditions is related to sophisticated learning. Second,as Nachbar (1996) shows, Bayesian learning players with the same degree ofsophistication may fail to satisfy the grain of truth condition each other.
Another approach is proposed by Foster and Vohra (1993): they introduce
aconcept of calibrated learning and construct acalibrated.learning rule.
Their leaning rule is explained by the following example: suppose that a
Graduate School of Economics, Hitotsubashi University, 2-1 Naka, Kunitachi, Tokyo
186-8601, Japan. -mail:ynoguchi@econ.hit-u.ac.jp$\mathrm{E}$54
player or forecaster, who follows the learning rule, predicts aprobability ofrain, say , at the beginning of every period. Now fix any and pick up
periods in which aplayer predicts $p_{t}=p$ . Then, empirical distributionsof rain in those periods always converge to :the prediction is $emp$ iricall
correct. The Foster and Vohra learning rule is calibrated to empirical datain this sense. They consider repeated games where every player takes theleaning rule, and show that the path converges to acorrelated equilibrium.
Inspired by their work, Fudenberg and Levine (1995) and (1999) introduce a
concept of the universal consistency and show that smooth fictitious play has
the universal consistency property. In the following, Isummarize Fudenberg
and Levine’s work and then explain Noguchi (1999) that develops their work
to study the problem of learning to optimize against many regularities.
$p_{t}$

$p$

$p$

$y$

2The Model
We consider one player who plays an infinitely repeated game against an
opponent, where he may observe only apast history of actions in each period;
the opponent may be amachine, nature, or consist of multiple players. The
player’s (resp. opponent’s) pure action in astage game is denoted by (resp.
$y)$ and aset of all player’s pure
actions (resp. all opponent’s pure actions)is finite, denoted by(resp. ), Aset of probability distributions over$\in\Delta(A)$is denoted by. A(resp.
) denotes the player’s mixed
action (resp. the opponent mixed action).
is written for the player’s
expected utility of astage game. Histories of the repeated game consist of
sequences of actions by the player and his opponent. Afinite history withtime length is denoted by $h_{T}=$and an infinite history
is denoted by; we define. Let $H$ denote asetof all finite histories anddenote a set of all infinite histories.
Our player takes an action on the basis of apast history in every period, sothat his strategy may be represented by abehavior strategy : $Harrow\Delta(A)$ .
Aset of all player’s behavior strategies is denoted by. We assume thatthe player does not know about any opponent characteristic except that the
opponent plays abehavior strategy and takes an action in at each period.
An opponent behavior strategy is denoted by and aset of all opponent
behavior strategies is denoted by $Bp$ .
$a$

$A$

$S$

$\mathrm{Y}$$\Delta(S)$$\pi\in\Delta(\mathrm{Y})$$u(\lambda, \pi)$$T$$(a_{1}, y_{1}, \cdots, a_{T}, y_{T})$$h_{\infty}=$$(a_{1}, y_{1}, a_{2}, y_{2}, \cdots)$$h_{0}=\emptyset$$H_{\infty}$$\sigma$$B_{P}$$\mathrm{Y}$$\rho$

55
3Universal Consistency
Aword “consistency” has several definitions in the learning literature. We
first describe adefinition of consistency in statisitics and then introduce thatin the economic context. In statistics, the consistency means that prediction
averages asymptotically coincide with empirical frequencies. For example,
be aconsider sequences of two possible states, rain and no rain. Let
an empirical frequency
prediction of rain at the beginning of period andof rain up to period . Then, the consistecy criterion requires that$\mu_{t}(R)$$t$$D_{t}$$t$$\frac{\sum_{s=1}^{t}\mu_{S}}{t}-D_{t}arrow 0$, as t$arrow\infty$, a.s.

This criterion is unsatisfactory because the criterion cannot deal withregulariteis. Suppose that the weather is very cyclical, for example, it is rainin even periods and it is no rain in odd periods. Let aprediction always
give 50-50 probability. That is, the prediction is so naive that it does notlearn the weather simple pattern. However, it is clear that the prediction
passes the consistency criterion. Note that this criticism may be applied to
the Foster and Vohra learning rule.
Fudenberg and Levine (1995) introduce the universal consistency in theeconomic sense (precisely its definition dates badc to Hannan (1959)).
Definition 1A behavior strategy
for all opponent strategies$\sigma$

is said to be$\epsilon$-universally consistent, if$\rho$$\lim_{Tarrow\infty}\sup V(D(h_{T}))-\frac{1}{T}\sum_{s=1}^{T}u(a_{s},y_{s})<\epsilon$,$\mathrm{a}.\mathrm{s}.$,where $D(h_{T})$ is an empirical tribution of all opponent actions up to period
and $V(D(h_{T}))= \max${$u(\lambda,D(h_{T}))|$ A $\in\Delta(A)$ }.
$d\dot{u}$$T$

We need to give several remarks. First, the criterion is concerned with
optimization, not prediction. Second, the criterion is universal in the sensethat it requires single behavior strategy pass the criterion for all opposing
behavior strategies. Third, atarget in the criterion is amaximum payoff
against an empirical distribution up to the current period, so that realized
average payoffs eventually becomes at least as high as the minmax payoff inastage game. Those mean that the criterion may be interpreted as asafet

56
criterion in the sense that the player’s strategy does not perform very poorly
against any opposing behavior strategy.
Fudenberg and Levine (1995) show that smooth fictitious play, that is, asmooth approximate best response against an empirical distribution of past
opponent actions, is universally consistent.
Proposition 1($hdenbe\eta$ and Levine (1995)) For any
fictitious play is universally consistent.

$\epsilon$$>0$ ,

a smooth$\epsilon$

It is important to note that avery simple strategy such as smooth fictitious play has the property. Unfortunately, this criterion may be criticized by
the same argument as in the statistical deffiition: it does not assure aplausible optimality against even very simple patterns. To understand it, consider
arepeated matching pennies game where an opponent takes an alternating
strategy :
, ,

, ,

, ,

, , $H,T$, $H$, $T$, $H,T$, $H,T$, $H$ , $T$, $H$ , $T$,

$H$ $T$ $H$ $T$ $H$ $T$ $H$ $T$$\cdots$

Then, the universal consistency criterion only assures that player’s average payoffs may attain the minmax payoff 0. However, when the player is alittle smart, he may recognize the simple pattern and try to adjust to it, sothat he must earn much more payoffs than 0. It means that the universal
consistency is too weak to evaluate sophisticated learning behaviors.

4Conditional Universal Consistency
Fudenberg and Levine have been aware of the weakness, and then they pr0pose “conditional” universal consistency (1999). They show that conditionalsmooth fictitious play has the conditional universal consistency property under mild assumptions. Let us explain conditional (smooth) fictitious play. It
is just a(smooth approximate) best response to conditional empirical distributions. For example, suppose that aplayer consider samples should be
separated according to even and odd periods. It may be represented by aclassification rule 72 that is defined as apartition of $H$ . In this example,, where{ $h_{t}|t$ is even} and{ $h_{t}|t$ is odd}. Then,conditional fictitious play on 72 is a(smooth approximate) best response toan empirical distribution of past odd period samples in any odd peirod and abest reponse to an empirical distribution of past even period samples in any$\mathcal{R}=\{\gamma_{e}, \gamma_{\mathit{0}}\}$$\gamma_{e}=$$\gamma_{\mathit{0}}=$57
even period. In ageneral case, aclass in aclassification ruleis called acategory, denoted by ;if apast history, we say that is active in pe
riod $T+1$ . We assume that each 7has prior samples represented by avector
is called aprior sample size for . Thus, when apast history
belongs to 7, aplayer considers that acurrent period is
activeHe collects observed samples
in past -active periods and plays a(smooth approximate) best resposne to an empirical distributionof prior and observed samples conditional on,
wheredenotes an empirical distribution
only of observed samples in -active periods.
$\mathcal{R}$$h_{T}\in\gamma$$\gamma$$d_{\gamma}^{0};n_{\gamma}^{0}= \sum d_{\gamma,y}^{0}$$\gamma$$\gamma$$T$$h_{T-1}$$d_{\gamma}(h_{T-1})$$\gamma$$\gamma$$\tilde{D}_{\gamma}(h_{T-1})$$\gamma:\tilde{D}_{\gamma}(h_{T-1})=\frac{d_{\gamma}(h_{T-1})+d_{\gamma}^{0}}{n_{\gamma}(h_{T-1})+||\varpi\gamma}$$n_{\gamma}(h_{T-1})= \sum_{y}d_{\gamma,y}(h_{T-1});D_{\gamma}(h_{T})$$\gamma$

Proposition 2($Phdenk\tau y$ ared Levine (1999)) Suppose that a dassification$nde$satisfies the following two assumptiou:$(\mathcal{R}, (d_{0}^{\gamma})_{\gamma\in \mathcal{R}})$(1)$\lim_{Tarrow\infty}\frac{K^{R}(h_{T-1})}{T}=0$

for all$h_{\infty}\in H_{\infty}$, and (2))$\sup_{\gamma\in \mathcal{R}}n_{0}^{\gamma}<\infty$

where $K^{R}(h_{T-1})$ is the number of categories that have been active up to period, a conditional smooth fictitious play on
T. Then, for any
satisfies
the conditional universal consistency: for any opposing behavior strategy$\epsilon>0$$\mathcal{R}$$\rho$, $a.s$ .

$\lim_{Tarrow\infty}\sup||\frac{1}{T}\sum n_{T}^{\gamma}V(D_{\gamma}(h_{T}))-\sum_{\epsilon=1}^{T}u(a_{s},y_{\epsilon})||<\epsilon$

where$n_{T}^{\gamma}$

is the nurnlter of times that is active up$\gamma$$\max\{u(\lambda,D_{\gamma}(h_{T}))|\lambda\in\Delta(A)\}$

$tp$

.

period$T$

and$V(D_{\gamma}(h_{T}))$

However, the conditional universal consistency is still weak to evaluateasophisticated player’s behavior, although it is very useful to prove
sitions in the next section: the target, amaximum
average payoff against conditional empirical distributions on , might not
be an upper bound to time average payoff, even if an opponent plays asimple
regular strategy. The following example explains it:$\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{e}\succ$$\frac{1}{T}\sum_{\gamma\in R}n_{T}^{\gamma}V(D_{\gamma}(h_{T}))$$\mathcal{R}$

Example 1Suppose that a player’s payoff rnati is:
$L$

$M$

$R$

3 –33
D –3
3 –3
$U$

58
Let the player’s classification rule be, where $\gamma_{RL}=\{h_{t}|y_{t}=$or } and $\gamma_{M}=\{h_{t}|y_{t}=M\}$ . Assume, however, that his opponent
follows a cyclical behavior LMRLMRLMR . Even though a conditionalsmooth fictitious play on 72 is supposed to perform well according to theconditional universal consistency criterion, the conditiond fictitious play on$\prime \mathcal{R}=\{\gamma_{RL}, \gamma_{M}\}$

$R$

$L$

$\cdots$$playe \mathrm{r}\dot{u}mah,\mathrm{f}\mathrm{f}\mathrm{i}mhecauldoece.ive\frac{\frac{2}{\mathrm{f}}}{3}3+\frac{\frac{1}{1^{3}}}{3}3+\frac{1}{3}3=3oeh\dot{u}h.me-avemge\mathcal{R}_{\mathit{9}}eneratesonlytime- averageutdity0+3=1bypmpo\dot{\Re}t\dot{\iota}\sigma n\mathit{2}.If\#\iota e$

by switching to a cyclical behavior, UDUUDUUDU . It is hard tobelieve that the smart player takes the conditional fictitious play on 72 forever.
$\cdots$$u\mathrm{f}\mathrm{f}\mathrm{l}.\dot{\iota}\mathrm{t}y$The example suggests (1) we need astronger criterion ofutilty especialy for regular opponent strategies and (2) if aplayer wants
to learn his opponent stratey, then his classification rule should be moresophisticated (or finer) than aconditioning rule of the opponent strategy in$\mathrm{t}\mathrm{i}\mathrm{m}\triangleright \mathrm{a}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}$some sense.

5Optimal Properties of Conditional Fictitious
Play
Taking into account the weakness of the consistency, Noguchi (1999) introe
duces two strong criteria: time average optimality and conditioning classoptimality. The first criterion requires that aplayer may eventually obtain
almost as high time average payoff as if he knew atrue opposing behavior
strategy. The second one insists that aplayer may eventually obtain almost
as high payoff as if he knew atrue opposing behavior strategy. Then, heconstruct an “optiml” classification rule for each of those criteria and showsthat conditional smooth fictitious play on the optimal rule pass the strong
criterion for all regular opposing strategies. We shall explain those criteria
and optimal classification rules.

5.1

Time average optimality

Recal the defect of the consistency: atarget might not be an upper boundto time average utility. We shall intorduce astrong criterion to resolve theproblem: time average optimality.

59
Definition 2A behavior strategy (7 H
average sense for SCB, for all pE $S$$\ovalbox{\tt\small REJECT}$$\ovalbox{\tt\small REJECT}$$\ovalbox{\tt\small REJECT}\ovalbox{\tt\small REJECT}(A)$is called e optimal in the$i\ovalbox{\tt\small REJECT}$,$\lim_{Tarrow\infty}\sup\frac{1}{T}[\sum_{-0}^{T-1}V(\rho(h_{e}))-\sum_{-1}^{T}u(a_{s},y_{e})]<\epsilon$where$\mu_{(\sigma,\rho)}$$V(\rho(h_{\epsilon}))$is a probability distribution on

is a maximum payoff against$\rho(h_{s})$$H_{\infty}$.

$\mu_{(\sigma,\rho)}-\mathrm{a}.\mathrm{s}$generated by$\sigma$.

and , and$\rho$

payoff against atrue op
In this definition, we put averageponent strategy as atarget, instead of that against empirical distributions.
Note that it is an asymptotic least upper bound of time average utility: forany player’s strategy and opposing strategy ,$\ovalbox{\tt\small REJECT}\iota \mathrm{m}$$\sigma$$\lim_{Tarrow\infty}\inf\frac{1}{T}$$\rho$[I$V( \rho(h_{s}))-\sum_{s=1}^{T}u(a_{s},y_{s})$]$\geq 0$, a.s.

and the equality holds $a.s$ . for withfor all $h_{s}\in H$ . It is easily obtained by combining the strong law.
of large numbers and afact that $V(\rho(h_{s}))\geq u(\lambda, \rho(h.))$ for all
Hence, if time average utility is almost as high as the target, aplayer, who isconcerned with time average utility, has no incentive to change his behavior,so that the weakness of the consistency is resolved. However, we cannot hopethe best result for the time average optimalty criterion if a player has no
weakly dominant action in astage game: there is no single strategy thatis optimal in the average sense for all opposing strategies; when there is adominant strategy, the optimality is alway obtained by playing the dominantaction.
$\overline{\sigma}$$\overline{\sigma}(h_{s})\in\arg\max\{u(\lambda,\rho(h_{s}))|\lambda\in$$\Delta(A)\}$$\lambda\in\Delta(A)$Proposition 3Assume that there is no weakly dominant action in a stage
there eexists no -optimal behavior strategy
game. Then, for somefor all opponent behavior strategies.
$\epsilon 0>0$$\epsilon 0$

Although it is impossible to obtain the best result, it is worthwhile studying aplayer’s optimal strategy for many opponent behavior strategies. Indeed, we may show that there exists atime average optimal strategy for all
opposing strategies generated by acountable family of finitely conditionin

60
Definition 3(1) A conditioning rule is a partition of $H$ , denoted by . Anelement of a conditioning $mle$ is called a conditioning class, denoted by .
conditioning rule $P$ is said to be finite if the number of classes in $P$ is finite.
(2) A conditioning rule of a behavior strategy is a partition, denoted by ,that is generated by the following equivalent relation:$\mathcal{P}$$\beta$$A$$P_{\rho}$$\rho$$h_{t}\sim\overline{h}_{\mathrm{f}1}\Leftrightarrow\rho(h_{t})=\rho(\overline{h}_{\mathrm{f}\mathrm{i}})$

The mixed action,.

$\rho(h_{t})$$\rho_{\beta}$

conditional on a class$\beta$is uniquely defined by$\rho_{\beta}=$$h_{\ell}\in\beta$

Definition 4We say that a behavior strategy is generated by a family
conditioning rules , if a conditioning rede of belongs to , that is,.

$oj$$\rho$$\Omega$$\Omega$$P_{\rho}\in\Omega$$\rho$

Proposition 4For any countable family finitely conditioning rules
there exists a classification $mle$
such that for any
some conditional
is -optimal in the average sense for all behaviorsmooth fictitious play on.
strategies generated by the family$oj$$\{\mathcal{P}_{i}\}_{i=1}^{\infty}$$\epsilon>0$$\mathcal{R}_{0}$$\mathcal{R}_{0}$$\epsilon$$\{\mathcal{P}:\}_{\dot{l}=1}^{\infty}$

This proposition implies that conditional smooth fictitious play is optimalin the avearge sense for all regular opposing strategies because conditioning
rules of regular strategies are at most countable; regular strategies are thosethat have computable finitely conditioning rules (see Noguchi (1999) for its
formal definition).
Instead of giving arigorous proof, we will explain abasic idea of con; without
structing an optimal rule
. It is based on agiven family
is ordered by the fineness asloss of generality, we may assume thatpartitions:. A
for allkey is that aplayer makes use of eachas atemporary classification rule
represents the following player behavior: at thein some periods. That is,
first stage he starts with employing
as atemporary rule. But he has anincentive to change finer conditioning rules because atrue conditioning rule; the player could not receive the maximum averagemight be finer thanpayoff against atrue strategy if he got stuck inand the true rule were finer
than (recall example 1). The argument may be applied to any period andany conditioning rule he employs in that period: there is always possibility
that atrue conditioning rule might be finer than acurrent employed one.
Therefore, he employs finer and finer rules as time proceeds. But he wants
to change the rules very slowly. Delayed changes of temporary rules allo$\{P_{i}\}_{i=1}^{\infty}$$\mathcal{R}_{0}$$\{\mathcal{P}:\}_{i=1}^{\infty}$$i(\mathcal{P}_{i}\prec \mathcal{P}_{\dot{\iota}+1}\Leftrightarrow\forall\beta\in P_{\dot{\iota}+1}\exists\tilde{\beta}\in \mathcal{P}_{i}(\beta\subset\tilde{\beta}))$$P_{i}\prec P_{\dot{\iota}+1}$$P_{\dot{l}}$$\mathcal{R}\circ$$P_{1}$$P_{1}$$P_{1}$$P_{1}$

61
the player to obtain enough samples for each rule, so that if his employing
rule is finer than atrue one, he could have agood prediction on the true
opponent strategy and obtain the maximum average payoff to it. The player
employs finer and finer rules, and eventually employed rules are finer thanatrue one. It means that the player may eventually obatin the maximum
avearge payoff.

Conditioning class optimality

5,2

Even the time average optimality may be weak as abehavior criterion ofasophisticated player. The weakness is that the time average optimality
criterion may ignore aperformance of aplayer’s strategy in active periods ofaconditiong class whose frequency vanishes relative to all periods. Consider
arepeated matching pennies game where aplayer always plays heads andhis opponent plays the following regular strategy:
,

,

,

,

,

,

$H$ $T$ $H$ $H$ $T$ $H$ $H$, $H$ , $T$, $H$ , $H$ , $H$ , $H$ , $T$, $H$ , $H$, $H$ , $H$, $H$ , $T$,$\cdots$

That is, the frequency of playing tails diminishes regularly as time proceeds.
The player suffers the worst
whenever the opponent plays tails,
although tails are played regularly. However, the player’s strategy passes thetime average optimality criterion for this opponent strategy because periodsof the worst outcome asymptotically vanish relative to aU periods. If theplayer is smart, then he would be aware of the regularity and very likely toplay tails when his opponent plays tails. The example suggests that we needastronger criterion in order to capture sophisticated behaviors: acriterion
that assures an optimality in each conditioning class. We shall only give thedefinition of the conditioning class optimality for acase of finitely conditioned
strategies, that is, strategies whose conditioning rules are finite (see Noguchi(1999) for ageneral case).
$\mathrm{o}\mathrm{u}\mathrm{t}\mathrm{c}\mathrm{o}\mathrm{m}\mathrm{e}-1$

Definition 5A behavior strategy : $Harrow\Delta(A)$ is called
optimal in theclasswise sense for , iffor any opponent strategy and any conditioning class$\sigma$$\epsilon$$S$$\beta\in P_{\rho}$, if$\tau\cdotarrow\infty \mathrm{h}\mathrm{m}\sup V(\rho_{\beta})-\frac{1}{n_{T}^{\beta}}$$n_{T}^{\beta}arrow\infty$$1 \sum_{h.\in\beta}u(a_{s+1}, y_{s+1})<\epsilon$, $a.s$ .

$0\leq\cdot\leq T-,$here
is the reurnber of -active periods up tomasimurn payoff against .
$w$$n_{T}^{\beta}$$\beta$$\rho_{\beta}$$\mu r\cdot od$ $T$and$V(\rho_{\beta})$

is

$a$

62
This optimality criterion assures that aplayer may eventually obtain
almost as high payoffs as if he knew atrue opponent strategy, and the above
problem is resolved.
Proposition 5For any countable family finite conditioning rules,
there exists a classification rulesuch that for any $>0$ , some conditionalsmooth fictitious play strategy on
is optimal for all behavior strategies
generated by the family, $i.e$ .
.
$oj$$\mathcal{R}_{1}$$\{\mathcal{P}_{i}\}_{i}$$\epsilon$$\mathcal{R}_{1}$$\mathcal{P}_{\rho}=P_{\dot{l}}\exists i$

This criterion is stronger than the time average optimality and it would
be the strongest as amyopic optimality criterion in the sense that the same
proposition with astronger criterion would not hold.
Abasic idea of constructing
is very similar to that of the time average. Adifference is that aplayer switches classwise: if aclassoptimal ruleobtains enough samples, then the class is changed to finer classes inin, while any other class inwith few samples keeps to be employed, i.e.
it is not switched to finer classes.
$\mathcal{R}_{1}$$\mathcal{R}_{0}$$\beta$$P_{i}$$P_{i+1}$$\mathcal{P}_{\dot{l}}$6Concluding Remarks
We conclude by giving several remarks. First, we need to emphasize that thetime average optimality and the conditioning class optimality (proposition
4and 5) may be obtained by using the conditional universal consistency.
This means that those optimalities may be attained without the cost of theuniversal consistency. Thus, single smooth fictitious play is not only optimalfor many opposing strategies but also safe against all other ones. Second, we
have focused on myopic optimality. However, nonmyopic cases are important in some economic situations. Noguchi (2001) extends the conditioningclass optimality to anonmyopic case and show that anonmyopic version of
conditional smooth fictitious play passes the nonmyopic optimality criterion
for many strategies.

References[1] Foster, D. and Vohra, R. (1993). “Calibrated Learning and Correlated
Equilibrium,” Mimeo, Wharton

63[2] Fudenberg, D. and Levine, D. (1995). “Universal Consistency and Cautious Fictitious Play,” Journal of Economic Dynamics and Control 19,1065-1089.
[3] Fudenberg, D. and Levine, D. (1999). “Conditional Universal Consistency,” Games and Economic Behavior 29, 104-130.
[4] Hannan, J. (1959). “Approximation to Bayes Risk in Repeated Plays,”in Contributions to the Theory of Games (M. Dresher, A.W. Tucker, andP. Wolfe, Eds.), Vol. 3, pp. 97-139, Princeton, NJ, Princeton Univ. Press.
[5] Kalai, E. and Lehrer, E. (1993). “Rational Learning Leads to Nash Equilibrium,” Econometrica 61, 1019-1045.
[6] Nachbar, J. (1996). “Prediction, Optimization, and Learning in RepeatedGames,” Econometrica 65, 275-309[7] Noguchi, Y. (1999). “Optimal Properties of Conditional Fictitious Play,”
Mimeo, Harvard University.

[8] Noguchi, Y. (2001). “Nonmyopic Optimality of Conditional Learning,”
Mimeo, Hitotsubashi University.