# Reinforcement Learning in a New Keynesian Model +
## 2 The Standard Behavioural NK Model
### The Brock-Hommes Behavioural NK Model  
In the Brock-Hommes framework, which we later follow, the model becomes behavioural by a departure from the RE assumption and the introduction of two groups of agents. One group is rational, and the other forms EL expectations through simple \"heuristic\" learning rules. RE agents form model-consistent expectations fully aware of the existence of BR agents in the composite model. A version of general adaptive learning rules (the authors of Reference [24] provided lab-based support for such rules, and the generalised heuristic rule we later adopt in Section 4 includes a \(t-2\) period and encompasses all the different behavioural group forecast heuristics) that encompasses those adopted by References [3; 4; 13; 19; 25] is  
\[\mathbb{E}_{t}^{*}y_{t+1} = \mathbb{E}_{t-1}^{*}y_{t}+\lambda_{y}(y_{t-j}-\mathbb{E}_{t-1}^{* }y_{t})\ ;\quad\lambda_{y}\in[0,1],\,j=0,1 \tag{4}\] \[\mathbb{E}_{t}^{*}\pi_{t+1} = \mathbb{E}_{t-1}^{*}\pi_{t}+\lambda_{\pi}(\pi_{t-j}-\mathbb{E}_{t -1}^{*}\pi_{t})\ ;\quad\lambda_{\pi}\in[0,1],\,j=0,1 \tag{5}\]  
where we can in principle allow for both current and lagged observations of output and inflation, \(j=0,1\), respectively. Throughout the rest of the paper, we make the following information assumptions: for observations of aggregateoutput and inflation, similar to the EL approach, we assume \(j=1\). Later in the AU approach, we need to model observations of market-specificvariables consisting of factor prices, profits and marginal costs. These we assume can be observed without a lag, and therefore, \(j=0\).  
Let \(n_{y,t}\), \(n_{\pi,t}\) be the proportions of rational agents forecasting output and inflation, respectively. The IS and NK Phillips curve equations then become  
\[y_{t} = n_{y,t}\mathbb{E}_{t}y_{t+1}+(1-n_{y,t})\mathbb{E}_{t}^{*}y_{t+1 }-[r_{n,t}-(n_{\pi,t}\mathbb{E}_{t}\pi_{t+1}+(1-n_{\pi,t})\mathbb{E}_{t}^{*} \pi_{t+1})]+u_{1,t} \tag{6}\] \[\pi_{t} = \beta[n_{\pi,t}\mathbb{E}_{t}\pi_{t+1}+(1-n_{\pi,t})\mathbb{E}_{t }^{*}\pi_{t+1}]+\lambda y_{t}+u_{2,t} \tag{7}\]  
To complete the model, we need expressions for the weights \(n_{y,t}\) and \(n_{\pi,t}\). These follow the reinforcement learning literature by choosing probabilities  
\[n_{x,t}=\frac{\exp(-\gamma\Phi_{x,t}^{RE}(\{x_{t}\}))}{\exp(-\gamma\Phi_{x,t}^ {RE}(\{x_{t}\}))+\exp(-\gamma\Phi_{x,t}^{AE}(\{x_{t}\}))} \tag{8}\]where \(-\Phi^{KE}_{x,t}(\{x_{t}\})\) and \(-\Phi^{AE}_{x,t}(\{x_{t}\})\) are \"fitness\" measures, respectively, of the forecast performance of the rational and non-rational predictor of outcome \(\{x_{t}\}=\{y_{t}\},\{\pi_{t}\}\) given by a discounted least-squares error predictor  
\[\Phi^{RE}_{x,t}(\{x_{t}\}) = \mu_{RE}\[PERSON]^{RE}_{x,t-1}(\{x_{t}\})+(1-\mu_{RE})([x_{t}-\mathbb{ E}_{t-1}\,x_{t}]^{2}+\mathbb{C}_{x}) \tag{9}\] \[\Phi^{AE}_{x,t}(\{x_{t}\}) = \mu_{AE}\[PERSON]^{AE}_{x,t-1}(\{x_{t}\})+(1-\mu_{AE})[x_{t-j}-\mathbb{ E}^{*}_{t-1-j}\,x_{t-1}]^{2};j=0,1 \tag{10}\]  
where \(\mu_{RE}\) and \(\mu_{AE}\) capture the memory of the agents forming RE and adaptive expectations (a measure of forgetfulness of past observations). \(\mathbb{C}_{x}\) represents the relative costs of being rational in learning about variable \(x_{t}\). Thus, the proportion of rational agents in the steady state is given by  
\[n_{x}=\frac{\exp(-\gamma\mathbb{C}_{x})}{\exp(-\gamma\mathbb{C}_{x})+1}\]  
which is pinned down by the \(\gamma\mathbb{C}_{x}\). Equations (3)-(10) constitute the linearised NK behavioural model (the authors of References [3; 4] constructed a rather different composite EL-type model consisting of \"fundamentalist\" rather than rational agents alongside adaptive learners. For the former RE, \(\mathbb{E}(\cdot)\) are replaced with \(\mathbb{E}^{\prime}y_{t+1}=y_{t}^{F}\) and \(\mathbb{E}^{\prime}\pi_{t+1}=0\). Thus, fundamentalists always believe that the next period's output gap is zero and that the net inflation rate will return to its steady-state value of zero. The same authors also assume \(\mathbb{C}_{x}=0\) in (9)).