# Stock Market Crisis Forecasting Using Neural Networks with Input Factor Selection
## 4 Model Building with Neural Networks
### Structure of the Neural Networks  
Throughout the model-building process, we use feed-forward neural networks with one input layer, one hidden layer with three neurons, and one output layer. Our neural networks are built from a data set with the following specifications:  
1. The monthly economic states, as defined and determined in Section 2, are our targeted output variable \(\mathcal{Y}\). The realizations of \(\mathcal{Y}\) form a vector \(Y=(y_{t})_{t=1,\dots,T}\). We conduct a binary classification, so \(Y\) only takes the values 0 and 1. Furthermore, we have \(T=605\) months for the total data set. For the walk-forward forecasting and testing, we train neural networks for different values of T, i.e., also for shorter periods. 2. The monthly transformed values of the input factors, as listed in Table 1, are our input variables \(\mathcal{X}\). The realizations of \(\mathcal{X}\) form a matrix \(X=(X_{n})_{n=1,\dots,N}\), where \(X_{n}=(x_{n,t})_{t=1,\dots,T}\), \(n=1,\dots,N\) are the input factors given to the neural network to train. Within our model-building approach, we train neural networks with various values for \(N\), i.e., for different subsets of our total set of input factors. Figure 2 visualizes our neural networks at a time \(t\)\((t=1,\dots,T)\). \begin{table}
\begin{tabular}{l l} \hline \hline
**Abbreviation** & **Variable Description** \\ \hline BAA-AAA & Difference: Moody’s Seasoned Ba vs. AAA Corporate Bond Yield \\ CPIAUCSI. Change & Relative change of the Consumer Price Index for All Urban Consumers: All Items \\ FEDFUNDS & Effective Federal Funds Rate \\ GSS-FEDFUNDS & Difference: 5-Year Treasury Constant Maturity Rate vs. Effective Federal Funds Rate \\ GSI0-FEDFUNDS & Difference: 10-Year Treasury Constant Maturity Rate vs. Effective Federal Funds Rate \\ HOUST & Housing Starts: Total: New Privately Owned Housing Units Started \\ INDPRO Change & Relative change of the Industrial Production Index \\ MINS Change & Relative change of the MI Money Stock \\ M2 NS Change & Relative change of the MI Money Stock \\ OECD CLI & Composite leading indicator for OECD + 6 major non-member economies \\ PAYEMS Change & Relative change of the AI Employees: Total Nonfarm Payrolls \\ PAI & ISM Manufacturing: PM Composite Index \\ PIATOCD Change & Relative change of the Producer Price Index for All Commodities \\ PSAFRET & Personal Saving Rate \\ ICU & Capacity Utilization: Total Industry \\ TOTALSI.Change & Relative change of the Total Consumer Credit Owned and Securitized, Outstanding \\ TREESGUSMOS52N & Relative change of the Total Reserves excluding Gold for United States \\ UMCSENT & University of Michigan: Consumer Sentiment \\ UNRATE & Civilian Unemployment Rate \\ VOALITILITY & Historical 20-day volatility of the S&P 500 \\ W823 RCI Change & Relative change of government social benefits to persons \\ WTHERChange & Relative change of the Spot Crude Oil Price: West Texas Intermediate (WTI) \\ GDP Quarterly Change & Relative quarterly change of the Gross Domestic Product \\ GFDEBIN Change & Relative change of the Federal Debt: Total Public Debt \\ EURO-DOLLA & Euro to US Dollar exchange rate \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Complete Variable List Before Input Factor Reduction (see Appendix A, Table A1 for the data sources). The training of a feed-forward supervised neural network starts with _forward-feeding_. _Forward-feeding_ works in the direction input layer - hidden layer - output layer on the training dataset (see, e.g., [23]). In this step, at each neuron of the hidden layer and the output layer, _propagation functions_ and _activation functions_ are applied as follows:  
1. At the hidden layer (\(h=1\)): 1. At neuron \(k_{1}\) (\(k_{1}=1,\ldots,3\)), we obtain a representation \(P^{1}_{k_{1},t}\) of the input values via the propagation function for each \(t=1,\ldots,T\): \[P^{1}_{k_{1},t}=\sum_{n=1}^{N}w^{1}_{n,k_{1}}x_{n,t}\,,\] (8) where \(w^{1}_{n,k_{1}}\) is the weight between the \(n\)-th input variable and the \(k_{1}\)-th neuron of the hidden layer. 2.