## 2 Neural Networks  
Neural networks (NNs), the primary methodological tool used in our analysis, have been in existence since the 1940s, beginning with the early work of [17], but have become immensely popular in the last decade due to dramatic increases in computing power and the explosion of available data. NNs are set up to imitate the decision-making process of the human brain. An example of a neural network is shown in Figure 1. This network has four layers: an input layer, two hidden layers, and an output layer.  
However, the simplest exemplar of a neural network is a single perceptron. A typical perceptron is implemented using the \"sigmoid\" function, also known as the \"output activation\" function.  
\[z(a)=\frac{1}{1+\exp(-a)}\in(0,1) \tag{1}\]  
where \(a\) is called the \"neuron input\" to the sigmoid function, and is a weighted sum of outputs from the preceding layer. There are several different types of neurons and another popular one that we use in this paper is known as the \"restricted linear unit\" or ReLU for short. This has a piecewise linear functional form.  
\[z(a)=\max(0,a) \tag{2}\]  
Therefore, this neuron emits an output of zero, unless its input \(a\) is positive, in which case it emits \(a\).  
The first layer with input values (usually denoted \(x\)) is called the input layer, the middle layers with function values (usually denoted \(y\)) are called the hidden layers and the last layer with \(z\) values is called the output layer. The functions at each node are called \"activation\" functions, borrowing from neuroscience terminology where neurons are said to fire or activate. There may be several hidden layers and multiple perceptrons (nodes) in each hidden layer. Neural nets with multiple layers (sometimes more than 100 layers) have resulted in the nomenclature of \"deep learning\" neural nets. Higher number of layers or nodes naturally lead to longer processing times. The output layer can either have multiple outputs or a single output (binary classifier). In the case of a binary output, the output layer is implemented with squashing functions that emit an output value between 0 and 1. The neural network we use in our paper is a fully connected feed-forward network, i.e., each input layer node is connected to each node in the first hidden layer. Each node in the first hidden layer is connected to every node in the subsequent layer, and so on, till we reach the output node(s).  
For our stock prediction algorithm, we would inject the information set (in our case, data histories on all stocks in the index) into the (left side) of the neural network. Each layer in the network comprises nodes, i.e., a set of mathematical functions that transform the data, which is then fed into the nodes (functions) in the next layer, until finally the output layer is reached. The output function in our case generates a value in the interval \((0,1)\), i.e., the probability that the market will go up in the forecast period. Intuitively, the neural network is a giant nonlinear function that takes in a large number of inputs and generates a prediction about the likelihood that the market will rise in the ensuing period.  
Figure 1: A simple neural network with two hidden layers of two nodes each, four inputs, and a single output node.