# Parallel Extreme Learning Machines Based on Frequency Multiplexing
## 1 Introduction  
Neural networks are usually trained by tuning the weights of each connection, which requires time and power and expensive algorithms such as gradient descent. Moreover, a scheme requiring each connection of the network to be modified during training is not easily implementable on non-conventional computational substrates. Randomized neural networks are systems in which most of the connections are selected at random and kept fixed and untrained, while only part of the weights are learned. While a randomized approach may reduce the network accuracy, it greatly increases both its training speed and physical implementability [1]. An Extreme Learning Machine (ELM) is a particular kind of randomized feed-forward neural network composed of a single hidden layer, where only output weights are trained; this allows for the formulation of the training as a linear problem [2; 3; 4; 5]. This network scheme can be implemented in a generic physical system; the input to the selected system constitutes the input layer of the network and is processed according to the physical laws of the system itself, while the output of the physical system is assumed to be the hidden layer of the network. Hence, the internal laws of the system are analogous to the fixed random connections between the input and the hidden layer of the ELM. The power of this approach consists in the fact that the system does not need to to be modified in any way since it represents the internal, untrained connections of the network, while the output weights, which have to be trained, can be set by the readout mechanism. However, not every system is expected to perform well: a \"good\" system should be able to project the input layer onto a higher dimensional space through a nonlinear transformation so that the output can be constructed as a linear combination of hidden neuron values. Photonics offers strong parallelization capabilities and a plethora of commercial-grade and potentially integrable tools to manipulate light degrees of freedom; hence, this platform is considered a good candidate for the implementation of fast non-electronicneural networks [6]. Different schemes for photonic implementations of randomized neural networks have already been explored. ELMs have been implemented in setups based on free-space propagation along scattering media [7; 8], multimode fibers [9], or time-multiplexed fiber loops [10]. Many of these substrates have also been exploited for Reservoir Computing (RC), which is a randomized approach to recursive neural networks. First, photonic implementations of RC exploited a single nonlinear node through time multiplexing [11; 12; 13], following the idea reported in [14]. Since time multiplexing implies a trade-off between elaboration speed and number of neurons, alternative schemes have been searched, investigating, for instance, free-space RCs [15; 16] or integrated passive networks [17]. It is attractive to also consider the wavelength degree of freedom for photonic information processing since it can be exploited via commercially available tools (e.g., WDM filters); hence, we developed an RC based on frequency multiplexing [18]. Our first work on frequency multiplexing ELM [19] was indeed derived from our frequency multiplexing RC scheme. The advantages of using the frequency degree of freedom have been exploited to realise perceptrons [20] and convolutional engines [21; 22] achieving very high processing speeds. In our frequency-multiplexing ELM implementation [19], the spectrum of the light travelling in the system has the shape of a frequency comb, and each comb line amplitude encodes the state of one neuron. When entering the system, the comb encodes the input layer; then, comb lines are made to interfere through periodic phase modulation, which results in a linear mixing of information encoded in the comb. Hence, the new comb generated by the mixing represents the hidden layer of the network.