\section{Introduction}

In the last decade Machine Learning (ML) has been rapidly evolving due to the profound performance improvements that Deep Learning (DL) has ushered. Deep Learning has outperformed previous state-of-the-art methods in many fields of Machine Learning, such as Natural Language Processing (NLP)~\cite{deng2018feature}, image processing~\cite{larsson2018robust} and speech generation~\cite{van2016wavenet}. As the number of new methods incorporating Deep Learning in many scientific fields increase, the proposed solutions begin to span across other disciplines where Machine Learning was used in a limited capacity. One such example is the quantitative analysis of the stock markets and the usage of Machine Learning to predict price movements or the volatility of the future prices or the detection of anomalous events in the markets.

In the field of quantitative analysis, the mathematical modelling of the markets has been the de facto approach to model stock price dynamics for trading, market making, hedging, and risk management. By utilizing a time series of values, such as the price fluctuations of financial products being traded in the markets, one can construct statistical models which can assist in the extraction of useful information about the current state of the market and a set of probabilities for possible future states, such as price or volatility changes. Many models, such as the Black-Scholes-Merton model~\cite{black1973pricing}, attempted to mathematically deduce the price of options and can be used to provide useful indications of future price movements.

However, at some point as more market participants started using the same model the behaviour of the price changed to the point that it could no longer be taken advantage of. Newer models, such as the stochastic modelling of limit order book dynamics \cite{cont2010stochastic}, the jump-diffusion processes for stock dynamics \cite{bandi2016price} and volatility estimation of market microstructure noise \cite{ait2009estimating} have been attempts predict multiple aspects of the financial markets. However such models are designed to be tractable, even at the cost of reliability and accuracy, and thus they do not necessarily fit empirical data very well.

The aforementioned properties put handcrafted models at a disadvantage, since the financial markets very frequently exhibit irrational behaviour, mainly due to the large influence of human activity, which frequently causes these models to fail. Combining Machine Learning models with handcrafted features usually improves the forecasting abilities of such models, by overcoming some of the aforementioned limitations, and improving predictions about various aspects of financial markets. This led many organizations that participate in the Financial Markets, such as Hedge Funds and investment firms, to increasingly use ML models, along with the conventional mathematical models, to make crucial decisions.

Furthermore, the introduction of electronic trading, that also led to the automation of trading operations, has magnified the volume of exchanges, producing a wealth of data. Deep Learning models are perfect candidates for analyzing such amounts of data, since they perform significantly better than the conventional Machine Learning methodologies when a large amount of data is available.  This is one of the reasons that Deep Learning is starting to have a role in analyzing the data coming from financial exchanges. \cite{kercheval2015modelling, tsantekidis2017using}

The most detailed type of data that financial exchanges are gathering is the comprehensive logs of every submitted order and event that is happening within their internal matching engine. This log can be used to reconstruct the Limit Order Book (LOB), which is explained further in Section \ref{data-section}. A basic task that can arise from this data is the prediction of future price movements of an asset by examining the current and past supply and demand of Limit Orders. This type of comprehensive logs kept by the exchanges is excessively large and traditional Machine Learning techniques, such as Support Vector Machines (SVMs) \cite{vapnik1995support}, usually cannot be applied out-of-the-box.
Utilizing this kind of data directly with existing Deep Learning methods is also not possible due to their non-stationary nature. Prices fluctuate and suffer from stochastic drift, so in order for them to be effectively utilized by DL methods a preprocessing step is required to generate stationary features from them.

The main contribution of this work is the proposal of a set of stationary features that can be readily extracted from the Limit Order Book. The proposed features are thoroughly evaluated for predicting future mid price movements from large-scale high-frequency Limit Order data using several different Deep Learning models, ranging from simple Multilayer Perceptrons (MLPs) and CNNs to Recurrent Neural Networks (RNNs). Also we propose a novel Deep Learning model that combines the feature extraction ability of Convolutional Neural Networks (CNNs) with the Long Short Term Memory (LSTM) networks' power to analyze time series.

In Section~2 related work which employs ML models on financial data is briefly presented. Then, the dataset used is described in detail in Section~3. In Section~4 the proposed stationary feature extraction methodology is presented in detail, while in Section~5 the proposed Deep Learning methods are described. In Section~6 the experimental evaluation and comparisons are provided. Finally, conclusions are drawn and future work is discussed in Section~7.

\section{Related Work}

The task of regressing the future movements of financial assets has been the subject of many recent works such as \cite{kazem2013support, hsieh2011forecasting, lei2018wavelet}. Proven models such as GARCH are improved and augmented with machine learning component such as Artificial Neural Networks \cite{michell2018stock}. New hybrid models are employed along with Neural Networks to improve upon previous performance \cite{huang2012hybrid}.

One of the most volatile financial markets is FOREX, the currency markets. In \cite{galeshchuk2016neural}, neural networks are used to predict the future exchange rate of major FOREX pairs such as USD/EUR. The model is tested with different prediction steps ranging from daily to yearly which reaches the conclusion that shorter term predictions tend to be more accurate. Other financial metrics, such as cash flow prediction, are very closely correlated to price prediction.

In \cite{heaton2016deep}, the authors propose the ``Deep Portfolio Theory'' which applies autoencoders in order to produce optimal portfolios. This approach outperforms several established benchmarks, such as the Biotechnology IBB Index. Likewise in \cite{takeuchi2013applying}, another type of autoencoders, known as Restricted Boltzmann Machine (RBM), is applied to encode the end-of-month prices of stocks. Then, the model is fine-tuned to predict whether the price will move more than the median change and the direction of such movement. This strategy is able to outperform a benchmark momentum strategy in terms of annualized returns.

Another approach is to include data sources outside the financial time series, e.g., \cite{xiong2015deep}, where phrases related to finance, such as ``mortgage'' and ``bankruptcy'' were monitored on the Google trends platform and included as an input to a recurrent neural network along with the daily S\&P 500 market fund prices. The training target is the prediction of the future volatility of the market fund's price. This approach can greatly outperform many benchmark methods, such as the autoregressive GARCH and Lasso techniques.

The surge of DL methods has dramatically improved the performance over many conventional machine learning methods on tasks, such as speech recognition \cite{graves2013speech}, image captioning\cite{xu2015show, mao2014deep}, and question answering \cite{zhu2016visual7w}. The most important building blocks of DL are the Convolutional Neural Networks (CNN) \cite{lecun1995convolutional}, and the Recurrent Neural Networks (RNNs). Also worth mentioning is the improvement of RNNs with the introduction of Long Short-Term Memory Units (LSTMs) \cite{hochreiter1997long}, which has made the analysis of time series using DL easier and more performant.

Unfortunately DL methods are prone to overfit especially in tasks such as price regression and many works exist trying to prevent such overfitting \cite{niu2012short, xi2014new}. Some might attribute overfitting to the lack of huge amounts of data that other tasks such as image and speech processing have available to them. A very rich data source for financial forecasting is the Limit Order Book. One of the few applications of ML in high frequency Limit Order Book data is \cite{kercheval2015modelling}, where several handcrafted features are created, including price deltas, bid-ask spreads and price and volume derivatives. An SVM is then trained to predict the direction of future mid price movements using all the handcrafted features. In \cite{tran2017temporal} a neural network architecture incorporating the idea of bilinear projection augmented with a temporal attention mechanism is used to predict LOB mid price.

Similarly in \cite{ntakaris2018mid, tran2017tensor} utilize the Limit Order Book data along with ML methods such as multilinear methods and smart feature selection to predict the future price movements. In our previous work~\cite{tsantekidis2017forecasting, tsantekidis2017using, passalis2017time} we introduced a large-scale high-frequency Limit Order Book dataset, that is also used in this paper, and we employed three simple DL models, the Convolutional Neural Networks (CNN), the Long-Short Term Memory Recurrent Neural Networks (LSTM RNNs) and the  Neural Bag-of-Features (N-BoF) model, to tackle the problem of forecasting the mid price movements. However, these approaches directly used the non-stationary raw Order Book data, making them vulnerable to distribution shifts and harming their ability to generalize on unseen data, as we also experimentally demonstrate in this paper.

To the best of our knowledge this is the first work that proposes a structured approach for extracting stationary price features from the Limit Order Book that can be effectively combined with Deep Learning models. We also provide an extensive evaluation of the proposed methods on a large-scale dataset with more than 4 million events. Also, a powerful model, that combines the CNN feature extraction properties with the LSTM's time series modelling capabilities, is proposed in order to improve the accuracy of predicting the price movement of stocks. The proposed combined model is also compared with the previously introduced methods using the proposed stationary price features.

\section{Limit Order Book Data}
\label{data-section}

In an order-driven financial market, a market participant can place two types of buy/sell orders. By posting a {\em limit order}, a trader promises to buy (sell) a certain amount of an asset at a specified price or less (more). The limit order book compromises on the valid limit order that are not executed or cancelled yet.

This Limit Order Book (LOB) contains all existing buy and sell orders that have been submitted and are awaiting to be executed. A limit order is placed on the queue at a given price level, where, in the case of standard limit orders, the execution priority at a given price level is dictated by the arrival time (first in, first out). A {\em market order} is is an order to immediately buy/sell a certain quantity of the asset at the best available price in the limit order book. If the requested price of a limit order is far from the best prices, it may take a long time for the execution of the limit order, in which case, the order can finally be cancelled by the trader. The orders are split between two sides, the bid (buy) and the ask (sell) side. Each side contains the orders sorted by their price, in descending order for the bid side and ascending order for the ask side.
\newcommand{\rho}{\rho}
\newcommand{\upnu}{\upnu}

Following the notation used in \cite{cont2010stochastic}, a price grid is defined as $\{\rho^{(1)}(t),\dots,\rho^{(n)}(t)\}$, where $\rho^{(j)}(t) > \rho^{(i)}(t)$ for all $j>i$. The price grid contains all possible prices and each consecutive price level is incremented by a single tick from the previous price level. The state of the order book is a continuous-time process $v(t) \equiv \left(v^{(1)}(t), v^{(2)}(t), \dots, v^{(n)}(t) \right)_{t \geq 0}$, where $|v^{(i)}(t)|$ is the number of outstanding limit orders at price $\rho^{(i)}(t)$, $1 \leq i \leq n$. If $v^{(i)}(t) < 0$, then there are $-v^{(i)}(t)$ bid orders at price $\rho^{(i)}(t)$; if $v^{(i)}(t)>0$, then there are $v^{(i)}(t)$ ask orders at price $\rho^{(i)}(t)$. That is, $v^{(i)}(t) > 0$ refers to ask orders and $v^{(i)}(t) < 0$ bid orders.

The location of the best ask price in the price grid is defined by:
\[
i_a^{(1)}(t) = \inf\{i = 1, \dots, n\ ;\ v^{(i)}(t)>0 \},
\]
and, correspondingly, the location of the best bid price is defined by:
\[
i_b^{(1)}(t) = \sup\{i = 1, \dots, n\ ;\ v^{(i)}(t)<0 \}.
\]
For simplicity, we denote the best ask and bid prices as $p_a^{(1)}(t) \equiv \rho^{\left(i_a^{(1)}(t) \right)}(t)$ and $p_b^{(1)}(t) \equiv \rho^{\left(i_b^{(1)} (t)\right)}(t)$, respectively. Notice that if there are no ask (bid) orders in the book, the ask (bid) price is not defined.

More generally, given that the $k$th best ask and bid prices exist, their locations are denoted as $i_a^{(k)}(t) \equiv i_a(t) + k-1$ and $i_b^{(k)}(t) \equiv i_b(t) + k-1$. The $k$th best ask and bid prices are correspondingly denoted by $p_a^{(k)}(t) \equiv \rho^{\left(i_a^{(k)}(t) \right)}(t)$ and $p_b^{(k)}(t) \equiv \rho^{\left(i_b^{(k)}(t) \right)}(t)$, respectively. Correspondingly, we denote the number of outstanding limit orders at the $k$th best ask and bid levels by $\upnu_a^{(k)}(t) \equiv v^{\left(i_a^{(k)}(t)\right)}(t)$ and $\upnu_b^{(k)}(t) \equiv v^{\left(i_b^{(k)}(t)\right)}(t)$, respectively.

Limit Order Book data can be used for a variety of tasks, such as the estimation of the future price trend or the regression of useful metrics, like the price volatility. Other possible tasks may include the early prediction of anomalous events, like extreme changes in price which may indicate manipulation in the markets. These examples are a few of multiple applications which can aid investors to protect their capital when unfavourable conditions exist in the markets or, in other cases, take advantage of them to profit.

Most modern methods that utilize financial time series data employ subsampling techniques, such as the well-known OHLC (Open-High-Low-Close) candles \cite{yang2000drift}, in order to reduce the number of features of each time interval. Although the OHLC candles preserve useful information, such as the market trend and movement ranges within the specified intervals, it removes possibly important microstructure information. Since the LOB is constantly receiving new orders in inconsistent intervals, it is not possible to subsample time-interval features from it in a way that preserves all the information it contains. This problem can be addressed, to some extent,  using recurrent neural network architectures, such as LSTMs, that are capable of natively handling inputs of varying size. This allows to directly utilize the data fully without using a time interval-based subsampling.

The LOB data used in this work is provided by Nasdaq Nordic and consists of 10 days worth of LOB events for 5 different Finnish company stocks, namely Kesko Oyj, Outokumpu Oyj, Sampo, Rautaruukki and Wärtsilä Oyj \cite{ntakaris2017benchmark,siikanen2016limit}. The exact time period of the gathered data begins from the 1st of June 2010 to the 14th of June 2010. Also, note that trading only happens during business days.

The data consists of consecutive snapshots of the LOB state after each state altering event takes place. This event might be an order insertion, execution or cancellation and after it interacts with the LOB and change its state a snapshot of the new state is taken. The LOB depth of the data that are used is $10$ for each side of Order Book, which ends up being 10 active orders (consisting of price and volume) for each side adding up to a total of $40$ values for each LOB snapshot. This ends up summing to a total of $4.5$ million snapshots that can be used to train and evaluate the proposed models.

In this work the task we aim to accomplish is the prediction of price movements based on current and past changes occurring in the LOB. This problem is formally defined as follows: Let $\mathbf{x}(t) \in \mathbb{R}^q$ denote the feature vector that describes the condition of the LOB at time $t$ for a specific stock, where $q$ is the dimensionality of the corresponding feature vector. The direction of the mid-price of that stock is defined as $l_k(t) = \{-1, 0, 1\}$ depending on whether the mid price decreased (-1), remained stationary (0) or increased (1) after $k$ LOB events occurred.
The number of orders $k$ is also called \textit{prediction horizon}. We aim to learn a model $f_k(\mathbf{x}(t))$, where $f_k: \mathbb{R}^{n} \rightarrow \{-1, 0, 1\} $, that predicts the direction $l_{k}(t)$ of the mid-price after $k$ orders.
In the following Section the aforementioned features and labels, as well as the procedure to calculate them are explained in depth.

\section{Stationary Feature and Label Extraction}

The raw LOB data cannot be directly used for any ML task without some kind of preprocessing. The order volume values can be gathered for all stocks' LOBs and normalized together, since they are expected to follow the same distribution. However, this is not true for price values, since the value of a stock or asset may fluctuate and increase with time to never before seen levels. This means that the statistics of the price values can change significantly with time, rendering the price time series non-stationary.

Simply normalizing all the price values will not resolve the non-stationarity, since there will always be unseen data that may change the distribution of values to ranges that are not present in the current data. We present two solutions for this problem, one used in past work where normalization is applied constantly using past available statistics and a new approach to completely convert the price data to stationary values.

\subsection{Input Normalization}
\label{sec:input-normalization}

The most common normalization scheme is standardization (z-score):
\begin{equation}
x_{\text{norm}} = \dfrac{{x} - \bar{x}}{\sigma_{\bar{x}}}
\label{zscore-eq},
\end{equation}
where ${x}$ is a feature to be normalized, $\bar{x}$ is the mean and $\sigma_{\bar{x}}$ is the standard deviation across all samples. Such normalization is separately applied to the order size values and the price values. Using this kind of ``global'' normalization allows the preservation of the different scales between prices of different stocks, which we are trying to avoid. The solution presented in \cite{tsantekidis2017forecasting,tsantekidis2017using} is to use z-score to normalize each stock-day worth of data with the means and standard deviations calculated using previous day's data of the same stock. This way a major problem is avoided which is the distribution shift in stock prices, that can be caused by events such as stock splits or the large shifts in price that can happen over longer periods of time.

Unfortunately this presents another important issue for learning. The difference between the price values in different LOB levels are almost always minuscule. Since all the price levels are normalized using z-score with the same statistics, extracting features at that scale is hard. In this work we propose a novel approach to remedy this problem. Instead of normalizing the raw values of the LOB depth, we modify the price values to be their percentage difference to the current mid price of the Order Book. This removes the non-stationarity from the price values, makes the feature extraction process easier and significantly improves the performance of ML models, as it is also experimentally demonstrated in Section~\ref{sec:experiments}. To compensate for the removal of the price value itself we add an extra value to each LOB depth sample which is the percentage change of the mid price since the previous event.

The mid-price is defined as the mid-point between the best bid and the best ask prices at time $t$ by
\begin{equation}
p_m^{(1)} (t) = \dfrac{p_a^{(1)}(t) + p_b^{(1)}(t)}{2}
\label{mid-price-def}.
\end{equation}
Let
\begin{align}
{p'}_a^{(i)}(t) =& \dfrac{p_a^{(i)}(t)}{p_m(t)} - 1, 	\label{stationary-price-a} \\
{p'}_b^{(i)}(t) =& \dfrac{p_b^{(i)}(t)}{p_m(t)} - 1,	\label{stationary-price-b}
\end{align}
and
\begin{equation}
{p'}_m(t) = \dfrac{p_m(t)}{p_m(t-1)} - 1.	\label{mid-price-change-def}
\end{equation}
Equations (\ref{stationary-price-a}) and (\ref{stationary-price-b}) serve as statistic features that represent the proportional difference between $i$th price and the mid-price at time $t$. Equation (\ref{mid-price-def}), on the other hand, serves as a dynamic feature that captures the proportional mid-price movement over the time period (that is, it represents asset's return in terms of mid-prices).

We also use the cumulative sum of the sizes of the price levels as a feature, also know as Total Depth:
\begin{align}
\upnu'^{(k)}_a(t) =& \sum_{i=1}^k{\upnu_a^{(i)}(t)}
\vspace{0.1cm} 	\label{size-cumsum-a}\\
\upnu'^{(k)}_b(t) =& \sum_{i=1}^k{\upnu_b^{(i)}(t)}
\label{size-cumsum-b}
\end{align}
where $\upnu^{(i)}_a(t)$ is number of outstanding limit order at the $i$th best ask price level and $\upnu^{(i)}_b(t)$ is number of outstanding limit order at the $b$th best ask price level.

The proposed stationary features are briefly summarized in Table \ref{features-table}. After constructing these three types of stationary features, each of them is separately normalized using standardization (z-score), as described in (\ref{zscore-eq}), and concatenated into a single feature vector $\myvec{x}_t$, where $t$ denotes the time step.

The input used for the time-aware models, such as the CNN, LSTM and CNN-LSTM, is the sequence of vectors $\myvec{X} = \{\myvec{x}_0, \myvec{x}_1, \dots , \myvec{x}_w\}$, where $w$ is the number of total number of events each one represented by a different time step input. For the models that need all the input into a single vector, such as the SVM and MLP models, the matrix $\myvec{X}$ is flatten into a single dimension so it can be used as input for these models.

\begin{table}[t]
	\caption{Brief description of each proposed stationary feature}
	\label{features-table}
	\begin{center}
		\begin{tabular}{ | c | c|}
			\hline
			\textbf{Feature} & \textbf{Description} \\
			\hline\hline
			Price level difference & \parbox[c]{10cm}{\vspace{0.2em}The difference of each price level to the current mid price, see Eq. (\ref{stationary-price-a}),(\ref{stationary-price-b})
				\[{p'}^{(i)}(t) = \dfrac{p^{(i)}(t)}{p_m(t)} - 1 \]
			} \\
			\hline
			Mid price change & \parbox[c]{10cm}{\vspace{0.2em} The change of the current mid price to the mid price of the previous time step, see Eq. (\ref{mid-price-change-def}) \\
				\[
				{p'}_m(t) = \dfrac{p_m(t)}{p_m(t-1)} - 1
				\]
			} \\
			\hline
			Depth size cumsum & \parbox[c]{10cm}{ \vspace{0.2em} Total depth at each price level, see Eq. (\ref{size-cumsum-a}), (\ref{size-cumsum-b})
				\[
				\upnu'^{(k)}(t) = \sum_{i=1}^k{\upnu^{(i)}(t)}
				\]
			} \\
			\hline
		\end{tabular}
	\end{center}
\end{table}

\subsection{Labels}
\label{sec:labels}
The proposed models aim to predict the future movements of the mid price. Therefore, the ground truth labels must be appropriately generated to reflect the future mid price movements. Note that the mid price is a ``virtual'' value and no order can be guaranteed to immediately executed if placed at that exact price. However being able to predict its upwards or downwards movement provides a good estimate of the price of the future orders. A set of discrete choices must be constructed from our data to use as target for our classification models. The labels for describing the movement denoted by $y_t \in \{-1, 0, 1\}$, where $t$ denotes the timestep.

Simply using $p_m(t + k) > p_m(t)$ to determine the upward direction of the mid price would introduce unmanageable amount of noise, since the smallest change would be registered as an upward or downward movement. To remedy this, in our previous work \cite{tsantekidis2017forecasting, tsantekidis2017using} the noisy changes of the mid price were filtered by employing two averaging filters. One averaging filter was used on a window of size $k$ of the past values of the mid price and another averaging was applied on a future window $k$:
\begin{align}
m_b(t) =& \dfrac{1}{k+1} \sum_{i=0}^k p_m(t-i) \label{m-b} \\
m_a(t) =& \dfrac{1}{k} \sum_{i=1}^k p_m(t+i) \label{m-a}
\end{align}
where $p_t$ is the mid price as described in Equation~(\ref{mid-price-def}).
The label $l_t$, that expresses the direction of price movement at time $t$, is extracted by comparing the previously defined quantities ($m_b$ and $m_a$). However, using the $m_b$ values to create labels for the samples, as in \cite{tsantekidis2017forecasting, tsantekidis2017using}, is making the problem significantly easier and predictable due to the slower adaptation of the mean filter values to sudden changes in price. Therefore, in this work we remedy this issue by replacing $m_b$ with the mid price. Therefore, the labels are redefined as:
\begin{equation}
l_t =
\begin{cases}
\ \ 1, & \text{if } \dfrac{m_a(t)}{p_m(t)} > 1 + \alpha
\vspace{0.2cm}\\
-1, & \text{if } \dfrac{m_a(t)}{p_m(t)} < 1 - \alpha
\vspace{0.2cm}\\
\ \ 0, & \text{otherwise}
\end{cases}
\label{direction-eq}
\end{equation}

where $\alpha$ is the threshold that determines how significant a mid price change $m_a(t)$ must be in order to label the movement as upward or downward. Values that do not satisfy this inequality are considered as insignificant and are labeled as having no price movement, or in other words being ``stationary''. The resulting labels present the trend to be predicted. This process is applied across all time steps of the dataset to produce labels for all the depth samples.

\section{Machine Learning Models}
In this section we explain the particular inner workings of the CNN and LSTM models that are used and present how they are combined to form the proposed CNN-LSTM model. The technical details of each model are explained along with the employed optimization procedure.

\begin{figure}
	\centering
	\includegraphics[scale=0.4]{CNN}
	\caption{A visual representation of the evaluated CNN model. Each layer includes the filter input size and the number of filters used.}
	\label{fig:cnn-model}
\end{figure}

\subsection{Convolutional Neural Networks}
\label{sec:conv-nets}

Convolutional Neural Networks (CNNs) consist of the sequential application of convolutional and pooling layers usually followed by some fully connected layers, as shown in Figure~\ref{fig:cnn-model}. Each convolutional layer $i$ is equipped with a set of filters $\mathbf{W}_i \in \mathbb{R} ^{S \times D \times N}$ that is convolved with an input tensor, where $S$ is the number of used filters, $D$ is the {filter size},  and $N$ is the number of the input channels. The input tensor $\mathbf{X} \in \mathbb{R}^{(B \times T \times F)}$ is consisted by the temporally ordered features described in Section \ref{sec:input-normalization}, where $B$ is the batch size, $T$ is the number of time steps and $F$ is the number of features per time step.

In this work we leverage the causal padding introduced in \cite{van2016wavenet} to avoid using future information to produce features for the current time step. Using a series of convolutional layers allows for capturing the fine temporal dynamics of the time series as well as correlating temporally distant features. After the last convolutional/pooling layer a set of fully connected layers are used to classify the input time series. The network's output expresses the categorical distribution for the three direction labels (upward, downward and stationary), as described in (\ref{direction-eq}), for each time-step.

We also employ a temporal batching technique, similar to the one used in LSTMs, to increase the computational efficiency and reduce memory requirements of our experiments when training with CNNs. Given the above described input tensor $\myvec{X}$ and convolution filters $\myvec{W}_i$ the last convolution produces a tensor with dimensions $ (B,T,S,N) $, which in most uses cases is flattened to a tensor of size $(B, T \times S \times N)$ before being fed to a fully connected layer. Instead we retain the temporal ordering by only reducing the tensor to dimension $(B, T, S \times N) $. An identical fully connected network with a softmax output is applied for each $S \times N$ vectors leading to $T$ different predictions.

Since we are using causal convolutions with "full" padding, all the convolutional layers produce the same time steps $T$, hence we do not need to worry about label alignment to the correct time step. Also the causal convolutions ensure that no information from the future leaks to past time step filters. This technique reduces the receptive field of the employed CNN, but this can be easily remedied by using a greater number of convolutional layers and/or a larger filter size $D$.

\subsection{Long Short Term Memory Recurrent Neural Networks}

One of the most appropriate Neural Network architectures to apply on time series is the Recurrent Neural Network (RNN) architecture. Although powerful in theory, this type of network suffers from the vanishing gradient problem, which makes the gradient propagation through a large number of steps impossible. An architecture that was introduced to solve this problem is the Long Short Term Memory (LSTM) networks~\cite{hochreiter1997long}. This architecture protects its hidden activation from the decay of unrelated inputs and gradients by using gated functions between its ``transaction'' points. The protected hidden activation is the ``cell state''  which is regulated by said gates in the following manner:

\begin{align}
\myvec{f}_t &= \sigma(\myvec{W}_{xf} \cdot \myvec{x} + \myvec{W}_{hf} \cdot \myvec{h}_{t-1} + \myvec{b}_f) \\
\myvec{i}_t &= \sigma(\myvec{W}_{xi} \cdot \myvec{x} + \myvec{W}_{hi} \cdot \myvec{h}_{t-1} + \myvec{b}_i) \\
\myvec{c}'_t &= tanh(\myvec{W}_{hc} \cdot \myvec{h}_{t-1} + \myvec{W}_{xc} \cdot \myvec{x}_t + \myvec{b}_c) \\
\myvec{c}_t &= \myvec{f}_t \cdot \myvec{c}_{t-1} + \myvec{i}_t \cdot \myvec{c}'_t  \\
\myvec{o}_t &= \sigma(\myvec{W}_{oc} \cdot \myvec{c}_t + \myvec{W}_{oh} \cdot \myvec{h}_{t-1} + \myvec{b}_o) \\
\myvec{h}_t &= \myvec{o}_t \cdot \sigma(\myvec{c}_t)
\end{align}
where $\myvec{f}_t$, $\myvec{i}_t$ and $\myvec{o}_t$ are the activations of the input, forget and output gates at time-step $t$, which control how much of the input and the previous state will be considered and how much of the cell state will be included in the hidden activation of the network. The protected cell activation at time-step $t$ is denoted by $\myvec{c}_t$, whereas $\myvec{h}_t$ is the activation that will be given to other components of the model. The matrices $\myvec{W}_{xf}, \myvec{W}_{hf}, \myvec{W}_{xi}, \myvec{W}_{hi}, \myvec{W}_{hc}, \myvec{W}_{xc}, \myvec{W}_{oc}, \myvec{W}_{oh}$ are used to denote the weights connecting each of the activations with the current time step inputs and the previous time step activations.

\subsection{Combination of models (CNN-LSTM)}

We also introduce a powerful combination of the two previously described models. The CNN model is identically applied as described in Section \ref{sec:conv-nets}, using causal convolutions and temporal batching to produce a set of features for each time step. In essence the CNN acts as the feature extractor of the LOB depth time series, which produces a new time series of features with the same length as the original one, with each of them having time steps corresponding to one another.

An LSTM layer is then applied on the time series produced by the CNN, and in turn produces a label for each time step. This works in a very similar way to the fully connected layer described in \ref{sec:conv-nets} for temporal batching, but instead of the Fully Connected layer the LSTM allows the model to incorporate the features from past steps. The model architecture is visualized in Figure~\ref{fig:cnnlstm}.

\subsection{Optimization}
\label{sec:optimization}
The parameters of the models are learned by minimizing the categorical cross entropy loss defined as:
\begin{equation}
\mathcal{L}(\myvec{W}) = -\sum_{i=1}^{L} y_i \cdot \log \hat{y}_i,
\end{equation}
where $L$ is the number of different labels and the notation $\myvec{W}$ is used to refer to the parameters of the models. The ground truth vector is denoted by  $\mathbf{y}$, while $\hat{\mathbf{y}}$ is the predicted label distribution. The loss is summed over all samples in each batch. Due to the unavoidable class imbalance of this type of dataset, a weighted loss is employed to improve the mean recall and precision across all classes:
\begin{equation}
\label{eq:loss}
\mathcal{L}(\myvec{W}) = -\sum_{i=1}^{L} c_{y_i} \cdot y_i \cdot \log \hat{y}_i,
\end{equation}
where $c_{y_i}$ is the assigned weight for the class of $y_i$. The individual weight   $c_i$ assigned to each class $i$ is calculated as:
\begin{equation}
c_i = \dfrac{|\mathcal{D}|}{n \cdot |\mathcal{D}_i|},
\end{equation}
where $ |\mathcal{D}| $ is the total number of samples in our dataset $\mathcal{D}$, $n$ is the total number of classes (which in our case is 3) and $\mathcal{D}_i$ is set of samples from our dataset that have been labeled to belong in class $i$.

The most commonly used method to minimize the loss function defined in (\ref{eq:loss}) and learn the parameters  $\myvec{W}$ of the model is gradient descent \cite{werbos1990backpropagation}:
\begin{equation}
\myvec{W}' = \myvec{W} - \eta \cdot \dfrac{\partial \mathcal{L}}{\partial \myvec{W}}
\end{equation}
where $\myvec{W}'$ are the parameters of the model after each gradient descent step and $\eta$ is the learning rate. In this work we utilize the RMSProp optimizer \cite{tieleman2012lecture}, which is an adaptive learning rate method and has been shown to improve the training time and performance of DL models.

\begin{figure}
	\centering
	\includegraphics[scale=0.3]{CNNLSTM}
	\caption{CNN-LSTM model}
	\label{fig:cnnlstm}
\end{figure}

The LSTM, CNN and CNN-LSTM models along with all the training algorithms were developed using Keras \cite{chollet2015keras}, which is a framework built on top of the Tensorflow library \cite{tensorflow2015-whitepaper}.

\section{Experimental Evaluation}
\label{sec:experiments}

All the models were tested for step sizes $k = 10, 50, 100,$ and $200$ in (\ref{m-a}), where the $\alpha$ value for each was set at $2 \times 10^{-5},\ 9 \times 10^{-5},\ 3 \times 10^{-4}$ and $ 3.5 \times 10^{-4} $ respectively. The parameter $\alpha$ was chosen in conjunction with the future horizon with the aim to have relatively balanced distribution of labels across classes. In a real trading scenario it is not possible to have a profitable strategy that creates as many trade signals as ``no-trade'' signals, because it would accumulate enormous commission costs. For that reason $\alpha$ is selected with the aim to get a logical ratio of about 20\% long, 20\% short and 60\% stationary labels. The effect of varying the parameter $\alpha$ on the class distribution of labels is shown in Table \ref{alpha-table}. Note that increasing the $\alpha$ allows for reducing the number of trade signals which should be changed depending on the actual commission and slippage costs that are expected to occur.

\begin{table}
	\caption{Example of sample distribution across classes depending on $\alpha$ for prediction horizon $k =100$}
	\label{alpha-table}
	\begin{center}

		\begin{tabular}{ | c |c| c| c|}
			\hline
			\hspace{2em}$\alpha$\hspace{2em} & \hspace{1em}Down\hspace{1em} & Stationary & \hspace{1.5em}Up\hspace{1.5em} \\
			\hline\hline
			$1.0 \times 10^{-5}$ & $0.39$&$0.17$&$0.45$ \\ \hline
			$2.0 \times 10^{-5}$ & $0.38$&$0.19$&$0.43$ \\ \hline
			$5.0 \times 10^{-5}$ & $0.35$&$0.25$&$0.41$ \\ \hline
			$1.0 \times 10^{-4}$ & $0.30$&$0.33$&$0.36$ \\ \hline
			$2.0 \times 10^{-4}$ & $0.23$&$0.49$&$0.28$ \\ \hline
			$3.0 \times 10^{-4}$ & $0.18$&$0.60$&$0.22$ \\ \hline
			$3.5 \times 10^{-4}$ & $0.15$&$0.66$&$0.19$ \\ \hline
		\end{tabular}
	\end{center}
\end{table}

We tested the CNN and LSTM models using the raw features and the proposed stationary features separately and compared the results. The architecture of the three models that were tested is described bellow.

The proposed CNN model consists of the following sequential layers:

\begin{center}
	\begin{minipage}{0.6\textwidth}
		\begin{enumerate}
			\item 1D Convolution with 16 filters of size $(10,42)$
			\item 1D Convolution with 16 filters of size $(10,)$
			\item 1D Convolution with 32 filters of size $(8,)$
			\item 1D Convolution with 32 filters of size $(6,)$
			\item 1D Convolution with 32 filters of size $(4,)$
			\item Fully connected layer with 32 neurons
			\item Fully connected layer with 3 neurons
		\end{enumerate}
	\end{minipage}
\end{center}
The activation function used for all the convolutional and fully connected layer of the CNN is the Parametric Rectifying Linear Unit (PRELU) \cite{he2015delving}. The last layer uses the softmax function for the prediction of the probability distribution between the different classes. All the convolutional layers are followed by a Batch Normalization (BN) layer after them.

The LSTM network uses 32 hidden neurons followed by a feed-forward layer with 64 neurons using Dropout and PRELU as activation function. Experimentally we found out that the hidden layer of the LSTM should contain 64 or less hidden neurons to avoid over-fitting the model. Experimenting with higher number of hidden neurons would be feasible if the dataset was even larger.

Finally the CNN-LSTM model applies the convolutional feature extraction layers on the input and then feeds them in the correct temporal order to an LSTM model. The CNN component is comprised of the following layers:
\begin{center}
	\begin{minipage}{0.6\textwidth}
		\begin{enumerate}
			\item 1D Convolution with 16 filters of size $(5,42)$
			\item 1D Convolution with 16 filters of size $(5,)$
			\item 1D Convolution with 32 filters of size $(5,)$
			\item 1D Convolution with 32 filters of size $(5,)$
		\end{enumerate}
	\end{minipage}
\end{center}

Note that the receptive field of each convolutional filter in the CNN module is smaller that the standalone CNN, since the LSTM can capture most of the information from past time steps. The LSTM module has the same exact architecture as the standalone LSTM. A visual representation of this CNN-LSTM model is shown in Figure~\ref{fig:cnnlstm}. Likewise, PRELU is the activation function used for the CNN and the fully connected layers, while the softmax function is used for the output layer of the network to predict the probability distribution of the classes.

\begin{figure}
	\centering
	\includegraphics[scale=0.70]{lstm_cost_per_step}
	\caption{Mean cost per recurrent step of the LSTM network}
	\label{bad-step-score}
\end{figure}

\begin{table*}
	\caption{Experimental results for different prediction horizons $k$. The values that are reported are the mean of each metric for the last 20 training epochs.}
	\label{results-table}

	\begin{center}
\footnotesize
\bgroup
\def\arraystretch{0.8
		\begin{tabular}{ |c|c|c|c|c|c|c|c|c|}
			\hline

			\multirow{1}{*}{\textbf{Feature Type}} &
			\multicolumn{1}{c|}{\textbf{Model}} &
			\multicolumn{1}{c|}{\textbf{Mean Recall}} &
			\multicolumn{1}{c|}{\textbf{Mean Precision}} &
			\multicolumn{1}{c|}{\textbf{Mean F1}} &  \multicolumn{1}{c|}{\textbf{Cohen's} $\kappa$} \\  \cline{1-6}

			\multicolumn{6}{|c|}{\multirow{2}{*}{Prediction Horizon $k=10$}} \\
			\multicolumn{6}{|c|}{} \\ \cline{1-6}
			\multirow{4}{*}{\textbf{Raw Values}}
			& SVM &  $0.35  $ & $0.43  $ & $0.33  $ & $0.04   $ \\ \cline{2-6}
			& MLP &  ${ 0.34   }$ & ${ 0.34   }$ & ${ 0.09   }$ & ${ 0.00   }$ \\ \cline{2-6}
			& CNN &  ${ 0.51   }$ & ${ 0.42   }$ & ${ 0.38   }$ & ${ 0.14   }$ \\ \cline{2-6}
			& LSTM &  ${ 0.49   }$ & ${ 0.41   }$ & ${ 0.35   }$ & ${ 0.12   }$ \\ \cline{1-6}

			\multirow{5}{*}{\textbf{Stationary Features}}
			& SVM &  $0.33  $ & $\mathbf{0.46  }$ & $0.30  $ & $0.011  $ \\ \cline{2-6}
			& MLP  &  ${ 0.34   }$ & ${ 0.35   }$ & ${ 0.09   }$ & ${ 0.00   }$ \\ \cline{2-6}
			& CNN  &  ${ 0.54   }$ & ${ 0.44   }$ & ${ 0.43   }$ & ${ 0.19   }$ \\ \cline{2-6}
			& LSTM  &  ${ 0.55   }$ & ${ 0.45   }$ & ${ 0.42   }$ & ${ 0.18   }$ \\ \cline{2-6}
			& CNNLSTM  &  $\mathbf{ 0.56   }$ & ${ 0.45   }$ & $\mathbf{ 0.44   }$ & $\mathbf{ 0.21   }$ \\ \cline{1-6}

			\multicolumn{6}{|c|}{\multirow{2}{*}{Prediction Horizon $k=50$}} \\
			\multicolumn{6}{|c|}{} \\ \cline{1-6}

			\multirow{4}{*}{\textbf{Raw Values}}
			& SVM &  $0.35  $ & $0.41  $ & $0.32  $ & $0.03   $ \\ \cline{2-6}
			& MLP  &  ${ 0.41   }$ & ${ 0.38   }$ & ${ 0.21   }$ & ${ 0.04   }$ \\ \cline{2-6}
			& CNN  &  ${ 0.50   }$ & ${ 0.42   }$ & ${ 0.37   }$ & ${ 0.13   }$ \\ \cline{2-6}
			& LSTM  &  ${ 0.46   }$ & ${ 0.40   }$ & ${ 0.34   }$ & ${ 0.10   }$ \\ \cline{1-6}

			\multirow{5}{*}{\textbf{Stationary Features}}
			& SVM &  $0.39   $ & $0.41   $ & $0.38   $ & $0.09    $ \\ \cline{2-6}
			& MLP &  $0.49  $ & $0.43  $ & $0.38  $ & $0.14   $ \\ \cline{2-6}
			& CNN  &  $0.55   $ & $0.45   $ & $0.43   $ & $0.20   $ \\ \cline{2-6}
			&LSTM  &  $\mathbf{0.56  } $ & $0.46  $ & $0.44   $ & $0.21  $ \\ \cline{2-6}
			& CNNLSTM &  $\mathbf{0.56  }$ & $\mathbf{0.47  }$ & $\mathbf{0.47  }$ & $\mathbf{0.24  } $ \\ \cline{1-6}

			\multicolumn{6}{|c|}{\multirow{2}{*}{Prediction Horizon $k=100$}} \\
			\multicolumn{6}{|c|}{} \\ \cline{1-6}

			\multirow{4}{*}{\textbf{Raw Values}}
			& SVM &  $0.35  $ & $0.46  $ & $0.33  $ & $0.05   $ \\ \cline{2-6}
			& MLP  &  ${ 0.45   }$ & ${ 0.39   }$ & ${ 0.26   }$ & ${ 0.06   }$ \\ \cline{2-6}
			& CNN  &  ${ 0.49   }$ & ${ 0.42   }$ & ${ 0.37   }$ & ${ 0.12   }$ \\ \cline{2-6}
			& LSTM  &  ${ 0.45   }$ & ${ 0.39   }$ & ${ 0.34   }$ & ${ 0.09   }$ \\ \cline{1-6}

			\multirow{5}{*}{\textbf{Stationary Features}}
			& SVM &  $0.36  $ & $0.46  $ & $0.35  $ & $0.07   $ \\ \cline{2-6}
			& MLP  &  ${ 0.50   }$ & ${ 0.43   }$ & ${ 0.39   }$ & ${ 0.14   }$ \\ \cline{2-6}
			& CNN  &  ${ 0.54   }$ & ${ 0.46   }$ & ${ 0.44   }$ & ${ 0.21   }$ \\ \cline{2-6}
			& LSTM  &  $\mathbf{ 0.56   }$ & ${ 0.46   }$ & ${ 0.44   }$ & ${ 0.20   }$ \\ \cline{2-6}
			& CNNLSTM  &  ${ 0.55   }$ & $\mathbf{ 0.47   }$ & $\mathbf{ 0.48   }$ & $\mathbf{ 0.24   }$ \\ \cline{1-6}

			\multicolumn{6}{|c|}{\multirow{2}{*}{Prediction Horizon $k=200$}} \\
			\multicolumn{6}{|c|}{} \\ \cline{1-6}

			\multirow{4}{*}{\textbf{Raw Values}}
			& SVM &  $0.35  $ & $0.44  $ & $0.31  $ & $0.04   $ \\ \cline{2-6}
			& MLP  &  ${ 0.44   }$ & ${ 0.40   }$ & ${ 0.32   }$ & ${ 0.08   }$ \\ \cline{2-6}
			& CNN  &  ${ 0.47   }$ & ${ 0.43   }$ & ${ 0.39   }$ & ${ 0.14   }$ \\ \cline{2-6}
			& LSTM  &  ${ 0.42   }$ & ${ 0.39   }$ & ${ 0.36   }$ & ${ 0.08   }$ \\ \cline{1-6}

			\multirow{5}{*}{\textbf{Stationary Features}}
			& SVM &  $0.38   $ & $0.46   $ & $0.36   $ & $0.10   $ \\ \cline{2-6}
			& MLP  &  ${ 0.49   }$ & ${ 0.45   }$ & ${ 0.42   }$ & ${ 0.17   }$ \\ \cline{2-6}
			& CNN  &  ${ 0.51   }$ & ${ 0.47   }$ & ${ 0.45   }$ & ${ 0.20   }$ \\ \cline{2-6}
			& LSTM  &  ${ 0.52   }$ & ${ 0.47   }$ & ${ 0.46   }$ & ${ 0.22   }$ \\ \cline{2-6}
			& CNNLSTM  &  $\mathbf{ 0.53   }$ & $\mathbf{ 0.48   }$ & $\mathbf{ 0.49   }$ & $\mathbf{ 0.25   }$ \\ \cline{1-6}

		\end{tabular}
\egroup

	\end{center}
\end{table*}

One recurring effect we observe when training LSTM networks on LOB data is that for the first steps of observation the predictions $y_i$ yield a bigger cross entropy cost, meaning worse performance in our metrics. We run a set of experiments where the LSTM was trained for all the steps of the input windows $T$. The resulting mean cost per time step can be observed in Figure \ref{bad-step-score}. As a result, trying to predict the price movement using insufficient past information is not possible and should be avoided since it leads to noisy gradients. To avoid this, a ``burn-in'' input is initially used to build its initial perception of the market before actually making correct decisions. In essence the first ``burn-in'' steps of the input are skipped, by not allowing any gradient to alter our model until after the 100th time step. We also apply the same method to the CNN-LSTM model.

\begin{figure*}
	\centering
	\includegraphics[width=1.02\linewidth]{all_training}
	\caption{F1 and Cohen's $\kappa$ metrics during training for prediction horizon $k=100$. Plots are smoothed with a mean filter with window=3 to reduce fluctuations.
	}
	\label{fig:f1-kappa-training}
\end{figure*}

For training the models, the dataset is split as follows. The first 7 days of each stock are used to train the models, while the final 3 days are used as test data. The experiments were conducted for 4 different prediction horizons $k$, as defined in  (\ref{m-a}) and (\ref{direction-eq}).

Performance is measured using Cohen's kappa \cite{cohen1960coefficient}, which is used to evaluate the concordance between sets of given answers, taking into consideration the possibility of random agreements happening. The mean recall, mean precision and mean F1 score between all 3 classes is also reported. Recall is the number of true positives samples divided by the sum of true positives and false negatives, while precision is the number of true positive divided by the sum of true positives and false positives. F1 score is the harmonic mean of the precision and recall metrics.

The results of the experiments are shown in Table \ref{results-table}. The results are compared for the models trained on the raw price features with the ones trained using the extracted stationary features. The results confirm that extracting stationary features from the data significantly improve performance of Deep Learning models such as CNNs and LSTMs.

We also trained a Linear SVM model and a simple MLP model and compared them to the DL models. The SVM model was trained using Stochastic Gradient Descent since the size of the dataset is too large to use a regular Quadratic Programming solver. The SVM model implementation is provided by the sklearn library \cite{pedregosa2011scikit}. The MLP model consists of three fully connected layers with sizes 128, 64, 32, and PRELU as activations for each layers. Dropout is also used to avoid overfitting and the softmax activation function was used in the last layer.

Since both the SVM and the MLP models cannot iterate over timesteps to gain the same amount of information as the CNN and LSTM-based models, a window of 50 depth events is used and is flattened into a single sample. This process is applied in a rolling fashion for all the dataset to generate a dataset upon which the two models can be trained. One important note is the training fluctuations that are observed in Figure \ref{fig:f1-kappa-training}, which are caused by the great class imbalance. Similar issues where observed in initial experiments with CNN and LSTM models but using the weighted loss described in \ref{sec:optimization} the fluctuations subsided.

The proposed stationary price features significantly outperform the raw price features for all the tested models. This can be attributed to a great extent to the stationary nature of the proposed features. The employed price differences provide an intrinsically stationary and normalized price measure that can be directly used. This is in contrast with the raw price values that requires careful normalization to ensure that their values remain into a reasonable range and suffer for significantly non-stationarity issues when the price increases to levels not seen before. By converting the actual prices to the their difference to the mid price and normalize that, this important feature is exaggerated to avoid being suppressed by the much larger price movements through time. The proposed combination model CNN-LSTM also outperforms its separated individual component models as shown in Figure \ref{fig:f1-kappa-training} and Table \ref{results-table} showing that it can better handle the LOB data and use them to take advantage of the microstructure existing within the data to produce more accurate predictions.

\section{Conclusion}

In this paper we proposed a novel method for extracting stationary features from raw LOB data, suitable for use with different DL models. Using different ML models, i.e., SVMs, MLPs, CNNs and LSTMs, it was experimentally demonstrated that the proposed features significantly outperform the raw price features. The proposed stationary features achieve this by making the difference between the prices in the LOB depth the main metric instead of the price itself, which usually fluctuates much more through time than the price level within the LOB. A novel combined CNN-LSTM model was also proposed for time series predictions and it was demonstrated that exhibits more stable behaviour and leads to better results that the CNN and LSTM models.

There are several interesting future research directions. As with all the DL application, more data would enable the use of bigger models that would not be at risk of being overtrained as it was observed in this work. An RNN-type of network could be also used to perform a form of ``intelligent'' r-sampling extracting useful features from a specific and limited time-interval of depth events, which would avoid losing information and allow for the later models produce prediction for a certain time period and not for a number of following events. Another important addition would be an attention mechanism \cite{xu2015show}, \cite{cho2015describing}, which would allow for the better observation of the features by the network allowing it to ignore noisy parts of the data and use only the relevant information.

\section*{Acknowledgment}
The research leading to these results has received funding from the H2020 Project BigDataFinance MSCA-ITN-ETN 675044 (http://bigdatafinance.eu), Training for Big Data in Financial Research and Risk Management.

\bibliographystyle{elsarticle-num}


\section{Introduction}
Given a data set and a model with some unknown parameters, the inverse problem aims to find the values of the model parameters that best fit the data.
In this work, in which we focus on systems of interacting elements,
 the inverse problem concerns  the statistical inference
 of the underling interaction network and of its coupling coefficients from observed data on the dynamics  of the system.
 Versions of this problem are encountered in physics, biology (e.g., \cite{Balakrishnan11,Ekeberg13,Christoph14}), social sciences and finance (e.g.,\cite{Mastromatteo12,yamanaka_15}), neuroscience (e.g., \cite{Schneidman06,Roudi09a,tyrcha_13}), just to cite a few, and are becoming more and more important due to the increase in the amount of data available from these fields.\\
 \indent
 A standard approach used in statistical inference is to predict the interaction couplings by maximizing the likelihood function.
 This technique, however, requires the evaluation of the

   partition function that, in the most general case, concerns a number of computations scaling exponentially with the system size.

    Boltzmann machine learning uses Monte Carlo sampling to compute the gradients of the Log-likelihood looking for stationary points \cite{Murphy12} but this method is computationally manageable only for small systems. A series of faster approximations, such as naive mean-field, independent-pair approximation \cite{Roudi09a, Roudi09b}, inversion of TAP equations \cite{Kappen98,Tanaka98}, small correlations expansion \cite{Sessak09}, adaptive TAP \cite{Opper01}, adaptive  cluster expansion \cite{Cocco12} or Bethe approximations \cite{Ricci-Tersenghi12, Nguyen12}  have, then, been developed. These techniques take as input means and correlations of observed variables and most of them assume a fully connected graph as underlying connectivity network, or expand  around it by perturbative dilution.  In most cases, network reconstruction turns out to  be not accurate for small data sizes and/or when couplings are  strong or, else, if the original interaction network is sparse.\\
\indent
 A further method, substantially improving performances for small data, is the so-called Pseudo-Likelyhood Method (PLM) \cite{Ravikumar10}. In Ref. \cite{Aurell12} Aurell and Ekeberg performed a comparison between PLM and some of the just mentioned mean-field-based algorithms on the pairwise interacting Ising-spin  ($\sigma = \pm 1$) model, showing how PLM performs sensitively better, especially on sparse graphs and in the high-coupling limit, i.e., for low temperature.

 In this work, we aim at performing statistical inference  on a model whose interacting variables are continuous $XY$ spins, i.e., $\sigma \equiv \left(\cos \phi,\sin \phi\right)$ with $\phi \in [0, 2\pi )$. The developed tools can, actually, be also straightforward applied  to the $p$-clock model  \cite{Potts52} where the phase $\phi$ takes discretely equispaced $p$ values  in the $2 \pi$ interval, $\phi_a =  a 2 \pi/p$, with $a= 0,1,\dots,p-1$. The $p$-clock model, else called vector Potts model, gives a hierarchy of discretization of the $XY$ model as $p$ increases. For $p=2$, one recovers the Ising model, for $p=4$ the Ashkin-Teller model \cite{Ashkin43}, for $p=6$ the ice-type model \cite{Pauling35,Baxter82} and the eight-vertex model \cite{Sutherland70,Fan70,Baxter71} for $p=8$.
It turns out to be very useful also for numerical implementations of the continuous $XY$ model.
Recent analysis on the multi-body $XY$ model has shown that for a limited number of discrete phase values ($p\sim 16, 32$) the thermodynamic critical properties of the $p\to\infty$ $XY$ limit are promptly recovered \cite{Marruzzo15, Marruzzo16}.
Our main motivation to study statistical inference is that these kind of models have recently turned out to be rather useful in describing the behavior of optical systems,
including standard mode-locking lasers \cite{Gordon02,Gat04,Angelani07,Marruzzo15} and random lasers \cite{Angelani06a,Leuzzi09a,Antenucci15a,Antenucci15b,Marruzzo16}.
In particular, the inverse problem on the pairwise XY model analyzed here might be of help in recovering images from light propagated through random media.

      This paper is organized as follows: in Sec. \ref{sec:model} we introduce the general model  and we discuss its  derivation also  as a model for light transmission through random scattering media.
    In Sec. \ref{sec:plm} we introduce the PLM with $l_2$ regularization  and with decimation, two variants of the PLM respectively introduced in Ref. \cite{Wainwright06} and \cite{Aurell12} for the inverse Ising problem.
    Here, we analyze these techniques for continuous $XY$ spins and we test them on thermalized data generated by Exchange Monte Carlo numerical simulations of the original model dynamics. In Sec. \ref{sec:res_reg} we  present the results related to the PLM-$l_2$. In Sec. \ref{sec:res_dec} the results related to the PLM with decimation are reported and its performances are compared to the PLM-$l_2$ and to a variational mean-field method analyzed in Ref. \cite{Tyagi15}. In Sec. \ref{sec:conc}, we outline conclusive remarks and perspectives.

        \section{The leading $XY$ model}
      \label{sec:model}
 The leading model we are considering is defined, for a system of $N$ angular $XY$ variables, by the Hamiltonian
 \begin{equation}
  \mathcal{H} = - \sum_{ik}^{1,N} J_{ik} \cos{\left(\phi_i-\phi_k\right)}
  \label{eq:HXY}

  \end{equation}

  The $XY$ model is well known in statistical mechanics, displaying important physical
  insights, starting from the Berezinskii-Kosterlitz-Thouless
  transition in two dimensions\cite{Berezinskii70,Berezinskii71,Kosterlitz72} and moving to, e.g., the
  transition of liquid helium to its superfluid state \cite{Brezin82}, the roughening transition of the interface of a crystal in equilibrium with its vapor \cite{Cardy96}. In presence of disorder and frustration \cite{Villain77,Fradkin78} the model has been adopted to describe synchronization problems as the Kuramoto model \cite{Kuramoto75} and in the theoretical modeling of Josephson junction arrays \cite{Teitel83a,Teitel83b} and arrays of coupled lasers \cite{Nixon13}.
  Besides several derivations and implementations of the model in quantum and classical physics, equilibrium or out of equilibrium, ordered or fully frustrated systems, Eq. (\ref{eq:HXY}), in its generic form,
  has found applications also in other fields. A rather fascinating example being the behavior of starlings flocks  \cite{Reynolds87,Deneubourg89,Huth90,Vicsek95, Cavagna13}.
  Our interest on the $XY$ model resides, though, in optics. Phasor and phase models with pairwise and  multi-body interaction terms can, indeed, describe the behavior of electromagnetic modes in both linear and nonlinear optical systems in the analysis of problems such as light propagation and lasing \cite{Gordon02, Antenucci15c, Antenucci15d}. As couplings are strongly frustrated, these models turn out to be especially useful to the study of optical properties in random media \cite{Antenucci15a,Antenucci15b}, as in the noticeable case of random lasers \cite{Wiersma08,Andreasen11,Antenucci15e}  and they might as well be applied to linear scattering problems, e.g., propagation of waves in opaque systems or disordered fibers.

  \subsection{A propagating wave model}
  We briefly mention a derivation of the model as a proxy for the propagation of light through random linear media.
 Scattering of light is held responsible to obstruct our view and make objects opaque. Light rays, once that they enter the material, only exit  after getting scattered multiple times within the material. In such a disordered medium, both the direction and the phase of the propagating waves are random. Transmitted light
 yields a disordered interference pattern typically  having low intensity, random phase and almost no resolution, called a speckle. Nevertheless, in recent years it has been realized that disorder is rather a blessing in disguise \cite{Vellekoop07,Vellekoop08a,Vellekoop08b}. Several experiments have made it possible to control the behavior of light and other optical processes in a given random disordered medium,
 by exploiting, e.g.,  the tools developed for wavefront shaping to control the propagation of light and to engineer the confinement of light \cite{Yilmaz13,Riboli14}.
 \\
 \indent
 In a linear dielectric medium, light propagation can be described through a part of the scattering matrix, the transmission matrix $\mathbb{T}$, linking the outgoing to the incoming fields.
 Consider the case in which there are $N_I$ incoming channels and $N_O$ outgoing ones; we can indicate with $E^{\rm in,out}_k$ the input/output electromagnetic field phasors of channel $k$. In the most general case, i.e., without making any particular assumptions on the field polarizations, each light mode and its polarization polarization state can be represented by means of the $4$-dimensional Stokes vector. Each $ t_{ki}$ element of $\mathbb{T}$, thus, is a $4 \times 4$ M{\"u}ller matrix. If, on the other hand, we know that the source is polarized and the observation is made on the same polarization, one can use a scalar model and adopt Jones calculus \cite{Goodman85,Popoff10a,Akbulut11}:
   \begin{eqnarray}
 E^{\rm out}_k = \sum_{i=1}^{N_I}  t_{ki} E^{\rm in}_i \qquad \forall~ k=1,\ldots,N_O
 \label{eq:transm}
 \end{eqnarray}
  We recall that the elements of the transmission matrix are random complex coefficients\cite{Popoff10a}. For the case of completely unpolarized modes, we can also use a scalar model similar to Eq. \eqref{eq:transm}, but whose variables are  the intensities of the outgoing/incoming fields, rather than the fields themselves.\\
In the following, for simplicity, we will consider Eq. (\ref{eq:transm}) as our starting point,
where $E^{\rm out}_k$, $E^{\rm in}_i$ and $t_{ki}$ are all complex scalars.
If Eq. \eqref{eq:transm} holds for any $k$, we can write:
  \begin{eqnarray}
  \int \prod_{k=1}^{N_O} dE^{\rm out}_k \prod_{k=1}^{N_O}\delta\left(E^{\rm out}_k - \sum_{j=1}^{N_I}  t_{kj} E^{\rm in}_j \right) = 1
  \nonumber
  \\
  \label{eq:deltas}
  \end{eqnarray}

 Observed data are a noisy representation of the true values of the fields. Therefore, in inference problems it is statistically more meaningful to take that noise into account in a probabilistic way,
 rather than looking  at the precise solutions of the exact equations (whose parameters are unknown).
 To this aim we can introduce Gaussian distributions whose limit for zero variance are the Dirac deltas in Eq. (\ref{eq:deltas}).
 Moreover, we move to consider the ensemble of all possible solutions of Eq. (\ref{eq:transm}) at given $\mathbb{T}$, looking at  all configurations of input fields. We, thus, define the function:

   \begin{eqnarray}
  Z &\equiv &\int_{{\cal S}_{\rm in}} \prod_{j=1}^{N_I}  dE^{\rm in}_j \int_{{\cal S}_{\rm out}}\prod_{k=1}^{N_O} dE^{\rm out}_k
  \label{def:Z}
\\
    \times
  &&\prod_{k=1}^{N_O}
   \frac{1}{\sqrt{2\pi \Delta^2}}  \exp\left\{-\frac{1}{2 \Delta^2}\left|
  E^{\rm out}_k -\sum_{j=1}^{N_I}  t_{kj} E^{\rm in}_j\right|^2
\right\}
\nonumber
 \end{eqnarray}
   We stress that the integral of Eq. \eqref{def:Z} is not exactly a Gaussian integral. Indeed, starting from Eq. \eqref{eq:deltas}, two constraints on the electromagnetic field intensities must be taken into account.

  The space of solutions is  delimited by the total power ${\cal P}$ received by system, i.e.,
  ${\cal S}_{\rm in}: \{E^{\rm in} |\sum_k I^{\rm in}_k = \mathcal{P}\}$, also implying  a constraint on the total amount of energy that is transmitted through the medium, i. e.,
  ${\cal S}_{\rm out}:\{E^{\rm out} |\sum_k I^{\rm out}_k=c\mathcal{P}\}$, where the attenuation factor  $c<1$ accounts for total losses.
  As we will see more in details in the following, being interested in inferring the transmission matrix through the PLM, we can omit to explicitly include these terms in Eq. \eqref{eq:H_J} since they do not depend on $\mathbb{T}$ not adding any information on the gradients with respect to the elements of $\mathbb{T}$.

 Taking the same number of incoming and outcoming channels, $N_I=N_O=N/2$, and  ordering the input fields in the first $N/2$ mode indices and the output fields in the last $N/2$ indices, we can drop the ``in'' and ``out'' superscripts and formally write $Z$  as a partition function
    \begin{eqnarray}
        \label{eq:z}
 && Z =\int_{\mathcal S} \prod_{j=1}^{N} dE_j \left(   \frac{1}{\sqrt{2\pi \Delta^2}} \right)^{N/2}
 \hspace*{-.4cm} \exp\left\{
  -\frac{ {\cal H} [\{E\};\mathbb{T}] }{2\Delta^2}
  \right\}
  \\
&&{\cal H} [\{E\};\mathbb{T}] =
-  \sum_{k=1}^{N/2}\sum_{j=N/2+1}^{N} \left[E^*_j t_{jk} E_k + E_j t^*_{kj} E_k^*
\right]
 \nonumber
\\
&&\qquad\qquad \qquad + \sum_{j=N/2+1}^{N} |E_j|^2+ \sum_{k,l}^{1,N/2}E_k
U_{kl} E_l^*
 \nonumber
 \\
 \label{eq:H_J}
 &&\hspace*{1.88cm } = - \sum_{nm}^{1,N} E_n J_{nm} E_m^*
 \end{eqnarray}
 where ${\cal H}$ is a real-valued function by construction, we have introduced the effective input-input coupling matrix
\begin{equation}
U_{kl} \equiv \sum_{j=N/2+1}^{N}t^*_{lj} t_{jk}
 \label{def:U}
 \end{equation}
 and the whole interaction matrix reads (here $\mathbb{T} \equiv \{ t_{jk} \}$)
 \begin{equation}
 \label{def:J}
 \mathbb J\equiv \left(\begin{array}{ccc|ccc}
 \phantom{()}&\phantom{()}&\phantom{()}&\phantom{()}&\phantom{()}&\phantom{()}\\
 \phantom{()}&-\mathbb{U} \phantom{()}&\phantom{()}&\phantom{()}&{\mathbb{T}}&\phantom{()}\\
\phantom{()}&\phantom{()}&\phantom{()}&\phantom{()}&\phantom{()}&\phantom{()}\\
 \hline
\phantom{()}&\phantom{()}&\phantom{()}&\phantom{()}&\phantom{()}&\phantom{()}\\
 \phantom{()}& \mathbb   T^\dagger&\phantom{()}&\phantom{()}& - \mathbb{I} &\phantom{()}\\
\phantom{a}&\phantom{a}&\phantom{a}&\phantom{a}&\phantom{a}&\phantom{a}\\
 \end{array}\right)
 \end{equation}

 Determining the electromagnetic complex amplitude configurations that minimize the {\em cost function} ${\cal H}$, Eq.  (\ref{eq:H_J}),  means to maximize the overall distribution peaked around the solutions of the transmission Eqs. (\ref{eq:transm}). As the variance $\Delta^2\to 0$, eventually, the initial set of Eqs. (\ref{eq:transm}) are recovered. The ${\cal H}$ function, thus, plays the role of an Hamiltonian and  $\Delta^2$ the role of a noise-inducing temperature. The exact numerical problem corresponds to the zero temperature limit of the statistical mechanical problem. Working with real data, though, which are noisy, a finite ``temperature''
  allows for a better representation of the ensemble of solutions to the sets of equations of continuous variables.

  Now, we can express every phasor in Eq. \eqref{eq:z}  as $E_k = A_k e^{\imath \phi_k}$. As a working hypothesis we will consider the intensities $A_k^2$ as either homogeneous or as \textit{quenched} with respect to phases.
The first condition occurs, for instance, to the input intensities $|E^{\rm in}_k|$ produced by a phase-only spatial light modulator (SLM) with homogeneous illumination \cite{Popoff11}.
With \textit{quenched} here we mean, instead, that the intensity of each mode is the same for every solution of Eq. \eqref{eq:transm} at fixed $\mathbb T$.
We stress that, including intensities in the model does not preclude the inference analysis but it is out of the focus of the present work and will be considered elsewhere.

If all intensities are uniform in input and in output, this amount to a constant rescaling for each one of the four sectors of matrix $\mathbb J$ in Eq. (\ref{def:J}) that will not change the properties of the matrices.
For instance, if the original transmission matrix is unitary, so it will be the rescaled one and the matrix $\mathbb U$ will be  diagonal.
Otherwise, if intensities are \textit{quenched}, i.e., they can be considered as constants in Eq. (\ref{eq:transm}),
they are inhomogeneous with respect to phases. The generic Hamiltonian element will, therefore, rescale as
  \begin{eqnarray}
  E^*_n J_{nm} E_m = J_{nm} A_n A_m e^{\imath (\phi_n-\phi_m)} \to J_{nm} e^{\imath (\phi_n-\phi_m)}
  \nonumber
  \end{eqnarray}
  and the properties of the original  $J_{nm}$ components are not conserved  in the rescaled one. In particular, we have no argument, anymore, to possibly set the rescaled $U_{nm}\propto \delta_{nm}$.
  Eventually, we end up with the complex couplings $XY$ model, whose real-valued Hamiltonian is written as
 \begin{eqnarray}
  \mathcal{H}& = &  - \frac{1}{2} \sum_{nm} J_{nm} e^{-\imath (\phi_n - \phi_m)}  + \mbox{c.c.}
    \label{eq:h_im}
\\    &=&  - \frac{1}{2} \sum_{nm} \left[J^R_{nm} \cos(\phi_n - \phi_m)+
  J^I_{nm}\sin (\phi_n - \phi_m)\right]
  \nonumber
 \end{eqnarray}
where $J_{nm}^R$ and $J_{nm}^I$ are the real and imaginary parts of $J_{nm}$. Being $\mathbb J$  Hermitian, $J^R_{nm}=J^R_{mn}$ is symmetric and $J_{nm}^I=-J_{mn}^I$ is skew-symmetric.

\begin{comment}
\textcolor{red}{
F: comment about quenched:
I think that to obtain the XY model, it is not necessary that the intensities are strictly quenched (that is also a quite unfeasible situation, I guess).
Indeed eq (2) does not deal with the dynamics of the modes, but just connect the in and out ones.
For this, what it is necessary to have the XY model, it is that the intensities are always the same on the different samples
(so that the matrix $t_{ij}$ is the same for different phase data). If the intensities are fixed, then they can be incorporated in $t_{ij}$ and eq (2) can be written just for phases as described. \\
}
\end{comment}

  \section{Pseudolikelihood Maximization}
  \label{sec:plm}
The inverse problem consists in the reconstruction of the parameters $J_{nm}$ of the Hamiltonian, Eq. (\ref{eq:h_im}).
Given a set of $M$ data configurations of $N$ spins
 $\bm\sigma = \{ \cos \phi_i^{(\mu)},\sin \phi_i^{(\mu)} \}$, $i = 1,\dots,N$ and $\mu=1,\dots,M$, we want to \emph{infer} the couplings:
 \begin{eqnarray}
\bm \sigma  \rightarrow  \mathbb{J}
\nonumber
 \end{eqnarray}
 With this purpose in mind,
 in the rest of this section we implement the working equations for the techniques used.
 In order to test our methods, we generate the input data, i.e., the configurations, by Monte-Carlo simulations of the model.
 The joint probability distribution of the $N$ variables $\bm{\phi}\equiv\{\phi_1,\dots,\phi_N\}$, follows the Gibbs-Boltzmann distribution:
 \begin{equation}\label{eq:p_xy}
 P(\bm{\phi}) = \frac{1}{Z} e^{-\beta \mathcal{H\left(\bm{\phi}\right)}} \quad \mbox{ where } \quad Z = \int \prod_{k=1}^N d\phi_k  e^{-\beta \mathcal{H\left(\bm{\phi}\right)}}
 \end{equation}
 and where we denote $\beta=\left( 2\Delta^2 \right)^{-1}$ with respect to Eq. (\ref{def:Z}) formalism.
 In order to stick to usual statistical inference notation, in the following we will rescale the couplings by a factor $\beta / 2$: $\beta J_{ij}/2 \rightarrow J_{ij}$.
 The main idea of the PLM is to work with the conditional probability distribution of one variable $\phi_i$ given all other variables,
 $\bm{\phi}_{\backslash i}$:

  \begin{eqnarray}
	\nonumber
   P(\phi_i | \bm{\phi}_{\backslash i}) &=& \frac{1}{Z_i} \exp \left \{ {H_i^x (\bm{\phi}_{\backslash i})
  	\cos \phi_i + H_i^y (\bm{\phi}_{\backslash i}) \sin \phi_i } \right \}
	\\
 \label{eq:marginal_xy}
	&=&\frac{e^{H_i(\bm{\phi}_{\backslash i}) \cos{\left(\phi_i-\alpha_i(\bm{\phi}_{\backslash i})\right)}}}{2 \pi I_0(H_i)}
  \end{eqnarray}
  where $H_i^x$ and $H_i^y$ are defined as
   \begin{eqnarray}
   H_i^x (\bm{\phi}_{\backslash i})  &=& \sum_{j (\neq i)} J^R_{ij} \cos \phi_j  - \sum_{j  (\neq i) } J_{ij}^{I} \sin \phi_j \phantom{+  h^R_i} \label{eq:26} \\
   H_i^y  (\bm{\phi}_{\backslash i})  &=&  \sum_{j  (\neq i)} J^R_{ij}  \sin \phi_j  + \sum_{j  (\neq i) } J_{ij}^{I}   \cos \phi_j \phantom{ + h_i^{I} }\label{eq:27}
   \end{eqnarray}
and $H_i= \sqrt{(H_i^x)^2 + (H_i^y)^2}$, $\alpha_i = \arctan H_i^y/H_i^x$   and we introduced the modified Bessel function of the first kind:
  \begin{equation}
  \nonumber
   I_k(x) = \frac{1}{2 \pi}\int_{0}^{2 \pi} d \phi e^{x \cos{ \phi}}\cos{k \phi}
  \end{equation}

   Given $M$ observation samples $\bm{\phi}^{(\mu)}=\{\phi^\mu_1,\ldots,\phi^\mu_N\}$, $\mu = 1,\dots, M$, the
   pseudo-loglikelihood for the variable $i$ is given by the logarithm of Eq. (\ref{eq:marginal_xy}),
   \begin{eqnarray}
   \label{eq:L_i}
   L_i &=& \frac{1}{M}  \sum_{\mu = 1}^M  \ln P(\phi_i^{(\mu)}|\bm{\phi}^{(\mu)}_{\backslash i})
   \\
   \nonumber
   & =&  \frac{1}{M}  \sum_{\mu = 1}^M  \left[ H_i^{(\mu)} \cos( \phi_i^{(\mu)} - \alpha_i^{(\mu)}) - \ln  2 \pi I_0\left(H_i^{(\mu)}\right)\right] \, .
    \end{eqnarray}
The underlying idea of PLM is that an approximation of the true parameters of the model is obtained for values that maximize the functions $L_i$.
The specific maximization scheme differentiates the different techniques.

   \subsection{PLM with $l_2$ regularization}
   Especially for the case of sparse graphs, it is useful to add a regularizer, which prevents the maximization routine to move towards high values of
   $J_{ij}$ and $h_i$ without converging. We will adopt an $l_2$ regularization so that the Pseudolikelihood function (PLF) at site $i$  reads:
   \begin{equation}\label{eq:plf_i}
  {\cal L}_i = L_i
  - \lambda \sum_{i \neq j} \left(J_{ij}^R\right)^2 - \lambda \sum_{i \neq j} \left(J_{ij}^I\right)^2
   \end{equation}
   with $\lambda>0$.
   Note that the values of $\lambda$ have to be chosen arbitrarily, but not too large, in order not to overcome $L_i$.
   The standard implementation of the PLM consists in maximizing each ${\cal L}_i$, for $i=1\dots N$, separately. The expected values of the couplings are then:
   \begin{equation}
   \{ J_{i j}^*\}_{j\in \partial i} :=  \mbox{arg max}_{ \{ J_{ij} \}}
    \left[{\cal L}_i\right]
   \end{equation}
   In this way, we obtain two estimates for the coupling $J_{ij}$, one from maximization of ${\cal L}_i$, $J_{ij}^{(i)}$, and another one from ${\cal L}_j$, say $J_{ij}^{(j)}$.
    Since the original Hamiltonian of the $XY$ model is Hermitian, we know that the real part of the couplings is symmetric while the imaginary part is skew-symmetric.

     The final estimate for $J_{ij}$ can then be obtained  averaging the two results:

   \begin{equation}\label{eq:symm}
   J_{ij}^{\rm inferred} = \frac{J_{ij}^{(i)} + \bar{J}_{ij}^{(j)}}{2}
   \end{equation}
   where with $\bar{J}$ we indicate the complex conjugate.
   It is worth noting that the pseudolikelihood $L_i$, Eq. \eqref{eq:L_i}, is characterized by the
   following properties: (i) the normalization term of Eq.\eqref{eq:marginal_xy} can be
   computed analytically at odd with the {\em full} likelihood case that
   in general require a computational time which scales exponentially
   with the size of the systems; (ii) the $\ell_2$-regularized pseudolikelihood
   defined in Eq.\eqref{eq:plf_i} is strictly concave (i.e. it has a single
   maximizer)\cite{Ravikumar10}; (iii) it is consistent, i.e. if $M$ samples are
   generated by a model $P(\phi | J*)$ the maximizer tends to $J*$
   for $M\rightarrow\infty$\cite{besag1975}. Note also that (iii) guarantees that
   $|J^{(i)}_{ij}-J^{(j)}_{ij}| \rightarrow 0$ for $M\rightarrow \infty$.
   In Secs. \ref{sec:res_reg}, \ref{sec:res_dec}
   we report the results obtained and we analyze the performances of the PLM having taken the configurations from Monte-Carlo simulations of models whose details are known.

   \subsection{PLM with decimation}
   Even though the PLM with $l_2$-regularization allows to dwell the inference towards the low temperature region and in the low sampling case with better performances that mean-field methods, in some situations some couplings are overestimated and not at all symmetric. Moreover, in the technique there is the bias of the $l_2$ regularizer.
   Trying to overcome these problems, Decelle and Ricci-Tersenghi introduced a new method \cite{Decelle14}, known as PLM + decimation: the algorithm maximizes the sum of the $L_i$,
   \begin{eqnarray}
    {\cal L}\equiv \frac{1}{N}\sum_{i=1}^N \mbox{L}_i
    \end{eqnarray}
    and, then, it recursively set to zero couplings which are estimated very small. We expect that as long as we are setting to zero couplings that are unnecessary to fit the data, there should be not much changing on ${\cal L}$. Keeping on with decimation, a point is reached where ${\cal L}$ decreases abruptly indicating  that relevant couplings are being decimated and under-fitting is taking place.
   Let us define  by $x$  the fraction of non-decimated couplings. To have a quantitative measure for the halt criterion of the decimation process, a tilted ${\cal L}$ is defined as,
   \begin{eqnarray}
  \mathcal{L}_t &\equiv& \mathcal{L}  - x \mathcal{L}_{\textup{max}} - (1-x) \mathcal{L}_{\textup{min}} \label{$t$PLF}
   \end{eqnarray}
   where
   \begin{itemize}
   \item $\mathcal{L}_{\textup{min}}$ is the pseudolikelyhood of a model with independent variables. In the XY case: $\mathcal{L}_{\textup{min}}=-\ln{2 \pi}$.
   \item
   $\mathcal{L}_{\textup{max}}$ is the pseudolikelyhood in the fully-connected model and it is maximized over all the $N(N-1)/2$ possible couplings.
   \end{itemize}
   At the first step, when $x=1$, $\mathcal{L}$ takes value $\mathcal{L}_{\rm max}$ and  $\mathcal{L}_t=0$. On the last step, for an empty graph, i.e., $x=0$, $\mathcal{L}$ takes the value $\mathcal{L}_{\rm min}$ and, hence, again $\mathcal{L}_t =0$.
   In the intermediate steps, during the decimation procedure, as $x$ is decreasing from $1$ to $0$, one observes firstly that $\mathcal{L}_t$ increases linearly and, then, it displays an abrupt decrease indicating that from this point on relevant couplings are being decimated\cite{Decelle14}. In Fig. \ref{Jor1-$t$PLF} we give an instance of this behavior for the 2D short-range XY model with ordered couplings. We notice that the maximum point of $\mathcal{L}_t$ coincides with the minimum point of the reconstruction error, the latter defined as
   \begin{eqnarray}\label{eq:errj}
   \mbox{err}_J \equiv \sqrt{\frac{\sum_{i<j} (J^{\rm inferred}_{ij} -J^{\rm true}_{ij})^2}{N(N-1)/2}} \label{err}
   \end{eqnarray}
  We stress that the ${\cal L}_t$ maximum is obtained ignoring the underlying graph, while the err$_J$ minimum can be evaluated once the true graph has been reconstructed.

     \begin{figure}[t!]
   	\centering
   	\includegraphics[width=1\linewidth]{Jor1_dec_tPLF_new.eps}
   	\caption{The tilted likelyhood ${\cal L}_t$ curve and the reconstruction error vs the number of decimated couplings  for an ordered, real-valued J on 2D XY model with $N=64$ spins. The peak of ${\cal L}_t$ coincides with the dip of the error.}
   	\label{Jor1-$t$PLF}
   \end{figure}

   In the next sections we will show the results obtained on the  $XY$ model analyzing the performances of the two methods and comparing them also with a mean-field method \cite{Tyagi15}.

   \section{Inferred couplings with PLM-$l_2$}
    \label{sec:res_reg}
    \subsection{$XY$ model with real-valued couplings}

    In order to obtain the vector of couplings, $J_{ij}^{\rm inferred}$ the function $-\mathcal{L}_i$ is minimized through the vector of derivatives ${\partial \mathcal{L}_i}/\partial J_{ij}$. The process is repeated for all the couplings obtaining then a fully connected adjacency matrix. The results here presented are obtained with $\lambda = 0.01$.
   For the minimization we have used the MATLAB routine \emph{minFunc\_2012}\cite{min_func}.

      \begin{figure}[t!]
    	\centering
    	\includegraphics[width=1\linewidth]{Jor11_2D_l2_JR_soJR_TPJR}
    	\caption{Top panels: instances of single site coupling reconstruction for the case of $N=64$ XY spins on a 2D lattice with ordered $J$ (left column) and bimodal distributed $J$ (right column).

    	Bottom panels: sorted couplings.}
    	\label{PL-Jor1}
    \end{figure}

To produce the data by means of numerical Monte Carlo simulations a system with $N=64$  spin variables  is considered on a deterministic 2D lattice with periodic boundary conditions.
Each spin has then connectivity $4$, i.e., we expect to infer an adjacency matrix with $N c = 256$ couplings different from zero.
The dynamics of the simulated model is based on the Metropolis algorithm and parallel tempering\cite{earl05} is used to speed up the thermalization of the system.
The thermalization is tested looking at the average energy over logarithmic time windows and
the acquisition of independent configurations
starts only after the system is well thermalized.

   For the values of the couplings we considered two cases: an ordered case, indicated in the figure as  $J$ ordered (e.g., left column of Fig. \ref{PL-Jor1}) where the couplings can take values $J_{ij}=0,J$, with $J=1$,
   and a quenched disordered case, indicated in the figures as  $J$ disordered (e.g., right column of Fig. \ref{PL-Jor1})
   where the couplings can take also  negative values, i.e.,
    $J_{ij}=0,J,-J$, with a certain probability.  The results here presented were obtained with bimodal distributed $J$s:

    $P(J_{ij}=J)=P(J_{ij}=-J)=1/2$.  The performances of the PLM have shown not to depend on $P(J)$.

    We recall that in Sec. \ref{sec:plm} we used the temperature-rescaled notation, i.e., $J_{ij}$ stands for $J_{ij}/T$.

    To analyze the performances of the PLM, in Fig. \ref{PL-Jor1} the inferred couplings, $\mathbb{J}^R_{\rm inf}$, are shown on top of the original couplings,  $\mathbb{J}^R_{\rm true}$.
     The first figure (from top) in the left column shows  the $\mathbb{J}^R_{\rm inf}$ (black) and the $\mathbb{J}^R_{\rm tru}$ (green) for a given spin
     at temperature $T/J=0.7$ and number of samples  $M=1024$. PLM appears to reconstruct the correct couplings, though zero couplings are always given a small inferred non-zero value.
     In the left column of Fig.  \ref{PL-Jor1},  both the $\mathbb{J}^R_{\rm{inf}}$ and the $\mathbb{J}^R_{\rm{tru}}$ are sorted in decreasing order and plotted on top of each other.
     We can clearly see that $\mathbb{J}^R_{\rm inf}$ reproduces the expected step function. Even though the jump is smeared, the difference between inferred couplings corresponding to the set of non-zero couplings
     and to the set of zero couplings can be clearly appreciated.
     Similarly, the plots in the right column of Fig. \ref{PL-Jor1} show the results obtained for the case with  bimodal disordered couplings, for the same working temperature and number of samples.
     In particular, note that the algorithm infers half positive and half negative couplings, as expected.

\begin{figure}
\centering
\includegraphics[width=1\linewidth]{Jor11_2D_l2_errJ_varT_varM}
\caption{Reconstruction error $\mbox{err}_J$, cf. Eq. (\ref{eq:errj}), plotted as a function of temperature (left) for three values of the number of samples $M$ and  as a function $M$ (right) for three values of temperature in the ordered system, i.e., $J_{ij}=0,1$.
The system size is $N=64$.}
\label{PL-err-Jor1}
\end{figure}

In order to analyze the effects of the number of samples and of the temperature regimes, we plot in Fig. \ref{PL-err-Jor1} the reconstruction error, Eq. (\ref{err}), as a function of temperature for three different sample sizes $M=64,128$ and $512$.
The error is seen to sharply rise al low temperature, incidentally, in the ordered case, for  $T<T_c \sim 0.893$, which is the Kosterlitz-Thouless transition temperature of the 2XY model\cite{Olsson92}.
 However, we can see that if only $M=64$ samples are considered, $\mbox{err}_J$ remains high independently on the working temperature.
 In the right plot of Fig. \ref{PL-err-Jor1},  $\mbox{err}_J$ is plotted as a function of $M$ for three different working temperatures $T/J=0.4,0.7$ and $1.3$. As we expect,
 $\mbox{err}_J$  decreases as $M$ increases. This effect was observed also with mean-field inference techniques on the same model\cite{Tyagi15}.

To better understand the performances of the algorithms, in Fig. \ref{PL-varTP-Jor1} we show several True Positive (TP) curves obtained for various values of $M$ at three different temperatures $T$. As $M$ is large and/or temperature is not too small,  we are able to reconstruct correctly all the couplings present in the system (see bottom plots).
The True Positive curve displays how many times the inference method finds a true link of the original network  as a function of the index of the vector of sorted absolute value of reconstructed couplings $J_{ij}^{\rm inf}$.
The index $n_{(ij)}$ represents the related spin couples $(ij)$. The TP curve is obtained as follows:
first  the values $|J^{\rm inf}_{ij}|$ are sorted in descending order and  the spin pairs $(ij)$ are ordered according to the sorting position of $|J^{\rm inf}_{ij}|$. Then,
     	a cycle over the ordered set of pairs $(ij)$, indexed by $n_{(ij)}$, is performed, comparing with the original network coupling $J^{\rm true}_{ij}$ and verifying whether it is zero or not. The true positive curve is computed as
\begin{equation}
\mbox{TP}[n_{(ij)}]= \frac{\mbox{TP}\left[n_{(ij)}-1\right] (n_{ij}-1)+ 1 -\delta_{J^{\rm true}_{ij},0}}{n_{(ij)}}
\end{equation}
As far as $J^{\rm true}_{ij} \neq 0$, TP$=1$. As soon as the true coupling of a given $(ij)$ couple in the sorted list is zero, the TP curve departs from one.
In our case, where the connectivity per spin of the original system is  $c=4$ and there are $N=64$ spins, we know that we will have  $256$ non-zero couplings.
     	If the inverse problem is successful, hence, we expect a steep decrease of the TP curve when  $n_{ij}=256$ is overcome.

In Fig. \ref{PL-varTP-Jor1}
it is  shown that,  almost independently of $T/J$, the TP score improves as $M$ increases. Results are plotted for three different temperatures, $T=0.4,1$ and $2.2$, with increasing number of samples $M = 64, 128,512$ and $1024$ (clockwise).
We can clearly appreciate the improvement in temperature if the size of the data-set is not very large: for small $M$, $T=0.4$ performs better.
When $M$ is high enough (e.g., $M=1024$), instead, the TP curves do not appear to be strongly influenced by the temperature.

\begin{figure}[t!]
	\centering
	\includegraphics[width=1\linewidth]{Jor11_2D_l2_TPJR_varT_varM}
	\caption{TP  curves  for 2D short-range ordered $XY$ model with $N=64$ spins at three different values of $T/J$ with increasing - clockwise from top - $M$.}
	\label{PL-varTP-Jor1}
\end{figure}

\subsection{$XY$ model with complex-valued couplings}
For the complex $XY$ we have to contemporary  infer $2$ apart coupling matrices,  $J^R_{i j}$ and $J^I_{i j}$.  As before, a system of $N=64$ spins is considered on a 2D lattice.
For the couplings we have considered both ordered and bimodal disordered cases.
In Fig. \ref{PL-Jor3}, a single row of the matrix $J$ (top) and the whole sorted couplings (bottom) are displayed for the ordered model (same legend as in Fig. \ref{PL-Jor1}) for the real, $J^R$ (left column), and the imaginary part, $J^I$.

\begin{figure}[t!]
	\centering
\includegraphics[width=1\linewidth]{Jor3_l2_JRJI_soJRJI_TPJRJI}
	\caption{Results related to the ordered complex XY model with $N=64$ spins on a  2D lattice. Top: instances of  single site reconstruction for the real, JR (left column), and
		the imaginary, JI (right column), part of $J_{ij}$. Bottom: sorted values of JR (left) and JI (right).}

	\label{PL-Jor3}
\end{figure}

  \section{PLM with Decimation}
 \label{sec:res_dec}

\begin{figure}[t!]
   	\centering
   	\includegraphics[width=1\linewidth]{Jor1_dec_tPLF_varT_varM}
    	\caption{Tilted Pseudolikelyhood, ${\cal L}_t$, plotted as a function of decimated couplings. Top: Different ${\cal L}_t$ curves obtained for different values of $M$ plotted on top of each other. Here $T=1.3$. The black line indicates the expected number of decimated couplings, $x^*=(N (N-1) - N c)/2=1888$. As we can see, as $M$ increases, the maximum point of ${\cal L}_t$ approaches $x^*$. Bottom: Different ${\cal L}_t$ curves obtained for different values of T with $M=2048$. We can see that, with this value of $M$, no differences can be appreciated on the maximum points of the different  ${\cal L}_t$ curves.}
   	\label{var-$t$PLF}
   \end{figure}

\begin{figure}[t!]
    \centering
    \includegraphics[width=1\linewidth]{Jor1_dec_tPLF_peak_statistics_varM_prob.eps}
    	\caption{Number of most likely decimated couplings, estimated by the maximum point of $\mathcal{L}_t$, as a function of the number of samples $M$. We can clearly see that the maximum point of $\mathcal{L}_t$ tends toward $x^*$, which is the right expected number of zero couplings in the system.}
    	\label{PLF_peak_statistics}
    \end{figure}

      For the ordered real-valued XY model we show in Fig. \ref{var-$t$PLF}, top panel, the outcome on the tilted pseudolikelyhood, $\mathcal{L}_t$ Eq. \eqref{$t$PLF}, of the progressive decimation: from a fully connected lattice  down to an empty lattice. The figure shows the behaviour of $\mathcal{L}_t$ for three different data sizes $M$. A clear data size dependence of the maximum point of  $\mathcal{L}_t$, signalling the most likely value for decimation, is shown. For small $M$ the most likely number of  couplings is overestimated and for increasing $M$ it tends to the true value, as displayed in Fig. \ref{PLF_peak_statistics}.  In the bottom panel of Fig. \ref{var-$t$PLF} we display instead different
 $\mathcal{L}_t$ curves obtained for three different values of $T$.
  Even though the values of $\mathcal{L}_t$ decrease with increasing temperature, the value of the most likely number of decimated couplings appears to be quite independent on $T$ with $M=2048$ number of samples.
In Fig. \ref{fig:Lt_complex} we eventually display the tilted pseudolikelyhood for a 2D network with complex valued ordered couplings, where the decimation of the real and imaginary coupling matrices proceeds in parallel, that is,
when a real coupling is small enough to be decimated its imaginary part is also decimated, and vice versa.
One can see that though the apart errors for the real and imaginary parts are different in absolute values, they display the same dip, to be compared with the maximum point of $\mathcal{L}_t$.

       \begin{figure}[t!]
    	\centering
      	\includegraphics[width=1\linewidth]{Jor3_dec_tPLF_new}
      	\caption{Tilted Pseudolikelyhood, ${\cal L}_t$, plotted with the reconstruction errors for the XY model with $N=64$ spins on a 2D lattice. These results refer to the case of  ordered and complex valued couplings. The full (red) line indicates ${\cal L}_t$. The dashed (green)
      		and the dotted (blue) lines show the reconstruction errors (Eq. \eqref{eq:errj}) obtained for the real and the imaginary couplings respectively. We can see that both ${\rm err_{JR}}$ and ${\rm err_{JI}}$ have a minimum at $x^*$.}
          	\label{fig:Lt_complex}
    \end{figure}

\begin{figure}[t!]
   	\centering
   	\includegraphics[width=1\linewidth]{Jor1_dec_JR_soJR_TPJR}
   	\caption{XY model on a 2D lattice with $N=64$ sites and real valued couplings. The graphs show the inferred (dashed black lines) and true couplings (full green lines) plotted on top of each other. The left and right columns refer to the
   		 cases of ordered and bimodal disordered couplings, respectively. Top figures: single site reconstruction, i.e., one row of the matrix $J$. Bottom figures: couplings are plotted sorted in descending order.}
   	\label{Jor1_dec}
   \end{figure}

\begin{figure}[t!]
    	\centering
    	\includegraphics[width=1\linewidth]{Jor3_dec_JRJI_soJRJI_TPJRJI}
    	\caption{XY model on a 2D lattice with $N=64$ sites and ordered complex-valued couplings.
    		The inferred and true couplings are plotted on top of each other. The left and right columns show the real and imaginary parts, respectively, of the couplings. Top figures refer to a single site reconstruction, i.e., one row of the matrix $J$. Bottom figures report the couplings sorted in descending order.}
    	\label{Jor3_dec}
    \end{figure}

       \begin{figure}[t!]
     	\centering
     	\includegraphics[width=1\linewidth]{MF_PL_Jor1_2D_TPJR_varT}
     	\caption{True Positive curves obtained with the three techniques: PLM with decimation, (blue) dotted line,  PLM with $l_2$ regularization, (greed) dashed line, and mean-field, (red) full line.  These results refer to real valued ordered couplings with $N=64$ spins on a 2D lattice. The temperature is here $T=0.7$ while the four graphs refer to different sample sizes: $M$ increases clockwise.}
     	\label{MF_PL_TP}
     \end{figure}

    \begin{figure}[t!]
    	\centering
    	\includegraphics[width=1\linewidth]{MF_PL_Jor1_2D_errJ_varT_varM}
    	\caption{Variation of reconstruction error, ${\rm err_J}$, with respect to temperature as obtained with the three different techniques, see Fig. \ref{MF_PL_TP}, for four different sample size:  clockwise from top   $M=512,1024, 2048$ and $4096$.}
    	\label{MF_PL_err}
    \end{figure}

     Once the most likely network has been identified through the decimation procedure, we perform the same analysis displayed in Fig. \ref{Jor1_dec}  for ordered and then quenched disordered real-valued couplings
and in Fig. \ref{Jor3_dec} for  complex-valued ordered couplings.  In comparison to the results shown in Sec. \ref{sec:res_reg},
  the PLM with decimation leads to rather cleaner results. In Figs. \ref{MF_PL_err} and \ref{MF_PL_TP} we compare the performances of the PLM with decimation in respect to ones of the PLM with $l_2$-regularization. These two techniques are also analysed in respect to a mean-field technique previously implemented on the same XY systems\cite{Tyagi15}.

    For what concerns the network of connecting links, in Fig. \ref{MF_PL_TP} we compare the TP curves obtained with the three techniques. The results refer to the case of ordered and real valued couplings, but similar behaviours were obtained for the other cases analysed.
  The four graphs are related to different sample sizes, with $M$ increasing clockwise. When $M$ is high enough, all techniques reproduce the true network.
  However, for lower values of $M$ the performances of the PLM with $l_2$ regularization and with decimation drastically overcome those ones of the previous mean field technique.
  In particular, for $M=256$ the PLM techniques still reproduce the original network while the mean-field method fails to find more than half of the couplings.
  When $M=128$, the network is clearly reconstructed only through the PLM with decimation while the PLM with $l_2$ regularization underestimates the couplings.
  Furthermore, we notice that the PLM method with decimation is able to clearly infer the network of interaction even when $M=N$ signalling that it could be considered also in the under-sampling regime $M<N$.

In Fig. \ref{MF_PL_err} we compare the temperature behaviour of the reconstruction error.
In can be observed that for all temperatures and for all sample sizes  the reconstruction error, ${\rm err_J}$, (plotted here in log-scale) obtained with the PLM+decimation is always smaller than
that one obtained with the other techniques. The temperature behaviour of ${\rm err_J}$ agrees with the one already observed for Ising spins in \cite{Nguyen12b} and for XY spins  in \cite{Tyagi15} with a mean-field approach:  ${\rm err_J}$ displays a minimum around $T\simeq 1$ and then it increases for very lower $T$; however,
 the error obtained with the PLM with decimation is several times smaller  than the error estimated by the other methods.

     \section{Conclusions}
     \label{sec:conc}

Different statistical inference methods have been applied to the inverse problem of the XY model.
After a short review of techniques based on pseudo-likelihood and their formal generalization to the model we have tested their performances against data generated by means of Monte Carlo numerical simulations of known instances
with diluted, sparse, interactions.

The main outcome is that the best performances are obtained by means of the  pseudo-likelihood method combined with decimation. Putting to zero (i.e., decimating) very weak bonds, this technique turns out to be very precise for  problems whose real underlying interaction network is sparse, i.e., the number of couplings per variable does not scale with number of variables.
The PLM + decimation method is compared to the PLM + regularization method, with $\ell_2$ regularization and to a mean-field-based method. The behavior of the quality of the network reconstruction is analyzed by looking at the overall sorted couplings and at the single site couplings, comparing them with the real network, and at the true positive curves in all three approaches. In the PLM +decimation method, moreover, the identification of the number of decimated bonds at which the tilted pseudo-likelihood is maximum allows for a precise estimate of the total number of bonds. Concerning this technique, it is also shown that the network with the most likely number of bonds is also the one of least reconstruction error, where not only the prediction of the presence of a bond is estimated but also its value.

The behavior of the inference quality in temperature and in the size of data samples is also investigated, basically confirming the low $T$ behavior hinted by Nguyen and Berg \cite{Nguyen12b} for the Ising model. In temperature, in particular, the reconstruction error curve displays a minimum at a low temperature, close to the critical point in those cases in which a critical behavior occurs, and a sharp increase as temperature goes to zero. The decimation method, once again, appears to enhance this minimum of the reconstruction error of almost an order of magnitude with respect to other methods.

The techniques displayed and the results obtained in this work can be of use in any of the many systems whose theoretical representation is given by Eq. \eqref{eq:HXY} or Eq. \eqref{eq:h_im}, some of which are recalled in Sec. \ref{sec:model}. In particular, a possible application can be the field of light waves propagation through random media and the corresponding problem of the  reconstruction of an object seen through an opaque medium or a disordered optical fiber \cite{Vellekoop07,Vellekoop08a,Vellekoop08b, Popoff10a,Akbulut11,Popoff11,Yilmaz13,Riboli14}.