Strong denoising of financial time-series

## Abstract

Abstract In this paper we introduce a method for significantly improving the signal to noise ratio in financial data. The approach relies on combining a target variable with different context variables and use auto-encoders (AEs) to learn reconstructions of the combined inputs. The objective is to obtain agreement among pairs of AEs which are trained on related but different inputs and for which they are forced to find common ground. The training process is set up as a “conversation” where the models take turns at producing a prediction (speaking) and reconciling own predictions with the output of the other AE (listening), until an agreement is reached. This leads to a new way of constraining the complexity of the data representation generated by the AE. Unlike standard regularization whose strength needs to be decided by the designer, the proposed mutual regularization uses the partner network to detect and amend the lack of generality of the learned representation of the data. The integration of alternative perspectives enhances the de-noising capacity of a single AE and allows us to discover new regularities in financial time-series which can be converted into profitable trading strategies.

## 1 Introduction

The manifold hypothesis states that many datasets of practical interest concentrate around a low-dimensional manifold Meilă & Zhang (). The hypothesis expresses the idea that valid data points can be found by smoothly transforming input data along the tangent directions of the manifold. The geometry of the manifold reflects structural constraints obeyed by the data which, in turn, are due to some (physical) mechanism that prepares the data. Financial markets are social systems in which multiple mechanisms continuously compete for the attention of investors. It is common to think of the market as operating in different “regimes” each having its own set of rules on how real-world observations (news) are converted into prices. Auto-encoders seem to be well suited to extract and represent these rules. They project inputs onto the manifold in the encoding stage and “lift” points on the manifold back into the high–dimensional input space during the decoding stage. The key is to make the encoder as insensitive to variations of the input as possible. Only variations that give rise to a new input pattern (instead of being a noisy version of a given pattern) must be encoded. These variations define the tangent directions of the manifold. Learning is unsupervised so the difference between signal and noise is not known but determined by the auto-encoder or, more precisely, its (in-)sensitivity to inputs.

Insensitivity to input data is typically achieved by placing a bottleneck layer at the end of the encoding stage such that the code dimension is lower than the input dimension (undercomplete auto-encoder). Alternatively (or in addition), a regularization term is added to the loss function which limits the complexity of the code e.g., by encouraging zeros in the encoding layer. Yet another method pursues local insensitivity to input variations by penalizing the derivative of the encoding function with respect to inputs Rifai et al. (). None of these methods incorporate an explicit statement of the generalization capability of the model. They all approach the problem indirectly by preventing the auto-encoder to represent arbitrary data sets.

### Common understanding requires generality

In this paper we approach the regularization problem in a different way. Every training step is followed by a reconciliation step in which two networks try and understand each other. The idea is that common understanding can only be obtained if the acquired knowledge (about the data) is general enough. If two AEs disagree on the representation of data they will be forced to alter the way they encode the data so that different aspects are extracted on which an agreement is possible. The agreement is measured in terms of the recurrence of combined encoding patterns: two networks are said to agree if a given encoding keeps appearing in conjunction with the same encoding by the partner AE. The idea is that this only happens if both networks extract defining features from the input data which itself keep reappearing. If spurious aspects are encoded then the probability of recurrence of specific pair of encodings is low. This means that the agreement level (AL) quantifies the generality of the representation.

The principal idea in this paper is to have two networks create two differing vantage points from which the data is examined. This is realized by a) choosing heterogeneous architectures i.e., networks that differ in the number and width of layers or activation functions and b) by combining a target variable with different context variables i.e., to provide the inputs $(y,\,x_{1})$ and $(y,\,x_{2})$ to the networks as shown in figure 1. $x_{1}$ and $x_{2}$ should be different but related signals which help create alternative perspectives on $y$. Agreement can only be achieved on an abstract level in the sense that the networks agree on the occurrence of data configuration to which they assign codes that can be translated between the networks using a stable dictionary. In the context of financial variables these configurations are often referred to as market regimes.

## 2 Related Work

As stated in the introduction, regularization has been an integral part of the development of autoencoders. Without it, they would merely produce a literal copy of the input without extracting valuable features from data or be able to reduce the dimensionality of its representation. Classic approaches involve L1 and L2 constraints on the network weights or robust training techniques like dropout Baldi & Sadowski () that avoid over-dependence on particular activation patterns. This paper takes an interactive approach which similar in spirit to adversarial regularization Makhzani et al. (), Zhao et al. (). The adversary sub-network encourages the main network to learn an unbiased representation. This has been shown to yield undesirable side-effects, including unstable gradients and reduced performance on in-domain examples Grand & Belinkov (). By contrast, our setting is a collaborative one in which the encoder outputs of two networks are compared given the same input data. Our idea is to reconcile these outputs in terms of the synchronicity of their appearance.

Figure: Figure 1: Two AEs communicate with each other (via translators) at the level of the encoding layer while trying to reproduce input data.
Refer to caption: x1.png

## 3 Regularization

Autoencoders project the input data on a low dimensional manifold $M$ whose coordinates are given by the codes $z$ in the hidden layer. They are typically designed to be undercomplete, i.e. the dimension of their encoder outputs is much smaller than the input dimension. This prevents them from learning the identity function. Because of the bottleneck, some aspects of the input have to be grouped into larger components from which the input is reconstructed while a literal reproduction is avoided. These components represent the features that define the objects contained in the data. However, the relationship between architectural constraints and simplification is quite loose and is typically determined empirically. In this paper, we propose a scheme in which the simplification is the result of a negotiation among two networks.

The networks only have weak bottlenecks and are actually encouraged to produce over-fitted predictions. The key point is that the models are different in the sense that predictions are obtained by decoding two different sets of codes $z$. If the networks assign codes to different perceived regularities in the data they can only agree on their (posterior) distribution $p(z|x)$ if the different codes repeatedly co-occur. If a code has been assigned to noise, the probability of its repeated co-occurrence with a code generated by the other network is very low. On codes for which an agreement on $p(z|x)$ is possible on the other hand must refer to a true regularities in the data.

The more the two networks differ the more effective they are as mutual regularizers. To achieve a different encoding behavior the networks can be of different complexity (e.g., by the number and width of layers). Also, independent noise sources $n_{1}$, $n_{2}$ can be added (as shown in figure 1) whose role is to encourage the networks to search in different directions. Notice that the noise is applied at the input and output during the training of the AE (unlike the de-noising setup). (Very loosely speaking,) it can be seen as an additional source of entropy which helps explore the space of possible code words. The objective is to find a common denominator with the other network, which may lie outside of the code regions explored by the networks individually. By mutual agreement we expect not only the noise to be rejected, but also the predictions to be more standardized as they are reproductions of the input data which are decoded from an agreed-upon posterior.

### 3.1 Training Setup

The training algorithm simultaneously improves the data fit and the mutual understanding on how to achieve it. Training is organized as a conversation among two AEs. The networks take turns at talk and either speak (S) or listen (L) to the other network. The S/L phases are asynchronous as in natural conversations. Translation layers T1 and T2 are placed between the two AEs to convert to and from the encoder outputs $z_{1}$ and $z_{2}$. In view of the desired dissimilarity the encoding layers have different output dimensions e.g., $\mbox{dim}z_{1}>\mbox{dim}z_{2}$ (without loss of generality). The loop starts with a speak action (S) by one of the networks, say AE1. Utterances are produced by sampling from a multivariate Gaussian with mean $\mu_{1}(x)$ using the familiar re-parametrization trick

$$ $z_{1}=\mu_{1}(x)+\sigma\,\varepsilon\quad\varepsilon\sim\mathcal{N}(0,1)$ (1) $$

where $\mu_{1}(x)$ is learned by the encoder and $\sigma>0$ is fixed for simplicity. We write $z_{1}\sim q_{1}(z_{1}|x)$ where $q_{1}(z_{1}|x)$ is the a posteriori distribution of codes $z_{1}$ after seeing the data $x$. It depends on the network parameters of AE1. A deterministic decoder maps $z_{1}$ to the network output $x_{1}^{\prime}$ which represents the prediction of the input data $x$. During (S), no parameters are updated. While AE1 speaks, AE2 listens.

### Training the autoencoder

Training only occurs during the listening phase. The following training procedure is described from the perspective of AE2 but equally applies to AE1 once it listens (with the roles of all variables switched accordingly). By means of the translation layer T1, AE2 obtains an estimate $z_{2}^{\prime}$ of its own code $z_{2}$ as produced by AE1:

$$ $\mbox{T}_{1}:\ z_{2}^{\prime}=g(z_{1})\quad z_{1}\sim q_{1}(\cdot|x)$ (2) $$

This transformation gives $z_{2}^{\prime}\sim p_{1}(\cdot)$ where $p_{1}(\cdot)$ is no longer a posterior distribution from the perspective of AE2 but represents data-independent noise. It depends on the parameters of AE1 which, however, are not updated while AE1 speaks. Predictions $x^{\prime}$ of $x$ will be generated by sampling codes $z_{2}\sim q_{2}(\cdot|x)$ from the posterior distribution of AE2. We assume that there exists a decoder $p_{2}(x|z_{2})$ depending on AE2’s parameters that describes the joint distribution $p_{2}(x,z_{2})=p_{2}(x|z_{2})p_{1}(z_{2})$ well. This means that

$$ $\mathbb{E}_{z_{2}\sim q_{2}(\cdot|x)}\underbrace{\left[\frac{p_{2}(x,z_{2})}{q _{2}(z_{2}|x)}\right]}_{\xi(z)}$ $$

is an unbiased estimator of the data-likelihood $p_{2}(x)$. From this we obtain $\ln p_{2}(x)=\ln\mathbb{E}_{z\sim q_{2}(\cdot|x)}\xi(z)\geq\mathbb{E}_{z\sim q
_{2}(\cdot|x)}\ln\xi(z)\coloneq L_{2}(x,q_{2})$ by Jensen’s inequality, where $L_{2}$ is the ELBO associated with AE2. $L_{2}$ may be re-written as

$$ $L_{2}(x,q_{2})=\mathbb{E}_{z_{2}\sim q_{2}(\cdot|x)}\left[\ln p_{2}(x|z_{2})-D _{KL}(q_{2}(\cdot|x)\|p_{1}(\cdot)\right]$ (3) $$

We measure $\ln p_{2}(x|z_{2})$ with the help of isotropic Gaussians $\mathcal{N}(x_{2}^{\prime},\mbox{diag}\,\sigma^{2})$ centered at predictions $x_{2}^{\prime}(z_{2})$ and using the same variance $\sigma^{2}$ as in equation (1) so the term becomes proportional to $-1/\sigma^{2}\|x-x_{2}^{\prime}\|^{2}$. The KL divergence between the approximate posterior $q_{2}(z_{2}|x)$ and the data independent prior $p_{1}(z_{2})$ is the divergence between two Gaussians

$$ $D_{KL}(q_{2}\|p_{1})=0.5\left[\ln\frac{|\Sigma_{p_{1}}|}{|\Sigma_{q_{2}}|}- \mbox{dim}(z_{2})+(\mu_{q_{2}}-\mu_{p_{1}})^{T}\Sigma_{p_{1}}^{-1}(\mu_{q_{2}} -\mu_{p_{1}})+\mbox{tr}[\Sigma_{p_{1}}^{-1}\Sigma_{q_{2}}]\right]$ (4) $$

We let

$$ $z_{2}=\mu_{2}(x)+\sigma\,\varepsilon\quad\varepsilon\sim\mathcal{N}(0,1)$ (5) $$

using the same scaling factor $\sigma$ as in (1)
so the parameter dependent part of $D_{KL}(q_{2}\|p_{1})$ becomes proportional to $1/\sigma^{2}\|\mu_{q_{2}}-\mu_{p_{1}}\|^{2}$. In summary, we obtain:

$$ $L_{2}(x,q_{2})=-\mathbb{E}_{z_{2}\sim q_{2}(\cdot|x)}\left[\|x-x_{2}^{\prime}( z_{2})\|^{2}+\|\mu_{q_{2}}-\mu_{p_{1}}\|^{2}\right]$ (6) $$

which is maximized during the listening (L) phase of AE2. The presence of $p_{1}(\cdot)$ acts as a (regularization) obstacle that prevents the encoder from fitting an arbitrarily complex posterior $q_{2}(\cdot|x)$

### The translator

Figure: Figure 2: The translator
Refer to caption: x2.png

$g(\cdot)$ in equation (2) is realized by training a separate network that learns an (approximate) lookup table which allows a per-sample translation among representations. This is in contrast to learning a smooth functional relationship between $z_{2}$ and $z_{1}$ which lacks the sharpness needed for regularization. This is illustrated by the dashed line in figure 2. The shape of the line is dominated by the three data patches in the lower, middle and upper part of the diagram. Bearing in mind that both dim$z_{1}$ and dim$z_{2}\gg 1$, the natural situation is that there will be less populated areas or “gaps” which are only occupied by a few samples. These samples do not contribute materially to the loss function of the hypothetical model that is fitted to learn $g(\cdot)$ and are therefore ignored. However, in the diagram, they form an equivalence class (versions of the letter “a”) which are all mapped to the same code (calligraphic a) as AE2 perceives them as equal. This is precisely the information that will regularize AE1 and must therefore be retained rather than “averaged” out. In other words, the translator must yield a per-sample association between $z_{1}^{t}$ and $z_{2}^{t}$ for every data point $x_{t}$, $t=1,\dots,T$.

The translator function $g(\cdot)$ is updated every time a new complete set of pairs $\{(z_{1}^{t},z_{2}^{t})\}_{t=1,\dots,T}$ becomes available i.e., after every training epoch of the AEs. The first dictionary is created by training the auto-encoders independently (without regularization). After that, the codes obtained from the previous training cycle (with regularization) are used. The same is true for

$$ $\mbox{T}_{2}:\ z_{1}^{\prime}=h(z_{2})\quad z_{2}\sim q_{2}(\cdot|x)$ (7) $$

which is the analog of equation (2) in reverse direction. AE2 will use T2 in the subsequent speak (S) phase to produce $z_{1}^{\prime}$ which AE1 obtains as a regularization input. AE1 then assumes the role of the listener (L) and the same update rules (with swapped variables) as described a above are applied. The training procedure is summarized in figure 3: standard training of separate networks only contains “speak” phases while mutual training features turn-taking among the networks.

The right hand side of the figure presents a typical evolution of the training error. The need to incorporate translated codes received from the other network introduces a non-monotonicity which has to be overcome by aligning the codes. The sigmoid activation functions in the encoding layer have a discretization effect on the effective codes that need to be aligned. By consequence, the error landscape is more rugged than the one obtained with standard training. The experiments behind these data are further described in the next section.

Figure: Figure 3: Separate (in blue) vs. mutual learning (in red) of two AEs
Refer to caption: x3.png

## 4 Application: Extract regularities from financial time-series

Financial markets are in general very efficient at converting new information into prices of traded securities. This means that changes in prices are essentially driven by the random arrival of new information. As a consequence, ’noise’ dominates the evolution of financial time-series. Regularities arise when market participants irrationally disregard available information or have difficulties interpreting it. The regularity represents a market inefficiency which becomes a source of risk-free return (alpha) if exploited by a trading strategy. For example, the prices of two related commodities may temporarily be out of sync which creates a price spread that eventually has to disappear because the more expensive commodity may be substituted by the cheaper one creating demand for the latter. Most systematic trading strategies available today are handcrafted, i.e. they start with an inefficiency which is underpinned by an economic theory, e.g. behavioral finance. It is reasonable to expect that many more regularities exist in financial times years which however are hidden behind high levels of noise. An auto-encoder with strong de-noising capabilities opens the door to a potentially large strategies space and may help discover new sources of alpha. This is illustrated, in the following experiments.

The regularities we seek to discover in time-series consist of longitudinal and cross-sectional ones. We employ a convolutional auto-encoder to capture them, see figure 4. The bottleneck layer (output of the encoder) flattens the time-step dimension (i.e., the dimension representing the rolling window over data used in the convolutions) and transforms the result via a dense layer into a low-dimensional code making the overall network under-complete. The network is trained using an ADAM optimizer with learning rate 0.01. The data consists of weekly macro and price data starting in the mid 1980’s which gives about 2000 samples. This relatively small data set corresponds to the typical information available to traders operating on the mid-to long-term horizon. We subdivide the dataset into 10 mini-batches and train for 150 batches. The translator is implemented as a simple dense network with a single, wide, hidden layer. As mentioned above, the objective of the translator is not simplification but to implement a one-to-one dictionary between the codes generated by the partner networks. Again, ADAM is used to optimize with step-size 0.1 over 10 epochs. All training phases (see figure 3) are incremental, i.e. the network weights obtained in a given epoch form the initial weights at which the next training epoch starts.

In the experiments, we will study how predictions obtained from an autoencoder with this architectural constraint improve by adding the mutual regularization constraint. In particular we are interested in emergent patterns among the context variables once the AEs interact with each other. The experimental setup consists in a target variable $y$ being combined with different context variables $x_{i}$ $,i=1,2,\dots,N$. The variables refer financial time-series where $y$ corresponds to the returns of a target market (to be traded) and $x_{i}$ to environment (e.g., macroeconomic) data. The returns of the target market are shifted forward $h$ time-steps during the training phase. The motivation behind this set-up is that if a relationship between today’s environment and the $h$-steps forward returns of a target market is found, then $x_{i}$ provides a trading signal for $y$. In such a case, $h$ corresponds to the investment horizon of the trading strategy based on $x_{i}$.

Figure: Figure 4: Architecture of one AE: encoding and decoding stages employ (1D) convolutional layers and are linked through a bottleneck layer in which the “time-step” dimension of the convolution output is flattened. Heterogeneity among different AEs is obtained by varying the number of the input convolution channels between 16 and 32.
Refer to caption: x4.png

For every pair $(y,x_{i})$, an autoencoder AEi is trained which produces a denoised version $(y^{\prime},x_{i}^{\prime})$ of the input. The idea is that after denoising the relationship between $x_{i}$ and $y$ is more stable and apparent. The objective is to learn “typical” evolutions of $x_{i}$ that precede market up– or down–moves. These typical evolutions are obtained by clustering (e.g., by standard K-means) the denoised $x_{i}^{\prime}$ conditional on $y>0$ (up-market) or $y<0$ (down-market). The evolutions are truncated at a length that corresponds to the “depth” dimension of the convolution layers (32 timesteps in this case, see figure 4). The clustered patterns form a library of signals that can be used out-of-sample to decide whether to build a long or short position in the target market. Since $y$ is shifted forward, out-of-sample data cannot be auto-encoded, so only the original (noisy) $x_{i}$ can be compared to the library. However, as every pattern corresponds to a (truncated) time-series our feature space is fairly high dimensional which means that we can robustly determine the relative closeness of noisy signals towards the up– or down–patterns in the library.

Indeed, we define a trading position in terms of relative distances as

$$ $\theta_{i}=d_{iu}^{-1}/(d_{iu}^{-1}+d_{id}^{-1})$ (8) $$

where $d_{iu}=\sum_{\xi_{u}}$MSE$(x_{i},\xi_{u})$ and $\xi_{u}$ is a library entry corresponding to an up-market and MSE stands for “mean squared error”. $d_{id}$ refers to the distance to all down-market library patterns. This definition of $\theta_{i}$ corresponds to a simple (directional) trading strategy which directly exploits relationships extracted from de-noised configurations $(y,x_{i})$. The strategy can be refined by combining libraries obtained from multiple AEs. At this stage, finding common ground among the AEs is an important prerequisite as it not only leads to more abstract reconstructions of the target–context pairs but also allows us to mix contexts. When mixing signals it is practical to proceed in a pairwise fashion. This provides an interesting use-case for mutual regularization as introduced in this paper.

The target variable ($y$) is the S&P 500 (total return USD). The context variables ($x_{i}$) used in figures 4-6 are: Y10 = 10-Year Treasury Yield, CAPE = Cyclically Adjusted Price/Earnings Ratio, NYF = New York Fed Economic Activity Index, MG = US Corporate Margins (YoY), Y02 = 2-Year Treasury Yield/ short-term rates, STP = Steepness of the Treasury Yield Curve, M2 = Money Supply (YoY). In figure 5, we study configurations of context variables namely Y10 (used by AE1) and CAPE (used by AE2) while the target market (S&P 500) is color coded as up (red) or down (blue). For illustration purposes homogeneous up or down regions are red or blue shaded. We find that the separate reconstructions achieve some sorting relative to the original data. The output of AEs with mutual regularization, in turn, reveal that up and down phases can be very clearly separated in Y10-CAPE space. It should be noted that all three representations yield the same reconstruction of the target variable but differ in the location its up/down values in the space spanned by context variables. Strong denoising is also apparent as the data occupies a smaller region in Y10-CAPE space.

Figure: Figure 5: Reconstructions of context variables $x_{1}=$Y10, $x_{2}=$CAPE; left: literal reconstruction (no regularization), middle: separate training of AEs), right: training of AEs with mutual regularization. Up/ down values of the target variable $y=$S&P 500 are color coded as red and blue.
Refer to caption: x5.png

Figure 6 illustrates how the sorting of context variables allows for the extraction of characteristic profiles associated with up or down markets. The profiles consist of sequences of length 24 (weeks) and are extracted by clustering the reconstructed (i.e., de-noised) context variables conditional on positive/ negative reconstructed future returns of the target market. The profile form references against which new context data will be compared. Notice that the new data cannot be processed by the AE as the pair $(y^{t+h},\,x_{i}^{t})$ contains future (unknown) returns $y^{t+h}$ which are only available if $t$ is $h$ steps in the past ($h$ is the investment horizon). The role of denoising (by the AE) is to create stable and sufficiently different reference profiles against which noisy (unprocessed) data can be compared out-of-sample. If trading positions in the underlying market are entered according to (8) a payout profile emerges with interesting diversifying properties relative to the market in that it avoids the 2020 and 2022 draw-downs induced by the onset of the pandemic or the rate hike cycle respectively.

Figure: Figure 6: Strategy discovery from denoising environment variables: characteristic profiles are extracted to which new data is compared to define a trading position in the market for which a context representation has been found.
Refer to caption: x6.png

By forming random pairs from the available context variables we can efficiently create other strategies which derive trading decisions from different configurations of context variables. Figure 7 provides examples of strategies obtained by combining other pairs of contexts. We see that the resulting trading positions and P/L (profit and loss) time-series are quite different and respond to different macro-events. While CAPE/NYF and CAPE/MG respond to fundamental economic activity and margin growth (compared to what is priced into the valuation multiple) the three other strategies introduce the vantage point of rates (Y02, or RR) as well as money supply (M2). Interest rates determine how future cash-flows received from owning the stocks should be discounted and thereby impact the valuation multiple. The middle strategy PE/Y02 stayed out of the market after the first rate hikes occured before 2020. it is also much less sensitive to rates when the hiking resumed after the pandemic. This is very different from strategy STP/M2 which recognizes that the enormous expansion of money supply during the pandemic eventually inflates prices including those of stocks. We see that despite the abstract nature of the AE itself, its output can be understood and interpreted especially when forming pairs of AEs that coordinate their learning through mutual regularization.

Figure: Figure 7: Overview of trading strategies obtained by de-noising pairs of context variables together with a (shared) target variable.
Refer to caption: x7.png

## 5 Conclusion

The paper discusses collaborative regularization where two networks compare notes in the form of encodings generated upon seeing the same input data. A useful analogy is to think of the codes as words in different languages denoting the same object. The expressions might have different connotations which however have to be neglected in order make an agreement on their usage possible. This implies a degree of standardization in what can be predicted using agreed-upon codes. The codes correspond to recurring features in the input data, which are therefore also the defining elements of the objects contained in the data. Every input data point gives rise to a code and if this code is of the “agreed-upon” type, it means that the data point is representative of the object or pattern to be identified. This conclusion can be reached in an entirely unsupervised fashion. In some sense, the networks mutually supervise themselves by trying to match their use of encodings. Agreed-upon codes define equivalence classes of inputs (one for each code).

The idea opens up a to a vast strategy space which has yet to unfold entirely as more and more stable relationships are identified and traded. It will contribute to making financial markets more complete by providing liquidity and putting a price on more assets in more states of the world. Many active strategies are based on well-known (stylized) market inefficiencies or they operate within significant constraints (e.g., within a tracking error limit vs. a benchmark). If all market participants search along a similar dimension, alpha capture becomes a zero-sum game. A market with more heterogeneous trade positions will make all participants better off –according to their own criteria– without having to make someone else worse off (Pareto optimality with differential preferences).

## Peer Reviews

confidence: 2
contribution: 2
initial_rating: 5
presentation: 1
questions: **Method Clarity**  
Can the authors clarify where $\bar{z}_1$ and $\bar{z}_2$ come from? I was confused by this because their definition was deferred in their initial introduction (L170), but this was not defined in the next section. 

How exactly does the attention mechanism with key-value pairs work? What is the rationale behind using the specific kernel function in equation (7)? Why the RBF kernel instead of the usual exponential over dot products? Why is the translator non-trainable?
* Suggestions: To improve clarity, I'd include:
  * a step-by-step explanation of the attention mechanism and its implementation.
  * Justification for the choice of kernel function and any hyperparameters involved.
  * Clarification on why the translator is non-trainable and how this affects its performance.
  * A redrawn visual diagram to illustrate exact mapping between x1, x2, z1, z2, and y


**Experiments**  
Is there a way you can quantify the effect of denoising? I wasn't quite sure how to interpret the visuals in Figure 5 



**Presentation nits**  
What are the x and y-axis in figure 2 correspond to? x-axis is training steps? As nit, would be helpful to label in figure.
soundness: 2
strengths: **Novel Approach to Denoising**   
The paper introduces a novel concept of using a "conversational" framework with two distinct AEs and translation layers for mutual regularization. This approach differs from traditional methods and offers a potentially powerful way to filter out noise by seeking agreement between independently trained AEs

**Intriguing Application to Trading Strategy Discovery**   
The application of this denoising technique to discover trading strategies is a compelling demonstration of its potential. The idea of identifying "typical" patterns in denoised context variables to predict future market movements is interesting and could have practical implications.

**Focus on Practical Relevance**   
The paper acknowledges the challenge of the low signal-to-noise ratio in financial data and attempts to address it with a method that aims to extract real market inefficiencies for profitable trading. This focus on practical relevance and potential impact on financial markets is a strength
summary: This paper proposes a method for learning denoised representations of financial time series. They use distinct autoencoders (AEs) that mutually regularize each other’s learned representations through a collaborative process. To learn these representations, the authors propose training AEs to reconstruct distinct input features corresponding to the same underlying time series (e.g., 10-year treasury yield and cyclically adjusted price/earnings ratio), where the decoder of one AE takes as input the “translated” hidden states of the other AE. This translation is done via an attention layer that associates the two hidden states together via high-similarity. The authors validate their approach with  empirical results showing that their method effectively identifies profitable trading strategies based on denoised relationships between market returns and various macroeconomic indicators.
weaknesses: **Limited evaluation**  
The empirical results are based on a specific set of context variables and a limited time period. It's unclear how well this method generalizes to other markets, asset classes, or timeframes. More extensive experiments and robustness checks are needed to assess the generalizability of the findings. 
* The paper relies heavily on the profitability of the discovered trading strategies as evidence of successful denoising. While this is an interesting application, a more direct and quantitative evaluation of the denoising performance itself is necessary. Comparing the denoised outputs to a benchmark or using metrics specific to time-series denoising would strengthen the claims

**Lack of baselines / comparison to related works**  
Similarly, I was unable to track the progress of this work versus prior works discussed in the related work. Did the authors run their evaluation on similar denoising methods?

**Lack of clarity / justification in the method**  
It was hard for me to really understand the method and its justifications. While the paper describes the architecture of the AEs and the training process, the implementation and function of the "translator" remain unclear. A more detailed explanation of how this component works, including the choice of kernel function and the impact of its non-trainable nature, would be appreciated. After multiple readings, it was still unclear to me how exactly the two autoencoders could learn regularized representations. 
* For example, if AE1 and AE2 were both trained with a "self-reconstruction" term MSE_self and "cross-reconstruction" term MSE_cross, then the presented intuition would have made sense, i.e.,:
  * MSE_self = (AE1(x1) - x1)^2 + (AE2(x2) - x2)^2  
  * MSE_cross = (Decoder1(z2) - x1)^2 +  (Decoder2(z1) - x2)^2)  
  * AE1(x) = Encoder1(Decoder1(x))  (same for AE2)
  * Encoder1(x1) = z1 (same for Encoder 2)     
* But I could not tell if such a setup was being used, and the motivation behind instead using an attention mechanism felt unclear to me.

Similarly, how is y learned?

---

confidence: 2
contribution: 1
initial_rating: 3
presentation: 2
questions: Could the authors provide a complexity analysis or benchmarks comparing the proposed dual-AE approach with single AE models? Understanding the trade-offs in terms of training time or computational resources would be valuable.

The model employs several parameters (e.g., layer configurations, mutual regularization strength). Could the authors discuss the sensitivity of the model’s performance to these parameters? It would be helpful to know which parameters have the most impact on denoising effectiveness.
To strengthen the evaluation, it would be beneficial to include quantitative metrics to objectively assess the model’s denoising effectiveness. 

Additionally, comparing the proposed approach with other established denoising techniques (e.g., wavelet thresholding, stacked autoencoders) would contextualize its performance and help highlight any unique strengths.
soundness: 1
strengths: The paper introduces an innovative regularization mechanism, where two AEs mutually regularize each other’s representations. This approach is novel and avoids the drawbacks of traditional regularization methods.

The methodology is detailed, with a clear description of the interaction between the AEs and their translation layers. Additionally, empirical results on financial time-series demonstrate how the denoised signals can be applied to real-world trading scenarios.

The paper is generally well-written, especially in the sections explaining the conversational setup between the AEs and how mutual agreement results in effective noise reduction.

The approach could significantly impact finance by improving time-series predictability and revealing hidden data patterns, contributing to more robust trading strategies. The mutual regularization approach could also have broader applications beyond finance, potentially in any domain involving high-noise data.
summary: This paper presents a novel method for enhancing the signal-to-noise ratio in financial time-series data through a unique application of auto-encoders (AEs). Unlike traditional methods, which often rely on regularization by complexity penalties, this approach uses a dual-AE system where two AEs iteratively refine their reconstructions through a conversational process, leading to mutual regularization. By enforcing agreement between the networks, only genuine data regularities are captured, reducing noise. The authors validate the approach through experiments showing the effectiveness of this denoising technique in creating interpretable signals, benefiting systematic trading strategies.
weaknesses: The dual-AE setup may introduce additional **computational overhead**, especially with high-dimensional data. The paper does not discuss scalability or computational trade-offs.

The paper lacks **quantitative metrics** to objectively evaluate the performance of the denoising method. Only qualitative results are presented. Additionally, there is no **comparison with other state-of-the-art** denoising techniques, such as those based on wavelet-thresholding or stacked autoencoders. Including these would provide a clearer understanding of how the proposed method stands relative to existing approaches in terms of accuracy and denoising effectiveness.

The authors do not provide the **code** implementing the method presented in the paper, making it difficult to reproduce the results.

---

confidence: 3
contribution: 1
initial_rating: 3
presentation: 1
questions: What is the difficulty of training such method? It seems like the models may collapse.
soundness: 2
strengths: This work presents a new method to tackle the problem of time series prediction.
summary: This paper addresses a method for modeling time-dependent signal data, such as financial time series. The method employs two autoencoders with different network architectures, assuming they learn different features. The emprical results are demonstrated mainly on S&P500 data.
weaknesses: - I find some of the components of this paper not well-defined. Terms "listen" and "speak" are fine to use as long as they are well defined. I understand the intuition but cant verify if what I understand is correct.

- Some components of this paper are not well-defined. Terms like "listen" and "speak" are acceptable as long as they are clearly defined. While I understand the intuition, I cannot verify if my understanding is correct. 

- There is a lack of quantitative results (I am not familiar with financial benchmarks). The authors present strategies obtained by the model in Figure 7, which, from my perspective, seem to mostly predict following the present (i.e., the prediction is a one-day shift from the ground truth). This observation might be incorrect, but I cannot verify it. Therefore, I am missing a few metrics that indicate the model's quality, such as the profit over 1 or 5 years compared to simply staying invested in the S&P 500.

- The paper includes a lot of details that are obvious at this point, such as reparameterization trick derivations.

- I see no reason to use this method exclusively on financial data. While financial data is a challenging domain, it would be beneficial to see this method applied to more popular modalities or benchmarks.

- There is no comparison to prior works.

I could not find any code attached to this work. I believe that providing code that reproduces the paper's results adds significant value to any research work.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 1
questions: 1. Could you clarify why you think the test on this relatively small dataset provides any signifiance? 

2. How would the proposed method perform against similar approaches? Are there particular benefits that could be highlighted more explicitly?

3. Is there a plan to release the code for reproducibility?

4. What exactly do we see in Figure 3? Validation Loss?
soundness: 1
strengths: **Originality**: The paper introduces a method for enhancing financial data representations by using a regularization between two auto-encoders, hereby I think the conversational aspect is novel.

**Clarity**: The methodology, especially the "speaking and listening" phases of the auto-encoders, is well explained

**Significance**: In general i think the work could have significant implications for more robust trading strategies. The proposed approach could lead to new insights in financial forecasting by enhancing the clarity of underlying market patterns.
summary: This paper proposes a  method using autoencoders to improve the signal-to-noise ratio in financial time-series data. The approach involves using two AEs that collaborate through a "conversation," alternating between producing predictions and reconciling differences to enhance data representation. The goal is to extract meaningful features and denoise financial data, which has a  low signal-to-noise ratio. Experiments suggest that this  regularization approach supports finding market regularities and helps the development of new trading strategies.
weaknesses: My background is not in the context of financial time-series, however I really struggled understanding what the authors try to describe with their provided figures, e.g., what is a strat and mkt? In the following i outline some areas which might allow for improvements

**Figure Quality**: 
- In general, the figures in the manuscript need a significant revision
- Figure 2 uses hand-drawn letters, which is not appropriate for a venue like ICLR.
- Axis labels are overlapping, and there is no legend for the blue, red, or black series in Figure 6.
- Figure 4 does not provide substantial value; a table would suffice. Additionally, the use of color in the layer names is not explained, making it misleading.
- Figure 3 lacks axis labels, which makes interpretation difficult.

**Reproducibility**: The absence of a code or detailed procedural steps limits the reproducibility of the experiments. Providing a link to code or pseudocode would help mitigate this issue.

**Experimental Scope**: The experimental dataset used (around 2000 samples) is relatively small, especially in the context of financial data. Broader testing with larger datasets is needed to substantiate the claims made in the paper. Additionally, I think qualitative evaluations alone are insufficient in this case. A more thorough quantitative evaluation is required. Showing improvement over older/other denoising methods (e.g., traditional filters or more advanced nn approaches) would provide stronger support for the novelty and effectiveness of the proposed method.

Method Justification: The design choices are not well-supported by ablation studies or references to related literature. It would be helpful to mention similar concepts like Siamese Networks (e.g., Dong et al.) and mutual information alignment by Lee et al.

Related Work: Basing on the pure number of references and the amount of demand in financial forecasting (as the authors motivated), i do not believe the related work is nowhere near exhaustive.

### References:

Lee et al., Soft Contrastive Learning for Time Series (ICLR 2024)

Dong et al., TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling (2024)

Meta-review: The paper introduces a method for learning denoised representations of financial time series. It leverages dual autoencoders trained on a reconstruction task, where they mutually refine their reconstructions through a collaborative process. The autoencoders are constrained to agree on their reconstructions, capturing only the underlying data regularities and thus reducing noise. The approach is validated through its application to trading strategies based on the filtered series.

The reviewers commend the originality of the proposed approach and its target application. However, they raise several concerns, including the lack of clarity in the technical description, a limited evaluation—particularly with respect to quantitative metrics—and insufficient comparisons with baselines. The authors are encouraged to refine their positioning and enhance their experimental evaluation.

Decision: Reject