WATTNet: Learning to Trade FX via Hierarchical Spatio-Temporal
Representation of Highly Multivariate Time Series
Michael Poli1,2 , Jinkyoo Park2 , Ilija Ilievski1
2

1
Neuri Pte Ltd, Singapore, Singapore
Department of Industrial & Systems Engineering, KAIST, Daejeon, South Korea
michael@neuri.ai, jinkyoo.park@kaist.ac.kr, ilija139@neuri.ai

Abstract
Finance is a particularly challenging application area fordeep learning models due to low noise-to-signal ratio,
non-stationarity, and partial observability. Non-deliverableforwards (NDF), a derivatives contract used in foreign exchange (FX) trading, presents additional difficulty in the formof long-term planning required for an effective selection ofstart and end date of the contract. In this work, we focus
on tackling the problem of NDF tenor selection by leveraging high-dimensional, non-homogeneous sequential data consisting of spot rates, technical indicators and historical NDF
transactions. To this end, we construct a dataset from the Depository Trust & Clearing Corporation (DTCC) NDF datathat includes a comprehensive list of NDF volumes and daily
spot rates for 64 FX pairs. We introduce WaveATTentionNet(WATTNet), a novel temporal convolution (TCN) model for
spatio-temporal modeling of highly multivariate and nonhomogeneous time series, and validate it across NDF markets with varying degrees of dissimilarity between the training and test periods in terms of volatility and general market
regimes. The proposed method achieves a significant positive
return on investment (ROI) in all NDF markets under analysis, outperforming recurrent and classical baselines by a wide
margin. Finally, we propose two orthogonal interpretability
approaches to verify noise stability and detect the driving factors of the learned tenor selection strategy.

1

Introduction

Following recent trends of successful AI adoption, the financial world has seen a significant surge of attempts at leveraging deep learning and reinforcement learning techniques
across various application areas. Slowing down progress inthis field are the particular properties of financial data: low
signal-to-noise ratio (Guhr and Kälber 2003), partial observability, and irregular sampling. Furthermore, AI breakthroughs in finance often go unpublished due to monetary
incentives. Additional challenges are caused by the scarcity
of datasets available, which are often limited in scope, difficult to acquire or for some application areas missing altogether.
Copyright c 2020, Association for the Advancement of Artificial
Intelligence (www.aaai.org). All rights reserved.

As an attempt to alleviate some of these concerns, we release both a curated dataset and a novel model for foreignexchange (FX) futures trading. We focus our attention ona particular class of FX trading methods, non-deliverableforward (NDF) contracts, which constitute an important
open problem in finance and can serve as a challenging
benchmark for supervised or reinforcement learning models. We formulate the learning problem as an optimal selection problem in which the model is tasked with selecting theend date of the forward contract (tenor) from a rich input
containing past human trade patterns as well as spot rates
and technical indicators. In particular, tenor selection is castinto a direct imitation learning (Judah, Fern, and Dietterich2012) framework, where the model learns policy π directlyfrom a set of execution trajectories of a demonstration policy π ∗ without receiving a reward signal from the environment. The demonstrations are derived in a greedy fashion
from spot rate data and the resulting input-output tuple isutilized to perform standard supervised learning.
A key difference of our approach compared to existing
FX trading algorithms lies in the type of data relied upon
for learning, which includes expert tenor patterns in addition
to standard technical indicators. Such patterns are extracted
from a large dataset containing trades from competitive market players assumed to be informed about market state andto act rationally in order to achieve higher returns. Leveraging this additional information allows the models to differentiate between profitable and non-profitable market conditions with improved accuracy, ultimately leading to higher
returns.
Fundamentally important for finance are models capableof capturing inter and intra-dependencies in highly multivariate time series. Many, if not most, of such interaction
terms are nonlinear and thus challenging to analyze withstandard statistical approaches. A direct consequence has
been the new-found popularity of data-driven models for financial forecasting tasks, in particular recurrent neural networks (RNN) and their variants. Recurrent models, while
offering an intuitive approach to time series modeling, lackan explicit module to capture inter-dependencies and perform relational reasoning (Santoro et al. 2018). A different
approach to time series modeling relies on temporal con-volutions (TCN) as its fundamental computational block.
Particularly successful in this area of research is WaveNet(van den Oord et al. ), originally developed as a generativemodel for speech data. However, vanilla WaveNet and its
derivative models are primarily designed to handle univariate time series and thus are ill-suited for highly multivariate
financial time series. To bridge this gap, we introduce a new
TCN model called WaveATTentionNet (WATTNet) that incorporates computationally efficient dilated convolutions for
temporal learning of autoregressive effects and self-attention
modules to learn spatial, inter-time series interaction terms.
We summarize our main contributions as follows:• We curate, analyze, and release a new dataset containing
spot rates for 64 FX currencies, along with technical indicators and hourly frequency NDF contract trade dataspanning the period from 2013 to 2019. Several models,
including classical baselines (Momentum-1, Momentum90) and recurrent baselines (GRUs, LSTMs) are evaluatedagainst expert NDF data.
• We introduce WATTNet, a novel temporal convolution(TCN) architecture for spatio-temporal modeling. WATTNet is designed to extend WaveNet models to settings
with highly multivariate time series data.
• We provide two orthogonal approaches to evaluate noise
stability and explain driving factors of the learned trading
strategy, along with examples to highlight their efficacy.

2

Related Work and Background

Deep Learning for FX trading Earlier attempts at utilizing the expressivity of neural networks in forex (FX) trading
have been carried out in (Chan and Teong 1995), which predicts technical indicators via shallow fully-connected neuralnetworks. (Yu, Lai, and Wang 2005) designs a hybrid trading system capable of providing suggestions based on qualitative expert knowledge and price forecasting data obtained
from a neural network. More recently (Czekalski, Niezabitowski, and Styblinski 2015), (Galeshchuk and Mukherjee 2017), , (Petropoulos et al. 2017) and (Pathberiya,
Tilakaratne, and Hansen 2017) leverage fully-connectedneural networks, CNNs and classical autoregressive modeling techniques. However, these approaches focus on regularforex markets and short-term predictions and rely only on
spot rates and technical indicators as informative features.
Incorporating additional sources of data has been exploredin (Nassirtoussi et al. 2015), (Vargas, De Lima, and Evsukoff 2017) (Hu et al. 2018), in which additional textual
information obtained from financial news articles or online
discussion is included in the input features.
While the literature has no shortage of works in which
reinforcement learning is applied to portfolio management(Yu et al. 2019) and optimal trade execution, the FX markets remain comparatively unexplored. (Carapuço, Neves,and Horta 2018) develops a short-term spot trading system
based on reinforcement learning and obtains positive ROI in
the EURUSD market. (Sornmayura 2019) offers an analysis
of deep Q-learning (DQN) performance on two FX instruments. We are not aware of any published work where deeplearning or reinforcement systems are introduced to tackle
FX trading in an NDF setting.
Spatio temporal modeling WaveNet (van den Oord etal. ) is an autoregressive model based on dilated temporalconvolutions (TCN) in which the joint probability of the
input sequence is modeled as a factorized product of conditional probabilities. SNAIL (Mishra et al. 2017) obtains
improvements over vanilla WaveNet by adding a temporal attention layer between dilated convolutions. However,both vanilla WaveNet and SNAIL are originally designed
to process univariate time series data and are thus unable
to learn interaction terms between time series. ConvLSTM(Xingjian et al. 2015) introduce a convolution operation inside the LSTM cell to capture spatiotemporal information.
A weakness of ConvLSTMs is given by the prior assumption of structure in the spatial domain where features closer
together are prioritized by the convolution operation, as is
the case for example with video data. In general applications, the time series are arbitrarily concatenated as input
data and locality assumptions do not hold. Long-Short Term
Network (LSTNet) (Lai et al. 2018) extracts local featuresin temporal and spatial domain with convolutions and adds
a recurrent layer for longer-term dependencies. Similarly to
ConvLSTM, LSTNet assumes spatial locality. A more recent approach to spatio-temporal modeling based on Graph
Neural Networks (GNNs) is Spatio-Temporal Graph Convolutional Network (STCGN) (Yu, Yin, and Zhu 2018) which
utilizes graph convolution to carry out learning in both spatial and temporal dimensions.

2.1

BackgroundWe briefly introduce the necessary background regarding
different types of forex trading.
Foreign Exchanges Trading in forex (FX) markets is generally done via spot exchanges or forward exchanges, where
spot rate indicates the present expected buying rate. The
spot market can be volatile and is affected by news cycles,
speculation, and underlying market dynamics. On the other
hand, forward exchanges contain a long-term planning component: two parties fix a binding amount and date of exchange and the profits are calculated by comparing currency
rates at the start date and fix date. The difference between
start date and fix date is commonly referred to as tenor.
Non-Deliverable-Forward An NDF operates similarly toforward exchange contracts and exists as a replacement toforward FX trades in emerging markets. NDF markets areover-the-counter, meaning they operate directly between involved parties without supervision, and are generally morevolatile due to limited market-depth. In NDF trades the parties agree on notional amounts of primary and secondary
currency (e.g. dollar USD and korean won KRW) which
define the forward rate. The currency amounts are not exchanged at the end of the contract: instead, NDF trades
are cash-settled in USD, and the cash flow is computed as
Rt,a = (xt+a − xt )vt where xt is the spot rate at time t, a

NDF contracts
7,580,814
Trading hours
35,712
Trading days
1,488
Number of features per trade hour
1,123
Number of FX spot rates
64
Table 1: Summary of dataset statistics. Additional details areincluded in Appendix A.
is the tenor and v is the notional amount. Longer tenors are
generally more profitable at the expense of a higher volatility, commonly referred to as risk-premia. A successful trading agent thus has to find a difficult balance between risky,high return and safer, low return actions.

3

NDF Dataset

Notation A multivariate time series of length T and dimension M is indicated as {X}. We use {xi } for individual
time series indexed by i. xi,t selects a scalar element of timeseries i at time index t . In particular, we indicate a slice
across all time series at time t with {x1,t . . . xM,t }. Whenever the operations on {X} are batched, we add superscript j
for single samples in the batch: {Xj }. With batch size N , theresulting tensor {X1 } . . . {XN } has dimensions N ×T ×M .
We refer to tenors as a and to the set of admissible tenor
choices as A.
Expert benchmarks The NDF trade records have beencollected from the The Depository Trust Clearning Corporation (DTCC) database. These records contain start and end
dates of each NDF contract, along with currency amounts.
For each trading day t and admissible tenor a ∈ A we obtain trading volumes vt,a . We refer to Expert as a trading
agent that chooses tenors corresponding to maximum volumes at = arg maxa∈A vt,a . In addition to Expert we obtaina fictitious agent based on NDF records which is assumed
to have partial future knowledge of the market dynamics,
which we refer to as Expert oracle. Expert oracle is a filtered version of Expert: at each trading day t it selects theshortest tenor with positive return:at = arg min {a|(xt+a − xt ) > 0}

(1)

a∈A

In particular, Expert oracle is designed to select shortest
tenors to avoid a perfect accuracy exploitation of risk-premia
which would set an unrealistic benchmark for any model.
Expert and Expert oracle are used as human trader benchmarks.
Multivariate input data In order to learn how to effectively choose NDF tenors, the models have access to time series that can be broadly categorized in three groups: FX spot
rates, technical indicators, and NDF tenor volumes. Daily
FX spot rates serve as contextual market information andprovide a frame of reference that aids the model in identifying profitable states. We include spot rates of 64 major andminor FX currency pairs, with a complete list provided inAppendix A. Raw financial market data is often augmented
with hand-crafted features to help combat noise and nonstationarity (Ntakaris et al. 2019). To this end, we choose
the following technical indicators:• Simple moving average (SMA)• Exponential moving average (EMA)• Moving Average Convergence Divergence (MACD)• Rolling standard deviation (RSD)• Bollinger Bands (BB)• ARIMA 1-day spot rate forecast
The last category of input features, NDF tenor volumes, isobtained from DTCC NDF records. For each NDF pair under consideration and each admissible tenor a ∈ A, we generate a time series of volumes vt,a which includes a summation over all NDF records at a specific day t. In particular,
given a choice of maximum tenor of 90 days, each NDF paircontributes with a 90-dimensional multivariate volume timeseries to the input, which further emphasizes the need fora model capable of processing and aggregating information
across highly multivariate time series. The code for downloading and preprocessing the data will be released after
publication.

4

Model

Spatio-temporal modeling with WATTNet WaveATTentionNet (WATTNet) is a novel model designed for highly
multivariate, non–homogeneous time series inputs. WATTNet includes temporal modules, tasked with independently
aggregating information across time steps of univariate timeseries {x} ∈ {X} and spatial modules which aggregatefeatures across slices of all time series at a specific timet {x1,t , . . . xM,t }. Temporal and spatial modules are alternated and allow for learning a hierarchical spatio-temporal
representation. An overview of the model is given in Figure 1.
Temporal learning Temporal learning is achieved by applying temporal dilated convolutions (TCN) to univariate
time series {x} ∈ {X}. In particular, given a convolution
with kernel size k and dilation coefficient d, we compute theoutput at time t of a dilated convolution of {x} as:
zt =

k
X

wi ∗ xt−i∗d

(2)

i=1

where wi is the ith weight of the convolutional kernel. Eachunivariate time series has access to its own set of convolutional weights w as temporal convolution operations are carried on independently. We note that independence between
convolutions is necessary to provide the model with enoughflexibility to treat time series with different characteristics.
The outputs of the TCN operation are then concatenated as
to form a multivariate latent time series {Z}. In particular,
WATTNet includes gated convolutions, a standard architectural component for sequential data. Two dilated TCNs areapplied to {X} and the results {Z}α , {Z}β are passed to

…

Self
Attention

𝑧
𝑧ǁ

𝑧𝑀

𝑧𝑚

𝑧ǁ 𝑇 ∈ ℝ𝑀

Self
Attention

Spatial module

Temporal module
Dilation = 2*d…

Self
Attention

𝑧𝑀

𝑥

𝑥

𝑧ǁ 𝑇
𝑧𝑀𝑇

Self
Attention

𝑧𝑚𝑇

Spatial module𝑧𝑇

Temporal module
Dilation = d

𝑥 𝑇− 𝑥 𝑇

Time 𝑡Figure 1: WATTNet overview: dilated TCNs are independently applied to each univariate input time series. A single dotproduct attention head subsequently aggregates information across slices {z1,t . . . zM,t } and the result {Z̃} is passed to the next
WATTBlock or used directly for the task at hand.
non-linear activation functions and then multiplied elementwise:{Z} = σ({Z}α ) tanh({Z}β )(3)where σ indicates a sigmoid activation. The output {Z} isthen fed into a spatial learning module after which the process repeats for a number of times depending on WATTNet’s
layer depth.
Spatial learning A single-head scaled-dot product attention mechanism (Vaswani et al. 2017) is placed between
dilated TCN layers and allows the model to exchange information across different input time series at a specific timeslice. We compute key K, query Q and value V by considering a slice {z1,t . . . zM,t } ∈ {Z} of latent time series attime t as the input of learnable linear transformation of thetype ψ({z1,t . . . zM,t }) with weights Wk , Wq , Wv . The resulting matrices K, Q, and V are then used in the standard
scaled-dot product attention to return M weighted averagesof values V:
QKT{z̃1,t . . . z̃M,t } = softmax( √ )Vdk

(4)

where dk is a scaling factor given by the second dimension of K. The process is repeated for latent feature slices{z1,t . . . zM,t }, t = 1, . . . T and the results are concatenated into {Z̃}, a spatio-temporal latent representation of
input data {X}. Weights Wq , Wk , Wv are shared acrossthe entire sequence length T , allowing the attention headto capture time-invariant features that incorporate information from multiple time series. Output {Z̃} can be used directly for different tasks to perform decision making conditioned on multivariate time series data; alternatively, if thetask at hand benefits from deeper models, {Z̃} can instead
be passed to the following TCN layer to perform additional
cycles of temporal and spatial learning.
Hierarchical representation A single temporal and spatial module constitute a full WATTNet layer of computation and is referred to as WATTBlock. WATTBlocks can bestacked, in which case output {Z̃} becomes a hierarchicalspatio-temporal representation of {X}. As is the case withother TCN-based models, the dilation coefficient is doubled
each temporal module as to provide an increasing receptive field which allows for a computationally inexpensive
way to model long sequences. An additional benefit of thegradual dilation increase is the slow introduction of interaction terms between time series which include less lagged
values for early WATTBlocks and more for later ones. At
layer i, the dilated TCN for scalar output zt has a receptive
field of 2i k, with k being the size of the convolutional kernel. During spatial learning, the information flow across aslice of latent TCN output {Z} at time t {z1,t , . . . zM,t } isthus limited to 2i k lagged values of the raw {X}, given by{x1,[t−2i k:t] , . . . xM,[t−2i k:t] }. We observe that a gradually
increasing the size of this interaction window is key in learning a hierarchical representation of the data that strongly intertwines spatial and temporal causal effects.

5

NDF Tenor Selection

Selecting a profitable tenor is challenging since it burdens
the model with a choice of short tenors with smaller returns
or long, risky tenors with a potentially greater return. One
approach to training a tenor selection model is performing
imitation learning on Expert or Expert oracle labels. Both

have advantages and disadvantages; training on Expert allows for daily online training and thus reduces the needfor the model to extrapolate to periods further into the future. This aspect can be particularly beneficial for turbulent
markets that display frequent regime switches. Expert oracle labels, on the other hand, require information from up
to N days in the future, with N being the maximum allowed tenor, since positive return filtering can only be performed by leveraging spot rate data. Expert oracle labels
can be advantageous since they teach the model to be riskaverse; however, both approaches potentially include unwanted human-bias in the learned strategy. We propose analternative approach in which WATTNet learns from optimal greedy tenor labels obtained directly from market data.
Given the spot rate value for target FX pair {y}, we extract
the optimal tenor label āt at time t as:āt = arg max (yt+a − yt )(5)a∈A

Policy divergence (Ross, Gordon, and Bagnell 2011) isa performance degrading issue often present in imitation
learning where the agent accumulates small expert imitation
errors along the rollout and ends up in unexplored regions of
the state space. To sidestep this issue, we base our strategy
on a conditional independence assumption between tenor atand state dynamics st+1 : p(st+1 |st ) = p(st+1 |st , at ). The
training procedure is then carried out as follows. Input timeseries {X} is split into overlapping length T slices which arethen shuffled into a training dataset. At each trading day t,
the model φ is trained via standard gradient-descent methods
to minimize the cross-entropy loss of outputs φ({X[t−T :t] })and tenor labels āt .

6Experimental Results

NDF Markets The experimental evaluation covers the following 6 major NDF markets: Chinese Yuan (USDCNY), Indonesian Rupiah (USDIDR), Indian Rupee (USDINR), Korean Won (USDKRW), Philippine Peso (USDPHP), Taiwan
Dollar (USDTWD). We elaborate on the results for USDCNY, USDKRW, USDIDR and include discussion of the remaining 3 markets in Appendix C. The selection has been
carried out to test the proposed method on markets with different characteristics as shown in Table 3.
Recurrent and classical baselines Two-layer stacked
GRU and LSTMs are used as additional baselines for tenor
and referred to as GRU-I and LSTM-I. Both recurrent baselines and WATTnet are connected to a fully-connected head
which takes as input the latent representation of {X} produced by the model. A probability distribution over tenor
actions is then obtained via softmax. Additionally, we include the following classical trading baselines:• Momentum-1: 1-day lag of expert tenor actions. Effective
in markets where monotonic behavior in the spot rate isfrequent and the monotonic sequences span several trading periods.
• Momentum-90: best performing tenor from 90 days prior.
Effective in markets with trends whose duration is longer
compared to the maximum tenor.

USDKRW: Expert (oracle), % return gradientBuy<90d

0.08

Buy<80d

0.06

Buy<70d

0.04

Buy<60d

0.02

Buy<50d

0.00

Buy<40d

0.02

Buy<30d

0.04

Buy<20d

0.06

Buy<10d
Hold

0.08

USDKRW: LSTM-I, % return gradientBuy<90d

0.08

Buy<80d

0.06

Buy<70d

0.04

Buy<60d

0.02

Buy<50d

0.00

Buy<40d

0.02

Buy<30d

0.04

Buy<20d

0.06

Buy<10d
Hold

0.08

USDKRW: WATTNet, % return gradientBuy<90d

0.08

Buy<80d

0.06

Buy<70d

0.04

Buy<60d

0.02

Buy<50d

0.00

Buy<40d

0.02

Buy<30d

0.04

Buy<20d

0.06

Buy<10d
Hold

Trading Days

0.08

Figure 2: Tenor actions across USDKRW test set. Background gradient is used to indicate the ROI of different
tenors. The gradient is slanted since the raw return at day
t with a tenor of 90 days is the same as the return from trading day t + i with tenor of 90 − i days. WATTNet learns
to correctly exploit long periods of positive trend and places
less actions than LSTM-I in negative (red) return regions.

Training setup The models are implemented in PyTorchand trained using Adam (Kingma and Ba 2015) and a learning rate cosine decay schedule from 6e−4 down to 3e−4 .
To avoid overfitting uninformative noisy patterns in stale
data input sequence length is set to 30 days. In addition,
to enable a fair comparison and avoid additional overfitting
we employ an early stopping scheme based on training loss
that is motivated by different convergence times of different
models. We use a static testing approach with a long period of 446 out-of-sample trading days to test stability of
the learned trading strategy under turbulent market conditions and a wider distributional shift between in-sample andout-of-sample data. PyTorch code for models and training
procedure will be made available before the workshop.
Metrics The following metrics are used to benchmark the
performance of trading models and baselines:• Return on investment (ROI): given a tenor action a at timet and spot rate
 value xt , the percent ROI is calculated as
ROIt = 100 xt+axt−xt . The returns are not risk-adjusted
and are contextualized by Optimal (2), the best available
return achievable on the test data.

USDCNYUSDKRW

USDIDR

Model

ROI

opt.accnn.accROI

opt.acc.

nn.acc.

ROI

opt.acc.

nn.acc.

OptimalExpert (oracle)

759.8
77.6

100
21.6

100
100

844.2
139.4

100
13.7

100
100

1260.0
152

100
3.6

100
100

Expert
0.0
1.4
47.8
12.7
Momentum-1
14.6
1.4
48.3
10.7
Momentum-90
4.9
6.7
54.1
31.9
GRU-I
26.7. ± 48.5 4.9 ± 0.7 54.7 ± 1.0 −98.7 ± 35.4
LSTM-I
74.3 ± 37.3 3.7 ± 0.9 58.7 ± 2.2 74.6 ± 30.0
WATTNet219.1 ± 25.5 6.7 ± 0.7 59.5 ± 1.6 142.4 ± 16.9

1.9
49.9
230
0.4
67.0
0.9
49.7
201
0.7
66.5
2.6
56.2
338
1.9
69.2
1.5 ± 0.5 52.1 ± 1.4 83.5 ± 33.1 0.6 ± 0.1 62.9 ± 2.6
2.6 ± 0.6 56.0 ± 2.5 146.4 ± 40.4 1.1 ± 0.3 66.5 ± 1.1
2.7 ± 0.2 59.5 ± 1.0 280.2 ± 39.9 1.3 ± 0.3 69.5 ± 0.9

Table 2: Test results in percentages (average and standard error). Best performance is indicated in bold.
Marketµtrainσtrainµval

σval

USDCNY 11.33 231.63 -2.96 232.88
USDIDR 22.04 506.17
3.48 335.75
USDINR
2.88 297.40
8.19 286.79
USDKRW 0.50 471.25 -12.75 387.49
USDPHP 10.44 271.37 -0.79 240.37
USDTWD -3.73 307.28 -2.09 235.29

Table 3: Statistics of 1-step (daily) percent returns. Units reported are scaled up by 1e5 for clarity.
• Optimal accuracy: standard supervised learning accuracyof model outputs ŷt versus optimal tenor labels yt .
• Non-negative return accuracy: accuracy of model outputsyˆt compared to tenor actions with positive or zero return. At time t, there are generally multiple tenor actions
with non-negative return, thus rendering non-negative accuracy a less strict metric compared to optimal accuracy.
It should be noted that it is possible for a high ROI trading model to show poor optimal accuracy but competitive
positive return accuracy since non-negative accuracy also
captures positive ROI strategies that differ from optimal
tenor labels.
Discussion of results We characterize the 6 NDF markets
under evaluation based on mean µ and standard deviation
σ of their 1-day returns and performance of classical baselines. Mean-variance statistics given in Table 3 show easier
markets with similar volatility in training and test periods(e.g. USDINR) as well as markets that appear more erratic
and thus challenging to trade profitably in. From Table 2,
we determine USDCNY and USDKRW to be challenging
for Momentum and recurrent baselines, in addition to being
barely profitable for expert traders. GRU-I is unable to consistently get positive ROI and we suspect this is caused by
its insufficient ability to exchange information between timeseries. LSTM-I, on the other hand, fares better by leveraging
its memory module to perform rudimentary spatial learning (Santoro et al. 2018). USDIDR appears to be highly
profitable for Momentum-90, a phenomenon highlighting alonger positive trend and a higher density of positive return
trades in the test data.

• USDCNY: The results in Table 2 show that all classical baselines, including expert actions, perform poorly
in terms of ROI and optimal tenor accuracy. LSTM and
WATTNet, on the other hand, are able to generalize better, with WATTNet surpassing ROI of oracle trades.
• USDKRW: The USDKRW train-test split presents themost challenging trading period across all 6 NDF markets considered in this work. The return statistics in Table
3 show a decrease in mean return as well as a significant
change in volatility. Traditional baselines perform poorly
whereas WATTNet surpasses Expert oracle ROI. Figure 2
highlights the ability of WATTNet to adjust its tenor actions depending on trading spot rate trends.
• USDIDR: USDIDR has been chosen to evaluate the baselines under a profitable trading period. All classical baselines trade with positive ROI, even surpassing oracle ROI
due to their propensity to trade at longer tenors. WATTNet
achieves performance competitive with Momentum baselines, showing that it is capable of fully exploiting trading periods with long positive trends and a wide shift involatility between training and test data.

6.1

Explainability

Model explainability is particularly important in application areas where the models are tasked with critical decision
making, as is the case for algorithmic trading. Understanding driving factors behind a trading decision is necessary
to properly assess the risks involved. We tackle this issue
by proposing two orthogonal approaches for evaluating thetenor selection strategy in terms of noise stability and driving factors.
Feature importance by input gradients To pinpoint thedriving factors of trades at different tenors we propose sorting the features by their gradient magnitude. In the case oftenor selection, each input feature carries a specific meaning which can be leveraged by domain experts to confirm
whether the model outputs actions consistent with market
dynamics.
Given a batch of N multivariate input sequences{X1 } . . . {XN } with tenor labels equal to a, we computethe cross-entropy loss of model L and derive the empirical
expectation for the absolute value of time series {xj } input

EURSGD : -0.17

1.5

GBPAUD : -0.35

USDCHF : 0.29

1.0

1.00

0.8
1.0

0.75

0.6

0.50

0.4

0.5

0.25

0.2

EURDKK : 0.02

USDSGD : 0.5

AUDNZD: 0.08
1.00

1.00
1.0

0.5

0.75

0.75

0.50

0.50

0.25

0.25

Figure 3: Highest impact spot rate features behind 90 day tenor actions in USDCNY testing period. Background color shows
20 day rolling standard deviation.
WATTNet, UMAP

LSTM-I, UMAP

60
40
20
0

Embedding dim: 080
Embedding dim 1

Embedding dim 1

80

60
40
20

Embedding dim: 00

Figure 4: UMAP embedding of model latents. The points arelabeled according to the final tenor output of the model.

gradient as follows:
Gj =T
1 X ∂LT t=1 ∂xjt

(6)

To illustrate the effectiveness of this approach we select 6 spot rate features with highest absolute gradientvalues for tenor actions of 90 days in the USDCNY
test data: EURSGD, GBPAUD, USDCHF, EURDKK, USDSGD, AUDNZD (Figure 3). The Pearson’s correlation coefficient ρ between USDCNY and each of the above-listedfeatures is computed with training and test sets. In the background, 20 day rolling standard deviation highlights regionsof low and high volatility. Input sequences {x} ∈ {X}
which are mapped by the model to 90 tenor actions are colored in red, whereas the actions themselves are indicated as
black dots. The model learns to trade on long tenors whencurrencies that are positively correlated with USDCNY, suchas USDSGD, undergo periods of growth. The degree towhich such trends affect the model is directly reflected in ρ:
USDCHF, still positively correlated with USDCNY, shows aless decisive positive trend, with additional ups and downs.
Moreover, the model learns to favor trading periods with low
volatility. This analysis can be extended by domain experts
to additional input features, such as technical indicators or
past tenor actions, and can boost confidence in the decisions
made by the model.

Latent space representation Desired properties of thelearned trading strategy are input coherence and stability.
Input coherence is achieved by a model that outputs similar tenors for similar states. Stability, on the other hand,is concerned with how much noise perturbation is required
to cause a tenor switch from a certain state. We perform
a visual inspection of these properties via a uniform manifold approximation and projection (UMAP) which excels
at capturing both local and global structure of the highdimensional data (McInnes, Healy, and Melville 2018). For
each model, latent vectors of their last layer are embedded
into two-dimensional space. UMAP outputs compact clusters of labels for input coherent models and more volumetric clusters for stable models. From Figure D we observethat WATTNet learns a coherent latent representation thatclusters low and high tenors correctly and covers a larger
volume of the embedding space. Instability of GRU-I and
LSTM-I observed in the results of Table 2 can in part be explained by noticing that their learned representation lies onthin lower-dimensional manifolds with mixed tenor labels.
As a result, small noise perturbations can cause wide jumps
in tenor actions, potentially causing a drop in performance.

7

Conclusion

We introduced a challenging imitation learning problem,
tenor selection for non-deliverable-forward (NDF) contracts. With the goal of promoting further research in this direction, we constructed and released a comprehensive NDF
dataset and designed WaveATTentioNet (WATTNet), a novel
model for spatio-temporal data which outperforms expert
benchmarks and traditional baselines across several NDF
markets. Finally, we employed two explainability techniques
to determine driving factors and noise stability of the learned
tenor strategy. Future work includes defining and solving order sizing of an NDF contract, as well as augmenting thetenor selection model with a reinforcement learning agent
head. Reminiscent of world models (Ha and Schmidhuber 2018), such an approach would sacrifice explainability
for additional flexibility; the agent could for example beequipped with a memory module consisting of a recurrent
network to keep track of its previous trades. A different di-rection would be exploring the use of multi-head attention
instead of single-head as an effective modification to WATTNet in the case of long input sequences or multiple relations between features.

References
Carapuço, J.; Neves, R.; and Horta, N. 2018. Reinforcement
learning applied to forex trading. Applied Soft Computing
73:783–794.
Chan, K. C., and Teong, F. K. 1995. Enhancing technical analysis in the forex market using neural networks. In
Proceedings of ICNN’95-International Conference on Neural Networks, volume 2, 1023–1027. IEEE.
Czekalski, P.; Niezabitowski, M.; and Styblinski, R. 2015.
Ann for forex forecasting and trading. In 2015 20th International Conference on Control Systems and Computer Science, 322–328. IEEE.
Galeshchuk, S., and Mukherjee, S. 2017. Deep networks
for predicting direction of change in foreign exchange rates.
Intelligent Systems in Accounting, Finance and Management24(4):100–110.
Guhr, T., and Kälber, B. 2003. A new method to estimate thenoise in financial correlation matrices. Journal of Physics A:
Mathematical and General 36(12):3009.
Ha, D., and Schmidhuber, J. 2018. Recurrent world models
facilitate policy evolution. In Advances in Neural Information Processing Systems, 2450–2462.
Hu, Z.; Liu, W.; Bian, J.; Liu, X.; and Liu, T.-Y. 2018. Listening to chaotic whispers: A deep learning framework fornews-oriented stock trend prediction. In Proceedings of theEleventh ACM International Conference on Web Search andData Mining, 261–269. ACM.
Judah, K.; Fern, A. P.; and Dietterich, T. G. 2012. Active
imitation learning via reduction to iid active learning. In
2012 AAAI Fall Symposium Series.
Kingma, D. P., and Ba, J. 2015. Adam: A method forstochastic optimization. In 3rd International Conference on
Learning Representations, ICLR 2015, San Diego, CA, USA,May 7-9, 2015, Conference Track Proceedings.
Lai, G.; Chang, W.-C.; Yang, Y.; and Liu, H. 2018. Modeling long-and short-term temporal patterns with deep neuralnetworks. In The 41st International ACM SIGIR Conference
on Research & Development in Information Retrieval, 95–104. ACM.
McInnes, L.; Healy, J.; and Melville, J. 2018. Umap: Uniform manifold approximation and projection for dimension
reduction. arXiv preprint arXiv:1802.03426.
Mishra, N.; Rohaninejad, M.; Chen, X.; and Abbeel, P.
2017. A simple neural attentive meta-learner. arXiv preprint
arXiv:1707.03141.
Nassirtoussi, A. K.; Aghabozorgi, S.; Wah, T. Y.; and Ngo,
D. C. L. 2015. Text mining of news-headlines for forex
market prediction: A multi-layer dimension reduction algorithm with semantics and sentiment. Expert Systems with
Applications 42(1):306–324.

Ntakaris, A.; Mirone, G.; Kanniainen, J.; Gabbouj, M.; andIosifidis, A. 2019. Feature engineering for mid-price prediction with deep learning. IEEE Access 7:82390–82412.
Pathberiya, H. A.; Tilakaratne, C. D.; and Hansen, L. 2017.
An intelligent system for forex trading: Hybrid ann withgarch and intrinsic mode functions. In 2017 Intelligent Systems Conference (IntelliSys), 436–445. IEEE.
Petropoulos, A.; Chatzis, S. P.; Siakoulis, V.; and Vlachogiannakis, N. 2017. A stacked generalization system
for automated forex portfolio trading. Expert Systems with
Applications 90:290–302.
Ross, S.; Gordon, G.; and Bagnell, D. 2011. A reduction ofimitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international
conference on artificial intelligence and statistics, 627–635.
Santoro, A.; Faulkner, R.; Raposo, D.; Rae, J.; Chrzanowski,
M.; Weber, T.; Wierstra, D.; Vinyals, O.; Pascanu, R.; andLillicrap, T. 2018. Relational recurrent neural networks. In
Advances in Neural Information Processing Systems, 7299–
7310.
Sornmayura, S. 2019. Robust forex trading with deep q
network (dqn). ABAC Journal 39(1).
van den Oord, A.; Dieleman, S.; Zen, H.; Simonyan, K.;
Vinyals, O.; Graves, A.; Kalchbrenner, N.; Senior, A.; andKavukcuoglu, K. Wavenet: A generative model for raw audio. In 9th ISCA Speech Synthesis Workshop, 125–125.
Vargas, M. R.; De Lima, B. S.; and Evsukoff, A. G. 2017.
Deep learning for stock market prediction from financial
news articles. In 2017 IEEE International Conference on
Computational Intelligence and Virtual Environments for
Measurement Systems and Applications (CIVEMSA), 60–65.
IEEE.
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones,L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017. Attention is all you need. In Advances in neural information
processing systems, 5998–6008.
Xingjian, S.; Chen, Z.; Wang, H.; Yeung, D.-Y.; Wong, W.K.; and Woo, W.-c. 2015. Convolutional lstm network: A
machine learning approach for precipitation nowcasting. In
Advances in neural information processing systems, 802–
810.
Yu, P.; Lee, J. S.; Kulyatin, I.; Shi, Z.; and Dasgupta,
S.
2019.
Model-based deep reinforcement learning for dynamic portfolio optimization. arXiv preprint
arXiv:1901.08740.
Yu, L.; Lai, K. K.; and Wang, S. 2005. Designing a hybrid ai system as a forex trading decision support tool. In
17th IEEE International Conference on Tools with Artificial
Intelligence (ICTAI’05), 5–pp. IEEE.
Yu, B.; Yin, H.; and Zhu, Z. 2018. Spatio-temporalgraph convolutional networks: a deep learning framework
for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, 3634–
3640. AAAI Press.

6 context FX instruments
Currency 1
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar

58 context FX instruments

Currency 2

Name

Chinese Yuan
Indonesian Rupiah
Indian Rupee
Korean Won
Philippine Peso
Taiwan Dollar

USDCNYUSDIDRUSDINRUSDKRW
USDPHPUSDTWD

Table 4: NDF FX instruments

A

Dataset details

Here we report more information about the dataset.
• Period: 2013-09-10 to 2019-06-17• Number of features (M dimension of {X}): 1123. 64 FX
pairs, 519 technical indicators, 540 NDF volume features.
More specifically, 90 NDF volume features per NDF pair
(90 * 6)The list of context NDF spot rates is given in Table 1, andcontext FX spot rates in Table 2. Context spot rates havebeen obtained via the Oanda Developer API.
Technical Indicators Here we include more details about
the technical indicators, including their dimensionality as
number of features.
• Simple moving average (SMA): SMA is a trend-following
indicators that filters out high frequency oscillations bysmoothing spot rate time series. We use both 7 day and 21
day averages (i.e N = 7, N = 21). Included for all 64 FX
pairs (total: 64 * 2 = 128 features).
• Exponential moving average (EMA): similarly to SMA,EMA is a lagging indicator. The weight for each values
exponentially decreases, with bigger weight assigned tomore recent values of the time series. We include 12 day
and 26 day EMAs (total: 128 features).
n
xt
t=1
µt =
(7)
αxt + (1 − α)µt−1 t > 1• Moving Average Convergence Divergence (MACD): A
filtering function with a bigger time constant is subtracted
to another with a smaller one in order to estimate thederivative of a time series. In particular, a common choiceof filtering functions are 12 day and 26 day exponential
moving averages: MACD = EMA12 − EMA26 (total: 64
features).
• Rolling standard deviation (RSD): a window of 20 days isused to compute standard deviation at time t = 20 (total:
64 features).
• Bollinger Bands (BB): commonly used to characterizevolatility over time. We include both an upper band derived and a lower band as features BB = SMA ± RSD(total: 64 * 2 = 128 features)

Currency 1
Euro
Pound Sterling
United States Dollar
United States Dollar
United States Dollar
Euro
Euro
Austrial Dollar
Austrial Dollar
Euro
Pound Sterling
Euro
Euro
Euro
Euro
Euro
Euro
Euro
Euro
Euro
Euro
Euro
Euro
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
United States Dollar
Pound Sterling
Pound Sterling
Pound Sterling
Pound Sterling
Australian Dollar
Australian Dollar
Canadian Dollar
Swiss FrancNew Zealand Dollar
New Zealand Dollar
Singapore Dollar
South African Rand
Pound Sterling
Pound Sterling
Pound Sterling
Australian Dollar
Canadian Dollar
Canadian Dollar
Swiss FrancNew Zealand Dollar
New Zealand Dollar
Turkish Lira
United States Dollar

Currency 2

Name

United States Dollar
United States Dollar
Canadian Dollar
Swiss Franc
Japanese Yen
Pound Sterling
Swiss Franc
United States Dollar
Canadian Dollar
Japanese Yen
Japanese Yen
Australian Dollar
Czech Koruna
Hungarian Forint
New Zealand Dollar
Swedish Krona
Singapore Dollar
Canadian Dollar
Danish Krone
Norwegian Krone
Polish Zloty
Turkish Lira
South African Rand
Danish Krone
Hungarian Forint
Mexican Peso
Poland Zloty
Swedish Krona
Thai Baht
South African Rand
Czech Koruna
Hong Kong Dollar
Norwegian Krone
Saudi Riyal
Singapore Dollar
Turkish Lira
Australian Dollar
Swiss FrancSouth African Rand
Singapore Dollar
Japanese Yen
Singapore Dollar
Japanese Yen
Japanese Yen
Canadian Dollar
United States Dollar
Japanese Yen
Japanese Yen
Canadian Dollar
New Zealand Dollar
Poland Zloty
New Zealand Dolar
Swiss Franc
Singapore Dollar
South African Rand
Japanese Yen
Singapore Dollar
Japanese Yen
Malaysian Ringgit

EURUSD
GBPUSDUSDCADUSDCHF
USDJPYEURGBPEURCHF
AUDUSDAUDCADEURJPY
GBPJPYEURAUD
EURCZKEURHUF
EURNZDEURSEKEURSGDEURCADEURDKKEURNOKEURPLNEURTRYEURZARUSDDKKUSDHUFUSDMXNUSDPLNUSDSEKUSDTHBUSDZARUSDCZKUSDHKDUSDNOKUSDSARUSDSGDUSDTRY
GBPAUD
GBPCHF
GBPZAR
GBPSGD
AUDJPYAUDSGDCADJPYCHFJPYNZDCADNZDUSD
SGDJPYZARJPYGBPCAD
GBPNZD
GBPPLNAUDNZDCADCHFCADSGDCHFZARNZDJPYNZDSGDTRYJPYUSDMYR

Table 5: 58 FX instruments used as context in the dataset

LayerRecurrent-1
Recurrent-2
FC-1
FC-2

Input dim. Output dim.
1123
512
512
128

512
512
128
91

Table 6: Layer dimensions for recurrent models GRU-I and
LSTM-I
Layer

M-in M-out T-in T-out

FC-cmp1123
WATTBlock-1
90
WATTBlock-2
90
WATTBlock-3
90
WATTBlock-4
90
WATTBlock-5
90
WATTBlock-6
90
WATTBlock-7
90
WATTBlock-8
90
FC-1
5*90
FC-2
512

90
90
90
90
90
90
90
90
90
512
91

30
30
27
23
21
17
15
11
9
1
1

30
27
23
21
17
15
11
9
5
1
1

Table 7: Layer dimensions (M and T) for WATTNet• ARIMA spot rate forecast: an ARIMA model is trained
to perform 1-day forecasts of spot rates using data fromperiods preceding the start of the training set to avoid information leakage. The forecasts are added as time series
features. Included for NDF pairs and for USDMYR (total:
7 features)
Training hyperparameters The data is normalized as
xt = xtσ−µ where µ and σ are 60-day rolling mean andstandard deviation respectively. The dataset is split into ovelapping sequences of 30 trading days. Batch size is set to32.

B

Architectural hyperparameters

We herein provide detailed information about the model design.
Recurrent Models GRU-I and LSTM-I share the samestructure given in Table 6. Layer depth has been chosen
as the best performing in the range 1 to 6. Latent spatiotemporal representation {Z̃} of input time series {X} is obtained as the output of Recurrent-2. {Z̃} is the tensor transformed via UMAP and shown in Appendix D.
WATTNet Details about the WATTNet used are foundin Table 7. We employ fully-connected layer (FC-emp)
for compression in order to constrain GPU memory usage to less than 6GB. Improved results can be obtained bylifting this restriction and increasing the WATTBlock M dimension. {Z̃} is the output of WATTBlock-8. Due to therelatively short input sequence length, the dilation is scheduled for reset every 2 WATTBlocks. More specifically, thedilation coefficients for temporal learning are 2, 4, 8, 16,
2, 4, 8, 16 for WATTBlock-1 to 8. As is common in otherdilated TCN models, dilation introduces a reduction of T
dimension for deeper layers. This effect is observable in
Table 7. We utilize a residual architecture for the attention
module where the output is the summation of pre and post
attention tensors, i.e:{Z̃out } = σ({z̃1,t . . . z̃M,t }) + ({x1,t . . . xM,t })(8)Computational requirements of WATTNet WATTNet
can be used as a general lightweight tool for spatio-temporalmodeling. The temporal modules are fully parallelizable
due to complete independence of inputs and convolutional
weights across different univariate time series. To leveragean already existing fast CUDA implementation of parallel
TCNs in PyTorch, we utilize grouped-convolutions. In particular, the M dimension of input time series {X} becomesthe channel dimension for the TCN, and different convolutional kernels are applied to each input channel to obtainthe corresponding output. On the spatial learning front, theattention modules have a computational cost of O(T M 2 )which is comparable to the standard quadratic attention costof O(M 2 ) when T << M .

CAdditional results

We provide additonal results and discussion of USDINR,USDPHP and USDTWD. Training and test periods for all
NDF pairs are visualized in Figures 1-2 for reference. Figure
7 shows Expert and Expert oracle tenor action distributions.
The results are given in Table 5.
USDINR USDINR shows positive trend in mean returnand its volatility remains constant between training and test.
All baselines achieve high ROI in this scenario. Of particular interest is GRU-I, which surpasses LSTM-I likely due
to the relative simplicity of USDINR dynamics. With a stable, positive trend GRU-I is able to focus on the spot rates
directly, thus circumventing its inability to perform proper
spatial learning.
USDPHP USDPHP is slightly more challenging due to its
minimal volatility shift. WATTNet outperforms other models in terms of ROI and optimal accuracy.
USDTWD USDTWD has negative mean return andshows a reduction in volatility in the test set. WATTNet isable to exploit this phenomenon and significantly outperforms all other baselines.
Rolling testing and online training In general, static testing can turn out to be particularly challenging for financial data with longer test sets, since the models are tasked
with extrapolating for long periods, potentially under nonstationary condition of the market. A rolling testing approach can be beneficial; however, the size of window andUSDINR
Model
OptimalExpert (oracle)

USDPHPUSDTWD

ROI

opt.accnn.accROI

opt.acc.

nn.acc.

ROI

opt.acc.

nn.acc.

1288.7
123.2

100
6.7

100
100

900.0
114.1

100
6.7

100
100

612.0
91.4

100
9.8

100
100

Expert
216.1
0.7
62.7
117.1
0.2
57.2
43.3
Momentum-1
218.1
2.6
62.5
110.9
0.7
58.9
44.0
Momentum-90
300.9
2.2
57.4
134.9
1.2
58.9
10.0
GRU-I
273.5 ± 29.0 2.0 ± 0.5 63.2 ± 2.3 244.2 ± 27.5 2.3 ± 0.1 62.5 ± 1.4 46.6 ± 34.9
LSTM-I239.9 ± 67.3 2.4 ± 0.5 61.6 ± 2.0 307.1 ± 16.8 1.8 ± 0.2 71.3 ± 1.0 46.8 ± 27.9
WATTNet313.0 ± 28.1 3.7 ± 0.2 65.9 ± 1.0 329.7 ± 31.2 3.3 ± 0.6 70.0 ± 0.7 162.9 ± 13.9

0.9
50.2
0.5
50.0
1.4
55.2
2.5 ± 0.3 57.6 ± 1.2
2.8 ± 0.6 59.6 ± 1.4
2.8 ± 0.3 59.8 ± 1.4

Table 8: Test results in percentages (average and standard error). Best performance is indicated in bold.
test periods require ad-hoc tuning for each FX currency. A
key factor influencing the optimal choice of window size isthe average length of market regimes. We leave the discussion on optimal window selection for rolling testing as future work. Figure 7 shows distributions of tenor labels andhighlights the difference in average tenor length.

D

Tenor actions and latent UMAP
embeddings

In this section we provide a complete collection of tenor action plots and UMAP embeddings for all NDF markets under consideration. Background gradient is used to indicate
the ROI of different tenors. The gradient is slanted since theraw return at day t with a tenor of 90 days is the same as thereturn from trading day t + i with tenor of 90 − i days.

7.0
6.8

70

USDINR spot rate
Training
Validation

65
60
55
2014

2015

2016

2017

2018

2019

USDKRW spot rate

1250

Training
Validation

1200
1150
1100
1050

USDCNY spot rate

1000

Training
Validation

2014

2015

2016

2017

2018

2019

2018

2019

USDPHP spot rate

6.6
6.4
6.2
2014

2015

2016

2017

2018

2019

USDIDR spot rate
15000
14000
13000
12000
11000
10000

75

Training
Validation

54
52
50
48
46
44
42
40

Training
Validation

2014

2015

2016

2017

USDTWD spot rate

34

Training
Validation

33
32
2014

2015

2016

2017

2018

2019

Figure 5: Training and test periods for NDF pairs

31
30
29

2014

2015

2016

2017

2018

2019

Figure 6: Training and test periods for NDF pairs

USDCNY, EXPERT
120
100
80
60
40
20
0 0 10 20 30 40 50 60 70 80 90

USDIDR, EXPERT

350
300
250
200
150
100
50
0 0 10 20 30 40 50 60 70 80 90

USDINR, EXPERT

400
350
300
250
200
150
100
50
0 0 10 20 30 40 50 60 70 80 90

USDKRW, EXPERT

400
350
300
250
200
150
100
50
0 0 10 20 30 40 50 60 70 80 90

USDPHP, EXPERT200

USDCNY, EXPERT ORACLE
200
150
100

0 0 10 20 30 40 50 60 70 80 90

USDIDR, EXPERT ORACLE

200
150
100
50
0 0 10 20 30 40 50 60 70 80 90

USDINR, EXPERT ORACLE

200
175
150
125
100
75
50
25
0 0 10 20 30 40 50 60 70 80 90
250

USDKRW, EXPERT ORACLE

200
150
100

0 0 10 20 30 40 50 60 70 80 90
250

USDPHP, EXPERT ORACLE

100

50

50

0 0 10 20 30 40 50 60 70 80 90

0 0 10 20 30 40 50 60 70 80 90

USDTWD, EXPERT

350
300
250
200
150
100
50
0 0 10 20 30 40 50 60 70 80 90

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

50

150

100

USDCNY: Optimal tenors, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

0.08

50

200

150

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

USDTWD, EXPERT ORACLE

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

200
150
100
50
0 0 10 20 30 40 50 60 70 80 90

Figure 7: Distribution of tenor actions for Expert and Expert
Oracle.

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

USDCNY: Expert (oracle), % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06
0.08

USDCNY: GRU, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDCNY: LSTM, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDCNY: WATTNet, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

Figure 8: USDCNY tenor actions

0.08

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

USDIDR: Optimal tenors, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDIDR: Expert (oracle), % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06
0.08

USDIDR: GRU, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDIDR: LSTM, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDIDR: WATTNet, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

Figure 9: USDIDR tenor actions

0.08

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

USDINR: Optimal tenors, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDINR: Expert (oracle), % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06
0.08

USDINR: GRU, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDINR: LSTM, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDINR: WATTNet, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

Figure 10: USDINR tenor actions

0.08

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10dHold

USDKRW: Optimal tenors, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading DaysUSDKRW: Expert (oracle), % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06
0.08

USDKRW: GRU, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading DaysUSDKRW: LSTM, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading DaysUSDKRW: WATTNet, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

Figure 11: USDKRW tenor actions

0.08

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

USDPHP: Optimal tenors, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDPHP: Expert (oracle), % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06
0.08

USDPHP: GRU, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDPHP: LSTM, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDPHP: WATTNet, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

Figure 12: USDPHP tenor actions

0.08

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

Buy<90d
Buy<80d
Buy<70d
Buy<60d
Buy<50d
Buy<40d
Buy<30d
Buy<20d
Buy<10d
Hold

USDTWD: Optimal tenors, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDTWD: Expert (oracle), % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06
0.08

USDTWD: GRU, % return gradient

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDTWD: LSTM, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

USDTWD: WATTNet, % return gradient

0.08

0.08
0.06
0.04
0.02
0.00
0.02
0.04
0.06

Trading Days

Figure 13: USDTWD tenor actions

0.08

Embedding dim: 1

WATTNet, `USDCNY`, UMAP

Embedding dim: 1

90
80
70
60
50
40
30
20
10
0

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 2

LSTM-I, `USDCNY`, UMAP

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 1
Embedding dim 1

Embedding dim: 1

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

GRU-I, `USDCNY`, UMAP

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Embedding dim 1

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Figure 14: UMAP of latent representation - USDCNY

Embedding dim: 1

WATTNet, `USDIDR`, UMAP

Embedding dim: 1

90
80
70
60
50
40
30
20
10
0

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 2

LSTM-I, `USDIDR`, UMAP

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 1
Embedding dim 1

Embedding dim: 1

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

GRU-I, `USDIDR`, UMAP

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Embedding dim 1

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Figure 15: UMAP of latent representation - USDIDR

Embedding dim: 1

WATTNet, `USDINR`, UMAP

Embedding dim: 1

90
80
70
60
50
40
30
20
10
0

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 2

LSTM-I, `USDINR`, UMAP

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 1
Embedding dim 1

Embedding dim: 1

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

GRU-I, `USDINR`, UMAP

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Embedding dim 1

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Figure 16: UMAP of latent representation - USDINR

Embedding dim: 1

WATTNet, `USDKRW`, UMAP

Embedding dim: 1

90
80
70
60
50
40
30
20
10
0

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 2

LSTM-I, `USDKRW`, UMAP

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 1
Embedding dim 1

Embedding dim: 1

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

GRU-I, `USDKRW`, UMAP

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Embedding dim 1

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Figure 17: UMAP of latent representation - USDKRWGRU-I, `USDPHP`, UMAP

Embedding dim: 1

90
80
70
60
50
40
30
20
10
0

WATTNet, `USDPHP`, UMAP

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 2

Embedding dim: 1

90

80

80

70

70

70

60
50
40

60
50
40

30

Embedding dim 09080
Embedding dim 2

Embedding dim 1
Embedding dim 1

90

GRU-I, `USDPHP`, UMAP

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Embedding dim 1

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

60
50
40

30

Embedding dim: 030

Embedding dim: 1

Embedding dim: 2

Figure 18: UMAP of latent representation - USDPHP

Embedding dim: 1

WATTNet, `USDTWD`, UMAP

Embedding dim: 1

90
80
70
60
50
40
30
20
10
0

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim: 2

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 2

LSTM-I, `USDTWD`, UMAP

90
80
70
60
50
40
30
20
10
0

Embedding dim 0

Embedding dim 1
Embedding dim 1

Embedding dim: 1

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

GRU-I, `USDTWD`, UMAP

Embedding dim 2

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Embedding dim 1

Embedding dim: 0

90
80
70
60
50
40
30
20
10
0

Figure 19: UMAP of latent representation - USDTWD