# DAuGAN: An Approach for Augmenting Time Series Imbalanced Datasets via Latent Space Sampling Using Adversarial Techniques

Authors: Andrei Bratu; Gabriela Czibula
Journal: Scientific Programming
 (2021)

---

## Abstract

Data augmentation is a commonly used technique in data science for improving the robustness and performance of machine learning models. The purpose of the paper is to study the feasibility of generating synthetic data points of temporal nature towards this end. A general approach named DAuGAN (Data Augmentation using Generative Adversarial Networks) is presented for identifying poorly represented sections of a time series, studying the synthesis and integration of new data points, and performance improvement on a benchmark machine learning model. The problem is studied and applied in the domain of algorithmic trading, whose constraints are presented and taken into consideration. The experimental results highlight an improvement in performance on a benchmark reinforcement learning agent trained on a dataset enhanced with DAuGAN to trade a financial instrument.


---

## Body

## 1. Introduction

Data augmentation is a vast and often used method for enhancing the amount of data available for training a machine learning (ML) model. It is well known that the amount and quality of data available are closely bounded to the success of any ML project, independent of the application domain. There are multiple data augmentation procedures, which are often specific to the application domain and the specific dataset that is used.For example, image-based machine learning tasks often employ operations of contrast adjustment, flipping, translation, cropping, rotation, color space manipulation, etc. These present new contexts to the model, helping it to better generalize and to avoid overfitting [1]. Another example of data augmentation is the SMOTE, or the Synthetic Minority Oversampling Technique, whose purpose is to alter the dataset presented to the algorithm by presenting minority-class data points to the classifier more often than they naturally occur (oversampling), while minimising the rate at which the majority class appears (undersampling). This increases the sensibility of the model for the subrepresented data class and has applications such as identifying fraud credit card transactions [2].A more recent augmentation method involves using agenerative adversarial neural network (GAN) architecture, whose ability to reproduce a statistical distribution is repurposed for creating new, convincing examples of a poorly represented class, or generally any point of the dataset [3]. GANs are an important machine learning paradigm. Two neural networks engage in a zero-sum game where the generator network attempts to generate new samples, while the discriminator discerns between real samples and generated samples. The end goal is to train the generator into reproducing the initial train distribution as close as possible. GANs have been used to great effect, with examples such as reproducing the effects of dark matter on astronomical observations [4], generating photorealistic human faces [5], or applying style transfer operations in the audiodomain [6].Identifying imbalanced classes and enhancing their presence in the time series would not be devoid of practical applications. One such example issecurities trading. Securities are defined as any financial instrument that can be bought or sold via an accredited intermediary, creating a supply and demand market. An example is the stock market, which allows buying and selling “shares”: discreet units of ownership in a company. While the price of any share has a correlation with the business performance, there is research that indicates the market’s sentiment and domain-specific factors such as national interest rate create a cyclical effect on the price evolution [7, 8].The current paper is based on work originating from two research questions:(1)
RQ1. Is it possible to improve the performance ofreinforcement learning- (RL-) based trading algorithms through augmenting training data using adversarial techniques?(2)
RQ2. What is the performance gain of the RL agent trained on the enhanced data over a baseline RL agent trained on the initial data, without augmentation?To this end, a general approach namedData Augmentation using GANs is proposed for identifying poorly represented sections of security-related time series. Leveraging that autoencoder neural network architectures can encode and decode complex temporal dependencies to and from a latent space, the proposal reduces the problem of identifying poorly represented time series periods into a clustering problem [9] and synthesises new examples of the minority class using a GAN architecture. A method of integrating synthesised data points into the original time series, respecting the original constraints of the data, is presented, and experiments are carried out for measuring the performance improvement on benchmark reinforcement learning algorithms.Parallels between the proposed method and the TimeGAN method proposed by Yoon et al. for generating arbitrary time-series [10] are acknowledged. The proposed method improves by particularising the problem to the constraints of security-related time series on one hand and the problem of data series minority data augmentation on the other one.Section2 introduces fundamental concepts used by the approach, along with a literature review on time series generation. DAuGAN is introduced in Section 3 along with proposed methodology, whilst experimental results and their analysis are presented in Section 4. The conclusions of the paper and directions to further improve and extend DAuGAN are outlined in Section 5.


## 2. Background

This section presents a literature overview of the technical notions used in this paper. The importance and evolution of generative adversarial and autoencoder networks are presented, together with a brief review on reinforcement learning. Historical approaches related to data augmentation and data synthesis on time series are also presented.


### 2.1. Generative Adversarial Networks

GANs is a deep learning architecture that have been first introduced by Ian Goodfellow et al. [11], which has been heavily used in image-based tasks, from synthesising new images [12] to reproducing the content of one image in the style of another.At a very high level, the generative adversarial network technique pits two deep neural networks against each other in a zero-sum game. One of the networks, thegenerator, acts as a map from a latent space towards a desired distribution, sampling noise from the latent space that is synthesised as closely as possible to a point in the distribution. Its counterpart, the critic, is fed samples from both the real distribution and from the generator, with the goal of deciding whether the sample is “real” or “fake.” Over time, the two networks improve at their goal, resulting in not only better fakes from the generator but also a better ability to discern the fakes from the critic. Ideally, the system converges towards equilibrium where the critic can no longer separate between the two classes; i.e., it assign equal probability for any sample to be either one of the classes.Formally, the generator attempts to minimise the value of the following loss function, while the discriminator attempts to maximise it:ExlogDx+Ezlog1−DGz, where x is a random variable corresponding to the real distribution, z is a random variable assigned to the generated distribution, Gx is the generator’s output, Dx is the discriminator’s output, and Ev denotes the expected value over all instances v.The expected value is used to indicate that the loss is the average over all samples of the batch at a given training step. The critic assigns values from 0 to 1, estimating the probability that a sample is real. Lettingx denote the real distribution and z denote the synthesised distribution, the critic attempts to maximise this loss, the upper bound being obtained when all labels are properly assigned, while the generator attempts to minimise it by controlling the second term, i.e., generating more convincing examples, signified by the Gz term. The log operations are derived from the cross entropy between the real and fake distributions.Notorious problems affecting GANs aremode collapse and vanishing gradients. Vanishing gradients is a general issue in machine learning, where gradients prove insufficient for the machine learning model to update meaningfully, while this issue has classically occurred in neural models with high depth [13] or recurrent networks such as long-short term memory architecture [14]. However, the issue manifests particularly in the case of generative adversarial networks: the unadjusted loss formulation presented above will result in a critic that converges faster than the generator. Thus, the critic cannot offer constructive feedback for the generator to improve on, since it perfectly discerns between real and fake.A connected issue with vanishing gradients that is faced by generative adversarial networks ismode collapse. The problem manifests on the generator’s side by mapping all latent points to the same synthesised sample. From the perspective of game theory, both mode collapse and vanishing gradients issues are caused by the two players, the critic and the generator, converging to a local, undesired optimum of the game that does not offer enough incentives for any of the networks to update their weights [15].Several improvements on the domain transfer subproblem have been addressing the mode collapse issue. CycleGAN [16] introduces the following adjustments: instead of sampling from a latent random space, the generator samples from the input space of the input domain, with output in the target domain. The critic is fed both generated images and those belonging to the target domain, thus encouraging the generator to learn a better mapping between input and target. Thus, the improvement resides in translating the task into an unsupervised task, as the generator is ideally able to map any image from the first domain to an image in the target domain. A limitation of the CycleGAN paper is the domains being required to be homogeneous [17]. An improvement over the CycleGAN is represented by TraVeLGAN [18], which adds to the classical two network architectures formed of a generator and critic, a third, siamese network, and eliminates the domain homogeneity constraint [19].The Wasserstein variation of the generative adversarial network architecture (WGAN), authored by Arjovsky et al. [20] offers a robust method to train GAN architectures. The WGAN improves the stability of learning, eliminates problems such as mode collapse, and provides meaningful learning curves useful for hyperparameter searches.


### 2.2. Autoencoders

The autoencoder (AE) architecture uses a two-part neural network to transform the input in a compressed and meaningful representation using the encoder part, recreating the input using the decoder part [21]. The technique proves to be immensely flexible and is of interest to the purpose of this thesis since former research proved that autoencoders are able to encode and decode complex temporal features into the latent space [9]. Furthermore, there are no special theoretical aspects to be considered over a general-purpose deep learning architecture. The autoencoder is presented as two symmetrical parts, with a small, latent representation in the middle, usually trained to minimise the mean squared error between the distribution and itself.AEs can be interpreted as an improvement over the statistical technique of principal component analysis. Principal component analysis withp dimensions identifies an orthonormal base of p vectors that best identify the variance of the input distribution [22]. This technique is limited to linear representations, unlike the manifold organized by the autoencoder. Thus, the AE is able to construct higher fidelity correlations between the original and latent spaces, preserving relationships. Furthermore, there are multiple accounts in the literature in using the latent representation over the initial dataset with great effect for increased classification performance, better interpretability of the obtained clusters, or better ability to generalize over the latent representation [23–25].Salakhutidnov and Hinton restrict the latent representation to a binary code which is interpreted as the output of a black-box hash function modelled by the encoder. The hashing is applied in the field of document retrieval, where the hashing of the query is used to retrieve the directly associated documents plus documents from neighbouring hashes. The task has also been approached from a generative approach by Hansen et al. inUnsupervised Neural Generative Semantic Hashing [26].


### 2.3. Reinforcement Learning

Reinforcement learning (RL) is a paradigm of the machine learning field where problems are modelled around two fundamental notions: agents and environments. Agents are able to interact with the environment via a defined set of “actions” which change the environment’s “state.” Defining the problem’s solution as a desirable environment state, the agent is conditioned via “rewards” and “punishments” to reach this favourable state. RL purpose is to teach an agent the optimal policy of acting inside an environment. The environment can at any moment be in a certain state that can be changed by the agent’s actions. The agent receives feedback from the environment in the form of a reward. Using a tradeoff between reward and value (future reward received by the agent by taking a certain action in a particular state), the agent learns a policy that decides the best course of action for a given state.This flexible framework has allowed to model complex real-world situations: scheduling drug administration to patients with chronic conditions in order to minimise risk of negative interactions [27, 28], minimising energy costs associated with cooling of data centers [29], or outmatching human players in games such as Go [30].RL is facing a growing interest in the discipline of algorithmic trading [31, 32]. The interest can be explained by the ease with which the problem can be modelled: given the price fluctuation of a certain instrument, an agent’s purpose is to maximise the overall profit. Current frontier in reinforcement learning focuses on improved training performance, particularly incentivising the agent to explore multiple action courses and breaking the state causality effect on training by sampling and replaying random past states [33]. Intuitively, these improvements focus on offering the agent the ability to retrospect and decide on past better courses of action.


### 2.4. Time Series Generation

GAN-based methods or generative adversarial network models have emerged as a popular technique for generating or augmenting datasets, especially with images and videos. However, GANs give poor fidelity in networking data, which has both complex temporal correlations and mixed discrete-continuous data types. Although GAN-based time series generation exists—for instance for medical time series—such techniques fail on more complex data exhibiting poor autocorrelation scores on long sequences while prone to mode collapse.TimeGAN architecture introduced by Yoon et al. [10] is of strong interest for the proposed method, as it reinforces the idea that latent spaces can be used to better understand the original time series distribution of the data. Specifically, the paper proposes using two latent spaces, HS and HX, where S represents the mathematical space of static features of the time series, e.g., gender, while X represents the space of temporal dependencies of the time series, e.g., the cholesterol level as the person ages. The paper asserts that instead of using only a generator-discriminator system for creating new samples, introducing supervised learning to the unsupervised generation will increase the fidelity of generated data. The supervised loss comes from two encoder-decoder pairs (h:S⟼HS,e:HS⟼S) and (hX:X⟼HX,eX:HX⟼X) between the initial space and latent space, with the GAN networks learning to directly replicate the latent vectors: (g:ZS⟼HS,d:HS⟼IR) and (gX:ZS⟼HS,d:HS⟼IR), where ZS and ZX are the space the random noise is sampled from. Of particular interest is the use of recurrent neural networks throughout the architecture. Notably, gX features the use of a 2-step autoregression dependency for creating the temporal latent vector. Recurrent neural networks are also used for encoding and decoding between X and HX.DoppelGANger architecture introduced by Lin et al. [34] represents a current benchmark in time series generation. It tackles a similar problem with TimeGAN as both separate the generation of static attributes from the time series measurements, although focusing on privacy over accurate reproduction of the target distribution time series. Specifically, the generation procedure for the time series implies a conditional process akin to prior work with Conditional Generative Adversarial Nets [35].The network further improves by providing a normalization approach that avoids mode collapse on long time series: instead of normalizing the entire data set at once, using the global minimum and maximum, the algorithm normalizes using the per-sample minimum and maximum. Furthermore, the minimum and maximum are attached as static metadata describing the associated time series. Thus, the generator is responsible for creating the static features and is also in charge of creating the features that normalize the time series.The final DoppelGANger architecture uses three networks for generating data: a generator network used for generating static attributes and a generator network used for generating the minimum and maximum of each time series, as described above. The data generated by the two networks is fed into the third, a recurrent neural network which leverages the provided information plus its internal state to generate measurements. A stacked discriminator critiques the generator’s work: one model focuses on the generated static features (also called metadata), while the other is a recurrent neural network critiquing the generated measurements.


## 2.1. Generative Adversarial Networks

GANs is a deep learning architecture that have been first introduced by Ian Goodfellow et al. [11], which has been heavily used in image-based tasks, from synthesising new images [12] to reproducing the content of one image in the style of another.At a very high level, the generative adversarial network technique pits two deep neural networks against each other in a zero-sum game. One of the networks, thegenerator, acts as a map from a latent space towards a desired distribution, sampling noise from the latent space that is synthesised as closely as possible to a point in the distribution. Its counterpart, the critic, is fed samples from both the real distribution and from the generator, with the goal of deciding whether the sample is “real” or “fake.” Over time, the two networks improve at their goal, resulting in not only better fakes from the generator but also a better ability to discern the fakes from the critic. Ideally, the system converges towards equilibrium where the critic can no longer separate between the two classes; i.e., it assign equal probability for any sample to be either one of the classes.Formally, the generator attempts to minimise the value of the following loss function, while the discriminator attempts to maximise it:ExlogDx+Ezlog1−DGz, where x is a random variable corresponding to the real distribution, z is a random variable assigned to the generated distribution, Gx is the generator’s output, Dx is the discriminator’s output, and Ev denotes the expected value over all instances v.The expected value is used to indicate that the loss is the average over all samples of the batch at a given training step. The critic assigns values from 0 to 1, estimating the probability that a sample is real. Lettingx denote the real distribution and z denote the synthesised distribution, the critic attempts to maximise this loss, the upper bound being obtained when all labels are properly assigned, while the generator attempts to minimise it by controlling the second term, i.e., generating more convincing examples, signified by the Gz term. The log operations are derived from the cross entropy between the real and fake distributions.Notorious problems affecting GANs aremode collapse and vanishing gradients. Vanishing gradients is a general issue in machine learning, where gradients prove insufficient for the machine learning model to update meaningfully, while this issue has classically occurred in neural models with high depth [13] or recurrent networks such as long-short term memory architecture [14]. However, the issue manifests particularly in the case of generative adversarial networks: the unadjusted loss formulation presented above will result in a critic that converges faster than the generator. Thus, the critic cannot offer constructive feedback for the generator to improve on, since it perfectly discerns between real and fake.A connected issue with vanishing gradients that is faced by generative adversarial networks ismode collapse. The problem manifests on the generator’s side by mapping all latent points to the same synthesised sample. From the perspective of game theory, both mode collapse and vanishing gradients issues are caused by the two players, the critic and the generator, converging to a local, undesired optimum of the game that does not offer enough incentives for any of the networks to update their weights [15].Several improvements on the domain transfer subproblem have been addressing the mode collapse issue. CycleGAN [16] introduces the following adjustments: instead of sampling from a latent random space, the generator samples from the input space of the input domain, with output in the target domain. The critic is fed both generated images and those belonging to the target domain, thus encouraging the generator to learn a better mapping between input and target. Thus, the improvement resides in translating the task into an unsupervised task, as the generator is ideally able to map any image from the first domain to an image in the target domain. A limitation of the CycleGAN paper is the domains being required to be homogeneous [17]. An improvement over the CycleGAN is represented by TraVeLGAN [18], which adds to the classical two network architectures formed of a generator and critic, a third, siamese network, and eliminates the domain homogeneity constraint [19].The Wasserstein variation of the generative adversarial network architecture (WGAN), authored by Arjovsky et al. [20] offers a robust method to train GAN architectures. The WGAN improves the stability of learning, eliminates problems such as mode collapse, and provides meaningful learning curves useful for hyperparameter searches.


## 2.2. Autoencoders

The autoencoder (AE) architecture uses a two-part neural network to transform the input in a compressed and meaningful representation using the encoder part, recreating the input using the decoder part [21]. The technique proves to be immensely flexible and is of interest to the purpose of this thesis since former research proved that autoencoders are able to encode and decode complex temporal features into the latent space [9]. Furthermore, there are no special theoretical aspects to be considered over a general-purpose deep learning architecture. The autoencoder is presented as two symmetrical parts, with a small, latent representation in the middle, usually trained to minimise the mean squared error between the distribution and itself.AEs can be interpreted as an improvement over the statistical technique of principal component analysis. Principal component analysis withp dimensions identifies an orthonormal base of p vectors that best identify the variance of the input distribution [22]. This technique is limited to linear representations, unlike the manifold organized by the autoencoder. Thus, the AE is able to construct higher fidelity correlations between the original and latent spaces, preserving relationships. Furthermore, there are multiple accounts in the literature in using the latent representation over the initial dataset with great effect for increased classification performance, better interpretability of the obtained clusters, or better ability to generalize over the latent representation [23–25].Salakhutidnov and Hinton restrict the latent representation to a binary code which is interpreted as the output of a black-box hash function modelled by the encoder. The hashing is applied in the field of document retrieval, where the hashing of the query is used to retrieve the directly associated documents plus documents from neighbouring hashes. The task has also been approached from a generative approach by Hansen et al. inUnsupervised Neural Generative Semantic Hashing [26].


## 2.3. Reinforcement Learning

Reinforcement learning (RL) is a paradigm of the machine learning field where problems are modelled around two fundamental notions: agents and environments. Agents are able to interact with the environment via a defined set of “actions” which change the environment’s “state.” Defining the problem’s solution as a desirable environment state, the agent is conditioned via “rewards” and “punishments” to reach this favourable state. RL purpose is to teach an agent the optimal policy of acting inside an environment. The environment can at any moment be in a certain state that can be changed by the agent’s actions. The agent receives feedback from the environment in the form of a reward. Using a tradeoff between reward and value (future reward received by the agent by taking a certain action in a particular state), the agent learns a policy that decides the best course of action for a given state.This flexible framework has allowed to model complex real-world situations: scheduling drug administration to patients with chronic conditions in order to minimise risk of negative interactions [27, 28], minimising energy costs associated with cooling of data centers [29], or outmatching human players in games such as Go [30].RL is facing a growing interest in the discipline of algorithmic trading [31, 32]. The interest can be explained by the ease with which the problem can be modelled: given the price fluctuation of a certain instrument, an agent’s purpose is to maximise the overall profit. Current frontier in reinforcement learning focuses on improved training performance, particularly incentivising the agent to explore multiple action courses and breaking the state causality effect on training by sampling and replaying random past states [33]. Intuitively, these improvements focus on offering the agent the ability to retrospect and decide on past better courses of action.


## 2.4. Time Series Generation

GAN-based methods or generative adversarial network models have emerged as a popular technique for generating or augmenting datasets, especially with images and videos. However, GANs give poor fidelity in networking data, which has both complex temporal correlations and mixed discrete-continuous data types. Although GAN-based time series generation exists—for instance for medical time series—such techniques fail on more complex data exhibiting poor autocorrelation scores on long sequences while prone to mode collapse.TimeGAN architecture introduced by Yoon et al. [10] is of strong interest for the proposed method, as it reinforces the idea that latent spaces can be used to better understand the original time series distribution of the data. Specifically, the paper proposes using two latent spaces, HS and HX, where S represents the mathematical space of static features of the time series, e.g., gender, while X represents the space of temporal dependencies of the time series, e.g., the cholesterol level as the person ages. The paper asserts that instead of using only a generator-discriminator system for creating new samples, introducing supervised learning to the unsupervised generation will increase the fidelity of generated data. The supervised loss comes from two encoder-decoder pairs (h:S⟼HS,e:HS⟼S) and (hX:X⟼HX,eX:HX⟼X) between the initial space and latent space, with the GAN networks learning to directly replicate the latent vectors: (g:ZS⟼HS,d:HS⟼IR) and (gX:ZS⟼HS,d:HS⟼IR), where ZS and ZX are the space the random noise is sampled from. Of particular interest is the use of recurrent neural networks throughout the architecture. Notably, gX features the use of a 2-step autoregression dependency for creating the temporal latent vector. Recurrent neural networks are also used for encoding and decoding between X and HX.DoppelGANger architecture introduced by Lin et al. [34] represents a current benchmark in time series generation. It tackles a similar problem with TimeGAN as both separate the generation of static attributes from the time series measurements, although focusing on privacy over accurate reproduction of the target distribution time series. Specifically, the generation procedure for the time series implies a conditional process akin to prior work with Conditional Generative Adversarial Nets [35].The network further improves by providing a normalization approach that avoids mode collapse on long time series: instead of normalizing the entire data set at once, using the global minimum and maximum, the algorithm normalizes using the per-sample minimum and maximum. Furthermore, the minimum and maximum are attached as static metadata describing the associated time series. Thus, the generator is responsible for creating the static features and is also in charge of creating the features that normalize the time series.The final DoppelGANger architecture uses three networks for generating data: a generator network used for generating static attributes and a generator network used for generating the minimum and maximum of each time series, as described above. The data generated by the two networks is fed into the third, a recurrent neural network which leverages the provided information plus its internal state to generate measurements. A stacked discriminator critiques the generator’s work: one model focuses on the generated static features (also called metadata), while the other is a recurrent neural network critiquing the generated measurements.


## 3. Methodology

This section presents the steps undertaken in obtaining the augmented dataset. The dataset is discussed, along with the data preprocessing operations carried out. Afterwards, theLSTM-based autoencoder architecture employed for translating between the initial and the latent space is detailed upon. The latent space obtained is explored, and underrepresented sequences are identified using an OPTICS clustering algorithm. Moving on, the adversarial network (WGAN) used to sample new examples is presented, and the process of integrating the new samples into the initial dataset is discussed.A high-level overview of the DAuGAN approach is depicted in Figure1. The code, models, and dataset used are publicly available in [36].Figure 1
DAuGAN approach.


### 3.1. Dataset

For the purpose of the paper, a dataset describing the evolution of the price for Apple’s company stock, denoted by the AAPL ticker on New York stock exchange, has been chosen.The dataset presents samples at every 15 minutes, covering the company’s price evolution starting from1st of January 1998 until 3rd of December 2021, totalling 289487 time steps. The dataset features 7 initial columns: date, time, open, high, low, close, and volume. These features are often used in the domain of algorithmic trading and offer indicators on the price’s evolution per time step. The open feature describes the price at the start of the time step, and high and low describe the maximum and minimum reached throughout the time-step, while the close price describes the price at interval’s end. The volume feature represents the number of trades executed in the given period.Table1 presents a sample fragment from the beginning of the time series.Table 1
An excerpt from the beginning of the time series.
DateTimeOpenHighLowCloseVolume1998/02/0109:3013.625013.750013.500013.6875202701998/02/0109:4513.687513.750013.500013.62503340001998/02/0110:0013.625013.750013.562513.75002999001998/02/0110:1513.750014.000013.625014.00004302011998/02/0110:3013.937514.812513.750014.62509442001998/02/0110:4514.625014.750014.375014.4375218103Since the time steps are continuous, there are constraints that apply for any time stept.(1)∀t≥1:closet−1=opent,(2)xt≤hight,∀t,∀x∈low,close,open,(3)xt≥closet,∀t,∀x∈low,high,open.However, real world imposes exceptions to these constraints. First, the data used come only from the trading hours action, starting from 09 : 30 until 16 : 00, Monday to Friday, when all traders can take part in the market. However, the NYSE, and in general, the exchanges located in the United States, also present a “before-market” and “after-market” period, limited to institutional investors such as pension funds, hedge funds, or banks. Furthermore, shares can be traded between any two interested parties, without the exchange as an intermediary.It is beyond the scope of the paper to identify and enumerate all possible sources of discontinuity that could violate Equation (1). The simplifying assumption that the condition holds for any two consecutive steps is made.Basic statistics for the numerical features of the dataset are calculated. The analysis from Table2 reveals that volume features a very wide, 102,107, domain.Table 2
Count, mean, standard deviation, and minimum values for the entire dataset plus upper bounds for each quarter of the dataset in sorted order.
OpenHighLowCloseVolumeCount2.894870e + 052.894870e + 052.894870e + 052.894870e + 052.894870e + 05Mean187.757776188.038462187.466230187.7584914.580510e + 05Std160.514466160.689723160.326415160.5135989.206810e + 05Min12.55000012.95000011.31250012.8500001.000000e + 0225%78.75000078.92000078.50000078.7500005.900000e + 0350%131.040000131.330000130.790000131.0400001.033630e + 0575%248.160000248.605000247.625000248.1500005.635505e + 05Max704.800000705.070000704.530000704.8000007.514145e + 07Columnsopen and volume are plotted, observing that columns high, low, and close will trend in correlation and close to open column, leading to Figure 2. Figure 2(a) presents the histogram of volume column in the initial dataset, while Figure 2(b) depicts the open column evolution in time.Figure 2
(a) Histogram ofvolume feature in the initial dataset; (b) open feature evolution in time.
(a)(b)The keen observer will be very interested in the two sudden drops in price illustrated inopen column evolution illustrated in Figure 2(b). They represent a domain-specific event called share splitting. In a X:1 share split, the price of one share is divided by X while each share presplit is replaced with X times more. This preserves the value of the investment while lowering the financial bar for buying one share, attracting interest and activity from smaller investors with the better price per action.


#### 3.1.1. Data Preprocessing

Date and time features are discarded, since the augmented dataset obtained at the end of the procedure contains a larger number of samples and the two features must be mocked. For training open, high, low, close, and volume columns are used. Considering the exponential distribution of the volume highlighted in Figure 2, with values in the domain of 102,107, a logarithm transformation is applied column-wise, followed two independent statistical normalization operations: one for the open,high,low,close columns and one for volume. Applying normalization on all columns would violate the constraint of Equation (1), as the features follow different distributions. Training on nonlogarithmized leads to an autoencoder collapse, with most values outputted be decoder for volume being zero.Figure3 illustrates the dataset after normalization. One observes that the transformation of volume feature (Figure 3(a)) is notable, compared to the initial data (Figure 2(a)).Figure 3
(a) Histogram ofvolume feature in the normalized dataset; (b) open feature evolution in time in the dataset after normalization.
(a)(b)In preparation for the training of the models, time series are split into chunks of twenty time steps and an overlap of eight time steps between two chunks formed of the forty consecutive time steps.


### 3.2. Autoencoder

It is difficult to identify outliers in the original, time series problem space. The issue is resolved via an autoencoder architecture that translates the time series samples into a latent, Euclidean space, where clustering can be applied in order to identify outliers. The autoencoder is tasked with translating between the two spaces. The dimensions of the latent space are not significant in themselves, and the cardinality has been chosen in order to minimise Pearson correlation, reducing autoencoder training time.The architecture of the proposed autoencoder is illustrated in Table3 (the architecture for the encoder) and Table 4 (the architecture for the decoder). The encoder and decoder parts of the architecture mirror each other. When reproducing the experiments, one should expect training and validation losses in the domain of 10−4 after 1000 epochs of training. Besides the valloss metric, Person correlation between dimensions of latent space is employed in order to determine the minimum number of nonredundant dimensions.Table 3
Encoder architecture.
Layer (type)Output shapeParam #input_1 (InputLayer)[(None; 20; 5)]0conv1d (Conv1D)(None; 20; 256)5376p_re_lu (PReLU)(None; 20; 256)5120dropout (Dropout)(None; 20; 256)0conv1d_1 (Conv1D)(None; 20; 256)262400p_re_lu_1 (PReLU)(None; 20; 256)5120dropout_1 (Dropout)(None; 20; 256)0conv1d_2 (Conv1D)(None; 20; 256)262400p_re_lu_2 (PReLU)(None; 20; 256)5120dropout_2 (Dropout)(None; 20; 256)0lstm (LSTM)(None; 20; 256)525312p_re_lu_3 (PReLU)(None; 20; 256)5120dropout_3 (Dropout)(None; 20; 256)0lstm_1 (LSTM)(None; 20; 256)525312p_re_lu_4 (PReLU)(None; 20; 256)5120dropout_4 (Dropout)(None; 20; 256)0lstm_2 (LSTM)(None; 12)12912In the encoder, one-dimensional convolutions usesame padding mode, increasing the number of dimensions fed into the LSTM layers. It has been observed that the expansion in dimensionality aids the convergence of LSTM layers. Three stacked LSTM layers are used to capture temporal dependencies and encode them in the condensed latent form. The decoder operates in the same manner, with stacked LSTM layers expanding the temporal correlations encoded in latent and reverse convolutions condensing dimensions back to the original form. Following best practices from the literature, dropout layers with a rate β=0.3 are intertwined for regularization purposes [37], and PReLu function is employed as activation between all layers [38]. The optimization algorithm involved in training is Stochastic Gradient Descent with a learning rate of α=0.0001 that uses Nesterov momentum [39]. The choice of the optimizer is motivated by the fact that Adam is not guaranteed to converge [40].Table 4
Decoder architecture.
Layer (type)Output shapeParam #input_2 (InputLayer)[(None; 12)]0repeat_vector (RepeatVector)(None; 20; 12)0lstm_3 (LSTM)(None; 20; 256)275456p_re_lu_5 (PReLU)(None; 20; 256)5120dropout_5 (Dropout)(None; 20; 256)0lstm_4 (LSTM)(None; 20; 256)525312p_re_lu_6 (PReLU)(None; 20; 256)5120dropout_6 (Dropout)(None; 20; 256)0conv1d_3 (Conv1D)(None; 20; 256)262400p_re_lu_7 (PReLU)(None; 20; 256)5120dropout_7 (Dropout)(None; 20; 256)0conv1d_4 (Conv1D)(None; 20; 256)262400p_re_lu_8 (PReLU)(None; 20; 256)5120dropout_8 (Dropout)(None; 20; 256)0conv1d_5 (Conv1D)(None; 20; 256)262400p_re_lu_9 (PReLU)(None; 20; 256)5120dropout_9 (Dropout)(None; 20; 256)0time_distributed (TimeDistributed)(None; 20; 5)1285time_distributed_1 (TimeDistributed)(None; 20; 5)0


#### 3.2.1. Analysing the Latent Space

The Pearson correlation heat map between the features is illustrated in Figure4. The latent features have weak correlation, indicating an optimum number of features. An OPTICS algorithm [41] is applied for identifying minority clusters, sampling the epsilon hyperparameter linearly from the 0.05,0.5 range. The Elbow method [42] is applied on total variance in order to choose an optimum number of clusters. Analysis yields a majority cluster of ≈ 20000 points, with the rest being classified as outliers or noise. Principal component analysis [43] is employed for reducing dimensionality to three axes, thus allowing visualization.Figure 4
Pearson correlation between features.


### 3.3. Synthesising New Samples

The generative architecture presented in Tables5 and 6 is able to synthesise credible examples that resemble the minority class. For statistical confirmation, a multivariate Wilcoxon test between the real minority points and the generated points is used [44, 45]. A p-value of 0.7275 is obtained at a significance level α=0.05, unable to refute the null hypothesis, proving there is no significant difference between the real and the generated points. The resemblance between distributions can be observed in Figure 5.Table 5
WGAN generator architecture.
Layer (type)Output shapeParam #input_4 (InputLayer)[(None; 15; 1)]0dense_3 (Dense)(None; 15; 20)40conv1d_9 (Conv1D)(None; 8; 16)1296leaky_re_lu_3 (LeakyReLU)(None; 8; 16)0conv1d_10 (Conv1D)(None; 4; 16)1040leaky_re_lu_4 (LeakyReLU)(None; 4; 16)0flatten_1 (Flatten)(None; 64)0dense_4 (Dense)(None; 100)6500dense_5 (Dense)(None; 100)10100dense_6 (ense)(None; 12)1212Figure 5
Distribution of latent points before and after synthesising new examples.Table 6
WGAN critic architecture.
Layer (type)Output shapeParam #input_3 (InputLayer)[(None; 12; 1)]0conv1d_6 (Conv1D)(None; 6; 16)80leaky_re_lu (LeakyReLU)(None; 6; 16)0conv1d_7 (Conv1D)(None; 3; 16)1040leaky_re_lu_1 (LeakyReLU)(None; 3; 16)0conv1d_8 (Conv1D)(None; 2; 16)1040leaky_re_lu_2 (LeakyReLU)(None; 2; 16)0flatten (Flatten)(None; 32)0dense_1 (Dense)(None; 100)3300dense_2 (Dense)(None; 1)101Figure6 depicts the correlation heat map for the synthesised examples.Figure 6
Correlation heat map for generated examples.It is worth noting that the feature correlation for synthesised latent vectors slightly differs from the minority features’ correlation, as shown in Figure7.Figure 7
Absolute difference between the correlation maps of minority and generated examples.


#### 3.3.1. Smoothing the Synthesised Examples

Despite statistical resemblance of the generated samples, the WGAN is unable to model the constraints from formulae (1), (2), and (3).The issue is approached as an optimization problem: the closest point to the initially synthesised data point is found that satisfies all constraints simultaneously. Bayesian search [46] in tandem with a greedy algorithm is employed. The time steps corresponding to the latent point are iteratively “smoothed.” If the time step’s values do not respect the imposed constraints, the latent point will be assigned a negative value and a new neighbouring latent point will be verified. Should the time step be found adequate, it is assigned the inverse of the distance between the original location of the sampled synthetic point and the current location of the data point, and this metric is maximised. It should be noted that the original data point is kept as reference throughout all explorations.Formally, letd=clv−olv2 (clv denotes the current latent vector, and olv represents the original latent vector); Bayesian search must optimize the black-box function γ for each time step, minimising the distance d at the overall data point level. After fixing a time step, the open price of the next time step is set to be equal to the close price of the current one in order to preserve continuity.(4)γopen,high,low,close=−1,if¬time step passes,1d,otherwise.The search space of the Bayesian process overγ is constrained to X±X, where X is the value for any of the features. The point with the maximum score after a set number of steps is selected and integrated in the time series. For time steps with index t≥1, the search is carried out only for high,low,close, since open price has been set at the first step.In order to integrate a smoothed chunkζ into the original time series, a tuple of consecutive chunks, defined by index t<lentime_series−1, is identified such that the objective is minimised:(5)closet,opent+1,γvolumet:volumet+1−openζ,closeζ,γvolumeζ2.Minimising the objective is equivalent to minimising the distance betweencloset,openζandcloseζ,opent+1, preserving the overall smoothness of the time series. Table 7 presents a chunk fragment obtained in the generation process.Table 7
Generated chunk fragment.
OpenHighLowCloseVolume0326.366272341.629822314.827942330.429718517.6384891330.429718336.826141310.153687325.654053899.5401612325.654053338.448029311.383545327.1006472897.6733403327.100647351.454468313.967712351.0909423598.6789554351.090942340.434845313.185944329.0170594032.8588875329.017059340.508057313.215607329.0681155119.4243166329.068115335.061981308.940491324.127441153.9409647324.127441331.650665305.705658320.777008144.4495708320.777008335.563629308.537842324.2147224437.6513679324.214722340.812073313.569183329.3977363368.68652310329.397736340.041931312.984558328.7131652112.14819311328.713165344.093750312.746857319.489136575.97119112319.489136339.126801312.318512327.8862921199.59729013327.886292344.057251316.442230332.4739389648.51757814332.473938346.419373318.820251334.8567815601.14257815334.856781345.538300318.073364334.0296634347.550781


## 3.1. Dataset

For the purpose of the paper, a dataset describing the evolution of the price for Apple’s company stock, denoted by the AAPL ticker on New York stock exchange, has been chosen.The dataset presents samples at every 15 minutes, covering the company’s price evolution starting from1st of January 1998 until 3rd of December 2021, totalling 289487 time steps. The dataset features 7 initial columns: date, time, open, high, low, close, and volume. These features are often used in the domain of algorithmic trading and offer indicators on the price’s evolution per time step. The open feature describes the price at the start of the time step, and high and low describe the maximum and minimum reached throughout the time-step, while the close price describes the price at interval’s end. The volume feature represents the number of trades executed in the given period.Table1 presents a sample fragment from the beginning of the time series.Table 1
An excerpt from the beginning of the time series.
DateTimeOpenHighLowCloseVolume1998/02/0109:3013.625013.750013.500013.6875202701998/02/0109:4513.687513.750013.500013.62503340001998/02/0110:0013.625013.750013.562513.75002999001998/02/0110:1513.750014.000013.625014.00004302011998/02/0110:3013.937514.812513.750014.62509442001998/02/0110:4514.625014.750014.375014.4375218103Since the time steps are continuous, there are constraints that apply for any time stept.(1)∀t≥1:closet−1=opent,(2)xt≤hight,∀t,∀x∈low,close,open,(3)xt≥closet,∀t,∀x∈low,high,open.However, real world imposes exceptions to these constraints. First, the data used come only from the trading hours action, starting from 09 : 30 until 16 : 00, Monday to Friday, when all traders can take part in the market. However, the NYSE, and in general, the exchanges located in the United States, also present a “before-market” and “after-market” period, limited to institutional investors such as pension funds, hedge funds, or banks. Furthermore, shares can be traded between any two interested parties, without the exchange as an intermediary.It is beyond the scope of the paper to identify and enumerate all possible sources of discontinuity that could violate Equation (1). The simplifying assumption that the condition holds for any two consecutive steps is made.Basic statistics for the numerical features of the dataset are calculated. The analysis from Table2 reveals that volume features a very wide, 102,107, domain.Table 2
Count, mean, standard deviation, and minimum values for the entire dataset plus upper bounds for each quarter of the dataset in sorted order.
OpenHighLowCloseVolumeCount2.894870e + 052.894870e + 052.894870e + 052.894870e + 052.894870e + 05Mean187.757776188.038462187.466230187.7584914.580510e + 05Std160.514466160.689723160.326415160.5135989.206810e + 05Min12.55000012.95000011.31250012.8500001.000000e + 0225%78.75000078.92000078.50000078.7500005.900000e + 0350%131.040000131.330000130.790000131.0400001.033630e + 0575%248.160000248.605000247.625000248.1500005.635505e + 05Max704.800000705.070000704.530000704.8000007.514145e + 07Columnsopen and volume are plotted, observing that columns high, low, and close will trend in correlation and close to open column, leading to Figure 2. Figure 2(a) presents the histogram of volume column in the initial dataset, while Figure 2(b) depicts the open column evolution in time.Figure 2
(a) Histogram ofvolume feature in the initial dataset; (b) open feature evolution in time.
(a)(b)The keen observer will be very interested in the two sudden drops in price illustrated inopen column evolution illustrated in Figure 2(b). They represent a domain-specific event called share splitting. In a X:1 share split, the price of one share is divided by X while each share presplit is replaced with X times more. This preserves the value of the investment while lowering the financial bar for buying one share, attracting interest and activity from smaller investors with the better price per action.


### 3.1.1. Data Preprocessing

Date and time features are discarded, since the augmented dataset obtained at the end of the procedure contains a larger number of samples and the two features must be mocked. For training open, high, low, close, and volume columns are used. Considering the exponential distribution of the volume highlighted in Figure 2, with values in the domain of 102,107, a logarithm transformation is applied column-wise, followed two independent statistical normalization operations: one for the open,high,low,close columns and one for volume. Applying normalization on all columns would violate the constraint of Equation (1), as the features follow different distributions. Training on nonlogarithmized leads to an autoencoder collapse, with most values outputted be decoder for volume being zero.Figure3 illustrates the dataset after normalization. One observes that the transformation of volume feature (Figure 3(a)) is notable, compared to the initial data (Figure 2(a)).Figure 3
(a) Histogram ofvolume feature in the normalized dataset; (b) open feature evolution in time in the dataset after normalization.
(a)(b)In preparation for the training of the models, time series are split into chunks of twenty time steps and an overlap of eight time steps between two chunks formed of the forty consecutive time steps.


## 3.1.1. Data Preprocessing

Date and time features are discarded, since the augmented dataset obtained at the end of the procedure contains a larger number of samples and the two features must be mocked. For training open, high, low, close, and volume columns are used. Considering the exponential distribution of the volume highlighted in Figure 2, with values in the domain of 102,107, a logarithm transformation is applied column-wise, followed two independent statistical normalization operations: one for the open,high,low,close columns and one for volume. Applying normalization on all columns would violate the constraint of Equation (1), as the features follow different distributions. Training on nonlogarithmized leads to an autoencoder collapse, with most values outputted be decoder for volume being zero.Figure3 illustrates the dataset after normalization. One observes that the transformation of volume feature (Figure 3(a)) is notable, compared to the initial data (Figure 2(a)).Figure 3
(a) Histogram ofvolume feature in the normalized dataset; (b) open feature evolution in time in the dataset after normalization.
(a)(b)In preparation for the training of the models, time series are split into chunks of twenty time steps and an overlap of eight time steps between two chunks formed of the forty consecutive time steps.


## 3.2. Autoencoder

It is difficult to identify outliers in the original, time series problem space. The issue is resolved via an autoencoder architecture that translates the time series samples into a latent, Euclidean space, where clustering can be applied in order to identify outliers. The autoencoder is tasked with translating between the two spaces. The dimensions of the latent space are not significant in themselves, and the cardinality has been chosen in order to minimise Pearson correlation, reducing autoencoder training time.The architecture of the proposed autoencoder is illustrated in Table3 (the architecture for the encoder) and Table 4 (the architecture for the decoder). The encoder and decoder parts of the architecture mirror each other. When reproducing the experiments, one should expect training and validation losses in the domain of 10−4 after 1000 epochs of training. Besides the valloss metric, Person correlation between dimensions of latent space is employed in order to determine the minimum number of nonredundant dimensions.Table 3
Encoder architecture.
Layer (type)Output shapeParam #input_1 (InputLayer)[(None; 20; 5)]0conv1d (Conv1D)(None; 20; 256)5376p_re_lu (PReLU)(None; 20; 256)5120dropout (Dropout)(None; 20; 256)0conv1d_1 (Conv1D)(None; 20; 256)262400p_re_lu_1 (PReLU)(None; 20; 256)5120dropout_1 (Dropout)(None; 20; 256)0conv1d_2 (Conv1D)(None; 20; 256)262400p_re_lu_2 (PReLU)(None; 20; 256)5120dropout_2 (Dropout)(None; 20; 256)0lstm (LSTM)(None; 20; 256)525312p_re_lu_3 (PReLU)(None; 20; 256)5120dropout_3 (Dropout)(None; 20; 256)0lstm_1 (LSTM)(None; 20; 256)525312p_re_lu_4 (PReLU)(None; 20; 256)5120dropout_4 (Dropout)(None; 20; 256)0lstm_2 (LSTM)(None; 12)12912In the encoder, one-dimensional convolutions usesame padding mode, increasing the number of dimensions fed into the LSTM layers. It has been observed that the expansion in dimensionality aids the convergence of LSTM layers. Three stacked LSTM layers are used to capture temporal dependencies and encode them in the condensed latent form. The decoder operates in the same manner, with stacked LSTM layers expanding the temporal correlations encoded in latent and reverse convolutions condensing dimensions back to the original form. Following best practices from the literature, dropout layers with a rate β=0.3 are intertwined for regularization purposes [37], and PReLu function is employed as activation between all layers [38]. The optimization algorithm involved in training is Stochastic Gradient Descent with a learning rate of α=0.0001 that uses Nesterov momentum [39]. The choice of the optimizer is motivated by the fact that Adam is not guaranteed to converge [40].Table 4
Decoder architecture.
Layer (type)Output shapeParam #input_2 (InputLayer)[(None; 12)]0repeat_vector (RepeatVector)(None; 20; 12)0lstm_3 (LSTM)(None; 20; 256)275456p_re_lu_5 (PReLU)(None; 20; 256)5120dropout_5 (Dropout)(None; 20; 256)0lstm_4 (LSTM)(None; 20; 256)525312p_re_lu_6 (PReLU)(None; 20; 256)5120dropout_6 (Dropout)(None; 20; 256)0conv1d_3 (Conv1D)(None; 20; 256)262400p_re_lu_7 (PReLU)(None; 20; 256)5120dropout_7 (Dropout)(None; 20; 256)0conv1d_4 (Conv1D)(None; 20; 256)262400p_re_lu_8 (PReLU)(None; 20; 256)5120dropout_8 (Dropout)(None; 20; 256)0conv1d_5 (Conv1D)(None; 20; 256)262400p_re_lu_9 (PReLU)(None; 20; 256)5120dropout_9 (Dropout)(None; 20; 256)0time_distributed (TimeDistributed)(None; 20; 5)1285time_distributed_1 (TimeDistributed)(None; 20; 5)0


### 3.2.1. Analysing the Latent Space

The Pearson correlation heat map between the features is illustrated in Figure4. The latent features have weak correlation, indicating an optimum number of features. An OPTICS algorithm [41] is applied for identifying minority clusters, sampling the epsilon hyperparameter linearly from the 0.05,0.5 range. The Elbow method [42] is applied on total variance in order to choose an optimum number of clusters. Analysis yields a majority cluster of ≈ 20000 points, with the rest being classified as outliers or noise. Principal component analysis [43] is employed for reducing dimensionality to three axes, thus allowing visualization.Figure 4
Pearson correlation between features.


## 3.2.1. Analysing the Latent Space

The Pearson correlation heat map between the features is illustrated in Figure4. The latent features have weak correlation, indicating an optimum number of features. An OPTICS algorithm [41] is applied for identifying minority clusters, sampling the epsilon hyperparameter linearly from the 0.05,0.5 range. The Elbow method [42] is applied on total variance in order to choose an optimum number of clusters. Analysis yields a majority cluster of ≈ 20000 points, with the rest being classified as outliers or noise. Principal component analysis [43] is employed for reducing dimensionality to three axes, thus allowing visualization.Figure 4
Pearson correlation between features.


## 3.3. Synthesising New Samples

The generative architecture presented in Tables5 and 6 is able to synthesise credible examples that resemble the minority class. For statistical confirmation, a multivariate Wilcoxon test between the real minority points and the generated points is used [44, 45]. A p-value of 0.7275 is obtained at a significance level α=0.05, unable to refute the null hypothesis, proving there is no significant difference between the real and the generated points. The resemblance between distributions can be observed in Figure 5.Table 5
WGAN generator architecture.
Layer (type)Output shapeParam #input_4 (InputLayer)[(None; 15; 1)]0dense_3 (Dense)(None; 15; 20)40conv1d_9 (Conv1D)(None; 8; 16)1296leaky_re_lu_3 (LeakyReLU)(None; 8; 16)0conv1d_10 (Conv1D)(None; 4; 16)1040leaky_re_lu_4 (LeakyReLU)(None; 4; 16)0flatten_1 (Flatten)(None; 64)0dense_4 (Dense)(None; 100)6500dense_5 (Dense)(None; 100)10100dense_6 (ense)(None; 12)1212Figure 5
Distribution of latent points before and after synthesising new examples.Table 6
WGAN critic architecture.
Layer (type)Output shapeParam #input_3 (InputLayer)[(None; 12; 1)]0conv1d_6 (Conv1D)(None; 6; 16)80leaky_re_lu (LeakyReLU)(None; 6; 16)0conv1d_7 (Conv1D)(None; 3; 16)1040leaky_re_lu_1 (LeakyReLU)(None; 3; 16)0conv1d_8 (Conv1D)(None; 2; 16)1040leaky_re_lu_2 (LeakyReLU)(None; 2; 16)0flatten (Flatten)(None; 32)0dense_1 (Dense)(None; 100)3300dense_2 (Dense)(None; 1)101Figure6 depicts the correlation heat map for the synthesised examples.Figure 6
Correlation heat map for generated examples.It is worth noting that the feature correlation for synthesised latent vectors slightly differs from the minority features’ correlation, as shown in Figure7.Figure 7
Absolute difference between the correlation maps of minority and generated examples.


### 3.3.1. Smoothing the Synthesised Examples

Despite statistical resemblance of the generated samples, the WGAN is unable to model the constraints from formulae (1), (2), and (3).The issue is approached as an optimization problem: the closest point to the initially synthesised data point is found that satisfies all constraints simultaneously. Bayesian search [46] in tandem with a greedy algorithm is employed. The time steps corresponding to the latent point are iteratively “smoothed.” If the time step’s values do not respect the imposed constraints, the latent point will be assigned a negative value and a new neighbouring latent point will be verified. Should the time step be found adequate, it is assigned the inverse of the distance between the original location of the sampled synthetic point and the current location of the data point, and this metric is maximised. It should be noted that the original data point is kept as reference throughout all explorations.Formally, letd=clv−olv2 (clv denotes the current latent vector, and olv represents the original latent vector); Bayesian search must optimize the black-box function γ for each time step, minimising the distance d at the overall data point level. After fixing a time step, the open price of the next time step is set to be equal to the close price of the current one in order to preserve continuity.(4)γopen,high,low,close=−1,if¬time step passes,1d,otherwise.The search space of the Bayesian process overγ is constrained to X±X, where X is the value for any of the features. The point with the maximum score after a set number of steps is selected and integrated in the time series. For time steps with index t≥1, the search is carried out only for high,low,close, since open price has been set at the first step.In order to integrate a smoothed chunkζ into the original time series, a tuple of consecutive chunks, defined by index t<lentime_series−1, is identified such that the objective is minimised:(5)closet,opent+1,γvolumet:volumet+1−openζ,closeζ,γvolumeζ2.Minimising the objective is equivalent to minimising the distance betweencloset,openζandcloseζ,opent+1, preserving the overall smoothness of the time series. Table 7 presents a chunk fragment obtained in the generation process.Table 7
Generated chunk fragment.
OpenHighLowCloseVolume0326.366272341.629822314.827942330.429718517.6384891330.429718336.826141310.153687325.654053899.5401612325.654053338.448029311.383545327.1006472897.6733403327.100647351.454468313.967712351.0909423598.6789554351.090942340.434845313.185944329.0170594032.8588875329.017059340.508057313.215607329.0681155119.4243166329.068115335.061981308.940491324.127441153.9409647324.127441331.650665305.705658320.777008144.4495708320.777008335.563629308.537842324.2147224437.6513679324.214722340.812073313.569183329.3977363368.68652310329.397736340.041931312.984558328.7131652112.14819311328.713165344.093750312.746857319.489136575.97119112319.489136339.126801312.318512327.8862921199.59729013327.886292344.057251316.442230332.4739389648.51757814332.473938346.419373318.820251334.8567815601.14257815334.856781345.538300318.073364334.0296634347.550781


## 3.3.1. Smoothing the Synthesised Examples

Despite statistical resemblance of the generated samples, the WGAN is unable to model the constraints from formulae (1), (2), and (3).The issue is approached as an optimization problem: the closest point to the initially synthesised data point is found that satisfies all constraints simultaneously. Bayesian search [46] in tandem with a greedy algorithm is employed. The time steps corresponding to the latent point are iteratively “smoothed.” If the time step’s values do not respect the imposed constraints, the latent point will be assigned a negative value and a new neighbouring latent point will be verified. Should the time step be found adequate, it is assigned the inverse of the distance between the original location of the sampled synthetic point and the current location of the data point, and this metric is maximised. It should be noted that the original data point is kept as reference throughout all explorations.Formally, letd=clv−olv2 (clv denotes the current latent vector, and olv represents the original latent vector); Bayesian search must optimize the black-box function γ for each time step, minimising the distance d at the overall data point level. After fixing a time step, the open price of the next time step is set to be equal to the close price of the current one in order to preserve continuity.(4)γopen,high,low,close=−1,if¬time step passes,1d,otherwise.The search space of the Bayesian process overγ is constrained to X±X, where X is the value for any of the features. The point with the maximum score after a set number of steps is selected and integrated in the time series. For time steps with index t≥1, the search is carried out only for high,low,close, since open price has been set at the first step.In order to integrate a smoothed chunkζ into the original time series, a tuple of consecutive chunks, defined by index t<lentime_series−1, is identified such that the objective is minimised:(5)closet,opent+1,γvolumet:volumet+1−openζ,closeζ,γvolumeζ2.Minimising the objective is equivalent to minimising the distance betweencloset,openζandcloseζ,opent+1, preserving the overall smoothness of the time series. Table 7 presents a chunk fragment obtained in the generation process.Table 7
Generated chunk fragment.
OpenHighLowCloseVolume0326.366272341.629822314.827942330.429718517.6384891330.429718336.826141310.153687325.654053899.5401612325.654053338.448029311.383545327.1006472897.6733403327.100647351.454468313.967712351.0909423598.6789554351.090942340.434845313.185944329.0170594032.8588875329.017059340.508057313.215607329.0681155119.4243166329.068115335.061981308.940491324.127441153.9409647324.127441331.650665305.705658320.777008144.4495708320.777008335.563629308.537842324.2147224437.6513679324.214722340.812073313.569183329.3977363368.68652310329.397736340.041931312.984558328.7131652112.14819311328.713165344.093750312.746857319.489136575.97119112319.489136339.126801312.318512327.8862921199.59729013327.886292344.057251316.442230332.4739389648.51757814332.473938346.419373318.820251334.8567815601.14257815334.856781345.538300318.073364334.0296634347.550781


## 4. Results and Discussion

With the goal of answering research question RQ2, Section4.1 presents the benchmark used to assess the effectiveness of the augmentation introduced in Section 3 in improving the performance of trading algorithms. The obtained results are then presented in Section 4.2.


### 4.1. Benchmark Methodology and Data Preparation

In order to assess the impact of augmentation, independent reinforcement learning trading algorithms and domain-specific data preparation procedures are provided by the open-sourceFinRL library [47].For the benchmark, 80% of the original time series is kept for training the trading strategies, while employing the rest of 20% for blind validation. For fair comparison, the augmentation procedure is employed only on the training portion, and the impact is measured on the validation period. The algorithms are trained on two distinct datasets, the augmented training dataset and original training dataset, until convergence, and score on validation is measured.FinRL preprocessing is applied on both datasets. The procedure adds technical indicators, whose purpose is to highlight trends in a security’s price evolution, such as relative volatility, magnitude of price shifts, or trends in price shift. The indicators added by FinRL are 12-MACD, Bollinger Bands, 30-RelativeStrengthIndex, 30-CommodityChannelIndex, 30-AverageDirectionalIndex, 30-CloseSimpleMovingAverage, and 60-CloseSimpleMovAvg.TheFinRL library requires the presence of two extra columns in order to calculate the technical indicators: the tic column which represents the security’s descriptor (FinRL is able to trade multiple securities at once, hence the requirement for this column) and the date column which represents the date and time of the column. With the simplifying assumption that the security has been traded continuously in intervals of 15 minutes starting from an arbitrary date, the two columns are added.The trading agent’s state is described by the tuple (shares, capital), where shares describes the number of owned shares, while capital describes the amount of monetary units available for buying shares. The state of the agent is completed by the time series, as seen until moment t of time. The initial state shares0,capital0=0,200000. The value of a portfolio value with shares is defined as capitalt+∑0,ropent.At any time step, the agent’s action space spans the integers−mink,sharest,k. k is a positive integer hyperparameter set in the environment. Negative integers indicate selling that amount of shares, receiving an amount of capital equal to the opening price for each sold share. Positive amounts indicate buying stock units and have the reverse effect on capital. The special case k=0, denoting the agent’s choice to hold its current position, should be noted.


### 4.2. Results

A positive correlation between the amount of samples introduced in the original time series, and the performance improvement of the algorithm is identified. The improvement is defined as the difference in portfolio value between training on the original time series and augmented series.Table8 summarizes the performance improvement of DAuGANa. Permutations of introduced samples and benchmark algorithm used: Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Advantage Actor Critic (A2C). Regarding the hyperparameter setting, the algorithms were trained using the values depicted in Table 9.Table 8
Improvement in trading performance, measured with respect to the number of introduced samples and algorithm used.
Algorithm# of introduced samples200040006000DDPG+2.71%+3.44%+4.25%PPO+3.12%+3.47%+3.82%A2C+1.89%+2.31%+2.42%Table 9
Hyperparameters for DDPG, PPO, and A2C algorithms.
AlgorithmHyperparameterValueDDPGcritic_learning_rate1e −3actor_learning_rate1e −3l2_weight_regularization1e −6gradient_clippingNonetrain_batch_size256episodes_before_learn_start1500PPOsgd_lr5e −5epochs_per_train_batch30ppo_parameter_clipping0.3kl_divergence_target0.01value_function_clip_parameter10entropy_coeff0.0A2CAdam_learning_rate1e −4Adam_minibatch_size32batch_size12500gradient_steps3000tau_mov_avg0.01l2_weight_regularization1e −6critic_learning_rate1e −4actor_learning_rate1e −4


## 4.1. Benchmark Methodology and Data Preparation

In order to assess the impact of augmentation, independent reinforcement learning trading algorithms and domain-specific data preparation procedures are provided by the open-sourceFinRL library [47].For the benchmark, 80% of the original time series is kept for training the trading strategies, while employing the rest of 20% for blind validation. For fair comparison, the augmentation procedure is employed only on the training portion, and the impact is measured on the validation period. The algorithms are trained on two distinct datasets, the augmented training dataset and original training dataset, until convergence, and score on validation is measured.FinRL preprocessing is applied on both datasets. The procedure adds technical indicators, whose purpose is to highlight trends in a security’s price evolution, such as relative volatility, magnitude of price shifts, or trends in price shift. The indicators added by FinRL are 12-MACD, Bollinger Bands, 30-RelativeStrengthIndex, 30-CommodityChannelIndex, 30-AverageDirectionalIndex, 30-CloseSimpleMovingAverage, and 60-CloseSimpleMovAvg.TheFinRL library requires the presence of two extra columns in order to calculate the technical indicators: the tic column which represents the security’s descriptor (FinRL is able to trade multiple securities at once, hence the requirement for this column) and the date column which represents the date and time of the column. With the simplifying assumption that the security has been traded continuously in intervals of 15 minutes starting from an arbitrary date, the two columns are added.The trading agent’s state is described by the tuple (shares, capital), where shares describes the number of owned shares, while capital describes the amount of monetary units available for buying shares. The state of the agent is completed by the time series, as seen until moment t of time. The initial state shares0,capital0=0,200000. The value of a portfolio value with shares is defined as capitalt+∑0,ropent.At any time step, the agent’s action space spans the integers−mink,sharest,k. k is a positive integer hyperparameter set in the environment. Negative integers indicate selling that amount of shares, receiving an amount of capital equal to the opening price for each sold share. Positive amounts indicate buying stock units and have the reverse effect on capital. The special case k=0, denoting the agent’s choice to hold its current position, should be noted.


## 4.2. Results

A positive correlation between the amount of samples introduced in the original time series, and the performance improvement of the algorithm is identified. The improvement is defined as the difference in portfolio value between training on the original time series and augmented series.Table8 summarizes the performance improvement of DAuGANa. Permutations of introduced samples and benchmark algorithm used: Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Advantage Actor Critic (A2C). Regarding the hyperparameter setting, the algorithms were trained using the values depicted in Table 9.Table 8
Improvement in trading performance, measured with respect to the number of introduced samples and algorithm used.
Algorithm# of introduced samples200040006000DDPG+2.71%+3.44%+4.25%PPO+3.12%+3.47%+3.82%A2C+1.89%+2.31%+2.42%Table 9
Hyperparameters for DDPG, PPO, and A2C algorithms.
AlgorithmHyperparameterValueDDPGcritic_learning_rate1e −3actor_learning_rate1e −3l2_weight_regularization1e −6gradient_clippingNonetrain_batch_size256episodes_before_learn_start1500PPOsgd_lr5e −5epochs_per_train_batch30ppo_parameter_clipping0.3kl_divergence_target0.01value_function_clip_parameter10entropy_coeff0.0A2CAdam_learning_rate1e −4Adam_minibatch_size32batch_size12500gradient_steps3000tau_mov_avg0.01l2_weight_regularization1e −6critic_learning_rate1e −4actor_learning_rate1e −4


## 5. Conclusions and Future Work

The paper has introduced a novel augmentation method for identifying poorly represented sections of a time series, studied the synthesis of new data points and their integration into the time series, and assessed the performance improvement on a benchmark machine learning model.Data synthetisation is a valid training augmentation in the area of algorithmic trading, which has the potential to be extended to other domains involving time series, due to the generality of the latent space approach. Of interest for the future are mission-critical tasks such as detecting rare medical conditions or weather now-casting, where performance improvement is vital.As possible improvements, the authors hypothesise that the use of recurrent neural networks at the generation step [10], combined with the autonormalization trick using DoppelGANger discussed in [34], can result in longer synthesised time series, eliminating the need for interleaving generated samples back into the original time series. Instead, numerous independent “training episodes,” several chunks in length, could be fed to the reinforcement learning agent, a method known to improve training performance [48].



---

*Source: 7877590-2021-10-12.xml*