Learning Predictive, Online Approximations of Explanatory, Offline Algorithms
Authors: Mattson Thieme
Ammar Gilani
Han Liu
Venue: ICLR 2022 Submitted (2022)
Keywords: multi-task learning
machine learning
online algorithms
offline algorithms

Abstract: In this work, we introduce a general methodology for approximating offline algorithms in online settings. By encoding the behavior of offline algorithms in graphs, we train a multi-task learning model to simultaneously detect behavioral structures which have already occurred and predict those that may come next. We demonstrate the methodology on both synthetic data and historical stock market data, where the contrast between explanation and prediction is particularly stark. Taken together, our work represents the first general and end-to-end differentiable approach for generating online approximations of offline algorithms.

Under review as a conference paper at ICLR 2022
LEARNING PREDICTIVE, ONLINE APPROXIMATIONS OF EXPLANATORY, OFFLINE ALGORIHTMS
Anonymous authors Paper under double-blind review
ABSTRACT
In this work, we introduce general methodology for approximating offline al- gorithms in online settings. By encoding the behavior of offline algorithms graphs, train multi-task learning model to simultaneously detect behavioral structures which have already occurred and predict those that may come next. We demonstrate the methodology on both synthetic data and historical stock market data, where the contrast between explanation and prediction is particularly stark. Taken together, our work represents the first general and end-to-end differentiable approach for generating online approximations of offline algorithms
INTRODUCTION
As the saying goes, hindsight is 20/20. With symmetric access to past and future data, it's easier to see paths joining cause and effect. Exploiting this symmetry. offline algorithms can recognize patterns only visible in retrospect and produce rich explanations of historical data. However, those same algorithms fall short in real-time settings, where future data is inaccessible. To bring an offline algorithm online, it must either be manually adapted its requirement for future data satisfied through exogenous forecasting techniques. To more efficiently bridge this gap between offline algorithms and their online counterparts. we introduce an end-to-end differentiable methodology for approximating offline algorithms in real-time.
While a variety of approaches exist for translating particular offline algorithms into their online counterparts, ours is the first to approach the problem in the general case (Blum) 1994] Kakade Kalail 2006 Ben-David et al. 1997). Using a graph-structured labeling scheme, our method blends explanation and prediction approximate the offline requirement for future data. However, unlike most time series forecasting techniques, which predict trajectories of time series data directly, instead focus on the behavior that an offline algorithm may yield that data (Mahalakshmi cet al. 2016). Refocusing in this way is productive for several reasons: 1) it simplifies the prediction task, which allows us to forecast further into the future 2) it predicts the behavior of an algorithm explicitly designed to ignore irrelevant noise, and 3) recasts the offline-to-onlin transformation straightforward mapping problem in which we can exploit the end-to-end differentiability of modern machine learning (ML) tools to find a solution.
In our framework, any modeling scheme in which future data informs the representation interpre- tation of the past is amenable to approximation. Under this definition, the scope of offline analysis extends well beyond computer science and into fields from medicine to economics.
Medical diagnostics is likely the largest suite of offline algorithms in use today. As patients experience new symptoms additional test results arrive, both diagnosis and prognosis may be updated in response meaning symptoms are ascribed to their cause retroactively. Processes for making these updates are codified in diagnostic manuals and revised as new scientific and patient data become available (Jeffery et al. 2012). However, with rapidly growing volumes of clinical data, it is becoming increasingly difficult for individual healthcare providers to integrate the new information, and with clinical outcomes closely tied to the accuracy of these offline diagnostic procedures, finding ways to improve them of interest to both the medical community and the public at large (Berger 1999
Balogh et al. 2015).
'll code, data, and configuration scripts available under the MIT License at [GitHub repository redacted for review].

---

Under review as a conference paper at ICLR 2022
00
GO
X A(X)
02
Q3
@1
Figure 1: Left: ID time series X and the output of an offline piece-wise linear approximation (PLA) algorithm A(X). Right: The graphical decision-poing -oiinn representation (Section 2.1 of the structure ( )44 using a greater-than relation between decision points.
Similarly, when considering economic and societal health, economists and policy-makers have long relied on offline algorithms to interpret historical data. Recessions and credit crises appear obvious in hindsight, but as many of these offline techniques lack online counterparts, forecasting similar events in real-time is difficult (Colander, 2010). Indeed, economic models are often criticized for their overreliance explanatory processes when making forecasts, which may in part be driven by the absence a general framework for converting explanatory models into predictive ones (Lipsey) (001). With these forecasts driving trillions in government and private spending annually, consequences of inaccuracy are wide-ranging, and everyone stands to benefit from more informed, data-driven predictions (Lewis & Pain 2015). In this paper, we'll take a step towards these data-driven, predictive economic models by using our methodology to forecast behavioral structures in equity markets.
We begin Section by formalizing the notion of behavioral structures in characterizing offline algorithms. We explain how to use these behavioral structures to generate descriptive labels and then formulate the machine learning problem in those terms. In Section 3 we review the offline algorithm being approximated in our experiments synthetic and real-world data. We then detail the multi-task MI. model used demonstrate the methodology. We note that the experiments are designed to build intuition for the methodology, and such are confined to ID settings. However, in Section 4. we survey additional examples single and multidimensional algorithms and explain how our method can be equally applied any offline algorithm time series data. We conclude by reviewing Related Work, discussing directions for future research, and outlining the steps we 've taken ensure reproducibility. Supplementary model architecture and training details are available in Appendix A.
GENERAL SCHEMA
Here, we present the building blocks of our method and develop intuition with illustrative examples motivated by the experiments in Section 3
2.1 GRAPHICAL DECISION-POIN REPRESENTATION
Suppose we have a time series (.i,.... IT). where x, IR, and an offline algorithm A(-) which operates on X produce sequence of decision points A(X) {(zz, )z,),... (Ix, aze as in Figure I Note that A(X) can yield decision points ag, on only a subset of x, E X. We define behavioral structure to be an ordered set of n contiguous decision points in A(X):
((Ik-nt11 arx-nt1): (IOk, azz))
where l the position of the terminal decision point. These structures are what the MI. model will ultimately learn to classify, and as such we need a representational scheme to meaningfully differentiate structures s E S (the set all unique structures the dataset). To do so, we represent directed graph in which each decision point is vertex and edges defined by
each structure a relationship ciationnsii-indicatoo indicator function r. This indicator function will be pellicatioo-dependeed take any form that meaningfully differentiates behavioral structures in S. Once S has been populated with all the structures yielded by A (XX), each unique structure $2 E S, |S] = k. is assigned class label Cz.
and can
2

---

Under review as a conference paper at ICLR 2022
observed (past)
unobserved (future)
Cip
Ci
C2 yh
Cin
Ci.
Figure 2: Left: Extending the example from Figure L we show = 5 decision points visible in Decision points in the future (at times t > 2) are not visible. Right: The decision point structures of the next structures beginning in w,, followed by their corresponding class labels in C2, y4- Highlighted decision points define the progressively forward-looking structures we aim to classify.
The offline algorithm A(-) in Figure I is piece-wise linear approximation (PLA) algorithm. applied to ID time series X. To construct the graphical decision-point representation of the n = 5 visible decision points, we choose the ilaaionssip-indicaate function to be the "greater than" relationship Directed edges will then extend from nodes in with higher values to those with lower values.
In the section, explain how this labeling scheme characterize structures extending from the observed past into the unobserved future.
2.2 PROGRESSIVELY FORWARD-LOOKING TASKS
For each Ij E X, we define Wi = (x;- (I;-d+1,.... Hz} to be the window on X size terminating F₁. As shown in Figure 2 (Left), this window contains decision points, one full structure. Increasingly forward-looking structures are those which begin in and end in the unobserved future (Right) highlights increasingly forward-looking structures which refer by the
t >> i. Figure (Right) number of decision points À that lie in the unobserved future. The structure entirely visible in becomes C.., while the structure with four unobserved decision points is C.a. Each window w; is labeled with all the structures that begin within it y: (Ci.- Ci... J. The class label assigned to each Ci E ys represents one unique structures found the original dataset.
From here, we refer the task of labeling the A structure on w;, i.e. generating prediction Ci,, Task-A. For each wi our goal is to predict these increasingly forward-looking structures (C... Oin.1> E yi-
2.3 PROBLEM FORMULATION
Suppose again that we have a time series X = {xi,.... ØT). where 24 E R, and an of- fline algorithm A(-) which operates on X to produce sequence of decision points A(X)
{(z,, 02, (Iz; az, }. as in Figure Further suppose we have identified and labeled each of the l unique structure classes present the dataset, as described in Section 2.1
For each window w; goal is to find an estimator which yields class conditional probabilities over all classes for each Task-X. Formally, we are looking for an estimator f:
f w. E Rdd + (Co...,,C-11) where f(w,) = V/-
This formulation closely resembles the multi-task learning problem, as generating predictions for each of the À structures represent related but fundamentally distinct tasks. Task-O is entirely explanatory, as all relevant decision points are visible in w,, while Task-(n-1) is entirely predictive, as only single decision point is visible.
In this example is transitive relationship, and the graph can be simplified into ordered list (smallest to greatest) but we note that this labeling scheme can generalize to graphs of arbitrary size and relations of arbitrary dimension.
3

---

Under review as a conference paper at ICLR 2022
In the following section, outline experiments in which our methodology is used approximate common offline time series algorithm using a hard parameter sharing multi-task ML model.
3 EXPERIMENTS
Here we detail the offline algorithm we aim to approximate, as well as the synthetic and real-world datasets will use to train model. Experiments on synthetic data designed to verify the functionality of our method in a controlled setting, and experiments on historical market data show the method's utiliiy noisy real-world data.
To demonstrate the method, we use a simple and well-understood multi-task learning model inspired by the hard parameter sharing approach introduced by CCruaaa (1993). While this architecture performs well in experiments on both synthetic and real-world data, we choose it primarily because demonstrates the role of machine learning in our framework: map behavioral structures in the past to those yet to come. The model is a multi-headed, multi-task learning model, with each of the heads performing single Task-A. Full model details are available Section 3.3
3.1 OFFLINE ALGORITHM: & TREND FILTERING
l trend filtering TF) is batch piece-wise linear approximation (PLA) method (Kim et al., 2009)
which extracts linear components from time series by minimizing the following objective function:
(xt 34) 2 +P Σ
8 28, + f111,,
where 2, indicates a length-T time series, â, denotes the approximation, and is smoothness parameter. In equation as the -n-ormm is used for smoothness, the resulting approximation is piece- wise linear which makes it suitable for structure and change point detection (CPD) (Aminikhanghahi Cook 2017). Decision points are then defined by intersection of neighboring segments produced by equation 3 Crucially, this algorithm requires access to future states, and as such is strictly confined to offline settings.
In our experiments, decision points ds, E R are generated by the é TF algorithm and correspond to points in the time series at which the slope changes significantly. The value of each decision point is assigned to the value of the time series at that position, as shown in Figure !
We choose the & TF algorithm for it's clarity and utility across domains. However, in Section 4.J we review additional algorithms and explain how the method may be applied in multi-dimensional settings.
3.2 DATASETS
To validate our methodology, we use synthetic and real-world datasets. In both experiments, structures have n = 5 decision points, yielding À = 5 tasks as shown in Figure)
3.2.1 SYNTHETIC DATA
Our synthetic dataset Syynth E IR is ID time series as shown Figure Structural sequences in X Synthh are composed of n 5 decision points with window size d = 150. As these generated algorithmically, we can produce structural sequences with precise amounts of randomness in their ordering. When 7 0, the structure sequence deterministic, meaning that any structure class Ci is always followed by the same sequence of structures Ct,, ,........._.. C._-.- When 7 1, there are no meaningful temporal relationships between structures and the trajectory of Syntth entirely random. Once all the structures (10k in total) are generated, we normalize X Synthh to have mean zero and standard deviation 0.1. Then we jitter the series with Gaussian noise having standard deviation 0.02, and split it into non-overlapping train/valiiation/test sets in percentages 85% / %/14% respectively.
The non-overlapping requirement is crucial, overlap between windows in the training/validaiionncco could skew the final test results.
4

---

Under review as a conference paper at ICLR 2022
Prediction
Structure
Label
Aso
C.o
Ci
his
f
C₁
Ci
hig
Ro
f02
Cia
his
J
Cia
his
Cia
Figure 3: The hard parameter sharing multi-task network for the case in which the structure size 5, where is both the number of decision points within structure and the number of heads of the network. Under "Structure" we show the target structures for each Task-À.
We construct the test set such that it contains only structures present in the train set (so we can recognize them in Task-O), but with their sequential relationships randomized by the factor 7-
Decision points are produced using the & TF algorithm and structures are labeled using a greater-than relationship indicator function r as in Figure Each label yi E {Cig..... Cin-1) is constructed as per the procedure in Section [22 and contains the visible structure's class Co as well as the four class labels for structures beginning w, and extending into the future.
3.2.2 HISTORICAL STOCK MARKET DATA
Our real-world dataset X E R also a 1D time series as shown in Figure The raw data is price-action (changes in the price of security) in one minute intervals for five commonly traded stocks on the NYSE, and covers all trading days between 2004 and 202(5) Decision points are generated using the TF algorithm and labels for each structure are assigned according to the procedure outlined in Section 2.2 Once the data is processed and labeled, it is normalized to have mean zero and standard deviation O. then split into overraaalgg train/vllidationtess sets in percentages 85% %/14% respectively. For this real-world data, structures which cover more or less than 150 points are resized have length 150. total, we identify 190k structures size n 5 and 32 unique structure classes in the data.
3.3 MODEL ARCHITECTURE
Figure 3 details the multi-task ML model architecture. While this particular multi-task architecture was suitable to demonstrate the method, we emphasize that our method is general, and allows for arbitrary architectures to be implemented depending on the demands of the offline algorithm being approximated.
We adopt a hard parameter sharing scheme as introduced by Caruana (1993) for its simplicity, intrinsic regularization, and its performance in our experiments on synthetic and real world data. The representation learner RG is convolutional neural network (CNN) parameterized by shared parameters 0 that learns representation of the input sequence w;. It is composed of three max-pooled ID convolutional layers followed by batch normalization layer (Toffe Szegedy) 2015). Each of the A tasks is performed by MLP fox parameterized by task-specific trainable parameters Φx and maps the representation learned by the CNN to class conditional probabilities over all & classes. Full
4 distribution of price changes significantly different after trading has closed, only data during market hours are used.
Financial data obtained through Polygon.io
5

---

Under review as a conference paper at ICLR 2022
details for each layer of the represention learner as well the MLP heads available in Figure 4 in the Appendix. Compactly, the output of each of the heads of the network can be described as follows:
h4s = for Foo.(Roly (R((z)
(4)
The network described in equation is an implementation of equation 2 that minimizes the Cross- Entropy Loss L() between hi, and the ground truth labels C₁ E y (Zhang & Sabuncu, 2018). Crucially, losses L(h4, M/x) are calculated for cach task À individually preserve the option to use different loss functions for different tasks the application warrants it.
3.4 TRAINING METHODOLOGY
The model is built in PyTorch and weights are initialized randomly to begin training. Once the datasets are prepared, we train for 300 epochs with batch size = 128 and use the Adam Optimizer with learning rate 0.001 (Paszke et al.) 2019) Kingma & Bal 2015). To account for slight class imbalances, we pass label weights (inverse frequency) to the Cross Entropy Loss criterion. For complete description of all relevant model and training hyperparameters, see [GitHub repository redacted for review] or Appendix A.1
4 RESULTS AND DISCUSSION
Our results demonstrate the method's ability to capture meaningful temporal relationships between behavioral structures identified by offline algorithms, and to apply those learnings to make quality predictions in noisy, real-time settings. We present the results terms of Top-N accuracy for each T
Synthetic Data:
The goal of the synthetic experiments is to verify the functionality of the method. As expected, increasing the randomness in the sequential relationships between behavioral structures resulted in proportional decrease in model accuracy on Task-A as increased. Table sshows shows the steady dropoff in performance as the sequences of behavioral structures become more random.
Task-0
Task-1
Task-2
Task
Task
Top-N Ace
5
2
5
2
5
2
1
2
1
y=0 0
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
7= 0.5
72.5
72.5
72.4
72.4
72.4
72.3
72.2
66.0
64.8
72.0
59.4
56.8
71.8
53.6
50.0
7=1 1
89.9
71.5
51.0
90.4
71.1
51.2
80.1
35.1
18.4
23.2
10.0
5.7
4.7
1.6
0.7
Table I: Top-N accuracy (shown in percent) for each Task-À on synthetic datasets generated with varying amounts of randomness 7 in the structure sequences. As expected, the model is able to perfectly predict even the most forward-looking task when the structure sequences are deterministic (7 0). Similarly, when the structure sequences are entirely random (? 1) the model is unable to make any meaningful predictions.
As Task-0 is simply identifying the structure in view, this accuracy stays fairly high as y 1. The mon-monotonicity of the Top-5 accuracies for Tasks 0-2 as y - 1 is driven by two factors: 1) the number of unique structures [S] k in the dataset increases in proportion to the randomness in structure sequences, and 2) the output size of each head, and thus the total number trainable parameters in the model, must increase to accommodate the increased number of labels. We review this limitation more thoroughly in Section ??. The performance of the most forward-looking task, Task-4, falls linearly as 7 + 1. This is expected, Task-4 is entirely predictive (4/5 decision points being out of view leaves no explanatory component). Top-1 Accuracies for these entirely predictive results (bolded) verify the method is working properly, and learning up to the randomness inherent in the structural sequences.
Random initializations across experiments did not appreciably affect the final performance so, for clarity, we have left standard emorr out of the main paper. For complete information on the Standard Errors for each Task, see Appendix A.1.4
'Note:

---

Under review as a conference paper at ICLR 2022
Task-O
Task-I
Task-2
Task-3
Task-4
2
1
5
2
5
2
1
Top-N Acc
98.8 91.1
71.8
94.2
70.2
45.8
79.0
39.1
18.9
45.2
16.2
7.7
30.1
12.2
6.2
Table 2: Top-N accuracy (shown in percent) for each Task-A on the real-world price action data.
Historical Stock Market Data: We demonstrate our methodology on equities data for two reasons: 1) the structures in ID time series (often referred to as "patterns" in this context) are clear and understandable, and 2) the notion that some market structures are predictive of future behavior is widely held the trading community (Bremer et al., 1997 Niarchos & Alexakis 2003). While this notion of structural predictability exists, studies of such structures remain confined small subset of easily recognizable structures, and do not approach the problem in the general case (Savin et al.) 2006). As our methodology explicitly learns temporal relationships between all structures in the dataset, it is particularly well suited to testing the original notion that past behavioral structures are predictive of future behavior. Table shows the Top-N accuracy of our method on the real-world dataset presented in Section 3.2.2
Examining the Norward-looking tasks, we see that there do exist meaningful relationships between behavioral structures in equities data and that our model was able to learn them with Top-1 accuracies well above chance (the dataset contained k action data are noisier than those in the synthetic dataset, Top-1 Accuracy explanatory tasks are lower while Top-5 accuracies remains high. And while it is beyond the scope of this paper, one could continue this analysis and investigate prediction performance comparing the results with the existing literature on the predictive power of price-action patterns.
unique structures). As structures in raw price-
srncctrr-by-strucee basis,
This experiment also demonstrates how the methodology can be used to express common problems in the target domain. For example, a popular target when predicting stock prices is to judge the direction the price of stock will move at any moment (Shen & Shatiq 2020). In our framework, this question could be addressed by comparing the relative values of the decision points predicted by Task-1. If the th or terminal, decision point predicted by Task-1 was greater than the (n 1)th, second last, decision point in Task-1, then the prediction would be that the price would increase. Similarly, if the terminal decision point T were lower than the second to last, the prediction would be price decrease. Investigating the Top-1 1 predictions of Task-1 in the test set, our model predicts the direction correctly 99.2% of the time. The difference between this value and the accuracy reported in Table stems stems from the similarities between candidate structures. When structures differ by only one or two decision points, the model can struggle to decide between them.
The directional predictions of Task-1 meet or exceed the performance of many state-of titte-of-the-re -he-hhrrtt ML-based stock price prediction systems (Rezaei et al. 2021). However, the result is made more interesting when we recognize that our prediction reflects not the usual one-slep-ahead prediction, but one-decision -eweddeciii-o-i------ee point-ahead, which is an average of 16.2 steps ahead in our dataset.
4.1 BEYOND THE & TF ALGORITHM
Offline change point detection (CPD) algorithms are designed to locate transitions between states in a time series, and are implemented in wide range of domains from medical condition monitoring and meteorology to finance and human activity analysis (Truong et al., 2020). While our experi- ments implement the é TF algorithm detailed in Section 3.J our method may be readily applied to approximate any algorithm that generates such change points, without any a priori knowledge of the algorithm's 's inner workings. Below, we list two examples from a large class of offline CPD algorithms. For exhaustive survey of CPD algorithms, see van den Burg & Williams (2220).
Relative Censity-Rativ Estimation: Introduced by Liu et al. (2013). this algorithm solves the CPD problem using non- non-maramtrric divergence estimation between time series samples from retrospective segments. The authors demonstrate their methodology on human-activity sensing, speech and Twitter messages.
Energy Change Point: Also introduced in 2013 (Matteson & James 2013), this algorithm solves the multivariate CPD problem using hierarchical clustering in both divisive and agglomerative
7

---

Under review as a conference paper at ICLR 2022
configurations. The authors demonstrate both methods geneties and financial data, and show performance matching existing state of the art CPD algorithms while making fewer assumptions.
Additionally, our method requires an appropriate relationship indicator function to define behavioral structures. Fortunately, these relations can be trivially obtained from existing knowledge about the target algorithm and data.
Relationship Indicators: As change points represent transitions between states, the relationship indicator function must encode meaningful relations between those states. For human activity analysis, relationship indicator functions might be relative miles walked or calories burned (more/less/same)) transitions between "active" and nnot active" states (Aminikhanghahi & Cook 2017). We note that these relationship indicator functions can be applied equally in multivariate settings. For example, in medical condition monitoring, states are routinely defined as a function of multiple variables. One of the many such states is an unusual pattern of breathing known as Kussmaul breathing (Sapra) al. 202T) which known to indicate underlying pathology and is a function of both the depth of the breath and the breathing rate. Using existing multidimensional indicator functions defined in diagnostic procedures, one could easily map change points within each time series (depth of breath and breathing rate) onto behavioral structures and, in combination with the historical change point data, use our method to predict the onset of multivariate states like Kussmaul breathing.
5 RELATED WORK
While existing literature does not address the problem of converting offline algorithms to their online counterparts in the general case, a variety of approaches have been proposed for specific algorithms. In this section we review those approaches well frameworks introduced to study the differences between explanatory and predictive models.
5.1 BATCH TO ONLINE CONVERSION
Blum (1944) showed if one-way predictor functions exist for a given problem, regarding polynomial- time learning, online models are harder construct than similar offline model. [akade & Kalai (2006) proposed an algorithm for converting proper batch algorithm into transductive online algorithm using hallucination generate random labels for unlabeled data, though only in the agnostic (unrealizable) setting (Ben-David et al. 1997 Kearns et al. 1994] Blum & Hartline (005). This algorithm computationally expensive in practice as re-runs the batch algorithm for every online prediction. |uutter (2014) proposed two methods, namely Limit and Mixture, converting offline probability estimators into online probability estimators. Limit needs to compute lim which may be incomputable and is not guaranteed to exist. Mixture, which uses Bayesian mixture, provides good performance guarantees (unlike Limit), but they do not provide a way to convert to an efficient
algorithm (Santhanam 2006).
5.2 EXPLANATORY VS. PREDICTIVE MODELS
Shmueli (200)) differentiated predictive models from explanatory models based four aspects: causasion-association association, theory-data, gurroseecive-proopectve and bias-variance, [Sainani] (2014) also differentiated these two type of models according to goal, threats to validity, candidate variables, variable selection, measures of model performance, and validation. Srribonchhita et al. (2019) proved there exists no algorithm generating the best explanatory model for given data, while there does exist an algorithm generating the best predictive model for given data. This is why the best predictive models are usually not similar to the best explanatory models.
6 CONCLUSION AND FUTURE DIRECTIONS
In this work, we introduced the first end-to-end differentiable methodology for learning predictive models of descriptive algorithms, which is both novel in it's generality and practically useful many
A proper algorithm is defined an algorithm whose output is always hypothesis h where F is class of functions.
8

---

Under review as a conference paper at ICLR 2022
domains. We described how offline algorithms can be used to generate descriptive training labels and formulated the approximation as a multi-task learning problem. In experiments on synthetic and real-world data, showed our method is capable of learning up the randommess inherent in the structural sequences, and that it can capture meaningful temporal relationships between behavioral structures even in the noisy price-action of commonly traded equities. Additionally, showed that the model is capable of matching prediction, with the caveat that our method did so using a peee-dcciiion-oin---aiiee translates to 16.2 steps in our dataset. Future research will be directed towards approximating offline algorithms which operate on higher dimensional and potentially multi-modal time series data.
performance in one-step-ahead stock price approach, which
Given the abundance of explanatory. offline algorithms deployed in high -ighiimpct settings, there noted absence of research into methods for transforming these algorithms into their online counterparts. We hope that this research serves as a stepping stone towards more general inquiry into the problem of translating explanatory processes into predictive ones.
6.1 ETHICS
Offline algorithms used in many contexts and towards many ends, usually explain and interpret the past. As their scope is so broad, we confine our ethics review to the application of our method in medicine and economic policymaking, where sophisticated diagnostic algorithms are routinely used and where new predictive models could have significant societal impact.
Medicine: Electronic health records (EHRs) are sparse, multi-dimensional time series, and the structures present between those series determine what cause is ultimately assigned to each symptom: "Symptom X appeared, then test result Y arrived,, so we know now that X was actually a result of condition Z. Now we know to look for symptom X" as 7. progresse.." ". 'This process is precisely what our methodology is well suited for: learning relationships between retroactively defined structures in data and forecasting which structures may come next. In regards to explanation, this methodology could be used to augment existing diagnostic aids and improve patient outcomes by generating better candidate diagnoses more quickly (Elkin et al. 2010). In regards to prediction, this could be integrated into existing prognostic models for predicting disease progression |Vogenberg/ 2009). Broadly, by appropriately encoding EHR data, our methodology could help doctors narrow the field of candidate diagnoses and better predict disease progression (Ross et al. 2014).
Policy: Economic models often based inappropriate assumptions and lack empirical validity Lipsey 2001). By encoding historical events and their precedents, e.g. credit crises and interest rates, as multivariate time series, our methodology could serve data-driven way to learn the temporal relationships between various economic phenomena and produce better forecasts. Additionally, as the number of predictor variables has exploded in recent years, prediction becoming more difficult, making an ML-based solution more appealing (Elliott & Timmermann 2008).
We note that the potential negative impacts of our methodology are the same as those posed by ML at large. Namely, we must take care to understand the distributional qualities of our input data and test appropriately to map out the edge cases. Particularly in healthcare, exploring and understanding edge cases is of paramount importance.
7 REPRODUCIBILITY
To ensure reproducibility, we provide thoroughly tested bash and python scripts for every step of the data acquisition, model training, model inference, and experimentation process. We additionally ensure that our experiment code is agnostic of the underlying hardware, and can be run either CPU or GPU. Our GitHub repository [Redacted for review] contains the following:
1. Detailed README with a clear description of the repo, the methodology, all experiments, and how to interpret new results.
2. Bash scripts to build and configure Conda environments with all required packages.
3. Clear configuration files in which all hyperparameters can be viewed and edited.
4. Full synthetic and equities datasets to reproduce each experiment plus scripts to pull addi- tional equities data if desired.

---

Under review as a conference paper at ICLR 2022
5. Pre-trained models validate end-to-end functionality.
6. Python scripts for training new models, along with instructions for how additional data should be formatted.
Note to reviewers: we have not included processed data in the supplemental material as would make the file prohibitively large, however, all data is available for download from GitHub repository once public.
REFERENCES
Samaneh Aminikhanghahi and Diane Cook. A survey of methods for time series change point detection. Knowledge and information systems, 51(2)339--677 May 2017.
Erin P Balogh, Bryan T Miller, John Ball, Committee on Diagnostic Error, Board on Health Care Services, Institute of Medicine, and The National Academies of Sciences, Engineering, and Medicine. Overview of Diagnostic Error Health Care. National Academies Press (US), December 2015.
Shai Ben-David, Eyal Kushilevitz, and Yishay Mansour. Online learning versus offline learning. Machine Learning, 29(1):45-66 63, 1997.
D Berger. A brief history of medical diagnosis and the birth of the clinical laboratory, part 1 a-anceent times through the 19th century. MI.O: medical laboratory observer, 31 -0,8 32, 34 40, July 1999.
Avrim Blum and Jason D. Hartline. Near-optimal online auctions. In Proceedings of the Sixteenth Annual RCM-SIAM Symposium on Discrete Algorithms, SODA '05, pp. 1156-1163, USA, 2005. Society for Industrial and Applied Mathematics. ISBN 08987158.8
Avrim Blum. Separating distribution-free and mistake bound learning models over the boolean domain. SIAM Journal on Computing. 23(5):90011000, 1994.
Marc Bremer, Takato Hiraki, and Richard J. Sweeney. Predictable patterns after large stock price changes on the tokyo stock exchange. The Journal of Financial and Quantitative Analysis, 32 (3):345-365, 1997. ISSN 00221090, 17566916. URL //tttp://www... 2331204
www esttrroor//stallee. stale//
Richard Caruana. Multitask learning: A Knowledge-Based source of inductive bias. In Proceedings of the Tenth International Conference on Machine Learning. 1993.
David Colander. The economics profession, the financial crisis, and method. Journal of Economic Methodology, 17(4):419-427, December 2010.
Peter Elkin, Mark Liebow, Brent A Bauer, Swarna Chaliki, Dietlind Pahher--Redder, John Bundrick, Mark Lee, Steven H Brown, David Froehling, Kent Bailey, Kathleen Famiglietti, Richard Kim, Ed Hofferr Mitchell Feldman, and G Octo Barnett. The introduction of diagnostic decision support system (DXplain into the workflow of a teaching hospital service can decrease the cost of service for diagnostically challenging diagnostic related groups (DRGs). International jouraaalannaauurrlll of medical informatics, 79(1 (9(11))772-771) November 2010.
Graham Elliott and Allan Timmermann. Economic forecasting. Journal of economic literature, 46 (1):3-6,, March 2008
Marcus Hutter. Offline to online conversion. International Conference on Algorithmic Learning Theory, pp. 230-244. Springer, 2014.
Sergey Toffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. February 2015.
Rebecca Jeffery, Tamara Navarro, Cynthia Lokker, R Brian Haynes, Nancy L Wilczynski, and George Farjou. How current are leading evidence- based medical textbooks? an analytic survey of four online textbooks. Journal of medical Internet research, 14(6):e December 2012.
10

---

Under review as a conference paper at ICLR 2022
Sham Kakade and Adam Tauman Kalai. From batch to transductive online leamnin. In Y Weiss, B Schölkopf, and C Platt (eds.), Advances in Neural Information Processing Systems 18, pp. 611-618. MIT Press, 2006.
Michael J Kearns, Robert E Schapire, and Linda M Sellie. Toward efficient agnostic learning. Machine Learning, 17(2-3):115-141, 1994.
Seung-Jean Kim, Kwangmoo Koh, Stephen Boyd, and Dimitry Gorinevsky. \ell_ trend filtering. SIAM review, 51(2)333--60, 2009.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL
7/arxiv. org. abs 1112. 6980
Christine Lewis and Nigel Pain. Lessons from OECD forecasts during and after the financial crisis. OECD Journal Economic Studies, 2014(1):9-35) 9)99-9,, March 2015.
Richard G. Lipsey, Successes and failures in the transformation of economics, Journal of Economic CRLhhtps:/// /doi
Methodology, 8(2): 201 2001. doi:
Song Liu, Makoto Yamada, Nigel Collier, and Masashi Sugiyama. Change-point detection in time-
series data by relative density-ratio estimation. Neural Networks, 43:72 83, 2013. ISSN 0893-6080. doi: http:///bb.oo///00001///eeeeeeee com/ sc wwweeeeeeeaatteeeeeeeeeeec
URL https: wwwp//wwww... wwt.:./cienneeire...o
Mahalakshmi, S Sridevi, and S Rajaram. A survey on forecasting of time series data. In 2016 International Conference Computing Technologies and Intelligent Data Engineering (ICCTIDE' 16), pp. 1-8, January 2016.
David S Matteson and Nicholas A James. A nonparametric approach for multiple change point analysis of multivariate data. June 2013.
N. Niarchos and Alexakis. Intraday stock price patterns the greek stock exchange. Applied Financial Economics, 13(1):13-22, 2003. doi: 10. 1080/096031001 10088166. URL
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. dÁlché Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019. URL http::/papers. new.eerssseerriis.ccc....o.. ctcpppp.cccpppppppe 9015 pytorch an imperative style high performarfooceeeeeeee deep- learning Library.
pdf
Hadi Rezaei, Hamidreza Faaljou, and Gholamreza Mansourfar. Stock price prediction using deep learning and frequency decomposition. Expert systems with applications, (69:114332, May 2021.
M Ross, W Wei, and L Ohno-Machado. "big data" and the electronic health record. Yearbook of medical informatics, 9:97-104, August 2014.
Kristin L Sainani. Explanatory versus predictive modeling. PM&R, 6(9):841-84)) 2014.
Narayana Santhanam. Probability estimation and compression involving large alphabets. PhD thesis, UC San Diego, 2006.
Amit Sapra, Ahmad Malik, and Priyanka Bhandari. Vital sign assessment. In StatPearls. StatPearls Publishing, Treasure Island (FL.), May 2021.
11

---

Under review as a conference paper at ICLR 2022
Gene Savin, Paul Weller, and Jänis Zvingelis. The Predictive Power of Price Patterns in the U.S. Stock Market. Journal of Financial Econometrics, 5(2):243 265, 12 2006. ISSN 1479-8409.
doi: 10.1093/jfinee/nn001... URL http::///d....rr//1.....
jjfinec/nbl nttp//bll0 nb12
Jingyi Shen and M Omair Shafiq. Short-term stock market price trend prediction using a comprehen- sive deep learning system. Journal of big data, 7(1):66, August 2020.
Galit Shmueli. To explain or predict? Statistical science: review journal of the Institute of Mathematical Statisticy, 25(3):289-310, August 2010.
Songsak Sriboonchitta, Lue Longpré, Vladik Kreinovich, and Thongchai Dumrongpokaphan. Why the best predictive models are often different from the best explanatory models: A theoretical explanation. In International Conference of the Thailand Econometrics Society, pp. 163-171. Springer, 2019.
Charles Truong, Laurent Oudre, and Nicolas Vayatis. Selective review of offline change point detection methods. Signal processing. 167(107299):007299, February 2020.
Gerrit J van den Burg and Christopher K T Williams. An evaluation of change point detection algorithms. March 2020.
F Randy Vogenberg. Predictive and prognostic models: implications for healtheare decision-mann -making in a modern recession. American health & drug benefits, 216):218-22,, September 2009.
Zhilu Zhang and Mert R Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. May 2018.
12

---

Under review as a conference paper at ICLR 2022
A APPENDIX
A.1 TRAINING METHODOLOGY
All code and data are available under the MIT License at [GitHub repository redacted for review]. In this Appendix, we review relevant training hyperparameters and briefty discuss the structure of each of the model components. Hyperparameters were chosen based combination of restricted, local grid search and hardware limitations like GPU RAM. We do claim that the model is entirely optimized, but optimized enough to perform well in our experiments and demonstrate the value and functionality of the method.
Data: Stocks we consider: AAPL, MSFT, IBM, AMZN and PG. These were chosen because they are large-cap, US stocks frequently traded on the NYSE. With the scripts available at [GitHub repository redacted for review] it is easy download and process additional tickers if desired.
Training: To account for slight class imbalances
dataset, we pass label weights proportional to the 1/(label frequency) the Cross Entropy Loss caleulation. As mentioned in Section we train for 300 epochs on the synthetic data. However, on the real data (with far fewer unique structures than synthetic dataset) the model converged after 80 epochs.
the entirely random
To train for 80 epochs on our hardware (see below) took 20 minutes and utilized 80% of the GPU, or 220W.
The model used in our experiments is modestly sized with million trainable parameters. This could be made more compact, but found that this configuration worked well in setting. Figure I details the layers of the CNN representation learner and a single MI.P head. Ro outputs vector length 512 that ingested by each of the MLP heads. In each MLP, Dropoutl 0.5 and Dropout2 = 0.2, and it yields class conditional probabilities over each of the k unique structure classes present the dataset.
Model:
fox
R
Conv ID + ReLU
Conv ID + ReLU
Conv ID + ReLU
Conv ID + ReLU
Conv ID + ReLU
Conv ID + ReLU
BatchNorml
Linear ReLU
Linear + ReLU
Dropout 2
MaxPoolID
MaxPoolID
MaxPool
Dropout
Linear
Figure 4: The internal structure of the CNN representation learner as well as the internal structure of a single MLP head (all MLPs are identical). For a complete definition of the model including all input and output sizes of each layer, see [GitHub repository redacted for review].
A.1.1 HARDWARE
All training was performed on an internal cluster and utilized only single GPU. Hardware specifics are as follows:
Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz
13

---

Under review as a conference paper at ICLR 2022
NVIDIA GeForce RTX 2080 Ti
Driver Version: 460.73.01
Cuda Version: 112
A.1.2 SOFTWARE
All training and model development was done using the latest stable PyTorch release. Complete setup scripts for Linux and Mac can be found [GitHub repository redacted for review].
Python 3.8.8
PyTorch 1.8.1
Scipy 1.6.2
OpenCV 4.5.1.48 (used for resizing the input windows)
Interpolation: Cubic
A.1.3
HYPERPARAMETERS
Training hyperparameters are as follows:
Epochs: 300 for synthetic data, real-world data converged after 80 epochs.
Batch Size: 128
Optimizer: Adam
Learning Rate: 0.001
Weight Decay: 0
Amsgrad: False
Criterion: Weighted Cross Entropy Loss
A.1.4 STANDARD ERRORS
Performance was not appreciably affected by random initialization conditions. As this these errors calculated over four experiments, they would likely come down with additional trials. For clarity, they are expressed in terms percent and rounded to the nearest whole number.
Standard Errors produced on synthetic data X Synnth
Task-O
Task-1
Task-2
Task
Task-4
Top-N Ace
5
5
2
5
2
5
2
1
y 0
+0
+0
10
10
10
10
40
10
+0
10
10
+0
=0
+0
#0
0.5
+1
+1
+1
+1
+1
+0
+1
+0
+0
+0
+0
1=1
+1
+2
+3
+2
+0
+0
+0
+1
+0
Table 3: Standard Error for each Task- on synthetic datasets generated with varying amounts of randomness ? the structure sequences. All values are percent and rounded to the nearest whole number.
Standard Errors produced on real-world data Xx:
Task-0
Task-1
Task-2
Task-3
Task-4
5
2
1
5
5
2
5
2
1
1
Top-N Acc
+0
+0
+0
+1
+0
+0
+2
+0
+0
+3
+1
+1
Table 4: Standard Error for each Task-A on the real-world price-action data. All values are in percent and rounded the nearest whole number.
14


Reviews:
[{"review_id": "yIIn2AP2uCW", "reviewer": "Reviewer_zoKR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n\n+ A novel formulation and research topic. The idea of trying to predict the behavior of an offline algorithm in a online setting using multi-task learning is a new approach. The exact formulation and the the way to pose this as MTL is non-trivial. The bulk of the contributions of this work is to make this modelling approach. Once figured out the proposed algorithm itself is standard multi-task learning.\n\n+ This paper contributes to the now growing line of work on bridging classical algorithms with machine learning. In that line of work, this considered approach is novel. It gives a new perspective. The typical direction has been to use the ML model as hints to improve the online/offline algorithm. On the other hand here, the online to offline algorithm is bridged via a machine learning task.\n\n+ For the most part the paper is clear and well-written.\n\nWeakness\n\n- The first main weakness I find in this paper is that, it does not sufficiently motivate the problem well. In particular, the online problem and it being posed as a MTL seems very abstract to the reader. It is not clear, how to use the outcome of this modelling in an actionable form. In particular, how does one interpret the class prediction for a window? What happens if the number of classes are unknown/evolving? May be elaborating this on a toy/standard offline algorithm before making it abstract would help the reader a great bit. \n\n- Related to above, the formulation makes it seem like this applies to any offline algorithm. But it really only applies to offline algorithms that work on time-segmented data. So it comes of as over-selling the main contributions of the paper. Please correct me if I am wrong; if not, I would reword the introduction to make this aspect very clear.", "summary_of_the_paper": "This paper considers the problem of an offline algorithm that operates on a time-series X to obtain sequence of decisions in an online setting. That is, it tries to approximate the behavior of this offline algorithm in a setting where at time t the algorithm only has access to the input until t (whereas in the offline algorithm the algorithm can lookahead and optimize). The pose this as a multi-task learning problem, where they slice the input into windows of size d, and the goal is to map each d dimensional window to one of the k possible structures in the dataset. They propose a MTL algorithm and use simulations and real-world stock market data to study the effects of their approach.", "summary_of_the_review": "I like some of the ideas of this paper, but overall I think that it falls just below the bar because of the reasons I stated in the weakness. Please correct me if my understadning is incorrect.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "x7mFpGo4jdI", "reviewer": "Reviewer_gii5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Novelty and significance.**\n\nI am not an expert in this domain but to my knowledge the proposed approach is novel and presents an interesting method for using offline algorithms to create datasets for training machine learning models to approximate the outputs of the offline algorithms. I think the idea could be of interest to the community. \n\nThat said, the paper does not provide any way to evaluate the significance of the proposed result, as there are no empirical (or theoretical) comparisons to any other methods. Thus, it is impossible to situate the proposed method either relatively or absolutely to determine whether the method will be of any benefit to the community. The paper presents two datasets (a synthetic toy dataset and a constructed dataset of historical stock market data), neither of which seem to have been used in the literature before, and trains the proposed method on these datasets but compares to no other methods. The results show that the method has higher accuracy for the easier classification tasks and lower accuracy for the lower harder prediction tasks, and that the method seems to get above chance accuracy on most problems, but this does not tell the reader anything about the overall performance and behavior of the algorithm. \n\nIn future revisions of the paper, the authors should compare to other algorithms in this same space. A reasonable place to start is with the works discussed in the related work section. You can show generality by taking the proposed algorithm and comparing it with multiple different existing approaches on the different tasks that each of those existing approaches works on. If the scores of the proposed approach are reasonable, then we will have some evidence that it works as claimed. I urge the authors to also perform ablations on the method. What effects do changes in model architecture have? Or how does the choice of offline algorithm affect the method? How do variations of the synthetic dataset affect the proposed approach as opposed to other methods (i.e., is it more robust or more accurate in particular regimes, such as different values of n, |S|, \\gamma, and d)? Note that two of the three proposed values of $\\gamma = {0, 0.5, 1}$ are trivial and thus do not provide much information. I encourage the authors to also include $\\gamma=0.25$ and $0.75$ to better show trends, and to plot these values instead of just putting the numbers in a table. Further, showing top-$k$ for $k=5,2,1$ seems unnecessary. 5 and 1 would be sufficient.\n\nRegarding the claim of meeting or exceeding performance on ML-based stock prediction systems, there is no evidence given in this paper for this claim so it is unsubstantiated. As I understand it, the cited paper (Rezaei et al., 2021) uses an entirely different dataset, so comparisons of accuracy are meaningless.\n\n**Clarity.**\nOverall the method is fairly clearly explained, and the remainder of the paper is clear. I think the paper would benefit from providing a summary of the method at the beginning of section 2, and from some changes to notation to simplify the presentation and to fix some issues with the notation. The precise method to generate the synthetic dataset and create the stock market data should be detailed in the paper as well, without requiring readers to go to the (not yet provided) code.\n\n**Detailed questions and comments.**\n- Preprocessing both the train and test splits together is wrong as it allows information to bleed from test to train, both in the form of the normalization and the set of structures trained on. All pre-processing should be performed only on the train data, the statistics retained, and then these used on the test data.\n- The fact that the number of unique structures $|S|$ changes for different values of $\\gamma$ makes it difficult to compare trends across values of \\gamma. Instead, I would suggest the authors change the dataset generation process to first specify an alphabet of structures $S$ and then generate (noised) trajectories from this alphabet.\n- It appears that $\\lambda$ is used both as an index and a count, but the count value of $\\lambda$ always equals $n$, so why not just use $n$?\n- Defining $|S| = k$ is confusing as $k$ is already (and typically) used as an index variable and it is nonstandard and unclear to use it as a count.\n- Please define a domain for the class labels and use that directly to simplify notation.\n- The definition of a window seems to assume that decision points are uniformly spaced, but this is not made explicit anywhere.\n- The definition of the estimator $f$ in eq (2) does not match the text, as it should be mapping onto the simplex of the class label domain based on the corresponding text.\n- Please explain the method for computing the decision points (l1TF) in more detail.\n", "summary_of_the_paper": "The paper presents a novel method for approximating offline time-series algorithms in an online setting. The method achieves this by assigning each window of the time-series data to a set of discrete classes based on the behavioral structure in that window, where the behavioral structures are encodings of the relative placements of the decision points in that window as determined by an offline time-series algorithm. These classes then provide the targets for a series of connected classification problems. An approximate online algorithm is obtained by training a multi-task classification neural network to solve these. Results on one-dimensional synthetic and stock-market data show that the predictive behavior of this method matches our intuitions, where it is most accurate when explaining the data and least accurate when predicting into the future.", "summary_of_the_review": "Overall, this paper lacks an evaluation for the proposed method, and thus cannot be accepted. The proposed approach seems interesting, and I encourage the authors to resubmit after incorporating a proper evaluation by comparing to other methods on established datasets and addressing some of the other comments above (in particular the dataset issues).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "uaqzyl2Vh6", "reviewer": "Reviewer_aaFn", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general, I like the idea of approximating the behavior of offline algorithms through the lens of multiple progressively-forward looking tasks (which essentially predicts the trajectory of future actions of the offline algorithm), since this allows us to predict further into the future (as opposed to predicting one-step ahead in standard ML methodologies). To the best of my knowledge, the idea of encoding the behavior of offline algorithms in graph structures, and then predicting the occurrence of such structures for multiple actions ahead via a multi-task learning framework is novel. \n\nThe following are some questions/concerns:\n1. From my understanding, the ultimate goal of the whole paper is to approximate the behavior of offline algorithms in real time, as opposed to directly predicting the ground truth evolution of the time series. This seems to me that the proposed framework’s performance is primarily driven by how well the offline algorithm can fit the historical data. That being said, if the offline algorithm significantly overfits the offline data (e.g. some complex deep neural network), does this mean the offline-to-online framework can also perform arbitrarily well under certain conditions? If so, I find this hard to believe. I might be misunderstanding something here, and it would be great if the authors can provide some more explanations and insights in the paper (e.g. what are some key drivers for the proposed framework’s performance, and how does the proposed framework’s performance relate to that of the offline algorithm).\n\n2. From a practical perspective, it seems to me that the algorithm is very \"data hungry\" as the number of structures may grow exponentially in the number of decision points in each structure. Hence, I believe there is this inherent trade-off between the amount of data required for labeling structures and how far we can predict into the future. The paper seems to be lacking detailed discussions for this tradeoff, or, on a related note, for how one should choose the \"optimal\" number of decisions within a structure. \n\n3. I am confused about the occurrence moments of predicted future actions (since the proposed algorithm is predicting X actions ahead, instead of X moment ahead). Consider the stock market example, where we have task 1 that predicts 1 action ahead of some offline algo, and task 2 that predicts 2 actions ahead. How do we know that the last predicted action in task 2 is further away in the future than the (single) prediction action in task 1? In other words, from my understanding the predicted structures are completely agnostic to actual occurrence moments, and hence we cannot compare prediction actions across tasks? I might have missed related discussions in the paper, and it would be great if the authors can add some more emphasis. \n\n4. I find the discussions in Section 2 General Schema quite difficult to digest at first read, and not until I went through the entire paper did I better understand how the multi-task learning framework works. Perhaps instead of discussing pure concepts (e.g. structure, actions, etc.), introducing the methodological framework within the context of a simple concrete example (e.g. a simplified version of the stock market example with some dummy offline algorithm) would improve the overall clarity of this section. \n", "summary_of_the_paper": "This work studies a methodological framework to transform/approximate offline algorithms into their online counterparts. The main methodology is to predict an offline algorithm’s actions in the real time future via learning behavioral structures of the offline algorithm using past data. The work presents several experiments using both synthetic and real (stock market) data.", "summary_of_the_review": "To the best of my knowledge, the proposed offline-to-online framework by predicting behavioral structures of the offline algo through a multi-task learning scheme is novel. \n\nFor weaknesses, more explanations/discussions on the following aspects would improve the paper: 1. How the performance of the proposed framework relates to that of the offline algorithm; 2. Choice for number of decisions in a structure; 3. Comparing predictions across different tasks. The paper’s exposition in terms of explaining the key concepts can also be improved.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "mVZHAUeCOGo", "reviewer": "Reviewer_EgW9", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n+ The paper describes their method, experimental setup, and results very clearly.\n+ The paper presents an interesting research direction, using knowledge from offline algorithms to improve performance of online algorithms via learning.\n+ The paper highlights that leveraging these methods could be impactful for many domains.\n\n---\n**Weaknesses**\n+ The primary weakness is the main claim seem incorrect.\nThe authors claim to develop a general framework for **approximating offline algorithms using online algorithms**. But the online algorithms trained in this paper do not directly attempt to approximate the offline algorithms. The online algorithms do not even produce the same type of outputs as the offline algorithms.\n\nThe offline algorithms take time-series X={x_1, ..., x_T} as input and produces outputs in the form of decision points A(X) = {(x_i, a_{x_i}), ... (x_j, a_{x_j})}.\n\nAn online algorithm that approximates this could take as input a partial time-series X_t = {x_t-d+1, ..., x_t} and decide whether or not to produce a decision point at time t.\n\nInstead the online algorithms in this work predict class labels which are a lossy mapping of sequences of decision points. As a result it is not clear to me in what sense these online algorithms approximate the offline algorithms. \n\nCan you clarify this for me? In what sense is it approximating the offline algorithm? If classification accuracy is 100%, can we make any statements about how good the approximation is?\n\n+ Because the primary claim is not clear, it's not clear how to evaluate the proposed method, or what baselines to compare to.\n+ The related work section is short and only mentions offline to online conversion, and explanatory vs predictive models. Since the paper also mentions time-series forecasting, it would be good mention related work in that field too.\n\n**Suggestion**\n+ May I suggest the following claim: the paper develops a method which leverages offline algorithms to perform better online time-series prediction.\n+ Then the main evaluation metric should be time-series prediction. And baselines would include a range of methods for time-series prediction, and ablations which use offline outputs in different ways but have similar architecture.\n+ It would be good to report the performance of comparable ML time-series prediction algorithms trained on the same data, and with similar architectures. Currently the authors mention another paper but do not report numbers for it.\n+ Related: In Introduction paragraph 2, you compare your method to time-series forecasting techniques. And mention 3 benefits of your technique which focuses on behavior, vs techniques that directly predict time-series trajectories. It would be good to see this demonstrated experimentally.", "summary_of_the_paper": "This paper makes use of **offline algorithms** (i.e. algorithms that can view entire time-series) to produce outputs which are used to train an **online algorithm** (i.e. an algorithm that can only view past values of a time-series). The online algorithms are not trained to produce the outputs of the offline algorithm directly, instead windows of the outputs are mapped to class labels using a hand-crafted mapping specific to the domain. The online algorithms are then trained to predict the class labels of the current window and progressively forward-looking windows (i.e. a multi-task prediction problem) given a window of the time-series.\n\nThey apply this method to synthetic and real-world time-series data (historical stock market data), and report the classification accurately of each of the multi-task prediction problems.\n\nThey also mention this can be used to predict the direction the price of a stock will move. And state that their method is competitive with state-of-the-art ML methods on this task.\n", "summary_of_the_review": "While the ideas presented in this work could be very impactful, as the paper is currently written, its main claim seems incorrect which is grounds for rejection.\n\nThe paper claims to develop a general framework for approximating offline algorithms using online algorithms. But to me, it seems the online algorithms do not approximate the offline algorithms.\n\nI think the paper could be made substantially better in one of two ways:\n1) The authors clarify in what sense the online algorithms approximate the offline algorithms.\n2) The authors modify the claims to more accurately reflect the the method, and add additional experiments to support those claims.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "17eT71OxJp0", "reviewer": "Reviewer_s1H9", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Strengths*:\n  1.  The motivation of bridging the gap between offline algorithms and their online counterparts is clear and practical. Real-world examples are discussed in the introduction and conclusion, and help to further understand the motivation. \n  2.  The proposed approach is novel to my knowledge. I admire the idea to capture the behavior structure by multi-task learning model, which is interesting to create datasets using offline algorithm for training the online counterpart. \n  3.  The design is clearly presented. Figure 1, 2 are helpful to understand the high-level framework.\n\n*Weakness*:\n  1.  Why no baselines are presented in the experiment part. I am not an expert in this field, so I am not entirely convinced that it needs any comparison of other benchmarks.\n  2.  Is there any theoretical guarantees or insights behind the design?\n  3.  I personally think that the paper writing can be further enhanced. For example: 1) The sections and subsections does not follow a traditional manner, e.g., the experiment and experimental results are not in one section; the ethics is a subsection of conclusion. 2) Although the authors claim that the proposed method outperform the SOTA, however, the performance of the SOTA model is not present in the Table.\n\nMinor: \n``We review this limitation more thoroughly in Section ??’’ in page 6 —> Section ?? ", "summary_of_the_paper": "This work claims to propose a general methodology for approximating offline algorithms in online settings, in contrast to previous methods only for particular cases. To achieve this, the author prosed a multi-tasks-based method to learn from the datasets created by the offline algorithms. Experiments are conducted to verify the idea.", "summary_of_the_review": "I admire the motivation, idea, and possible impact of this paper. However, I am not entirely convinced that the experimental results are convincing enough. I would like to update the score after interacting with the authors and other reviewers.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: ## A Brief Summary
This paper uses offline algorithms that can see the entire time-series to approximate the online algorithms that can only view the past time-series. The way this is done is basically, the offline algorithm is used to provide discrete class targets to train the online algorithm. The paper presents results on synthetic and historical stock market data.

## Reviewer s1H9
**Strengths:**
- Practical problem.
- Novel approach.
- Clear presentation.
**Weaknesses:**
- No other baselines.
- No theoretical guarantees behind the approach.
- Writing could be improved.

## Reviewer EgW9
**Strengths:**
- Clear writing.
- Interesting research direction.
**Weaknesses:**
- The primary claim seems incorrect and unclear. 
- Due to the unclarity about the primary claim of this paper, it is difficult to evaluate the paper. 
- Lack of baselines.
- The lack of discussions of the related works.

## Reviewer gii5
**Strengths:**
- Interesting and novel approach.
**Weaknesses:**
- Difficult to evaluate, with no empirical baselines or theoretical evidence.
- The datasets used in the paper are not used in the literature before. Authors should provide experimental results on datasets from the literature as well.
- The paper needs to compare against the other baselines discussed in the related works.
- More ablations and analysis on the proposed algorithm is required.
- Unsubstantiated claims regarding being SOTA on the task, since the paper doesn't compare against any other baselines on these datasets.
- The paper can be restructured to improve the flow and clarity.

## Reviewer zoKR
**Strengths:**
- Novel and interesting research topic.
- Bridging classical algorithms and ML.
- Clearly written.
 
**Weaknesses:**
- Lack of motivation for the problem.
- The approach only works with offline algorithms that work on time-segmented data.

## Reviewer aaFn
**Strengths:**
- Novel algorithm.

**Weaknesses:**
- Potentially overfitting to the offline data.
- Data hungry approach.
- Confusion related to the occurrence moments of predicted future actions.
- Section 2 is difficult to understand.

## Key Takeaways and Thoughts
Overall, I think the problem setup is very interesting. However, as pointed out by reviewers gii5 and EgW5, due to the lack of baselines, it is tough to compare the proposed algorithm against other approaches, and this paper's evaluation is challenging. I would recommend the authors include more ablations in the future version of the paper and baselines and address the other issues pointed out above by the reviewers.
Decision: Reject