Original: Performative Prediction in Time Series: A Case Study
Revised: Performative Prediction in Time Series: A Case Study

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Performative prediction is a phenomenon where a model’s predictions, or the decisions based on these predictions, may inﬂuence the outcomes of the model.
Revised: Performative prediction is a phenomenon where a model’s predictions, or the decisions based on these predictions, may inﬂuence the outcomes of the model.

Original: Thisis especially conspicuous in a time series forecasting setting where interventionsoccur before outcomes are observed.
Revised: This is especially conspicuous in a time series prediction setting where interventions occur before outcomes are observed.

Original: These interventions dictate which data pointsin the time series can be used as inputs for future predictions.
Revised: These interventions dictate which data points in the time series can be used as inputs for future predictions.

Original: In this paper, werepresent a patient’s symptom levels along their cancer rehabilitation plight as atime series.
Revised: In this paper, we represent patient-reported symptom values collected during their oncology appointments as a time series.

Original: We use a decision-tree based model to predictthe future symptomvalues of a patient.
Revised: We use a decision-tree based model to predict a patient’s future symptom values.

Original: Based on these predictions, clinicians decide which symptomlevels will be observed in the future.
Revised: Based on these predictions, clinicians decide which symptom values will be observed in the future.

Original: We propose methods to mitigate the problemof performative prediction in time series.
Revised: We propose methods to provide robustness against the problem of performative prediction in time series.

Original: Our results show how performativeprediction may lead to a 29.4% to 40.7% higher error across different symptoms.
Revised: Our results characterise how performative prediction may lead to a 29.4% to 40.7% higher error across different symptoms.

Original:  Introduction
Revised:  Introduction

Original: Typically time series forecasting (TSF) uses observations of a time series (and perhaps other information) to forecast future values of that series.
Revised: Typically time series prediction (TSP) uses observations of a time series (and perhaps other information) to forecast future values of that series.

Original: As time passes, new data of the time series becomesavailable and is used to predict more future values.
Revised: As time passes, new data of the time series becomes available and is used to predict more future values.

Original: However, in some settings, the decisions based onthese predictions may inﬂuence future observations.
Revised: However, in some settings, the decisions based on these predictions may inﬂuence future observations.

Original: These predictions are called performative.
Revised: These predictions are called performative .

Original: In this work, we consider a concrete application setting where the goal is to develop a monitoringsystem of symptom levels for cancer patients during their rehabilitation journey focusing medicalcare on patients in critical states.
Revised: In this work, we consider a concrete application setting where the goal is to develop a monitoring system of symptom values for cancer patients, focusing medical care on patients in critical states.

Original: We, therefore, aim to predict future symptom values using historicalsymptom levels.
Revised: We, therefore, aim to predict future symptom values using historical ones.

Original: These predictions aid in deciding whether a patient should visit a rehabilitationcenter (where the patient gets their state assessed by a medical professional) or not.
Revised: These predictions aid in deciding whether a patient should visit a cancer center (where the patient gets their state assessed by a medical professional) or not.

Original: Therefore, theprediction model will impact which symptom values will be observed and may be used as inputs forfuture predictions.
Revised: Therefore, the prediction model will impact which symptom values will be observed and may be used as inputs for future predictions.

Original: This induces a distribution shift in the data, which corresponds to a performativeprediction problem [13].
Revised: This induces a distribution shift in the data, which corresponds to a performative prediction problem [Perdomo et al., 2020].

Original: Moreover, such systems introduce a partial observability problem whendeployed in the real world, making their evaluation especially tricky at deployment.
Revised: Moreover, such systems introduce a partial observability problem when deployed in the real world, making their evaluation especially tricky at deployment.

Original: Indeed, oncethe system is deployed, the true patient state is only available when the monitoring system sends thepatient to a rehabilitation center.
Revised: Indeed, once the system is deployed, the true patient state is only available when the monitoring system sends the patient to a cancer center.

Original: Otherwise, the true state remains unknown.
Revised: Otherwise, the true state remains unknown.

Original: It is therefore crucial toconduct an ofﬂine evaluation of the system that can simulate these effects.
Revised: It is therefore crucial to conduct an ofﬂine evaluation of the system that can simulate these effects.

Original: Our aim is to understandthe impact of this partial observability and study how this affects the accuracy of the system.
Revised: Our aim is to understand the impact of this partial observability and study how this may affect the accuracy of the system.

Original: The paper is organized as follows.
Revised: The paper is organized as follows.

Original: Section 2 provides an overview of the related work.
Revised: Section 2 provides an overview of the related work.

Original: Section 3formulates the general problem which we refer to as time series performative predictions.
Revised: Sectionformulates the general problem which we refer to as time series performative prediction .

Original: Section 4presents our proposed methodology for investigating the impact of the performative prediction effecton model performance along with solutions to train models with increased robustness to this effectand the data used in this study.
Revised: Sectionpresents our proposed methodology for investigating the impact of the performative prediction effect on model performance along with solutions to train models with increased robustness to this effect and the data used in this study.

Original: Section 5 ﬁnally shows results that highlight the need to consider theseaspects when training and evaluating time series predictive models that may induce performativeprediction behaviours at deployment.
Revised: Section 5 ﬁnally shows results that highlight the need to consider these aspects when training and evaluating time series predictive models that may induce performative prediction behaviours at deployment.

Original:  Related Work
Revised:  Related Work

Original: The problem of predicting future symptoms for oncology patients has been tackled with machinelearning in many forms, e.g., predictingthe severe symptoms in cancer patients [16, 12, 20] and usingpatient-reported symptom levels to improve the performance of models for emergency departmentvisits among cancer patients [17].
Revised: The problem of predicting future symptoms for oncology patients has been tackled with machine learning in many forms, e.g., predicting severe symptoms in cancer patients Seow et al. Papachristou et al. [2018], Vaz-Luis et al.

Revised: [2022] and using patient-reported symptom values to improve the performance of models for emergency department visits among cancer patients Sutradhar et al.

Original: However, these works do not consider the performative predictiondynamics that can occur at deployment but rather study the typical supervised learning problem.
Revised: However, these works do not consider the performative prediction dynamics that can occur at deployment, but rather study the typical supervised learning problem.

Original: Performative prediction has been observed in healthcare applications in the literature.
Revised: Performative prediction has been observed in healthcare applications in the literature.

Original: Liley et al.[11] discuss a typical healthcare setting where interventions lead to bias.
Revised: Liley et al. [2020] discuss a typical healthcare setting where post-intervention model updating leads to bias.

Original: Lenert et al.
Revised: Lenert et al.

Original: [10] show howprognostic models can become victims of their own success.
Revised: [2019] highlight the problem of when users respond to model predictions, downstream characteristics of the data, including the distribution of the outcome, may change and how these problems will need to be mitigated by systematically incorporating interventions into prognostic models to achieve robust performance surveillance of models in clinical use.

Original: However, these settings are not modeledas TSF as  our setting.
Revised: However, these settings are not modelled as TSP like our setting.

Original: TSF can often suffer from a distribution shift.
Revised: TSP can often suffer from a distribution shift.

Original: Duan et al.
Revised: Duan et al.

Original: [5] resolve this byusing a Hyper TimeSeries Forecasting model that jointly learns the time-varying distributions and thecorresponding forecasting models in an end-to-end fashion.
Revised: [2022] resolve this by using a Hyper TimeSeries Forecasting model that jointly learns the timevarying distributions and the corresponding forecasting models in an end-to-end fashion.

Original: Bennett and Clarkson [1] deal with TSFunder distribution shift using differentiable forgetting.
Revised: Bennett and Clarkson [2022] deal with TSP under distribution shift using differentiable forgetting.

Original: However, in these settings, the distributionshift is not induced by the decisions made by the model but is rather by the environment.
Revised: However these solutions may not be used in our setting because the distribution shift is not induced by the decisions made by the model but are rather by the environment.

Original:  Time Series Performative Predictions
Revised:  Time Series Performative Prediction

Original: Time series prediction essentially consists of predicting the future value of some process based onvalues observed for this process until now:
Revised: Time series prediction essentially consists of predicting the future value of some process based on values observed for this process until now:

Original: [Equation]
Revised: [Equation]

Original: where t denotes the current time.
Revised: where t denotes the current time.

Original: We consider the generic setting where observations are not assumedto be evenly spaced in time nor aligned for multiple time series data.
Revised: We consider the generic setting where observations are not assumed to be evenly spaced in time nor aligned for multiple time series data.

Original: Moreover, we do not make anyassumptions regarding the (in-)dependence between time series used for training a model and timeseries encountered at deployment.
Revised: Moreover, we do not make any assumptions regarding the (in-)dependence between time series used for training a model and time series encountered at deployment.

Original: Figure 1 illustrates time series data in the considered problem.
Revised: Figure 1 illustrates time series data in the considered problem.

Original: Such a problem will therefore require considering two cases at evaluation: time series that hadobservations in the training set (e.g. time series 2 and 3 in Fig. 1) and time series entirely encounteredat deployment (e.g. time series 4 in Fig. 1).
Revised: Such a problem will therefore require considering two cases at evaluation: time series that had observations in the training set (e.g. time series 2 and 3 in Fig. 1) and time series entirely encountered at deployment (e.g. time series 4 in Fig. 1).

Original: Let H ( t  ) =
Revised: Let H ( t k ) =

Original: { ( t 1 , x t 1 ) , ( t 2 , x t 2 ) , . . . 
Revised: { ( t 1 , x t 1 ) , ( t 2 , x t 2 ) , . . . , ( t k − 1 , x

Original:  } denote the history of previous observations available at timet ≥ 1 for a given time series, where t i denotes the time of the i -th observation and 1 ≤t i < t .
Revised: t k − 1 ) } denote the history of previous observations available at time k ≥ 1 for a given time series, where t i denotes the time of the i -th observation and 1 ≤ i < k .

Original: Theproblem is characterized by a decision function g : X (cid:55)→ [0 , 1] indicating whether an observationshould be acquired (1) or not (0) based on the current prediction.
Revised: The problem is characterized by a decision function g : X (cid:55)→ [0 , 1] indicating whether an [Figure] observation should be acquired (1) or not (0) based on the current prediction.

Original: At time t  , a predictive model outputsthe prediction ˆ x t  ∈ X using H ( t  ) and x t  is observed if g (ˆ x t  ) is true, in which case H ( t + 1) =
Revised: At time t k , a predictive model outputs the prediction ˆ x t k ∈ X using H ( t k ) and x t k is observed if g (ˆ x t k ) is true, in which case H ( t k +1 ) =

Original: H ( t  ) ∪ { ( t, x t  } .
Revised: H ( t k ) ∪ { ( t k , x t k ) } .

Original: Otherwise, x t  is not observed and H ( t + 1) =
Revised: Otherwise, x t k is not observed and H ( t k +1 ) =

Original: H ( t  ) .
Revised: H ( t k ) .

Original: In practice, one may allow some warm-up when starting to predict on a new time series.
Revised: In practice, one may allow some warm-up when starting to predict on a new time series.

Original: In this case,we deﬁne warm-up of duration N as systematically observing the ﬁrst N observations for a timeseries.
Revised: In this case, we deﬁne warm-up of duration N as systematically observing the ﬁrst N observations for a time series.

Original: Therefore, decisions on a new time series in a system with warm-up of N always begin attime t =
Revised: Therefore, decisions on a new time series in a system with warm-up of N always begin at time k

Original:  N + 1 with the history H (  N + 1) =
Revised: = N + 1 with the history H ( t N +1 )

Original:  { (1 , x  1 ) , (2 , x  2 ) , . ..
Revised: = { ( t 1 , x t 1 ) , ( t 2 , x t 2 ) , . .

Original: , ( N, x  N ) } .
Revised: , ( t N , x t N ) } .

Original:  Methodology
Revised:  Methodology

Original: To investigate the impact of the performative prediction dynamics of the problem, we conductexperiments where we simulate the training and deployment of monitoring systems of symptomlevels for cancer patients during their rehabilitation journey:
Revised: To investigate the impact of the performative prediction dynamics of the problem, we conduct experiments where we simulate the training and deployment of monitoring systems of symptom values for cancer patients:

Original: • We split the dataset in two as shown in Figure 1 (training and deployment).• We train a model on the training set using different training strategies.•
Revised: • We split the dataset into the training set and deployment set as shown in Figure 1. • We train a model on the training set using different training strategies.

Original:  We evaluate the model in the deployment setting simulated using the deployment data.
Revised: • We evaluate the model in the deployment setting simulated using the deployment set.

Original: Foreach time series contained in the deployment set, for each data point in the time series: ifthe history for that series contains less than N = 2 observations, then acquire observation;otherwise, predict and acquire (or not) observation based on the decision function 
Revised: For each time series contained in the deployment set, for each data point in the time series: if the history for that series contains less than N = 2 observations, then acquire observation; otherwise, predict and acquire (or not) observation based on the decision function g as described below.

Original: g .We consider a decision function  that acquires the observation when the prediction exceeds a threshold:g = (cid:26) 1 if I (ˆ x t ≥ τ ) 0otherwise , with threshold value τ = 5 .
Revised: We consider a decision function g that acquires the observation when the predicted symptom value exceeds a threshold τ : [Equation] When τ = 0 , all the symptom values are acquired, therefore there is full observability. On the other extreme, when τ = 10 , we observe only those symptom values that are equal to 10.

Revised: This would mean we would never observe the other symptom values, making their prediction very difﬁcult.

Original:  4.1 Data
Revised: Therefore, the higher the τ , the more difﬁcult it is to track the problem. We change the value of τ to gauge the robustness of the model. In our setting, we use τ = 5 as a medium-level difﬁculty task. 4.1 Data

Original: The data used in this study has been collected from 2013 to 2019 for 14586 patients with cancer typesbreast, colorectal, lymphoma, and head and neck.
Revised: The data used in this study has been collected at the Princess Margaret Cancer Centre fromto 2019 for 14586 patients with breast, colorectal, lymphoma, and head and neck cancer types.

Revised: Post cancer diagnosis, a patient visits the doctor or clinician either when they are undergoing initial diagnostic testing and treatment planning, undergoing treatments, or coming for follow-up surveillance appointments.

Revised: During a patient’s visit to a cancer center, they are asked to ﬁll an electronic survey (ﬁlled on iPads) documenting their symptoms using DART (Distress Assessment and Response

Original: We use demographic information of patients likesex and age, along with surgery and historical symptom level data for prediction.
Revised: Tool) [Li et al., 2016]. To predict future symptom values, we use demographic information of patients like sex and age from the cancer registry data, along with surgery and historical symptom value data from the surveys.

Original: Speciﬁcally, we usethe two previous symptom levels as input to the model.
Revised: We leverage the historical symptom values to calculate their gradient of change and use it as an input feature. As the average number of observations per patient is 4.76 and the minimum number of observations required to calculate the gradient of change in symptom value is two, we use the gradient between the two previous symptom values as input to the model.

Original: Therefore, we consider only those patients inthe dataset that have at least three observations of a symptom.
Revised: Therefore, we consider only those patients in the dataset that have at least three observations of a symptom.

Original: We consider two symptoms to predict,namely, pain and fatigue.
Revised: We consider two symptoms to predict, namely, pain and fatigue which were measured on the

Revised: Edmonton Symptom Assessment Scale-Revised [Watanabe et al., 2012].

Original: Each symptom level can be one of 11 possible values: 0 to 10, where 0means the lowest level of pain or fatigue and 10 means the highest level.
Revised: Each symptom level can be one of 11 possible values: 0 to 10, where 0 means the lowest level of pain or fatigue and 10 means the highest level.

Original: We observe that there areclass imbalance problems, where approximately 43% and 21% of the outputs belong to one class(level 0) for pain and fatigue, respectively.
Revised: We observe that there are class imbalance problems in the dataset, where approximately 43% and 21% of the outputs belong to one class (level 0) for pain and fatigue, respectively.

Original: Moreover, the number of samples in a class is inverselyproportional to the symptom level.
Revised: Moreover, the number of samples in a class is inversely proportional to the symptom level.

Revised: This class imbalance problem makes the prediction problem more complex.

Original: 4.2 Predictive model
Revised: 4.2 Predictive model

Original: Decision trees (DTs) are widely used in healthcare [14] and cancer-related applications [15, 6, 19, 4,3, 7].
Revised: Decision trees (DTs) are widely used in healthcare [Podgorelec et al., 2002] and cancer-related applications [Valdes et al., 2016, Deist et al., 2018, Coroller et al., 2017, Gennatas et al., 2018], making them a suitable predictive model for this problem.

Original: In this work, we consider LightGBM [8], a gradient boosting DT approach where new modelsare created to predict the residuals of prior models and are then combined to make the ﬁnal prediction.
Revised: Among several DT based algorithms, we consider LightGBM [Ke et al., 2017] because it uses gradient descent to minimize the loss when adding new models which is more beneﬁcial as compared to other DT algorithms that either train the different models separately before aggregating them or do not use multiple models at all.

Original: Inspired by previous works [21, 9], we rely on Synthetic Minority Oversampling TEchnique (SMOTE)[2] to deal with minority classes by generating synthetic samples  so that the trained model sees thesame number of data points of these classes as for the majority class.
Revised: Inspired by previous works [Yang Zhao and Tsui, 2018, Khushi et al., 2021], we rely on Synthetic Minority Oversampling TEchnique (SMOTE) [Chawla et al., 2002] to deal with the class imbalance problem which generates synthetic samples of the minority classes so that the trained model sees the same number of data points of these classes as of the majority class.

Original: A minority class is over-sampledby taking each of its samples and introducing synthetic examples along the line segments joiningany/all of the k minority class nearest neighbours.
Revised: A minority class is over-sampled by taking each of its samples and introducing synthetic examples along the line segments joining any/all of the k minority class nearest neighbours.

Original: For our experiments, we use the default valuek = 5 .
Revised: For our experiments, we use the default value k = 5 .

Original: To reduce the impact of the majority classes in the training, we also perform undersamplingwith Tomek Links [18] after oversampling.
Revised: To reduce the impact of the majority classes in the training, we also perform undersampling with Tomek Links [Tomek, 1976] after oversampling.

Original: 4.3 Training strategies
Revised: 4.3 Training strategies

Original: We evaluate different training strategies to investigate whether the models can be made more robustto the performative prediction dynamic:
Revised: We evaluate different training strategies to investigate whether the models can be made more robust to the performative prediction dynamic:

Original: Ideal Training the model using the two most recent observations.
Revised: Ideal Training the model using the two most recent observations.

Original: At deployment, we do not useg function.
Revised: At deployment, we set the threshold τ = 0 in the g function.

Original: Therefore, everything is observed.
Revised: Therefore, everything is observed.

Original: This strategy ignores the performative predictionproblem and can be seen as the upper bound in terms of observability.
Revised: This strategy ignores the performative prediction problem and can be seen as the upper bound in terms of observability.

Original: Oblivious This model is trained exactly like the ‘Ideal’ model, using the two most recent observations.
Revised: Oblivious This model is trained exactly like the ‘Ideal’ model, using the two most recent observations.

Original: However, at deployment, values are observed (or not) based on the g function.
Revised: However, at deployment, values are observed (or not) based on the g function.

Original: Warm-up One way to avoid the problem of performative prediction is to ignore all the data pointsaffected by the distribution shift.
Revised: Warm-up One way to avoid the problem of performative prediction is to ignore all the data points affected by the distribution shift.

Original: In this training strategy, we train the model using only the warm-uphistory to predict all future symptom levels.
Revised: In this training strategy, we train the model using only the warm-up history to predict all future symptom values.

Original: This may not be the best strategy because it does not userecent observations.
Revised: This may not be the best strategy because it does not use recent observations.

Original: At deployment, we use the g function.
Revised: At deployment, we use the g function.

Original: Random Training using two randomly chosen previous symptom levels as inputs.
Revised: Random Here, we train using two randomly chosen previous symptom values as inputs.

Original: This is anattempt in making the resulting model robust to unobserved symptom values.
Revised: This is an attempt in making the resulting model robust to unobserved symptom values.

Original: When deploying thismodel, we use the g function.
Revised: When deploying this model, we use the g function.

Original: 4.4 Evaluation Metrics
Revised: 4.4 Evaluation Metrics

Original: We measure performance using the Mean Absolute Error (MAE), which corresponds to the meanabsolute deviation in the symptom level making it interpretable for clinicians to infer and analyse.
Revised: We measure performance using the Mean Absolute Error (MAE), which corresponds to the mean absolute deviation in the symptom level making it interpretable for clinicians to infer and analyse.

Original: Let N denote the number of samples used for evaluating the performance, the MAE is given by:
Revised: Let N denote the number of samples used for evaluating the performance, the MAE is given by:

Original: MAE = 1 N (cid:80) Ni
Revised: [Equation]

Original: =1 | x i − ˆ x i | , where x i is the true value of the i th target observation, and ˆ x i is thevalue predicted by the model for this observation.
Revised: where x i is the true value of the i th target observation, and ˆ x i is the value predicted by the model for this observation.

Original: In the case of imbalanced classes, mistakes on a minority class can be hidden by good performanceon a dominant class.
Revised: In the case of imbalanced classes, mistakes on a minority class can be hidden by good performance on a dominant class.

Original: To this end, we focus on achieving good performance on individual classes aswell as overall.
Revised: To this end, we focus on achieving good performance on individual classes as well as overall.

Original: Let C denote the set of classes.
Revised: Let C denote the set of classes.

Original: For each class c ∈ C , we measure the MAE per class(MAE c ) and combine these MAE c values using the weighted-MAE (WMAE).
Revised: For each class c ∈ C , we measure the MAE per class (MAE c ) and combine these MAE c values using the weighted-MAE (WMAE) as described in eqn.

Original: The weight of a classgives value to the MAE cof that class inversely proportional to the number of samples in that class( N c ): w c = max c (cid:48)∈C N c (cid:48) N c .
Revised: The weight of a class w c as described in eqn 2 gives value to the MAE c inversely proportional to the number of samples in that class ( N c ).

Original: The weight willtherefore be of value 1 for the class with the largest amountof data and would be 10 times larger for a class with 10 times fewer data.
Revised: The weight will be of value 1 for the class with the largest amount of data and would be 10 times larger for a class with 10 times fewer data.

Original: We can then compute theglobal error as WMAE =
Revised: [Equation] At deployment, the performance is measured on all symptom values, whether observed or not.

Original: (cid:80) c ∈C w c MAE c (cid:80) c ∈C w c .
Revised: However, only the observed symptom values are used as inputs to the model for future predictions.

Original:  Results
Revised:  Results

Original: Our results are summarised in table 1.
Revised: Our results are summarised in table 1.

Original: For both symptoms, the ‘Ideal’ model has the lowest WMAE.
Revised: As expected, for both symptoms, the ‘Ideal’ model has the lowest WMAE.

Original: The other three models that consider performative prediction where some symptom values may not beobserved at deployment, perform worse overall than the ‘Ideal’ model as indicated by higher WMAE.
Revised: The other three models that consider performative prediction where some symptom values may not be observed at deployment, perform worse overall than the ‘Ideal’ model as indicated by higher WMAE.

Original: For predicting pain, WMAE is 29.4% to 32% higher than the ‘Ideal’ model.
Revised: When predicting pain, WMAE is 29.4% to 32% higher than the ‘Ideal’ model.

Original: For predicting fatigue, WMAE is 38.3% to 40.7% higher than the ‘Ideal’ model.
Revised: When predicting fatigue, WMAE is 38.3% to 40.7% higher than the ‘Ideal’ model.

Original: It is interesting to note that on lower classes( c < 4 ), the models considering performative prediction perform almost equivalent to and in somecases better than the ‘Ideal’ model.
Revised: It is interesting to note that on lower classes ( c < 4 ), the models considering performative prediction perform almost equivalent to and in some cases better than the ‘Ideal’ model.

Original: This shows that for some symptom levels, not observing recentvalues does not affect the prediction.
Revised: This shows that for some symptom values, not observing recent values does not affect the prediction.

Original: However, for higher symptom levels, the ‘Ideal’ models clearlyperform better.
Revised: However, for higher symptom values, the ‘Ideal’ models clearly perform better.

Original: The discrepancy between the WMAE of the ‘Ideal’ and other models highlights thenecessity to account for performative prediction for accurate prediction. 
Revised: The discrepancy between the WMAE of the ‘Ideal’ and other models highlights the necessity to account for performative prediction for accurate prediction. Moreover, achieving robustness to the performative dynamics is not trivial.

Original: [Table]
Revised: [Table]

Revised: Visual Representation of the Distribution shift Fig.

Revised: 2 shows the change in distribution due to performative prediction.

Revised: Fig.

Revised: 2a and 2b show the distribution of the train and test set, respectively.

Original: Conclusion
Revised: Their distributions are quite similar, making this problem a good candidate for a prediction model. Fig. 2b is the deployment set used for the ‘Ideal’ model. However, the distribution of the observed values using the ‘Oblivious’ training strategy shown in ﬁg. 2c has a different distribution from the train set which makes the prediction non-trivial.

Original:  In this study, we introduced an important healthcare application where we exhibit performativeprediction in time series.
Revised: [Figure]  Conclusion In this study, we introduced an important healthcare application where we exhibit performative prediction in time series.

Original: We described baseline methods to mitigate the problem of performativeprediction in time series for cancer symptom prediction.
Revised: We described baseline methods to provide robustness against the problem of performative prediction in time series for cancer symptom prediction.

Original: We observe that for some symptom levels,the model is not affected by performative prediction.
Revised: We observe that for some symptom values, the model is not affected by performative prediction.

Original: However, overall performative prediction is animportant feature of a problem setting that needs to be dealt with, otherwise, it can lead to higherlevels of error.
Revised: We characterise the extent that performative prediction can impact evaluation can be up to 40.7% increase in error values. Indeed, performative prediction is an important feature of the problem setting that needs to be dealt with.

Original: Future work may be required to formulate strategies that are more robust to thisproblem.
Revised: Future work may be required to formulate strategies that are more robust to this problem.

Original: We hope that our work will elicit discussion of this interesting problem, as machine learningmodels become more ubiquitously used to guide decisions by policymakers.
Revised: We hope that our work will elicit discussion of this interesting problem, as machine learning models become more ubiquitously used to guide decisions by policymakers.

Original: A Appendix
Revised: Acknowledgments and Disclosure of Funding

Original: Visual Representation of the Distribution shift Fig.

Original: 2 shows the change in distribution due toperformative prediction.
Revised: We would like to thank Mehrnaz Mofakhami and Joey Bose for our insightful discussions.

Original: The distribution of the train and deployment set are quite similar, makingthis problem a good candidate for a prediction model. This is the deployment set used for the ‘’Ideal’model. However, the distribution of the observed values using the ’Oblivious’ training strategy shownin the right-most column of the ﬁgure has a different distribution from the train set.
Revised: We would also like to thank CIFAR for the CCAI Chair funding and CIHR CCS #706699 and CIHR CAS #168602.