Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies the task of finding optimal intervention time for failures. The problem can be viewed abstractly as a sequence of stochastically generated features. A failure happens at some point for this task, and the goal is to intervene optimally. This is done by a policy that either allows the process to run or decides to intervene at a given time. Optimal intervention is framed as deciding to intervene such that one does not intervene too early before the actual failure event, while also ensuring that the probability of late intervention (i.e., intervention after the failure) is below a threshold. 

Two processes are then defined: a survival process and a hazard process. The survival process models the probability that failure will happen after a certain time from the current time. The hazard process $h_j(k)$ models the probability that failure will happen at a given time $j+k$ in the future when the current time is $j$. The optimal intervention time problem is then framed as finding the optimal stopping problem on the survival/hazard process. Under assumption that features evolve using Markovian dynamics, the optimal stopping problem can be solved. 

Experiments are presented on two datasets and compare with two baselines. The model used for prediction is a  Recurrent Neural Network that converts a history of features $\{X_1, \cdots, X_j\}$ into a vector representation. This vector representation is used by a decoder to generate a sequence of vectors $\{Z_{jk}\}_{k}$ that is then directly converted to the hazard rates $h_j(k)$. Experiments show that broadly the new approach outperforms the baselines on both dataset.

 
strengths_and_weaknesses: **Strength:**
1. Problem is relevant to machine learning community from both theoretical and real-world impact
2. Solution is novel and well-motivated
3. Paper is mostly well-written

**Weakness**
1. The paper can use more details on the experiments. E.g., how the RNN model is used to make predictions will be useful and how is the RNN model trained, and are the equations Theorem 3.1 solved exactly or approximately? Details of computational aspects can also be useful since if the approach takes a long time to predict then it may be not useful for real-world applications where intervention must be done in real-time. 

2. Discussion of Theorem 3 would be useful. Perhaps the other theorem can be removed to make space to discuss theorem 3 in more detail. E.g., a proof sketch can be provided, or a discussion of the theoretical results.
questions: 1. How is the DRSA loss computed? Is my understanding correct that the model is trained using DRS, then kept fixed, and used to solve equations in theorem 3 to find the optimal stopping time? As mentioned earlier, these experimental details would be helpful.

2. How will be $C_\alpha$ set in practice. This hyperparameter is less intuitive than $\alpha$ which might be easy to set based on domain knowledge.

3. Is there any particular reason why $(L-\tau)^+$ loss is used instead of square loss or any other loss?

4. Why is TTE seem to be doing better than OTI for large values of $C$ in Figure 3a?

Minor presentation:

1. "Corrolary" spelling error on Line 294. It should be "corollary" 

2. Space in "Allmodels" on Line 320

3. In Figure 1, the sub-captions for the two figures seem to be wrong. 
limitations: N/A
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper studies the problem of deciding when to intervene and “stop” a stochastic process before it transitions to a critical event. A new problem setting is defined called optimally timed intervention (OTI), where the goal is to minimize the expected time between intervention and the event L under the constraint that the probability of missing is small. An optimal policy is derived through the introduction of survival and hazard processes. The theory is then used to motivate an RNN approach to predict these processes. The method is evaluated on real-world data and appears to marginally outperform a baseline and significantly outperform a naive strategy.
strengths_and_weaknesses: Strengths
- This is not within my area so I cannot comment on the significance compared to related work, but the paper seems to do a good job differentiating itself.
- The OTI seems to be an interesting and well-motivated problem to study (again I can not really comment on novelty)
- The theory about computation of the optimal policies given the known survival/hazard rates is also fairly comprehensive.
- The experiments are promising, suggesting that the theory and insights developed in the paper have strong practical implications.

Weaknesses
- It doesn’t seem that it is ever described formally what the relationship is between the covariates of the process and the event time L. All that is known in the problem definition is that L is some random variable and a stopping time. Nevertheless in the experiments this is implicitly posed as a supervised learning problem where the covariates of the stochastic process are somehow predictive of L. While this makes intuitive sense and seems to be sufficient for the experiments, I think it would be helpful to make this more rigorous so that we can reason about familiar characteristics of statistical learning problems like realizability, noise, etc. In general the shift from the OTI theory to the experiments is a bit abrupt and I think it would have been helpful to explain in math more clearly how that jump is made. Some of these details are in the appendix already, but it would be helpful to bolster the formalization and have them in the main paper.
- It doesn’t seem that one can “ask” for a specific probability of failure (missing L) and then pay the price of this ask in the expected time to intervene, like one would expect to see in many similar learning/decision problems. Instead this is controlled implicitly by $C_{\alpha}$ which is a hyperparameter that must be set.
- I think that the experiments could be a lot more compelling if there were a plot showing the trade-off between the probability of failure and the expected time to intervene so that one could see visually what shape this trade-off curve takes (by setting different values of $C_{\alpha}$). Perhaps this is not feasible with the real-world data. However, even synthetic experiments could suffice to exhibit this.

questions: - Is it true that the original problem can be equivalently formulated as a standard optimal stopping problem (before even considering the survival and hazard processes)? In that case, what prevents one from directly using existing tools from optimal stopping and how would they compare with the optimal policy derived in this paper through the survival/hazard processes?
- Can the authors clarify the novelty of the survival and hazard processes? These seem to be presented as new ideas, but also the paper by Ren et al also seems to use the hazard process.

limitations: Limitations were addressed.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper studies the problem of triggering stopping interventions in dynamical systems before a critical event occurs, named the optimally timed intervention problem (OTI). The authors formulate this problem as that of minimizing the expected time to event (remaining time lost when the intervention is triggered before the critical event), while also minimizing the expected cost of an event happening (fixed cost when no intervention is triggered before the critical event). They show that two popular approaches, window-based intervention (WBI) and time-to-event (TTE), are sub-optimal for OTI, and they propose a new approach similar to TTE but with dynamic thresholds, which they show is optimal. They conduct experiments on two datasets which also support the superiority of their approach.
strengths_and_weaknesses: ### Originality

The proposed approach seems original, although I am not very familiar with the OTI problem.

### Quality

While the paper appears quite technical, I found the technical quality rather poor. The mathematical definitions used in the paper are loose and ambiguous at times, which affects readability and also brings doubt about the rigor of the presented results. Some parts of the reasoning are missing, such as how the optimal intervention $\tau^\star$ (Theorem 3.1, Corollaries 3.1.1 and 3.1.2) translates into an actual policy, or what the authors mean by a stationary process or a stationary policy. The experimental setup seems ok, but the experimetns are lacking a bit in size (only 2 datasets, only 5 and 3 values respectively for the cost $C$ for each dataset), and in presentation (no error bars or standard deviation).

### Clarity

I found the paper hard to read, due to several ambiguities in the notations and definitions. I also found the presentation of the proposed approach unnecessarily heavy and redundant. The authors derive their method twice from two equivalent concepts: the hazard rate and the survival function. I believe the presentation could be greatly simplified. Finally, the paper is missing a conclusions section.

### Significance

The presented analysis and approach might be significant to the OTI community. Still, the technical quality and presentation of the results is too poor to correctly assess significance in my opinion.
questions: I would like the authors to clarify the following points:
 - how does your derivation of the optimal intervention $\tau^\star$ translate into an optimal policy ?
 - what is a stationary process ? A stationary policy ?
 - given that any Markov process can be made homogeneous by including the index $j$ into each observation $X_j$, what is the point of considering non-homogeneous processes ?

Detailed comments:

p1 l21: networks(RNNs) -> typo

p2 l38-44: This whole description is a bit vague. "The critical event occurs" -> So there can only be one such event ? "The sequence of observations terminates after the occurrence of the critical event" -> what if there is no critical event ? Is there always a critical event ? If proper interventions happen at the right time, can't the system be kept alive for ever ? It seems what you describe does not allow that. "An intervention is considered late [...] if no intervention occurs" -> you say a late intervention is when there is no intervention... This sentence is contradictory by itself. I'd suggest to rework and clarify your setup description.

p2 l47: $\mathcal{X}_j \to \\{0,1\\}$ -> this notation is inconsistent. Earlier you define $\mathcal{X}_j$ as a single point in $\mathbb{R}^{j \times d}$. Now you use the same notation to depict the input space of a function.

p2 l53: again, the math here is quite loose and ambiguous. The index $j$ is defined twice in the left-hand-side and right-hand-side of the equation. On the left $j$ is given and constant, on the right $j$ is a free variable in a minimization problem. Are $\mathcal{X}_j$ on the left and on the right the same objects ? If not, how can $\tau$ be a function of $\mathcal{X}_j$ ? Also, what about the case where there is no intervention ? If the process reaches a terminal state $X_L$ without any intervention, then which value does the policy stopping time take ? Finally, the random variable $\tau$ does not depend only on a history at time $j$, it depends on the entire realization of the stochastic process and the policy $\phi$. Expressing $\tau$ as a function of $\mathcal{X}_j$ is really misleading.

p2 l58-60: This definition of OTI seem arbitrary. A motivation is missing, as to why such an objective makes sense in practice, or a reference to the literature if this objective is commonly accepted.

p2 eq1: Again, this equation is ill-defined. First, what is $L_\textit{max}$ ? What is $(L-\tau)^+$ ? These are not defined. Second, why isn't this  expression conditioned on the history $\mathcal{X}_j$ ? As it is written, equation (1) suggests that you are looking for a universal optimal time to intervene, $\tau^\star$, regardless of the current realization of the process. Surely this is sub-optimal, and at any time step $j$ the decision to intervene or not should be conditioned on the history $\mathcal{X}_j$, which can reduce the uncertainty about $L$.

p2 l67:  can be directly be -> typo

p3 l120: "system evolution towards a critical event is a non-stationary in general" -> I hardly get the meaning of that statement. Do you mean that your training or testing data are not i.i.d. realizations of random variables ($X_j$, $L$) ? Do you mean that each random variable $X_j$ follows a different distribution ? Is stationarity equivalent to homogeneousity ? A proper definition of stationarity is missing here.

p4 eq2: According to your setup description in Section 1.1, the existence of a history $\mathcal{X}_j$ implies that the process is not terminated yet, and hence $L>j$ is always true (or $L \geq j$, that is ambiguous from the definition of $L$). Thus I do not understand why the survival function is conditioned on $L>j$ in addition to $\mathcal{X}_j$ (or why the hazard rate is conditioned on $K \geq j$), as it does not bring any additional information.

p4 l160: From eq (2) I even have $S_j(0)=1$.

p4 l164: Again, $L_\textit{max}$ has not been defined yet.

p4 prop2.1: It seems that the notions of survival process and hazard rate process can be used interchangeably to define your OTI problem. I'd suggest to choose and stick to one of the two notions, this would simplify the manuscript a lot.

p5 eq5: I think the definition is missing $\forall j$. Otherwise this statement does not make much sense.

p5 l186: Is homogeneousity your definition of stationarity ? Furthermore, it seems that any Markovian survival process can be made homogeneous trivially, by including the index $j$ into each observed random variable $X_j$. What is the point of considering non-homogeneous Markov processes then ?

p5 l189: risk process risk process -> typo

p5 eq6: It is unclear here how this random variable $\tau^\star$ is related to a decision rule. At each time step $j$, a decision rule has to decide whether to stop or not, given only information from the past ($\mathcal{X}_j$). In equation (6), the optimal stopping time depends on the entire realization of the process (every $j$ has to be considered, in the right-hand-side of the equation). A clarification is missing, as to how equation (6) translates into a proper decision rule.

p5 eq7: $\prod_{m=0}^k (1-h_j(l+k))$ -> the $m$ index does not appear inside the factors here. This product does not make sense.

p5 l193: the optimal policy is stationary -> what does it mean for a policy to be stationary ?

p6 l239: $\mathbf{H}_0)$ -> typo

p6 l241: "produce conditional hazard rate" -> there is a limitation here to how many hazard rates your model can output, which directly depends on the model architecture and in particular the width of the final dense layer. This limitation is not discussed.

p6 l241: "a novel sequential loss" -> a description of this novel loss function is missing.

p8 fig2: dynamicaly -> typo

p8 l287: If in the end you model everything with hazard rates, which can be equivalently represented as survival functions, then I really suggest that you use only one of the two concepts in the paper. I do not see the additional value of expressing all your results using both concepts each time. This really hinders the readability of the paper.

p8 l292: If the OTI policy can be equivalently expressed as an adaptive threshold on the estimated time-to-event, then why bothering introducing hazard rates and survival functions in the first place ? This make me feel like the presented theoretical results are stated in an overly complicated fashion.

p9 fig3: These figures are missing information about the variance, such as boxplots or standard deviation curves.

p9 l320: Allmodels -> typo
limitations: Limitations are discussed very briefly. I see several limitations that are missing in the discussion. How should the cost C be chosen in practice ? What is the affect of the size of the predicted hazard rate vector ? What is the computational complexity of the proposed approach at test time ?

### rebuttal update

I have been through the author's response to my comments. I appreciate that the authors have now substantially simplified the paper, notably by moving the analysis in terms of survival process (a somewhat duplicate of the analysis in terms of hazard rate process) to the appendix. I also note that the authors have fixed several mathematical typos and errors from the original manuscript that I had pointed to.

However, I have also been through the updated manuscript, and I still find the mathematical notations and definitions to lack somewhat loose and ambiguous. I could not check in detail the entire revised paper due to the amount of changes and the limited time, but I have concerns already after reading the first few pages:

 - p2 l41: but bounded time-step $L < L\_{max}$ in the series. -> typo ? but at a bounded time step ?

 - p2 l43: before the the policy stops -> typo. Moreover, I still find this whole definition rather vague. What does "the policy stops" mean ? How different is that from saying "before an intervention" ? Does the policy stop / intervention necessarily triggers the end of the time series ? After an intervention at time step $\\tau$, is there still a final observation $X\_{\\tau + 1}$ ? If $X\_{L+1}=0$ indicates the end of the time series, then is this specific value forbidden, in the domain of the random variables $X\_j$ ? Despite the changes made, this whole definition still appears rather sloppy to me, which is problematic for a theoretical paper.

 - p2 l44: we can take $\\tau=L+1$ -> At this point $\\tau$ has not been introduced yet. Also, why not considering that the policy stops immediately, with $\\tau=L$ ? This would seem to make more sense, given that you consider interventions at $\\tau=L$ too late by definition.

 - p2 l50: Definition 1.1 -> First a suggestion. Because $j$ is somehow embedded into $\\mathcal{X}\_j$ (it is the length of the history), why not simply defining your intervention policy as $\\varphi(\\mathcal{X}\_j)$, instead of a sequence of $j$-indexed functions ? This would simplify a lot an otherwise heavy notation. Second, I again find the math a bit sloppy here. Earlier you defined $\\mathcal{X}\_j$ as a random variable, but here in $\\{\\mathcal{X}\_j\\} \\to \\{0,1\\}$ you seem to use it as the input space of a mapping. I understand the intention, but again this is a rather loose notation. A random variable and its support are not the same thing.

 - p2 l53: this means that intervention triggers after the first trigger are ignored -> typo ?

Again, I am not an expert in the field of OTI, but at the very least I should be able to read the problem setup unambiguously. To me the overall technical quality of the paper remains below the NeurIPS threshold.

Taking all of that into account I am willing to increase my rating to 4, but my recommendation remains rejection.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper formulates and studies the problem of optimally timed intervention (OPI) for the first time. More specifically, the problem is formulated as minimizing the expected time-to-event while keeping the probability of missing the critical event below a certain threshold. The authors show that the OTI problem is equivalent to an optimal stopping problem on the underlying survival process. Other theoretical characterizations are also performed showing that the optimal decision rule at each time step can be analytically determined by the conditional survival function and that the optimal policy is non-stationary in general (so static threshold policies are often sub-optimal). The authors also propose a dynamic deep recurrent survival analysis (DDRSA) architecture. Experimental results on two real-world datasets show that the proposed method provides a better tradeoff between the average time to event and event miss rate compared to some baselines.
strengths_and_weaknesses: Strengths
- formulate the OTI problem and link it to both survival analysis and optimal stopping theory for the first time, deriving the analytical expression of the optimal intervention 
- propose a method based on recurrent neural networks to lead an OTI policy
- promising empirical results of the proposed method compared to some minimum baselines
- the paper seems well written and the contributions were easy to follow


Weaknesses
- it seems that the experiment focuses only on RNN rather than trying to show the flexibility of the proposed framework. I could not see any particular reason why it is better to focus on RNN even though the proposed framework in section 3 does not necessarily rely on a particular model  (at least in my current understanding)

questions: - Is the proposed OTI policy applicable only to the RNN model, or can it be more flexible? If it is more flexible, I thought it would be interesting to see if the proposed framework can work with a range of models to estimate the hazard rate function rather than focusing only on RNN. 

- the OTI policy is expressed with the true hazard rate function for now. Would it be possible to theoretically characterize the accuracy of the function estimation and the performance of the resulting policy? In other words, how does the error of the function estimation propagate to the performance of the intervention policy?

limitations: I was unsure why it is necessary to focus only on the RNN model, even though the high-level OTI policy framework seems to be more flexible in my understanding. Then, it would be more comprehensive to show this flexibility in the experiment rather than focusing only on a single model.
review_title: null

Meta-review: metareview: The reviewers all appreciate the direction of this work, and while the merits and significance of the work have limitations -- as reflected in the weak scores -- all reviewers were positive and found no reason to reject. I agree with this and recommend acceptance on the basis that the merits of the contribution and having this as part of the program outweigh the concerns regarding significance. That said, I strongly encourage the authors to use the detailed feedback from the reviewers to improve their paper.
confidence: Less certain
Decision: Accept