Has the Deep Neural Network learned the Stochastic Process? An Evaluation Viewpoint

Authors: Harshit Kumar, Beomseok Kang, Biswadeep Chakraborty, Saibal Mukhopadhyay
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: evaluation, deep neural network, stochasticity, complex systems, forecasting

Abstract:
This paper presents the first systematic study of evaluating Deep Neural Networks (DNNs) designed to forecast the evolution of stochastic complex systems. We show that traditional evaluation methods like threshold-based classification metrics and error-based scoring rules assess a DNN's ability to replicate the observed ground truth but fail to measure the DNN's learning of the underlying stochastic process. To address this gap, we propose a new evaluation criteria called _Fidelity to Stochastic Process (F2SP)_, representing the DNN's ability to predict the system property _Statistic-GT_—the ground truth of the stochastic process—and introduce an evaluation metric that exclusively assesses F2SP. We formalize F2SP within a stochastic framework and establish criteria for validly measuring it. We formally show that Expected Calibration Error (ECE) satisfies the necessary condition for testing F2SP, unlike traditional evaluation methods. Empirical experiments on synthetic datasets, including wildfire, host-pathogen, and stock market models, demonstrate that ECE uniquely captures F2SP. We further extend our study to real-world wildfire data, highlighting the limitations of conventional evaluation and discuss the practical utility of incorporating F2SP into model assessment. This work offers a new perspective on evaluating DNNs modeling complex systems by emphasizing the importance of capturing underlying the stochastic process.

Decision: accept (poster)
Reviewer scores: 8, 6, 6, 8, 6

Reviews and discussion:
<thread>
creation date:
1729648510142

thread_type:
official_review

<thread_object>
creator: reviewer_iuwx

<content>
summary:
This paper introduces a novel stochasticity-compatible evaluation strategy for assessing existing models in the context of complex systems. The author justifies the Expected Calibration Error (ECE) as suitable for assessing the model fidelity of stochastic systems through both simulation environments and real-world data.

strengths:
1. Evaluating model fidelity on the stochastic system is significant and has wide applications.
2. The paper is well-motivated and both the dataset and experiments are thorough.

weaknesses:
1. Although the author attempts to explain the difference between their work and ECE in deep learning in Lines 282-288, it appears to me this work is still a direct application of using ECE to evaluate the model performance on a stochastic system. The author is encouraged to discuss more in-depth about the distinction between ECE in the proposed method (stochasticity comes from evolving in the environment, aka, Statistic-GT) and ECE in previous works (stochasticity comes from the output distribution).
2. In Lines 243-244, the author claims that Statistic-GT is more stable than classification-based metrics, but I could not find any evidence related to calculating ECE on Statistic-GT is less sensitive to the system variance than MSE. Is there any theoretical support for using ECE over MSE on stochastic systems with different noise levels and could the author clarify it a bit more?

questions:
I'm curious about how the author evaluates ECE at time $t$ based on Statistic-GT $P_{t}$. Do we have to simulate it again from $t=0$ for $N$ times or we can sample states from $t-1$ and go forward $N$ times (the system is Markov)? Can we still apply ECE on Statistic-GT when the system is not Markov?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730254258775

thread_type:
official_review

<thread_object>
creator: reviewer_hxsj

<content>
summary:
This work offers a new perspective on evaluating DNNs in stochastic complex systems by emphasizing  the importance of capturing underlying the stochastic process. Traditional evaluation methods assess the DNN’s ability to replicate the observed ground truth but fail to measure the DNN’s learning of the underlying stochastic process. This paper proposes a new property called Fidelity to Stochastic Process, representing the DNN’s ability to predict the ground truth of the stochastic process, and introduces an evaluation metric that exclusively assesses fidelity to  the ground truth of the stochastic process. The Expected Calibration Error is used to evaluate the fidelity to ground truth of statistic process. Empirical experiments on synthetic datasets (including wildfire, host-pathogen, and stock market models) and real-world wildfire data are used to show the measurement of fidelity to stochastic process by Expected Calibration Error.

strengths:
The paper offers a new perspective on evaluating DNNs by considering DNNs as stochastic processes and uses a widely used criteria in Bayesian Deep Learning application to assess the fidelity to stochastic process. This work clearly explains the Expected Calibration Error is used to assess DNN modes in three synthetic cases and one real world case.

weaknesses:
This paper is well organized and well written, several minor issues should be addressed: (1) The explaination of figures is not sufficient, e.g., in Figure 2 (1), the label for x-axis is not specified (I guess it is time?), either add a label or explain it in the captions. Same problems also exist in Figure 4. (2) This work examines ECE on three synthetic environments (forest fire, host-pathogen and stock market models) and a real world wildfire spread dataset. I can tell that these datasets are all multivariate either for classification or regression. Maybe due to the limit of pages, the authors didn't include the experiments on images. I suggest the authors add some discussions or comments in the paper.

questions:
As mentioned in the "Weaknesses" part.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730338026030

thread_type:
official_review

<thread_object>
creator: reviewer_snm1

<content>
summary:
This paper presents a study on evaluating deep neural networks designed to forecast the evolution of stochastic complex systems. The authors identify a gap in traditional evaluation methods—such as threshold-based classification metrics and error-based scoring rules—which focus on a model's ability to replicate observed ground truth but fail to assess how well the model has learned the underlying stochastic process. To address this issue, they introduce a new property called Fidelity to Stochastic Process, representing the DNN's ability to predict the statistical ground truth of the stochastic process.

The paper proposes using the Expected Calibration Error (ECE) as an evaluation metric that satisfies the necessary conditions for assessing fidelity to statistical ground truth. This work underscores the importance of capturing the underlying stochastic processes in deep neural networks  evaluations for complex systems.

strengths:
The paper makes a significant contribution by introducing the concept of Fidelity to Stochastic Process (F2SP), a novel evaluation criterion specifically designed to assess a DNN's ability to learn the underlying stochastic interactions in complex systems.

The authors provide a rigorous formalization of F2SP within a stochastic framework, establishing clear criteria for its valid measurement. The use of Expected Calibration Error (ECE) as an evaluation metric is well-justified.

weaknesses:
I found it hard to read the paper because there was a lack of consistency in the acronyms, the authors would redefine them in several parts of the text again and again. I addressed my comments on text in the questions section. 

In the tables, the best neural networks based on each criterion are not highlighted, which makes it difficult to the reader to infer and correlate the arguments in the text. I addressed my comments on text in the questions section. 

The focus of the paper is primarily on binary or discrete prediction tasks, leaving out regression tasks where the definition of calibration is more complex. While the authors acknowledge this and suggest it as an area for future work, the current scope limits the immediate applicability of the findings to a broader range of problems involving continuous outcomes.

Additionally, the use of the NDWS dataset, which is restricted to next-day predictions, prevents the assessment of ECE over longer time horizons, which are common in many complex systems. Could you elaborate on how future work might address this limitation? 

The paper highlights the lack of open-source complex system datasets as a barrier to broader validation. Are there any ongoing initiatives or plans to develop, collect, or standardize such datasets?

questions:
L50: Is --> is (lowercase)
Fig1: no need to write the whole name, you can use acronyms because they're already defined in the text, however MSE is not defined at this point.
L88: fidelity to realization --> F2R (it was already defined previously, so you can use the acronym)
L99: the notation of the dimension of the real vector O_t is confusing, what is (R^n)^(H x W), is n = H x W? If so, make that explicit.
Table 1: some rows end with full stop, other don't. Please make it consistent. Either all with or all without.
I find it odd to place Figures in columns as Figure 1 (which has a large top white margin) and Figure 3. I would suggest column figures into one row figure with multiple subfigures as you did with Figure 2. 
L201: Isn't the indicator variable already defined as B_t in L99? Why defining again with different notation?
L298: MSE already defined in text previously, no need to write the whole name again.
L516: ECE already defined in text previously, no need to write the whole name again.
Table 2 and Table 7: highlight the best performing DNNs.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730497423636

thread_type:
official_review

<thread_object>
creator: reviewer_7rfe

<content>
summary:
This paper presents a study evaluating deep neural networks (DNNs) within stochastic complex systems, emphasizing the importance of Expected Calibration Error (ECE) in measuring fidelity to stochastic processes. The findings are validated through multiple experiments and comparisons.

strengths:
The topic of evaluating DNNs within stochastic complex systems is both intriguing and important.

In the primary evaluations, the author conducted experiments across various settings, including different DNN architectures, comparisons with multiple evaluation metrics, and diverse simulation tasks.

The main text clearly explains the difference between ECE in classical assessment and stochastic process settings.

weaknesses:
The paper is somewhat difficult to follow. For example, providing a brief introduction to the structure of each section would enhance clarity, particularly in Sections 2 and 3. Additionally, it is difficult to grasp the main messages conveyed by the table in Figure 2(b). Furthermore, in lines 229–240, the macro-level concept is introduced abruptly, which may disrupt the clarity and readability of the main text.

The main findings' practical applicability appears limited. In real-world scenarios, data generally provides only a single observed outcome centered on observable ground truth (line 117). Since the primary evaluation is simulation-based, the controlled stochasticity falls short of capturing real-world complexity. The Statistic-GT is basically derived by normalizing the frequency of target state occurrences across multiple Monte Carlo simulations.

Minors:

M1. The original text for the abbreviation RV is not given.

M2. In Table 1, what about the possibility of recovery in the Host-Pathogen problem?

M3. In line 152, maybe consider using an alternative symbol for Moore neighborhood, instead of $\mathcal{N}$ (normally representing Gaussian distribution).

questions:
A question arises regarding Figure 1: Can ECE be an effective metric for measuring F2R compared to other available metrics? Figure 1 suggests that the answer may be *no*.

An important indicator that ECE is a reliable measure is its diagonal pattern, showing low scores only when training and test S-Levels align, as illustrated in Figure 4. Could the authors provide theoretical insights to support this indicator?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731034750282

thread_type:
official_review

<thread_object>
creator: reviewer_cbql

<content>
summary:
the metric of expected calibration error is introduced and studied as a way to capture fidelity of a learned representation to an underlying stochastic process (rather than a single realization of that process, as with typical metrics like AUC or MSE).

strengths:
Great paper, wonderfully practical and insightful; I've been looking for something like this for 5+ years! Nice eval on real-world data.
I started writing a thing I would like to you add and then discovered it was already in the paper (long horizon behaviour)

weaknesses:
While overall the paper is very clear, some of the captions and explanations of the experiments/insights from them and how they tie to the figures could be improved. 
Some specifics:
- first fig should say what you mean by realization, and F2R and F2SP should be bolded (not ital) to make them easy to find in the text. Observed GT should be explained a bit more, or maybe it would be enough to move the sentence currently after F2R to be the second sentence of the paragraph.
 - Fig 5 is unclear to me. What is the data, what is S-level, why is it "good" that the 20 vs 10 lines are far apart? All of this should be clear from the caption
 - the clarity wanes a bit as the paper goes on, and it's a bit confusing that you call it ECE vs. F2SP vs Statistic-GP. Do these different namings really serve something? It could be a lot more clear if you just have one naming.

questions:
- I don't understand the second part of the critical question, "is it encountering different stochastic behaviours" (different from what)? how is the "differentness" relevant?
- While it's pretty clear to me how to use this immediately in my work, I think anyone who wasn't already aware they wanted exactly this might struggle. Could you provide something like a "practical users guide" for non-domain experts?
 - if the clarity of the plots can be improved, the naming of the stat/metric you're introducing, and improve it's "usability" to the community, I would be happy to upgrade my score. You've done great work and this would bring the paper to the level it deserves.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732161686249

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Revisions

comment:
We thank the reviewers for their valuable feedback, which has significantly improved our paper. Peer review has been instrumental in clarifying and enhancing our findings, and we deeply appreciate the time and effort reviewers invested.

The reviewers acknowledged the paper's contribution to evaluating DNNs in stochastic complex systems, specifically the introduction of Fidelity to Stochastic Process (F2SP) and its rigorous formalization. They also noted the thorough experiments, the clear differentiation between classical and stochastic evaluation approaches, and the practical applicability demonstrated through real-world datasets.

We have revised the text in the paper, and the specific segments addressing reviewers’ comments have been highlighted in red. Key concerns and changes are summarized below. Revisions have been made in both the main paper and the Appendix.

---

## **Changes Made in the Main Paper**

### 1. **Practical Applicability and Usability**

**Pain Points**:  
Reviewers raised concerns about the clarity of the study’s practical applicability. There was also some confusion about whether Statistic-GT is required to measure ECE, raising questions about ECE’s applicability for testing F2SP in practical scenarios.

**Our Response**:  

- **Framework Visualization**: Redesigned Figure 1 into a cohesive evaluation framework that visually organizes metrics (AUC-PR, ECE, MSE), consolidating our insights.  
- **Practical User Guide**: Added a user guide in **Appendix H** with actionable steps for applying and interpreting the evaluation framework. This guide is cross-referenced in the main text and figures for easy navigation.  
- **Clarified Applicability**: Highlighted in the introduction and discussion sections that ECE can test F2SP using only the Observed-GT, demonstrating its practical applicability. To clarify, the concept of Statistic-GT was formalized in the paper to substantiate this claim.  

---

### 2. **Figures and Tables**

**Pain Points**:  
Figures and tables lacked clarity, with insufficient captions.

**Our Response**:  

- Added missing axis labels and expanded captions to ensure all figures are self-contained, understandable, and clearly convey their main messages, making it easier to correlate them with the text.  
- Merged the original Figures 1 and 3 into a **new Figure 1**, which provides a clearer visual overview of the evaluation framework, including:  
  - The two system properties, Observed-GT and Statistic-GT.  
  - The two evaluation criteria, F2R (Fidelity to Realization) and F2SP (Fidelity to the Stochastic Process).  
  - The evaluation metrics that measure these criteria.  
- Updated Figure 2.b to clarify its message by highlighting the different sources of randomness in the synthetic datasets used in our study.  
- Added color gradients to Table 2 to illustrate score transitions from good to bad and improve alignment with the text.  

---

### 3. **Theoretical Justification**

**Pain Points**:  
Need for theoretical support differentiating ECE and MSE with respect to sensitivity to system variance, explaining ECE’s behavior and its preference over MSE in certain contexts.

**Our Response**:  

- **Sensitivity Analysis**: Included ECE in the metric sensitivity analysis to macro-variance in **Appendix F.1**, demonstrating that while ECE and MSE exhibit similar sensitivity, their convergence asymptotes differ. Added Figure 17 to show that ECE's convergence behavior makes it better suited for testing fidelity to Statistic-GT compared to MSE. Revised discussions in Section 3.3 to cohesively connect these findings.  
- **Referenced Supporting Sections**: Updated figure captions and main text to directly reference sections providing theoretical insights, ensuring readers can easily locate and understand the explanations.  

---

### 4. **Scope and Limitations**

**Pain Points**:  
Limited discussion on extending the framework to image-based tasks.

**Our Response**:  
Expanded and revised the related works section (Section 6), **Appendix C.4**, and the future works section (Section 7) to discuss potential applications to vision tasks, such as segmentation map forecasting and stochastic video prediction.  

---

### 5. **Terminology and Notation Clarification**

**Pain Points**:  
Confusion around the terms ECE (Expected Calibration Error), F2SP, Statistic-GT, and their relationships.

**Our Response**:  

- Provided clear and explicit definitions of all key terms in the introduction.  
- Included a clarifying table in the updated Figure 1 to illustrate the distinctions between terms.  

---

### 6. **Clarity, Readability, and Minor Corrections**

**Pain Points**:  
Minor issues such as undefined abbreviations, inconsistent punctuation, notation errors, and formatting inconsistencies were also noted.

**Our Response**:  
We conducted a thorough sweep of the paper to address all these issues. Details of the revisions are provided in the individual responses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732606792567

thread_type:
official_comment

<thread_object>
creator: area_chair_vrrs

<content>
comment:
Dear all,

The deadline for the authors-reviewers phase is approaching (December 2).

@For reviewers, please read, acknowledge and possibly further discuss the authors' responses to your comments. While decisions do not need to be made at this stage, please make sure to reevaluate your score in light of the authors' responses and of the discussion.

- You can increase your score if you feel that the authors have addressed your concerns and the paper is now stronger.
- You can decrease your score if you have new concerns that have not been addressed by the authors.
- You can keep your score if you feel that the authors have not addressed your concerns or that remaining concerns are critical.

Importantly, you are not expected to update your score. Nevertheless, to reach fair and informed decisions, you should make sure that your score reflects the quality of the paper as you see it now. Your review (either positive or negative) should be based on factual arguments rather than opinions. In particular, if the authors have successfully answered most of your initial concerns, your score should reflect this, as it otherwise means that your initial score was not entirely grounded by the arguments you provided in your review. Ponder whether the paper makes valuable scientific contributions from which the ICLR community could benefit, over subjective preferences or unreasonable expectations.

@For authors, please respond to remaining concerns and questions raised by the reviewers. Make sure to provide short and clear answers. If needed, you can also update the PDF of the paper to reflect changes in the text. Please note however that reviewers are not expected to re-review the paper, so your response should ideally be self-contained.

The AC.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The reviewers unanimously recommend acceptance (8-6-6-8-6). The paper presents a significant contribution for the evaluation of neural networks designed to forecast the evolution of stochastic complex systems. Reviewers recognize the importance of the work and the quality of the results. The author-reviewer discussion has been constructive and has led to a number of improvements to the paper, in particular regarding its presentation. The reviewers have raised some concerns about the practical applicability of the findings (e.g., the focus on binary prediction tasks, the limited horizon in the experiments), but the authors have provided convincing arguments arguing for the relevance of their work nonetheless. No major concerns have been raised by the reviewers. For these reasons, I recommend acceptance. I encourage the authors to implement the changes discussed with the reviewers in the final version of the paper.