RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Claim: We see that p-dLDS broadly outperforms existing methods in all metrics and significantly improves inference for decomposed models.
Evidence type: table
Caption: Table 1: Metrics for synthetic dynamical systems. Bold means best performance. (\uparrow) indicates higher score is better while (\downarrow) indicates that lower is better. ✗ indicates that value diverged towards -\infty . Switch events for decomposed models are defined as times where the active set of DOs change from the previous time step.
Context: Table 1 summarizes our quantitative evaluations on three metrics: 1) the mean squared error (MSE) between the learned and ground truth latent dynamics, 2) the MSE between the inferred and true switch rate to determine agreement of the discrete switching behavior,
and 3) the 100-step inference R^{2} to demonstrate that the learned system generalizes beyond a single step on held-out data. (See Appendix D for mathematical definitions).
Verdict: Supported
Claim: Moreover, this leads to improved estimation of latent dynamics, a switching rate that agrees with the true system, and improved multistep inference performance as shown in Table 1 .
Evidence type: table
Caption: Table 1: Metrics for synthetic dynamical systems. Bold means best performance. (\uparrow) indicates higher score is better while (\downarrow) indicates that lower is better. ✗ indicates that value diverged towards -\infty . Switch events for decomposed models are defined as times where the active set of DOs change from the previous time step.
Context: In figure 2 D, we see that rSLDS does not distinguish between the different speeds along the outer and inner sections of the attractor. Instead, the discrete states obscure the continuum of speeds by incorrectly grouping all activity in each lobe into a single regime. Furthermore, we observe that dLDS is limited without an offset term, unable to accurately represent multiple fixed points. Instead of aligning with the two attractor lobes, transitions in the dominant coefficients occur radially relative to the origin and fail to reconstruct the two orbiting fixed points. Conversely, p-dLDS’s offset term enables learning a system where coefficients better match the true geometry. This representation correctly recovers differences between the outer and inner sections of the attractor while also accurately reconstructing the two orbiting fixed points.
Verdict: Supported