RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Claim: In contrast, models such as Qwen-2 or Gemma-1.1 tend to under-cite in their response.
Context: Figure 3 shows lollipop plots indicating the average heuristic-feature value ( y -axis) distribution across all languages ( x -axis). In English detection, smaller LLMs such as Gemma-1.1 (2B) do not generate output in the required target language, but rather rely on English. Next, for citation quality and support evaluation, OpenAI models achieve better Recall @10 and Entailment scores (except Llama-3 (70B) for a few languages), indicating baseline responses include grounded citations from relevant passages.
Caption: Figure 3: Lollipop plots denoting the average heuristic-based feature scores achieved by baselines in \mirage . x -axis denotes the languages in \mirage. whereas y -axis plots every heuristic feature value. models in the same family are represented as the same color in a lollipop (as multiple circles). Figure 9 provides lollipop plots for all eleven heuristic-based features.
Claim: Moreover, this leads to improved estimation of latent dynamics, a switching rate that agrees with the true system, and improved multistep inference performance as shown in Table 1 .
Context: In figure 2 D, we see that rSLDS does not distinguish between the different speeds along the outer and inner sections of the attractor. Instead, the discrete states obscure the continuum of speeds by incorrectly grouping all activity in each lobe into a single regime. Furthermore, we observe that dLDS is limited without an offset term, unable to accurately represent multiple fixed points. Instead of aligning with the two attractor lobes, transitions in the dominant coefficients occur radially relative to the origin and fail to reconstruct the two orbiting fixed points. Conversely, p-dLDS’s offset term enables learning a system where coefficients better match the true geometry. This representation correctly recovers differences between the outer and inner sections of the attractor while also accurately reconstructing the two orbiting fixed points.
Caption: Table 1: Metrics for synthetic dynamical systems. Bold means best performance. (\uparrow) indicates higher score is better while (\downarrow) indicates that lower is better. ✗ indicates that value diverged towards -\infty . Switch events for decomposed models are defined as times where the active set of DOs change from the previous time step.