Miracle-dz/UniFAR-FTUs huggingface dataset:
Deal, or no deal (or who knows)? Forecasting Uncertainty in Conversations using Large Language Models

Effective interlocutors account for the uncertain goals, beliefs, and emotions of others. But even the best human conversationalist cannot perfectly anticipate the trajectory of a dialogue. How well can language models represent inherent uncertainty in conversations? We propose FortUne Dial, an expansion of the long-standing"conversation forecasting"task: instead of just accuracy, evaluation is conducted with uncertainty-aware metrics, effectively enabling abstention on individual instances. We study two ways in which language models potentially represent outcome uncertainty (internally, using scores and directly, using tokens) and propose fine-tuning strategies to improve calibration of both representations. Experiments on eight difficult negotiation corpora demonstrate that our proposed fine-tuning strategies (a traditional supervision strategy and an off-policy reinforcement learning strategy) can calibrate smaller open-source models to compete with pre-trained models 10x their size.
Conditional cooperation with longer memory

Direct reciprocity is a wide-spread mechanism for evolution of cooperation. In repeated interactions, players can condition their behavior on previous outcomes. A well known approach is given by reactive strategies, which respond to the co-player's previous move. Here we extend reactive strategies to longer memories. A reactive-$n$ strategy takes into account the sequence of the last $n$ moves of the co-player. A reactive-$n$ counting strategy records how often the co-player has cooperated during the last $n$ rounds. We derive an algorithm to identify all partner strategies among reactive-$n$ strategies. We give explicit conditions for all partner strategies among reactive-2, reactive-3 strategies, and reactive-$n$ counting strategies. Partner strategies are those that ensure mutual cooperation without exploitation. We perform evolutionary simulations and find that longer memory increases the average cooperation rate for reactive-$n$ strategies but not for reactive counting strategies. Paying attention to the sequence of moves is necessary for reaping the advantages of longer memory.
With a Little Help from my (Linguistic) Friends: Topic segmentation of multi-party casual conversations

Topics play an important role in the global organisation of a conversation as what is currently discussed constrains the possible contributions of the participant. Understanding the way topics are organised in interaction would provide insight on the structure of dialogue beyond the sequence of utterances. However, studying this high-level structure is a complex task that we try to approach by first segmenting dialogues into smaller topically coherent sets of utterances. Understanding the interactions between these segments would then enable us to propose a model of topic organisation at a dialogue level. In this paper we work with open-domain conversations and try to reach a comparable level of accuracy as recent machine learning based topic segmentation models but with a formal approach. The features we identify as meaningful for this task help us understand better the topical structure of a conversation.
Self-Supervised Contrastive Learning for Long-term Forecasting

Long-term forecasting presents unique challenges due to the time and memory complexity of handling long sequences. Existing methods, which rely on sliding windows to process long sequences, struggle to effectively capture long-term variations that are partially caught within the short window (i.e., outer-window variations). In this paper, we introduce a novel approach that overcomes this limitation by employing contrastive learning and enhanced decomposition architecture, specifically designed to focus on long-term variations. To this end, our contrastive loss incorporates global autocorrelation held in the whole time series, which facilitates the construction of positive and negative pairs in a self-supervised manner. When combined with our decomposition networks, our contrastive learning significantly improves long-term forecasting performance. Extensive experiments demonstrate that our approach outperforms 14 baseline models in multiple experiments over nine long-term benchmarks, especially in challenging scenarios that require a significantly long output for forecasting. Source code is available at https://github.com/junwoopark92/Self-Supervised-Contrastive-Forecsating.
A theta operator for the group $\mathrm{GSp}_4$

We construct a differential operator on sheaves of $p$-adic modular forms defined over the locus of $p$-rank $\ge 1$ of the Siegel threefold, by applying a revisited version of the approach that Sean Howe recently introduced in his paper"A unipotent circle action on $p$-adic modular forms"(2020, Trans. Am. Math. Soc.) to construct the theta operator in the elliptic case.
Flexible Non-interactive Short-term Implicit Certificate Generation for VANETs

A leading industry standard for secure and trusted communication in vehicular ad-hoc networks (VANETs) is the Security Credential Management System (SCMS). It uses anonymous certificates, functioning as pseudonyms, to preserve the privacy of vehicles. With the rapid development of advanced applications in VANETs, such as crowdsensing and federated learning, vehicles need to communicate with each other or infrastructures more frequently, leading to a higher demand for pseudonyms. However, the current approach of certificate provisioning in SCMS is not able to fully support pseudonyms, due to storage limitation, cost of connectivity establishment, and communication overhead of certificate downloading. To tackle this challenge, we propose a non-interactive approach for SCMS, allowing vehicles themselves to generate short-term key pairs and anonymous implicit certificates. Our evaluation and comparison with previous work show that our solution not only effectively reduces the communication cost, but also grants vehicles greater flexibility in certificate generation and use. On the technical side, to the best of our knowledge, this is the first work which (1) applies sanitizable signature for non-interactive anonymous certificate generation, and (2) is specifically designed for SCMS, which opens up possibilities for extensions and applications in industry.
BECLR: Batch Enhanced Contrastive Few-Shot Learning

Learning quickly from very few labeled samples is a fundamental attribute that separates machines and humans in the era of deep representation learning. Unsupervised few-shot learning (U-FSL) aspires to bridge this gap by discarding the reliance on annotations at training time. Intrigued by the success of contrastive learning approaches in the realm of U-FSL, we structurally approach their shortcomings in both pretraining and downstream inference stages. We propose a novel Dynamic Clustered mEmory (DyCE) module to promote a highly separable latent representation space for enhancing positive sampling at the pretraining phase and infusing implicit class-level insights into unsupervised contrastive learning. We then tackle the, somehow overlooked yet critical, issue of sample bias at the few-shot inference stage. We propose an iterative Optimal Transport-based distribution Alignment (OpTA) strategy and demonstrate that it efficiently addresses the problem, especially in low-shot scenarios where FSL approaches suffer the most from sample bias. We later on discuss that DyCE and OpTA are two intertwined pieces of a novel end-to-end approach (we coin as BECLR), constructively magnifying each other's impact. We then present a suite of extensive quantitative and qualitative experimentation to corroborate that BECLR sets a new state-of-the-art across ALL existing U-FSL benchmarks (to the best of our knowledge), and significantly outperforms the best of the current baselines (codebase available at: https://github.com/stypoumic/BECLR).
Closed-Loop Unsupervised Representation Disentanglement with β-VAE Distillation and Diffusion Probabilistic Feedback

Representation disentanglement may help AI fundamentally understand the real world and thus benefit both discrimination and generation tasks. It currently has at least three unresolved core issues: (i) heavy reliance on label annotation and synthetic data -- causing poor generalization on natural scenarios; (ii) heuristic/hand-craft disentangling constraints make it hard to adaptively achieve an optimal training trade-off; (iii) lacking reasonable evaluation metric, especially for the real label-free data. To address these challenges, we propose a \textbf{C}losed-\textbf{L}oop unsupervised representation \textbf{Dis}entanglement approach dubbed \textbf{CL-Dis}. Specifically, we use diffusion-based autoencoder (Diff-AE) as a backbone while resorting to $\beta$-VAE as a co-pilot to extract semantically disentangled representations. The strong generation ability of diffusion model and the good disentanglement ability of VAE model are complementary. To strengthen disentangling, VAE-latent distillation and diffusion-wise feedback are interconnected in a closed-loop system for a further mutual promotion. Then, a self-supervised \textbf{Navigation} strategy is introduced to identify interpretable semantic directions in the disentangled latent space. Finally, a new metric based on content tracking is designed to evaluate the disentanglement effect. Experiments demonstrate the superiority of CL-Dis on applications like real image manipulation and visual analysis.
Revisiting the role of cosmic-ray driven Alfv\'en waves in pre-existing magnetohydrodynamic turbulence. I. Turbulent damping rates and feedback on background fluctuations

Alfv\'en waves (AWs) excited by the cosmic-ray (CR) streaming instability (CRSI) are a fundamental ingredient for CR confinement. The effectiveness of self-confinement relies on a balance between CRSI growth rate and damping mechanisms acting on quasi-parallel AWs excited by CRs. One relevant mechanism is the so-called turbulent damping, in which an AW packet injected in pre-existing turbulence undergoes a cascade process due to its nonlinear interaction with fluctuations of the background. The turbulent damping of an AW packet in pre-existing magnetohydrodynamic turbulence is re-examined, revised, and extended to include most-recent theories of MHD turbulence that account for dynamic alignment and reconnection-mediated regime. The case in which the role of feedback of CR-driven AWs on pre-existing turbulence is important will also be discussed. Particular attention is given to the nonlinearity parameter $\chi^w$ that estimates the strength of nonlinear interaction between CR-driven AWs and background fluctuations. We point out the difference between $\chi^w$ and $\chi^z$ that instead describes the strength of nonlinear interactions between pre-existing fluctuations. When $\chi^w$ is properly taken into account, one finds that (i) the turbulent damping rate of quasi-parallel AWs in anisotropic turbulence depends on the background-fluctuations' amplitude to the third power, hence is strongly suppressed, and (ii) the dependence on the AW's wavelength (and thus on the CR gyro-radius from which it is excited) is different from what has been previously obtained. Finally, (iii) when dynamic alignment of cascading fluctuations and the possibility of a reconnection-mediated range is included in the picture, the turbulent damping rate exhibits novel regimes and breaks. Finally, a criterion for CR-feedback is derived and simple phenomenological models of CR-modified turbulent scaling are provided.
Penalty-based Methods for Simple Bilevel Optimization under H\"{o}lderian Error Bounds

This paper investigates simple bilevel optimization problems where the upper-level objective minimizes a composite convex function over the optimal solutions of a composite convex lower-level problem. Existing methods for such problems either only guarantee asymptotic convergence, have slow sublinear rates, or require strong assumptions. To address these challenges, we develop a novel penalty-based approach that employs the accelerated proximal gradient (APG) method. Under an $\alpha$-H\"{o}lderian error bound condition on the lower-level objective, our algorithm attains an $(\epsilon,l_F^{-\beta}\epsilon^{\beta})$-optimal solution for any $\beta>0$ within $\mathcal{O}\left(\sqrt{\frac{L_{f_1}}{\epsilon }}\right)+\mathcal{O}\left(\sqrt{\frac{l_F^{\max\{\alpha,\beta\}}L_{g_1}}{\epsilon^{\max\{\alpha,\beta\}}}}\right)$ iterations, where $l_F$, $L_{f_1}$ and $L_{g_1}$ denote the Lipschitz constants of the upper-level objective, the gradients of the smooth parts of the upper- and lower-level objectives, respectively. If the smooth part of the upper-level objective is strongly convex, the result improves further. We also establish the complexity results when both upper- and lower-level objectives are general convex nonsmooth functions. Numerical experiments demonstrate the effectiveness of our algorithms.
State estimation of urban air pollution with statistical, physical, and super-learning graph models

We consider the problem of real-time reconstruction of urban air pollution maps. The task is challenging due to the heterogeneous sources of available data, the scarcity of direct measurements, the presence of noise, and the large surfaces that need to be considered. In this work, we introduce different reconstruction methods based on posing the problem on city graphs. Our strategies can be classified as fully data-driven, physics-driven, or hybrid, and we combine them with super-learning models. The performance of the methods is tested in the case of the inner city of Paris, France.
A free-space coupled, large-active-areasuperconducting microstrip single-photon detectorfor photon-counting time-of-flight imaging

Numerous applications at the photon-starved regime require a free-space coupling singlephoton detector with a large active area, low dark count rate (DCR), and superior time resolutions. Here,we developed a superconducting microstrip single-photon detector (SMSPD), with a large active area of 260 um in diameter, a DCR of ~5 kcps, and a low time jitter of ~171 ps, operated at near-infrared of 1550 nm. As a demonstration, we applied the detector to a single-pixel galvanometer scanning system and successfully reconstructed object information in depth and intensity using a time-correlated photon counting technology.
HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification

Hierarchical text classification (HTC) is a complex subtask under multi-label text classification, characterized by a hierarchical label taxonomy and data imbalance. The best-performing models aim to learn a static representation by combining document and hierarchical label information. However, the relevance of document sections can vary based on the hierarchy level, necessitating a dynamic document representation. To address this, we propose HiGen, a text-generation-based framework utilizing language models to encode dynamic text representations. We introduce a level-guided loss function to capture the relationship between text and label name semantics. Our approach incorporates a task-specific pretraining strategy, adapting the language model to in-domain knowledge and significantly enhancing performance for classes with limited examples. Furthermore, we present a new and valuable dataset called ENZYME, designed for HTC, which comprises articles from PubMed with the goal of predicting Enzyme Commission (EC) numbers. Through extensive experiments on the ENZYME dataset and the widely recognized WOS and NYT datasets, our methodology demonstrates superior performance, surpassing existing approaches while efficiently handling data and mitigating class imbalance. We release our code and dataset here: https://github.com/viditjain99/HiGen.
A Massive Protocluster Anchored by a Luminous Quasar at z = 6.63

Protoclusters, the progenitors of galaxy clusters, trace large scale structures in the early Universe and are important to our understanding of structure formation and galaxy evolution. To date, only a handful of protoclusters have been identified in the Epoch of Reionization. As one of the rarest populations in the early Universe, distant quasars that host active supermassive black holes are thought to reside in the most massive dark matter halos at that cosmic epoch and could thus potentially pinpoint some of the earliest protoclusters. In this Letter, we report the discovery of a massive protocluster around a luminous quasar at z = 6.63. This protocluster is anchored by the quasar and includes three [C ii] emitters at z ∼ 6.63, 12 spectroscopically confirmed Lyα emitters (LAEs) at 6.54 < z ≤ 6.64, and a large number of narrow-band-imaging selected LAE candidates at the same redshift. This structure has an overall overdensity of δ = 3.3 − 0.9 + 1.1 within ∼35 × 74 cMpc2 on the sky and an extreme overdensity of δ > 30 in its central region (i.e., R ≲ 2 cMpc). We estimate that this protocluster will collapse into a galaxy cluster with a mass of 6.9 − 1.4 + 1.2 × 10 15 M ⊙ at the current epoch, more massive than the most massive clusters known in the local Universe such as Coma. In the quasar vicinity, we discover a double-peaked LAE, which implies that the quasar has a UV lifetime greater than 0.8 Myrs and has already ionized its surrounding intergalactic medium.
Code Representation Learning At Scale

Recent studies have shown that code language models at scale demonstrate significant performance gains on downstream tasks, i.e., code generation. However, most of the existing works on code representation learning train models at a hundred million parameter scale using very limited pretraining corpora. In this work, we fuel code representation learning with a vast amount of code data via a two-stage pretraining scheme. We first train the encoders via a mix that leverages both randomness in masking language modeling and the structure aspect of programming language. We then enhance the representations via contrastive learning with hard negative and hard positive constructed in an unsupervised manner. We establish an off-the-shelf encoder model that persistently outperforms the existing models on a wide variety of downstream tasks by large margins. To comprehend the factors contributing to successful code representation learning, we conduct detailed ablations and share our findings on (i) a customized and effective token-level denoising scheme for source code; (ii) the importance of hard negatives and hard positives; (iii) how the proposed bimodal contrastive learning boost the cross-lingual semantic search performance; and (iv) how the pretraining schemes decide the downstream task performance scales with the model size.
A Study of the Generalized Naming Game and Bayesian Naming Game as Dynamical Systems

We study the $\beta$-model ($\beta$-NG) and the Bayesian Naming Game (BNG) as dynamical systems. By applying linear stability analysis to the dynamical system associated with the $\beta$-model, we demonstrate the existence of a non-generic bifurcation with a bifurcation point $\beta_c = 1/3$. As $\beta$ passes through $\beta_c$, the stability of isolated fixed points changes, giving rise to a one-dimensional manifold of fixed points. Notably, this attracting invariant manifold forms an arc of an ellipse. In the context of the BNG, we propose modeling the Bayesian learning probabilities $p_A$ and $p_B$ as logistic functions. This modeling approach allows us to establish the existence of fixed points without relying on the overly strong assumption that $p_A = p_B = p$, where $p$ is a constant.
How do software practitioners perceive human-centric defects?

Context: Human-centric software design and development focuses on how users want to carry out their tasks rather than making users accommodate their software. Software users can have different genders, ages, cultures, languages, disabilities, socioeconomic statuses, and educational backgrounds, among many other differences. Due to the inherently varied nature of these differences and their impact on software usage, preferences and issues of users can vary, resulting in user-specific defects that we term as `human-centric defects' (HCDs). Objective: This research aims to understand the perception and current management practices of such human-centric defects by software practitioners, identify key challenges in reporting, understanding and fixing them, and provide recommendations to improve HCDs management in software engineering. Method: We conducted a survey and interviews with software engineering practitioners to gauge their knowledge and experience on HCDs and the defect tracking process. Results: We analysed fifty (50) survey- and ten (10) interview- responses from SE practitioners and identified that there are multiple gaps in the current management of HCDs in software engineering practice. There is a lack of awareness regarding human-centric aspects, causing them to be lost or under-appreciated during software development. Our results revealed that handling HCDs could be improved by following a better feedback process with end-users, a more descriptive taxonomy, and suitable automation. Conclusion: HCDs present a major challenge to software practitioners, given their diverse end-user base. In the software engineering domain, research on HCDs has been limited and requires effort from the research and practice communities to create better awareness and support regarding human-centric aspects.
Uncertainty-Aware Testing-Time Optimization for 3D Human Pose Estimation

Although data-driven methods have achieved success in 3D human pose estimation, they often suffer from domain gaps and exhibit limited generalization. In contrast, optimization-based methods excel in fine-tuning for specific cases but are generally inferior to data-driven methods in overall performance. We observe that previous optimization-based methods commonly rely on projection constraint, which only ensures alignment in 2D space, potentially leading to the overfitting problem. To address this, we propose an Uncertainty-Aware testing-time Optimization (UAO) framework, which keeps the prior information of pre-trained model and alleviates the overfitting problem using the uncertainty of joints. Specifically, during the training phase, we design an effective 2D-to-3D network for estimating the corresponding 3D pose while quantifying the uncertainty of each 3D joint. For optimization during testing, the proposed optimization framework freezes the pre-trained model and optimizes only a latent state. Projection loss is then employed to ensure the generated poses are well aligned in 2D space for high-quality optimization. Furthermore, we utilize the uncertainty of each joint to determine how much each joint is allowed for optimization. The effectiveness and superiority of the proposed framework are validated through extensive experiments on two challenging datasets: Human3.6M and MPI-INF-3DHP. Notably, our approach outperforms the previous best result by a large margin of 4.5% on Human3.6M. Our source code will be open-sourced.