Miracle-dz/UniFAR-FTUs huggingface dataset:
Permafrost creep and rock glacier dynamics

This review paper examines thermal conditions (active layer and permafrost), internal composition (rock and ice components), kinematics and rheology of creeping perennially frozen slopes in cold mountain areas. The aim is to assemble current information about creep in permafrost and rock glaciers from diverse published sources into a single paper that will be useful in studies of the flow and deformation of subsurface ice and their surface manifestations not only on Earth, but also on Mars. Emphasis is placed on quantitative information from drilling, borehole measurements, geophysical soundings, photogrammetry, laboratory experiments, etc. It is evident that quantitative holistic treatment of permafrost creep and rock glaciers requires consideration of: (a) rock weathering, snow avalanches and rockfall, with grain‐size sorting on scree slopes; (b) freezing processes and ice formation in scree at sub‐zero temperatures containing abundant fine material as well as coarse‐grained blocks; (c) coupled thermohydro‐mechanical aspects of creep and failure processes in frozen rock debris; (d) kinematics of non‐isotropic, heterogeneous and layered, ice‐rich permafrost on slopes with long transport paths for coarse surface material from the headwall to the front and, in some cases, subsequent re‐incorporation into an advancing rock glacier causing corresponding age inversion at depth; and (e) the dynamics of rock glaciers, which include spatial and temporal variations in velocity that are manifested in the ridges, furrows and other surface structures typical of rock glaciers, as well as their down‐valley motion. Copyright © 2006 John Wiley & Sons, Ltd.
HierCas: Hierarchical Temporal Graph Attention Networks for Popularity Prediction in Information Cascades

Information cascade popularity prediction is critical for many applications, including but not limited to identifying fake news and accurate recommendations. Traditional feature-based methods heavily rely on handcrafted features, which are domain-specific and lack generalizability to new domains. To address this problem, researchers have turned to neural network-based approaches. However, existing methods follow a sampling-based modeling approach, potentially losing continuous dynamic information and structural-temporal dependencies that emerge during the information diffusion process. In this paper, we propose a novel framework called Hierarchical Temporal Graph Attention Networks for cascade popularity prediction (HierCas). Unlike existing methods, HierCas operates on the entire cascade graph by a dynamic graph modeling approach, enabling it to capture the full range of continuous dynamic information and explicitly model the interplay between structural and temporal factors. By leveraging time-aware node embedding, graph attention mechanisms and hierarchical pooling structures, HierCas effectively captures the popularity trend implicit in the complex cascade. Extensive experiments conducted on two real-world datasets in different scenarios demonstrate that our HierCas significantly outperforms the state-of-the-art approaches.
PatchCURE: Improving Certifiable Robustness, Model Utility, and Computation Efficiency of Adversarial Patch Defenses

State-of-the-art defenses against adversarial patch attacks can now achieve strong certifiable robustness with a marginal drop in model utility. However, this impressive performance typically comes at the cost of 10-100x more inference-time computation compared to undefended models -- the research community has witnessed an intense three-way trade-off between certifiable robustness, model utility, and computation efficiency. In this paper, we propose a defense framework named PatchCURE to approach this trade-off problem. PatchCURE provides sufficient"knobs"for tuning defense performance and allows us to build a family of defenses: the most robust PatchCURE instance can match the performance of any existing state-of-the-art defense (without efficiency considerations); the most efficient PatchCURE instance has similar inference efficiency as undefended models. Notably, PatchCURE achieves state-of-the-art robustness and utility performance across all different efficiency levels, e.g., 16-23% absolute clean accuracy and certified robust accuracy advantages over prior defenses when requiring computation efficiency to be close to undefended models. The family of PatchCURE defenses enables us to flexibly choose appropriate defenses to satisfy given computation and/or utility constraints in practice.
Packing forests

The seminal papers of Edmonds \cite{Egy}, Nash-Williams \cite{NW} and Tutte \cite{Tu} have laid the foundations of the theories of packing arborescences and packing trees. The directed version has been extensively investigated, resulting in a great number of generalizations. In contrast, the undirected version has been marginally considered. The aim of this paper is to further develop the theories of packing trees and forests. Our main result on graphs characterizes the existence of a packing of $k$ forests, $F_1, \ldots, F_k$, in a graph $G$ such that each vertex of $G$ belongs to exactly $h$ of the forests, and in addition, each $F_i$ has between $\ell(i)$ and $\ell'(i)$ connected components and the total number of connected components in the packing is between $\alpha$ and $\beta$. Finally, we extend this result to hypergraphs and dypergraphs, the latter giving a generalization of a theorem of B\'erczi and Frank \cite{BF3}.
The History and Risks of Reinforcement Learning and Human Feedback

Reinforcement learning from human feedback (RLHF) has emerged as a powerful technique to make large language models (LLMs) easier to use and more effective. A core piece of the RLHF process is the training and utilization of a model of human preferences that acts as a reward function for optimization. This approach, which operates at the intersection of many stakeholders and academic disciplines, remains poorly understood. RLHF reward models are often cited as being central to achieving performance, yet very few descriptors of capabilities, evaluations, training methods, or open-source models exist. Given this lack of information, further study and transparency is needed for learned RLHF reward models. In this paper, we illustrate the complex history of optimizing preferences, and articulate lines of inquiry to understand the sociotechnical context of reward models. In particular, we highlight the ontological differences between costs, rewards, and preferences at stake in RLHF's foundations, related methodological tensions, and possible research directions to improve general understanding of how reward models function.
LeTFuser: Light-weight End-to-end Transformer-Based Sensor Fusion for Autonomous Driving with Multi-Task Learning

In end-to-end autonomous driving, the utilization of existing sensor fusion techniques and navigational control methods for imitation learning proves inadequate in challenging situations that involve numerous dynamic agents. To address this issue, we introduce LeTFuser, a lightweight transformer-based algorithm for fusing multiple RGB-D camera representations. To perform perception and control tasks simultaneously, we utilize multi-task learning. Our model comprises of two modules, the first being the perception module that is responsible for encoding the observation data obtained from the RGB-D cameras. Our approach employs the Convolutional vision Transformer (CvT) \cite{wu2021cvt} to better extract and fuse features from multiple RGB cameras due to local and global feature extraction capability of convolution and transformer modules, respectively. Encoded features combined with static and dynamic environments are later employed by our control module to predict waypoints and vehicular controls (e.g. steering, throttle, and brake). We use two methods to generate the vehicular controls levels. The first method uses a PID algorithm to follow the waypoints on the fly, whereas the second one directly predicts the control policy using the measurement features and environmental state. We evaluate the model and conduct a comparative analysis with recent models on the CARLA simulator using various scenarios, ranging from normal to adversarial conditions, to simulate real-world scenarios. Our method demonstrated better or comparable results with respect to our baselines in term of driving abilities. The code is available at \url{https://github.com/pagand/e2etransfuser/tree/cvpr-w} to facilitate future studies.
Contrastive Preference Learning: Learning from Human Feedback without RL

Reinforcement Learning from Human Feedback (RLHF) has emerged as a popular paradigm for aligning models with human intent. Typically RLHF algorithms operate in two phases: first, use human preferences to learn a reward function and second, align the model by optimizing the learned reward via reinforcement learning (RL). This paradigm assumes that human preferences are distributed according to reward, but recent work suggests that they instead follow the regret under the user's optimal policy. Thus, learning a reward function from feedback is not only based on a flawed assumption of human preference, but also leads to unwieldy optimization challenges that stem from policy gradients or bootstrapping in the RL phase. Because of these optimization challenges, contemporary RLHF methods restrict themselves to contextual bandit settings (e.g., as in large language models) or limit observation dimensionality (e.g., state-based robotics). We overcome these limitations by introducing a new family of algorithms for optimizing behavior from human feedback using the regret-based model of human preferences. Using the principle of maximum entropy, we derive Contrastive Preference Learning (CPL), an algorithm for learning optimal policies from preferences without learning reward functions, circumventing the need for RL. CPL is fully off-policy, uses only a simple contrastive objective, and can be applied to arbitrary MDPs. This enables CPL to elegantly scale to high-dimensional and sequential RLHF problems while being simpler than prior methods.
Zone Evaluation: Revealing Spatial Bias in Object Detection

A fundamental limitation of object detectors is that they suffer from"spatial bias", and in particular perform less satisfactorily when detecting objects near image borders. For a long time, there has been a lack of effective ways to measure and identify spatial bias, and little is known about where it comes from and what degree it is. To this end, we present a new zone evaluation protocol, extending from the traditional evaluation to a more generalized one, which measures the detection performance over zones, yielding a series of Zone Precisions (ZPs). For the first time, we provide numerical results, showing that the object detectors perform quite unevenly across the zones. Surprisingly, the detector's performance in the 96\% border zone of the image does not reach the AP value (Average Precision, commonly regarded as the average detection performance in the entire image zone). To better understand spatial bias, a series of heuristic experiments are conducted. Our investigation excludes two intuitive conjectures about spatial bias that the object scale and the absolute positions of objects barely influence the spatial bias. We find that the key lies in the human-imperceptible divergence in data patterns between objects in different zones, thus eventually forming a visible performance gap between the zones. With these findings, we finally discuss a future direction for object detection, namely, spatial disequilibrium problem, aiming at pursuing a balanced detection ability over the entire image zone. By broadly evaluating 10 popular object detectors and 5 detection datasets, we shed light on the spatial bias of object detectors. We hope this work could raise a focus on detection robustness. The source codes, evaluation protocols, and tutorials are publicly available at \url{https://github.com/Zzh-tju/ZoneEval}.
Benchmarking and Improving Text-to-SQL Generation under Ambiguity

Research in Text-to-SQL conversion has been largely benchmarked against datasets where each text query corresponds to one correct SQL. However, natural language queries over real-life databases frequently involve significant ambiguity about the intended SQL due to overlapping schema names and multiple confusing relationship paths. To bridge this gap, we develop a novel benchmark called AmbiQT with over 3000 examples where each text is interpretable as two plausible SQLs due to lexical and/or structural ambiguity. When faced with ambiguity, an ideal top-$k$ decoder should generate all valid interpretations for possible disambiguation by the user. We evaluate several Text-to-SQL systems and decoding algorithms, including those employing state-of-the-art LLMs, and find them to be far from this ideal. The primary reason is that the prevalent beam search algorithm and its variants, treat SQL queries as a string and produce unhelpful token-level diversity in the top-$k$. We propose LogicalBeam, a new decoding algorithm that navigates the SQL logic space using a blend of plan-based template generation and constrained infilling. Counterfactually generated plans diversify templates while in-filling with a beam-search that branches solely on schema names provides value diversity. LogicalBeam is up to $2.5$ times more effective than state-of-the-art models at generating all candidate SQLs in the top-$k$ ranked outputs. It also enhances the top-$5$ Exact and Execution Match Accuracies on SPIDER and Kaggle DBQA.
On non-local electrical transport in anisotropic metals

We discuss various aspects of nonlocal electrical transport in anisotropic metals. For a metal with circular Fermi surface, the scattering rates entering the local conductivity and viscosity tensors are well-defined, corresponding to eigenfrequencies of the linearized collision operator. For anisotropic metals, we provide generalized formulas for these scattering rates and use a variational approximation to show how they relate to microscopic transition probabilities. We develop a simple model of a collision operator for a metal of arbitrary Fermi surface with finite number of quasi-conserved quantities, and derive expressions for the wavevector-dependent conductivity σ(q) and the spatially-varying conductivity σ(x) for a long, narrow channel. We apply this to the case of different rates for momentum-conserving and momentum-relaxing scattering, deriving closed-form expressions for σ(q) and σ(x) — beyond generalizing from circular to arbitrary Fermi surface geometry, this represents an improvement over existing methods which solve the relevant differential equation numerically rather than in closed form. For the specific case of a diamond Fermi surface, we show that, if transport signatures were interpreted via a model for a circular Fermi surface, the diagnosis of the underlying transport regime would differ based on experimental orientation and based on whether σ(q) or σ(x) was considered. Finally, we discuss the bulk conductivity. While the common lore is that “momentum”-conserving scattering does not affect bulk resistivity, we show that crystal momentum-conserving scattering — such as normal electron-electron scattering — can affect the bulk resistivity for an anisotropic Fermi surface. We derive a simple formula for this contribution.
Copnumbers of periodic graphs

A periodic temporal graph $\mathcal{G}=(G_0, G_1, \dots, G_{p-1})^*$ is an infinite periodic sequence of graphs $G_i=(V,E_i)$ where $G=(V,\cup_i E_i)$ is called the footprint. Recently, the arena where the Cops and Robber game is played has been extended from a graph to a periodic graph; in this case, the copnumber is also the minimum number of cops sufficient for capturing the robber. We study the connections and distinctions between the copnumber $c(\mathcal{G})$ of a periodic graph $\mathcal{G}$ and the copnumber $c(G)$ of its footprint $G$ and establish several facts. For instance, we show that the smallest periodic graph with $c(\mathcal{G}) = 3$ has at most $8$ nodes; in contrast, the smallest graph $G$ with $c(G) = 3$ has $10$ nodes. We push this investigation by generating multiple examples showing how the copnumbers of a periodic graph $\mathcal{G}$, the subgraphs $G_i$ and its footprint $G$ can be loosely tied. Based on these results, we derive upper bounds on the copnumber of a periodic graph from properties of its footprint such as its treewidth.
Analyzing the contribution of different passively collected data to predict Stress and Depression

The possibility of recognizing diverse aspects of human behavior and environmental context from passively captured data motivates its use for mental health assessment. In this paper, we analyze the contribution of different passively collected sensor data types (WiFi, GPS, Social interaction, Phone Log, Physical Activity, Audio, and Academic features) to predict daily self-report stress and PHQ-9 depression score. First, we compute 125 mid-level features from the original raw data. These 125 features include groups of features from the different sensor data types. Then, we evaluate the contribution of each feature type by comparing the performance of Neural Network models trained with all features against Neural Network models trained with specific feature groups. Our results show that WiFi features (which encode mobility patterns) and Phone Log features (which encode information correlated with sleep patterns), provide significative information for stress and depression prediction.
Quantifying the impact of urbanization on N and P inputs, transformations, removal and fluxes in watersheds draining to the Plum Island Estuary

x Introduction . 1 Chapter I: . 5 Introduction . 5 Methods . 8 Site Description . 8 Sample Collection. 10 Water chemistry . 10 Flow and Climate Data . 11 Load Estimates . 12 Data Analysis . 12 Results . 13 LOADEST Model Fits. 13 Long-term nutrient changes . 13
Assumption violations in causal discovery and the robustness of score matching

When domain knowledge is limited and experimentation is restricted by ethical, financial, or time constraints, practitioners turn to observational causal discovery methods to recover the causal structure, exploiting the statistical properties of their data. Because causal discovery without further assumptions is an ill-posed problem, each algorithm comes with its own set of usually untestable assumptions, some of which are hard to meet in real datasets. Motivated by these considerations, this paper extensively benchmarks the empirical performance of recent causal discovery methods on observational i.i. d. data generated under different background conditions, allowing for violations of the critical assumptions required by each selected approach. Our experimental findings show that score matching-based methods demonstrate surprising performance in the false positive and false negative rate of the inferred graph in these challenging scenarios, and we provide theoretical insights into their performance. This work is also the first effort to benchmark the stability of causal discovery algorithms with respect to the values of their hyperparameters. Finally, we hope this paper will set a new standard for the evaluation of causal discovery methods and can serve as an accessible entry point for practitioners interested in the field, highlighting the empirical implications of different algorithm choices.
Video Quality Assessment and Coding Complexity of the Versatile Video Coding Standard

In recent years, the proliferation of multimedia applications and formats, such as IPTV, Virtual Reality (VR, 360-degree), and point cloud videos, has presented new challenges to the video compression research community. Simultaneously, there has been a growing demand from users for higher resolutions and improved visual quality. To further enhance coding efficiency, a new video coding standard, Versatile Video Coding (VVC), was introduced in July 2020. This paper conducts a comprehensive analysis of coding performance and complexity for the latest VVC standard in comparison to its predecessor, High Efficiency Video Coding (HEVC). The study employs a diverse set of test sequences, covering both High Definition (HD) and Ultra High Definition (UHD) resolutions, and spans a wide range of bit-rates. These sequences are encoded using the reference software encoders of HEVC (HM) and VVC (VTM). The results consistently demonstrate that VVC outperforms HEVC, achieving bit-rate savings of up to 40% on the subjective quality scale, particularly at realistic bit-rates and quality levels. Objective quality metrics, including PSNR, SSIM, and VMAF, support these findings, revealing bit-rate savings ranging from 31% to 40%, depending on the video content, spatial resolution, and the selected quality metric. However, these improvements in coding efficiency come at the cost of significantly increased computational complexity. On average, our results indicate that the VVC decoding process is 1.5 times more complex, while the encoding process becomes at least eight times more complex than that of the HEVC reference encoder. Our simultaneous profiling of the two standards sheds light on the primary evolutionary differences between them and highlights the specific stages responsible for the observed increase in complexity.
Dually weighted multi-matrix models as a path to causal gravity-matter systems

We introduce a dually-weighted multi-matrix model that for a suitable choice of weights reproduce two-dimensional Causal Dynamical Triangulations (CDT) coupled to the Ising model. When Ising degrees of freedom are removed, this model corresponds to the CDT-matrix model introduced by Benedetti and Henson [Phys. Lett. B 678, 222 (2009)]. We present exact as well as approximate results for the Gaussian averages of characters of a Hermitian matrix $A$ and $A^2$ for a given representation and establish the present limitations that prevent us to solve the model analytically. This sets the stage for the formulation of more sophisticated matter models coupled to two-dimensional CDT as dually weighted multi-matrix models providing a complementary view to the standard simplicial formulation of CDT-matter models.
Sequence Length Independent Norm-Based Generalization Bounds for Transformers

This paper provides norm-based generalization bounds for the Transformer architecture that do not depend on the input sequence length. We employ a covering number based approach to prove our bounds. We use three novel covering number bounds for the function class of bounded linear transformations to upper bound the Rademacher complexity of the Transformer. Furthermore, we show this generalization bound applies to the common Transformer training technique of masking and then predicting the masked word. We also run a simulated study on a sparse majority data set that empirically validates our theoretical findings.
Continuous Transition Between Weak and Strong Thermalization using Rigorous Bounds on Equilibration of Isolated Systems

We analyze strong and weak thermalization regimes from a perspective of rigorous mathematical upper bounds on the equilibration of isolated quantum systems. We show that weak equilibration can be understood to be due to the small effective dimension of the initial state. Furthermore, analyzing the scaling of an upper bound on the fluctuations, we show that the observable fluctuations decay exponentially with the system size for both weak and strong thermalization indicating no sharp transitions between these two regimes.