Miracle-dz/UniFAR-FTUs huggingface dataset:
Extractive Summarization of Legal Decisions using Multi-task Learning and Maximal Marginal Relevance

Summarizing legal decisions requires the expertise of law practitioners, which is both time- and cost-intensive. This paper presents techniques for extractive summarization of legal decisions in a low-resource setting using limited expert annotated data. We test a set of models that locate relevant content using a sequential model and tackle redundancy by leveraging maximal marginal relevance to compose summaries. We also demonstrate an implicit approach to help train our proposed models generate more informative summaries. Our multi-task learning model variant leverages rhetorical role identification as an auxiliary task to further improve the summarizer. We perform extensive experiments on datasets containing legal decisions from the US Board of Veterans' Appeals and conduct quantitative and expert-ranked evaluations of our models. Our results show that the proposed approaches can achieve ROUGE scores vis-\`a-vis expert extracted summaries that match those achieved by inter-annotator comparison.
Algorithms with Prediction Portfolios

The research area of algorithms with predictions has seen recent success showing how to incorporate machine learning into algorithm design to improve performance when the predictions are correct, while retaining worst-case guarantees when they are not. Most previous work has assumed that the algorithm has access to a single predictor. However, in practice, there are many machine learning methods available, often with incomparable generalization guarantees, making it hard to pick a best method a priori. In this work we consider scenarios where multiple predictors are available to the algorithm and the question is how to best utilize them. Ideally, we would like the algorithm's performance to depend on the quality of the best predictor. However, utilizing more predictions comes with a cost, since we now have to identify which prediction is the best. We study the use of multiple predictors for a number of fundamental problems, including matching, load balancing, and non-clairvoyant scheduling, which have been well-studied in the single predictor setting. For each of these problems we introduce new algorithms that take advantage of multiple predictors, and prove bounds on the resulting performance.
Code4Struct: Code Generation for Few-Shot Structured Prediction from Natural Language

Large Language Model (LLM) trained on the mixture of text and code has demonstrated impressive capability in translating natural language (NL) into structured code. In this work, we propose C ODE 4S TRUCT to leverage such text-to-structure translation capability to tackle structured prediction tasks in NLP. For example, Event Argument Extraction (EAE) aims to convert text into event-argument structures that can be represented as a class object using code. This alignment between structures and code enables us to take advantage of Programming Language (PL) features such as inheritance 1 and type annotation 2 to introduce external knowledge or add constraints with ease. We exploit the analogy between PL and NLP problems, and, as a case study, we use C ODE 4S TRUCT to tackle the EAE task us-ing code generation. We ask a LLM to generate code to instantiate an event class with predicted arguments given a NL sentence. Despite only using 50 training instances for each event type, C ODE 4S TRUCT is comparable to fully-supervised models trained on 4,202 event instances and, when given the same 50-shot data, outperforms current state-of-the-art (SOTA) by 20.8% absolute F1. When prompted with hierarchical event types implemented using inheritance, C ODE 4S TRUCT can predict arguments for low-resource event types using 10-shot training instances from its sibling event type and outperforms zero-shot baseline by 12% absolute F1. 3
The Temperature Echo of the Superconducting Phase Transition

An analysis of experimental heat capacity at T > T c is presented for series of samples (R)Ba 2 Cu 3 O 6+ x (with x close to optimal). For all samples the anomaly was discovered which occurred steadily in the interval 250–290 K (anomaly T h ). The anomaly T h looks like a phase transition anomaly. It was shown that the anomaly T h correlates with superconducting anomaly T c , temperatures T h and T c being connected by the ratio T h ≈ 3 T c . The anomaly T h is interpreted as the origination of pairing the charge carriers. Anomalies at T ≈ 3 T c were also detected in heat capacity of low temperature superconductors Hg and Nb 3 Ge.
Standardly embedded train tracks and pseudo-Anosov maps with minimum expansion factor

We show that given a fully-punctured pseudo-Anosov map $f:S \to S$ whose punctures lie in at least two orbits under the action of $f$, the expansion factor $\lambda(f)$ satisfies the inequality $\lambda(f)^{|\chi(S)|} \ge \mu^4 \approx 6.85408$, where $\mu = \frac{1 + \sqrt{5}}{2} \approx 1.61803$ is the golden ratio. The proof involves a study of standardly embedded train tracks, and the Thurston symplectic form defined on their weight space.
Dichotomy of Control: Separating What You Can Control from What You Cannot

Future- or return-conditioned supervised learning is an emerging paradigm for offline reinforcement learning (RL), where the future outcome (i.e., return) associated with an observed action sequence is used as input to a policy trained to imitate those same actions. While return-conditioning is at the heart of popular algorithms such as decision transformer (DT), these methods tend to perform poorly in highly stochastic environments, where an occasional high return can arise from randomness in the environment rather than the actions themselves. Such situations can lead to a learned policy that is inconsistent with its conditioning inputs; i.e., using the policy to act in the environment, when conditioning on a specific desired return, leads to a distribution of real returns that is wildly different than desired. In this work, we propose the dichotomy of control (DoC), a future-conditioned supervised learning framework that separates mechanisms within a policy's control (actions) from those beyond a policy's control (environment stochasticity). We achieve this separation by conditioning the policy on a latent variable representation of the future, and designing a mutual information constraint that removes any information from the latent variable associated with randomness in the environment. Theoretically, we show that DoC yields policies that are consistent with their conditioning inputs, ensuring that conditioning a learned policy on a desired high-return future outcome will correctly induce high-return behavior. Empirically, we show that DoC is able to achieve significantly better performance than DT on environments that have highly stochastic rewards and transition
Large deviations for the volume of k-nearest neighbor balls

This paper develops the large deviations theory for the point process associated with the Euclidean volume of $k$-nearest neighbor balls centered around the points of a homogeneous Poisson or a binomial point processes in the unit cube. Two different types of large deviation behaviors of such point processes are investigated. Our first result is the Donsker-Varadhan large deviation principle, under the assumption that the centering terms for the volume of $k$-nearest neighbor balls grow to infinity more slowly than those needed for Poisson convergence. Additionally, we also study large deviations based on the notion of $\mathcal M_0$-topology, which takes place when the centering terms tend to infinity sufficiently fast, compared to those for Poisson convergence. As applications of our main theorems, we discuss large deviations for the number of Poisson or binomial points of degree at most $k$ in a geometric graph in the dense regime.
Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization

Efficient k-nearest neighbor search is a fundamental task, foundational for many problems in NLP. When the similarity is measured by dot-product between dual-encoder vectors or L2-distance, there already exist many scalable and efficient search methods. But not so when similarity is measured by more accurate and expensive black-box neural similarity models, such as cross-encoders, which jointly encode the query and candidate neighbor. The cross-encoders’ high computational cost typically limits their use to reranking candidates retrieved by a cheaper model, such as dual encoder or TF-IDF. However, the accuracy of such a two-stage approach is upper-bounded by the recall of the initial candidate set, and potentially requires additional training to align the auxiliary retrieval model with the cross-encoder model. In this paper, we present an approach that avoids the use of a dual-encoder for retrieval, relying solely on the cross-encoder. Retrieval is made efficient with CUR decomposition, a matrix decomposition approach that approximates all pairwise cross-encoder distances from a small subset of rows and columns of the distance matrix. Indexing items using our approach is computationally cheaper than training an auxiliary dual-encoder model through distillation. Empirically, for k > 10, our approach provides test-time recall-vs-computational cost trade-offs superior to the current widely-used methods that re-rank items retrieved using a dual-encoder or TF-IDF.
Elastic turbulence in a shell model of polymer solution

We show that, at low inertia and large elasticity, shell models of viscoelastic fluids develop a chaotic behaviour with properties similar to those of elastic turbulence. The low dimensionality of shell models allows us to explore a wide range both in polymer concentration and in Weissenberg number. Our results demonstrate that the physical mechanisms at the origin of elastic turbulence do not rely on the boundary conditions or on the geometry of the mean flow.
Semantic geometric fusion multi-object tracking and lidar odometry in dynamic environment

Simultaneous localization and mapping systems based on rigid scene assumptions cannot achieve reliable positioning and mapping in a complex environment with many moving objects. To solve this problem, this paper proposes a novel dynamic multi-object lidar odometry (MLO) system based on semantic object recognition technology. The proposed system enables the reliable localization of robots and semantic objects and the generation of long-term static maps in complex dynamic scenes. For ego-motion estimation, the proposed system extracts environmental features that take into account both semantic and geometric consistency constraints. Then, the filtered features can be robust to the semantic movable and unknown dynamic objects. In addition, we propose a new least-squares estimator that uses geometric object points and semantic box planes to realize the multi-object tracking (SGF-MOT) task robustly and precisely. In the mapping module, we implement dynamic semantic object detection using the absolute trajectory tracking list. By using static semantic objects and environmental features, the system eliminates accumulated localization errors and produces a purely static map. Experiments on the public KITTI dataset show that the proposed MLO system provides more accurate and robust object tracking performance and better real-time localization accuracy in complex scenes compared to existing technologies.
WOLD-TYPE DECOMPOSITION FOR DOUBLY NON-COMMUTING CONTRACTIONS

Let n > 1, and r ij ∈ S 1 := { z ∈ C : | z | = 1 } for 1 ≤ i, j ≤ n such that r ji = r ij for all i 6 = j . An n -tuple of contractions ( T 1 , . . . , T n ) on a Hilbert space H is called an n -tuple of doubly non-commuting contractions if T 1 , . , T n satisfy = ij T j T i for all i 6 = j . We obtain a recipe to calculate the orthogonal spaces of the Wold-type decomposition for n tuple of doubly non-commuting contractions on Hilbert spaces. As a by-product, a new proof as well as complete structure for pairs and an n -tuple of doubly non-commuting isometries have been established.
PHOTON INTERFERENCE AT BEAM SPLITTER WITH MICRO DETECTOR OF TRAJECTORIES

In this work we theoretically suggest and consider a new realistic experiment of single photon interference at beam splitter where (micro)detector of the photon quantum trajectories is presented by a pair of distant identical atoms. It admits studying of the quantum entanglement between photon and detector (Schrödinger cat effect) and modeling of the detection procedure (collapse) by spontaneous unitary symmetry (superposition) breaking. Practically, collapse can be considered as an especial case of the general formalism of spontaneous symmetry breaking.
An Affine Invariant Linear Convergence Analysis for Frank-Wolfe Algorithms

We study the linear convergence of variants of the Frank-Wolfe algorithms for some classes of strongly convex problems, using only affine-invariant quantities. As in Guelat & Marcotte (1986), we show the linear convergence of the standard Frank-Wolfe algorithm when the solution is in the interior of the domain, but with affine invariant constants. We also show the linear convergence of the away-steps variant of the Frank-Wolfe algorithm, but with constants which only depend on the geometry of the domain, and not any property of the location of the optimal solution. Running these algorithms does not require knowing any problem specific parameters.
LMPriors: Pre-Trained Language Models as Task-Specific Priors

Particularly in low-data regimes, an outstanding challenge in machine learning is developing principled techniques for augmenting our models with suitable priors. This is to encourage them to learn in ways that are compatible with our understanding of the world. But in contrast to generic priors such as shrinkage or sparsity, we draw inspiration from the recent successes of large-scale language models (LMs) to construct task-specific priors distilled from the rich knowledge of LMs. Our method, Language Model Priors (LMPriors), incorporates auxiliary natural language metadata about the task—such as variable names and descriptions—to encourage downstream model outputs to be consistent with the LM’s common-sense reasoning based on the metadata. Empirically, we demonstrate that LMPriors improve model performance in settings where such natural language descriptions are available, and perform well on several tasks that benefit from such prior knowledge, such as feature selection, causal inference, and safe reinforcement learning.
Deep Equilibrium Approaches to Diffusion Models

Diffusion-based generative models are extremely effective in generating high-quality images, with generated samples often surpassing the quality of those produced by other models under several metrics. One distinguishing feature of these models, however, is that they typically require long sampling chains to produce high-fidelity images. This presents a challenge not only from the lenses of sampling time, but also from the inherent difficulty in backpropagating through these chains in order to accomplish tasks such as model inversion, i.e. approximately finding latent states that generate known images. In this paper, we look at diffusion models through a different perspective, that of a (deep) equilibrium (DEQ) fixed point model. Specifically, we extend the recent denoising diffusion implicit model (DDIM; Song et al. 2020), and model the entire sampling chain as a joint, multivariate fixed point system. This setup provides an elegant unification of diffusion and equilibrium models, and shows benefits in 1) single image sampling, as it replaces the fully-serial typical sampling process with a parallel one; and 2) model inversion, where we can leverage fast gradients in the DEQ setting to much more quickly find the noise that generates a given image. The approach is also orthogonal and thus complementary to other methods used to reduce the sampling time, or improve model inversion. We demonstrate our method's strong performance across several datasets, including CIFAR10, CelebA, and LSUN Bedrooms and Churches.
Kennicutt-Schmidt Relation Variety and Star-forming Cloud Fraction

The observationally derived Kennicutt-Schmidt (KS) relation slopes differ from study to study, ranging from sublinear to superlinear. We investigate the KS-relation variety (slope and normalization) as a function of integrated intensity ratio, R31 = CO(J = 3–2)/CO(J = 1–0) using spatially resolved CO(J = 1–0), CO(J = 3–2), H i, Hα, and 24 μm data of three nearby spiral galaxies (NGC 3627, NGC 5055, and M83). We find that (1) the slopes for each subsample with a fixed R31 are shallower, but the slope for all data sets combined becomes steeper, (2) normalizations for high R31 subsamples tend to be high, (3) R31 correlates with star formation efficiency, therefore the KS relation depends on the distribution in R31–Σgas space of the samples: no Σgas dependence of R31 results in a linear slope of the KS relation, whereas a positive correlation between Σgas and R31 results in a superlinear slope of the KS relation, and (4) R31–Σgas distributions are different from galaxy to galaxy and within a galaxy: galaxies with prominent galactic structure tend to have large R31 and Σgas. Our results suggest that the formation efficiency of a star-forming cloud from molecular gas is different among galaxies as well as within a galaxy, and it is one of the key factors inducing the variety in galactic KS relation.
ECG Artifact Removal from Single-Channel Surface EMG Using Fully Convolutional Networks

Electrocardiogram (ECG) artifact contamination often occurs in surface electromyography (sEMG) applications when the measured muscles are in proximity to the heart. Previous studies have developed and proposed various methods, such as high-pass filtering, template subtraction and so forth. However, these methods remain limited by the requirement of reference signals and distortion of original sEMG. This study proposed a novel denoising method to eliminate ECG artifacts from the single-channel sEMG signals using fully convolutional networks (FCN). The proposed method adopts a denoise autoencoder structure and powerful nonlinear mapping capability of neural networks for sEMG denoising. We compared the proposed approach with conventional approaches, including high-pass filters and template subtraction, on open datasets called the Non-Invasive Adaptive Prosthetics database and MIT-BIH normal sinus rhythm database. The experimental results demonstrate that the FCN outperforms conventional methods in sEMG reconstruction quality under a wide range of signal-to-noise ratio inputs.
EpipolarNVS: leveraging on Epipolar geometry for single-image Novel View Synthesis

Novel-view synthesis (NVS) can be tackled through different approaches, depending on the general setting: a single source image to a short video sequence, exact or noisy camera pose information, 3D-based information such as point clouds etc. The most challenging scenario, the one where we stand in this work, only considers a unique source image to generate a novel one from another viewpoint. However, in such a tricky situation, the latest learning-based solutions often struggle to integrate the camera viewpoint transformation. Indeed, the extrinsic information is often passed as-is, through a low-dimensional vector. It might even occur that such a camera pose, when parametrized as Euler angles, is quantized through a one-hot representation. This vanilla encoding choice prevents the learnt architecture from inferring novel views on a continuous basis (from a camera pose perspective). We claim it exists an elegant way to better encode relative camera pose, by leveraging 3D-related concepts such as the epipolar constraint. We, therefore, introduce an innovative method that encodes the viewpoint transformation as a 2D feature image. Such a camera encoding strategy gives meaningful insights to the network regarding how the camera has moved in space between the two views. By encoding the camera pose information as a finite number of coloured epipolar lines, we demonstrate through our experiments that our strategy outperforms vanilla encoding.
On the optimization and pruning for Bayesian deep learning

The goal of Bayesian deep learning is to provide uncertainty quantification via the posterior distribution. However, exact inference over the weight space is computationally intractable due to the ultra-high dimensions of the neural network. Variational inference (VI) is a promising approach, but naive application on weight space does not scale well and often underperform on predictive accuracy. In this paper, we propose a new adaptive variational Bayesian algorithm to train neural networks on weight space that achieves high predictive accuracy. By showing that there is an equivalence to Stochastic Gradient Hamiltonian Monte Carlo(SGHMC) with preconditioning matrix, we then propose an MCMC within EM algorithm, which incorporates the spike-and-slab prior to capture the sparsity of the neural network. The EM-MCMC algorithm allows us to perform optimization and model pruning within one-shot. We evaluate our methods on CIFAR-10, CIFAR-100 and ImageNet datasets, and demonstrate that our dense model can reach the state-of-the-art performance and our sparse model perform very well compared to previously proposed pruning schemes.
Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems

Long-term fairness is an important factor of consideration in designing and deploying learning-based decision systems in high-stake decision-making contexts. Recent work has proposed the use of Markov Decision Processes (MDPs) to formulate decision-making with long-term fairness requirements in dynamically changing environments, and demonstrated major challenges in directly deploying heuristic and rule-based policies that worked well in static environments. We show that policy optimization methods from deep reinforcement learning can be used to find strictly better decision policies that can often achieve both higher overall utility and less violation of the fairness requirements, compared to previously-known strategies. In particular, we propose new methods for imposing fairness requirements in policy optimization by regularizing the advantage evaluation of different actions. Our proposed methods make it easy to impose fairness constraints without reward engineering or sacrificing training efficiency. We perform detailed analyses in three established case studies, including attention allocation in incident monitoring, bank loan approval, and vaccine distribution in population networks.