Miracle-dz/UniFAR-FTUs huggingface dataset:
PIPE : Parallelized Inference Through Post-Training Quantization Ensembling of Residual Expansions

Deep neural networks (DNNs) are ubiquitous in computer vision and natural language processing, but suffer from high inference cost. This problem can be addressed by quantization, which consists in converting floating point perations into a lower bit-width format. With the growing concerns on privacy rights, we focus our efforts on data-free methods. However, such techniques suffer from their lack of adaptability to the target devices, as a hardware typically only support specific bit widths. Thus, to adapt to a variety of devices, a quantization method shall be flexible enough to find good accuracy v.s. speed trade-offs for every bit width and target device. To achieve this, we propose PIPE, a quantization method that leverages residual error expansion, along with group sparsity and an ensemble approximation for better parallelization. PIPE is backed off by strong theoretical guarantees and achieves superior performance on every benchmarked application (from vision to NLP tasks), architecture (ConvNets, transformers) and bit-width (from int8 to ternary quantization).
Enhancing Perceptual Quality in Video Super-Resolution through Temporally-Consistent Detail Synthesis using Diffusion Models

In this paper, we address the problem of video super-resolution (VSR) using Diffusion Models (DM), and present StableVSR. Our method significantly enhances the perceptual quality of upscaled videos by synthesizing realistic and temporally-consistent details. We turn a pre-trained DM for single image super-resolution into a VSR method by introducing the Temporal Conditioning Module (TCM). TCM uses Temporal Texture Guidance, which provides spatially-aligned and detail-rich texture information synthesized in adjacent frames. This guides the generative process of the current frame toward high-quality and temporally-consistent results. We introduce a Frame-wise Bidirectional Sampling strategy to encourage the use of information from past to future and vice-versa. This strategy improves the perceptual quality of the results and the temporal consistency across frames. We demonstrate the effectiveness of StableVSR in enhancing the perceptual quality of upscaled videos compared to existing state-of-the-art methods for VSR. The code is available at https://github.com/claudiom4sir/StableVSR.
A Replica-BCS theory for dirty superconductors

Motivated by the discovery of the anomalous metal state in superconductor thin films, we revisit in this paper the problem of dirty superconductors using a replica-symmetric BCS (RS-BCS) theory for dirty metals with net attractive interactions. Within the RS-BCS mean field theory, we show that the (dirty) superconductor transits to a Cooper-pair-glass state beyond a critical strength of disorder. The single particle tunneling density of states and the superfluid density are computed within the RS-BCS theory for different strengths of disorder. We find that the single-particle spectral gap is strongly enhanced by disorder and the superfluid density reduces rapidly from the corresponding clean superconducting limit with increasing strength of disorder but remains finite in the Cooper-pair-glass state. The nature of the Cooper-pair-glass state and relevance of our result to the anomalous metal state are briefly discussed.
Efficient Rehearsal Free Zero Forgetting Continual Learning using Adaptive Weight Modulation

Artificial neural networks encounter a notable challenge known as continual learning, which involves acquiring knowledge of multiple tasks over an extended period. This challenge arises due to the tendency of previously learned weights to be adjusted to suit the objectives of new tasks, resulting in a phenomenon called catastrophic forgetting. Most approaches to this problem seek a balance between maximizing performance on the new tasks and minimizing the forgetting of previous tasks. In contrast, our approach attempts to maximize the performance of the new task, while ensuring zero forgetting. This is accomplished by creating a task-specific modulation parameters for each task. Only these would be learnable parameters during learning of consecutive tasks. Through comprehensive experimental evaluations, our model demonstrates superior performance in acquiring and retaining novel tasks that pose difficulties for other multi-task models. This emphasizes the efficacy of our approach in preventing catastrophic forgetting while accommodating the acquisition of new tasks
Localizing Lying in Llama: Understanding Instructed Dishonesty on True-False Questions Through Prompting, Probing, and Patching

Large language models (LLMs) demonstrate significant knowledge through their outputs, though it is often unclear whether false outputs are due to a lack of knowledge or dishonesty. In this paper, we investigate instructed dishonesty, wherein we explicitly prompt LLaMA-2-70b-chat to lie. We perform prompt engineering to find which prompts best induce lying behavior, and then use mechanistic interpretability approaches to localize where in the network this behavior occurs. Using linear probing and activation patching, we localize five layers that appear especially important for lying. We then find just 46 attention heads within these layers that enable us to causally intervene such that the lying model instead answers honestly. We show that these interventions work robustly across many prompts and dataset splits. Overall, our work contributes a greater understanding of dishonesty in LLMs so that we may hope to prevent it.
Task-Distributionally Robust Data-Free Meta-Learning

Data-Free Meta-Learning (DFML) aims to efficiently learn new tasks by leveraging multiple pre-trained models without requiring their original training data. Existing inversion-based DFML methods construct pseudo tasks from a learnable dataset, which is inversely generated from the pre-trained model pool. For the first time, we reveal two major challenges hindering their practical deployments: Task-Distribution Shift (TDS) and Task-Distribution Corruption (TDC). TDS leads to a biased meta-learner because of the skewed task distribution towards newly generated tasks. TDC occurs when untrusted models characterized by misleading labels or poor quality pollute the task distribution. To tackle these issues, we introduce a robust DFML framework that ensures task distributional robustness. We propose to meta-learn from a pseudo task distribution, diversified through task interpolation within a compact task-memory buffer. This approach reduces the meta-learner's overreliance on newly generated tasks by maintaining consistent performance across a broader range of interpolated memory tasks, thus ensuring its generalization for unseen tasks. Additionally, our framework seamlessly incorporates an automated model selection mechanism into the meta-training phase, parameterizing each model's reliability as a learnable weight. This is optimized with a policy gradient algorithm inspired by reinforcement learning, effectively addressing the non-differentiable challenge posed by model selection. Comprehensive experiments across various datasets demonstrate the framework's effectiveness in mitigating TDS and TDC, underscoring its potential to improve DFML in real-world scenarios.
Uncertainty Aware AI for 2D MRI Segmentation

Robust uncertainty estimations are necessary in safety-critical applications of Deep Learning. One such example is the semantic segmentation of medical images, whilst deep-learning approaches have high performance in such tasks they lack interpretability as they give no indication of their confidence when making classification decisions. Robust and interpretable segmentation is a critical first stage in automatically screening for pathologies hence the optimal solution is one which can provide high accuracy but also capture the underlying uncertainty. In this work, we present an uncertainty-aware segmentation model, BA U-Net, for use on MRI data that incorporates Bayesian Neural Networks and Attention Mechanisms to provide accurate and interpretable segmentations. We evaluated our model on the publicly available BraTS 2020 dataset using F1 Score and Intersection Over Union (IoU) as evaluation metrics.
On the generalized Ramanujan and Arthur conjectures over function fields

Let $G$ be a simple group over a global function field $K$, and let $\pi$ be a cuspidal automorphic representation of $G$. Suppose $K$ has two places $u$ and $v$ (satisfying a mild restriction on the residue field cardinality), at which the group $G$ is quasi-split, such that $\pi_u$ is tempered and $\pi_v$ is unramified and generic. We prove that $\pi_w$ is tempered at all unramified places $K_w$ at which $G$ is unramified quasi-split. More generally, the set of unitary spherical representations is partitioned according to nilpotent conjugacy classes in the Lie algebra of $G$. We show that if $\pi_v$ is in the set corresponding to the nilpotent class $N$, and if $\pi_u$ satisfies an analogous hypothesis, then $\pi_w$ belongs to the same class $N$, where $w$ is as above. These results are consistent with conjectures of Shahidi and Arthur. The proofs use the Galois parametrization of cuspidal representations due to V. Lafforgue to relate the local Satake parameters of $\pi$ to Deligne's theory of Frobenius weights. The main observation is that, in view of the classification of unitary spherical representations, due to Barbasch and the first-named author, the theory of weights excludes almost all complementary series as possible local components of $\pi$. This in turn determines the local Frobenius weights at all unramified places. In order to apply this observation in practice we need a result of the second-named author with Gan and Sawin on the weights of discrete series representations.
Three-dimensional finite-difference modeling of the San Andreas fault: Source parameterization and ground-motion levels

Olsen et al. (1995) recently simulated an Mw 7.75 earthquake on the San Andreas fault, predicting long-period (T > 2.5 sec) ground velocities of 140 cm/sec in the Los Angeles basin, about 60 km from the fault. These motions are much larger than estimates derived from empirical relations or other numerical simulations. Standard area-magnitude relations predict that the 170 × 16 km fault used in the simulations would produce an Mw 7.5 earthquake, giving a moment of 2.0 × 1027 dyne-cm, which is 2.4 times smaller than the moment used by Olsen et al. (1995). Further, self-similar scaling predicts a rise time of 3 sec for an Mw 7.75 event and 2.2 sec for an Mw 7.5 event. The filtered impulse slip function used by Olsen et al. (1995) has an effective rise time of 1.6 sec, yielding a response that is about 2 times larger than expected for periods less than 5 sec. This combination of high seismic moment and short rise time, along with the use of a uniform slip distribution, leads to the extreme ground-motion levels predicted by Olsen et al. (1995). To quantify the sensitivity of the long-period ground-motion response to source parameterization, we have performed 3D finite-difference (FD) simulations using various combinations of seismic moment, source rise time, and slip heterogeneity. These calculations incorporate the same grid dimensions, fault size, and bandwidth employed by Olsen et al. (1995). With a moment of 2.0 ՠ1027 dyne-cm, a rise time of 2 sec, and a smoothly heterogeneous slip distribution, we simulate peak long-period ground velocities of 155 cm/sec in the near-fault region and 40 cm/sec in the Los Angeles basin. These values are much closer to (although still higher than) empirical predictions. A uniform slip distribution produces the largest peak motions, both in the near-fault region and in the Los Angeles basin, whereas a rough asperity slip distribution noticeably reduces the maximum near-fault ground velocities. Our results indicate that the accurate simulation of long-period ground motions requires a realistic source parameterization, including appropriate choices of seismic moment and rise time, as well as the use of spatial and temporal variations in slip distribution.
Lightweight Face Recognition: An Improved MobileFaceNet Model

This paper presents an extensive exploration and comparative analysis of lightweight face recognition (FR) models, specifically focusing on MobileFaceNet and its modified variant, MMobileFaceNet. The need for efficient FR models on devices with limited computational resources has led to the development of models with reduced memory footprints and computational demands without sacrificing accuracy. Our research delves into the impact of dataset selection, model architecture, and optimization algorithms on the performance of FR models. We highlight our participation in the EFaR-2023 competition, where our models showcased exceptional performance, particularly in categories restricted by the number of parameters. By employing a subset of the Webface42M dataset and integrating sharpness-aware minimization (SAM) optimization, we achieved significant improvements in accuracy across various benchmarks, including those that test for cross-pose, cross-age, and cross-ethnicity performance. The results underscore the efficacy of our approach in crafting models that are not only computationally efficient but also maintain high accuracy in diverse conditions.
New Developments in the Numerical Conformal Bootstrap

The numerical conformal bootstrap has become in the last 15 years an indispensable tool for studying strongly coupled CFTs in various dimensions. Here we review the main developments in the field in the last 5 years, since the appearance of the previous comprehensive review \cite{Poland:2018epd}. We describe developments in the software ({\tt SDPB 2.0}, {\tt scalar\_blocks}, {\tt blocks\_3d}, {\tt autoboot}, {\tt hyperion}, {\tt simpleboot}), and on the algorithmic side (Delauney triangulation, cutting surface, tiptop, navigator function, skydive). We also describe the main physics applications which were obtained using the new technology.
NeuRAD: Neural Rendering for Autonomous Driving

Neural radiance fields (NeRFs) have gained popularity in the autonomous driving (AD) community. Recent methods show NeRFs' potential for closed-loop simulation, enabling testing of AD systems, and as an advanced training data augmentation technique. However, existing methods often require long training times, dense semantic supervision, or lack generalizability. This, in turn, hinders the application of NeRFs for AD at scale. In this paper, we propose NeuRAD, a robust novel view synthesis method tailored to dynamic AD data. Our method features simple network design, extensive sensor modeling for both camera and lidar -- including rolling shutter, beam divergence and ray dropping -- and is applicable to multiple datasets out of the box. We verify its performance on five popular AD datasets, achieving state-of-the-art performance across the board. To encourage further development, we will openly release the NeuRAD source code. See https://github.com/georghess/NeuRAD .
On the Cauchy problem for 3D Navier-Stokes helical vortex filament

This paper studies the Cauchy problem for a helical vortex filament evolving by the 3D incompressible Navier-Stokes equations. We prove global-in-time well-posedness and smoothing of solutions with initial vorticity concentrated on a helix. We provide a local-in-time well-posedness result for vortex filaments periodic in one spatial direction, and show that solutions with helical initial data preserve this symmetry. We follow the approach of [4], where the analogue local-in-time result has been obtained for closed vortex filaments in $\mathbb{R}^3$. Next, we apply local energy weak solutions theory with a novel estimate for helical functions in non-helical domains to uniquely extend the solutions globally in time. This is the first global-in-time well-posedness result for a vortex filament without size restriction and without vanishing swirl assumptions.
The 2nd Workshop on Maritime Computer Vision (MaCVi) 2024

The 2nd Workshop on Maritime Computer Vision (MaCVi) 2024 addresses maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicles (USV). Three challenges categories are considered: (i) UAV-based Maritime Object Tracking with Re-identification, (ii) USV-based Maritime Obstacle Segmentation and Detection, (iii) USV-based Maritime Boat Tracking. The USV-based Maritime Obstacle Segmentation and Detection features three sub-challenges, including a new embedded challenge addressing efficicent inference on real-world embedded devices. This report offers a comprehensive overview of the findings from the challenges. We provide both statistical and qualitative analyses, evaluating trends from over 195 submissions. All datasets, evaluation code, and the leaderboard are available to the public at https://macvi.org/workshop/macvi24.
Wave and current forces at a bottom-mounted submarine pipeline

ABSTRACT Aristodemo, F. , Tomasicchio, G.R. and Veltri, P., 2013. Wave and current forces at a bottom-mounted submarine pipeline. The present work deals with the analysis of in-line and transverse forces for a bottom-mounted slender pipeline subjected to periodic motions characterized by non-linear Stokes and cnoidal waves, and also with a superimposed positive current. Full-scale laboratory tests were conducted in the large wave flume of CSMI at Padua, Italy, and characterized by a longitudinal vertical wall which was built in the flume to create two separate channels where simultaneous measurements of pressures, velocities and surface elevations were performed. The experimental dataset was defined by 39 tests with periodic waves using a smoother cylinder, 14 tests with periodic waves using a rougher cylinder and 11 tests with periodic waves plus currents, leading to a Keulegan-Carpenter number ranging from 4 to 13. The hydrodynamic coefficients of Morison and transverse force models were deduced from the time records of kinematics and pressures at the cylinder through the evaluation of the performances of time and frequency domain techniques such as the ordinary and the weighted least squares. The comparisons between experimental and calculated non-linear forces in terms of peaks, associated phase shifts and spectral densities show acceptable accuracy as regards the Morison force components and a poor estimation of the vertical forces.
Suppressing Coherent Synchrotron Radiation Effects in Chicane Bunch Compressors

The most significant advances in the accelerator-based light sources (i.e., x-ray free electron lasers) are driven by the production of the high final peak current in the last several decades. As a prerequisite to attain the proposed high brightness, the symmetric C-chicane bunch compressor is typically exploited due to its simplicity, efficiency, and natural dispersion-free feature at all orders. However, during bunch compression for a high peak current requirement, a main contributing factor to the transverse emittance degradation is the emission of the coherent synchrotron radiation (CSR). Suppressing this effect is necessary to preserve the beam phase-space quality. To this end, this paper presents an analysis of one-dimensional CSR point-kick and derives the cancellation conditions in terms of compression factor. The CSR cancelation conditions indicate an asymmetric geometric design. We demonstrate concrete schemes for asymmetric C- and S-chicanes, and verify the CSR cancelation conditions using integration methods and ELEGANT simulations. Furthermore, the proposed asymmetric C- and S-chicanes can drastically suppress the emittance growth compared with the symmetric ones with identical bunch compression goals.
Sparsify-then-Classify: From Internal Neurons of Large Language Models To Efficient Text Classifiers

Among the many tasks that Large Language Models (LLMs) have revolutionized is text classification. However, existing approaches for applying pretrained LLMs to text classification predominantly rely on using single token outputs from only the last layer of hidden states. As a result, they suffer from limitations in efficiency, task-specificity, and interpretability. In our work, we contribute an approach that uses all internal representations by employing multiple pooling strategies on all activation and hidden states. Our novel lightweight strategy, Sparsify-then-Classify (STC) first sparsifies task-specific features layer-by-layer, then aggregates across layers for text classification. STC can be applied as a seamless plug-and-play module on top of existing LLMs. Our experiments on a comprehensive set of models and datasets demonstrate that STC not only consistently improves the classification performance of pretrained and fine-tuned models, but is also more efficient for both training and inference, and is more intrinsically interpretable.
Noisy Self-Training with Synthetic Queries for Dense Retrieval

Although existing neural retrieval models reveal promising results when training data is abundant and the performance keeps improving as training data increases, collecting high-quality annotated data is prohibitively costly. To this end, we introduce a novel noisy self-training framework combined with synthetic queries, showing that neural retrievers can be improved in a self-evolution manner with no reliance on any external models. Experimental results show that our method improves consistently over existing methods on both general-domain (e.g., MS-MARCO) and out-of-domain (i.e., BEIR) retrieval benchmarks. Extra analysis on low-resource settings reveals that our method is data efficient and outperforms competitive baselines, with as little as 30% of labelled training data. Further extending the framework for reranker training demonstrates that the proposed method is general and yields additional gains on tasks of diverse domains. \footnote{Source code is available at \url{https://github.com/Fantabulous-J/Self-Training-DPR}}
up: Exploring probation practitioners’ experiences managing high-risk

The COVID-19 pandemic has been, and still is, a worldwide health crisis. Despite the surge of literature on this phenomenon, little research has been conducted with the Probation Service during this time. The aim of this research was to explore Probation Practitioners ’ (PPs ’ ) experiences of the COVID-19 restrictions with a speci ﬁ c focus on those who access the Psychologically Informed Consultation Service (PICS). Further, to explore the experiences of key aspects of the COVID-19 pandemic through the lens of the Community Offender Personality Disorder Service. Semi-structured interviews were conducted with 9 PPs who represented a broad cross-section in terms of age and years of experience in the role. Interpretative Phenomenological Analysis was used to explore the experiences of PPs and revealed 5 main themes: unmet support needs, problematic working environments, an emotionally distressing time, the use of PICS, and a silver lining. These ﬁ ndings are discussed with implications for further research.