Miracle-dz/UniFAR-FTUs huggingface dataset:
Speaker-Independent Dysarthria Severity Classification using Self-Supervised Transformers and Multi-Task Learning

Dysarthria, a condition resulting from impaired control of the speech muscles due to neurological disorders, significantly impacts the communication and quality of life of patients. The condition's complexity, human scoring and varied presentations make its assessment and management challenging. This study presents a transformer-based framework for automatically assessing dysarthria severity from raw speech data. It can offer an objective, repeatable, accessible, standardised and cost-effective and compared to traditional methods requiring human expert assessors. We develop a transformer framework, called Speaker-Agnostic Latent Regularisation (SALR), incorporating a multi-task learning objective and contrastive learning for speaker-independent multi-class dysarthria severity classification. The multi-task framework is designed to reduce reliance on speaker-specific characteristics and address the intrinsic intra-class variability of dysarthric speech. We evaluated on the Universal Access Speech dataset using leave-one-speaker-out cross-validation, our model demonstrated superior performance over traditional machine learning approaches, with an accuracy of $70.48\%$ and an F1 score of $59.23\%$. Our SALR model also exceeded the previous benchmark for AI-based classification, which used support vector machines, by $16.58\%$. We open the black box of our model by visualising the latent space where we can observe how the model substantially reduces speaker-specific cues and amplifies task-specific ones, thereby showing its robustness. In conclusion, SALR establishes a new benchmark in speaker-independent multi-class dysarthria severity classification using generative AI. The potential implications of our findings for broader clinical applications in automated dysarthria severity assessments.
Dissecting Language Models: Machine Unlearning via Selective Pruning

Understanding and shaping the behaviour of Large Language Models (LLMs) is increasingly important as applications become more powerful and more frequently adopted. This paper introduces a machine unlearning method specifically designed for LLMs. We introduce a selective pruning method for LLMs that removes neurons based on their relative importance on a targeted capability compared to overall network performance. This approach is a compute- and data-efficient method for identifying and removing neurons that enable specific behaviours. Our findings reveal that both feed-forward and attention neurons in LLMs are specialized; that is, for specific tasks, certain neurons are more crucial than others.
Comment on"Controlled Bond Expansion for Density Matrix Renormalization Group Ground State Search at Single-Site Costs"(Extended Version)

In a recent Letter [Phys. Rev. Lett. 130, 246402 (2023)], Gleis, Li, and von Delft present an algorithm for expanding the bond dimension of a Matrix Product State wave function, giving accuracy similar to 2-site DMRG, but computationally more efficient, closer to the performance of 1-site DMRG. The Controlled Bond Expansion (CBE) algorithm uses the Hamiltonian projected onto two sites, and then further projected onto the two-site tangent space, to extract a set of $k$ vectors that are used to expand the basis between the two sites. CBE achieves this with a complicated sequence of five singular value decompositions (SVDs), in order to project onto the 2-site tangent space and reduce the bond dimension of the tensor network such that the contraction can be done in time $O(dwD^3)$. In this Comment, we show that (1) the projection onto the 2-site tangent space is unnecessary, and is generally not helpful; (2) the sequence of 5 SVDs can be replaced by a single $QR$ decomposition (optionally with one SVD as well), making use of the randomized SVD (RSVD) with high accuracy and significantly improved efficiency, scaling as $O(dwkD^2)$ i.e., the most expensive operations are only quadratic in the bond dimension $D$ and linear in the number of expansion vectors $k$; (3) several statements about the variational properties of the CBE algorithm are incorrect, and the variational properties are essentially identical to existing algorithms including 2-site DMRG and single-site subspace expansion (3S); (4) a similar RSVD approach can be applied to the 3S algorithm, which leads to many advantages over CBE, especially in systems with long range interactions. We also make some comments on the benchmarking MPS algorithms, and the overall computational efficiency with respect to the accuracy of the calculation.
Information Flow Routes: Automatically Interpreting Language Models at Scale

Information flows by routes inside the network via mechanisms implemented in the model. These routes can be represented as graphs where nodes correspond to token representations and edges to operations inside the network. We automatically build these graphs in a top-down manner, for each prediction leaving only the most important nodes and edges. In contrast to the existing workflows relying on activation patching, we do this through attribution: this allows us to efficiently uncover existing circuits with just a single forward pass. Additionally, the applicability of our method is far beyond patching: we do not need a human to carefully design prediction templates, and we can extract information flow routes for any prediction (not just the ones among the allowed templates). As a result, we can talk about model behavior in general, for specific types of predictions, or different domains. We experiment with Llama 2 and show that the role of some attention heads is overall important, e.g. previous token heads and subword merging heads. Next, we find similarities in Llama 2 behavior when handling tokens of the same part of speech. Finally, we show that some model components can be specialized on domains such as coding or multilingual texts.
The Bridged Posterior: Optimization, Profile Likelihood and a New Approach to Generalized Bayes

Optimization is widely used in statistics, thanks to its efficiency for delivering point estimates on useful spaces, such as those satisfying low cardinality or combinatorial structure. To quantify uncertainty, Gibbs posterior exponentiates the negative loss function to form a posterior density. Nevertheless, Gibbs posteriors are supported in a high-dimensional space, and do not inherit the computational efficiency or constraint formulations from optimization. In this article, we explore a new generalized Bayes approach, viewing the likelihood as a function of data, parameters, and latent variables conditionally determined by an optimization sub-problem. Marginally, the latent variable given the data remains stochastic, and is characterized by its posterior distribution. This framework, coined ``bridged posterior'', conforms to the Bayesian paradigm. Besides providing a novel generative model, we obtain a positively surprising theoretical finding that under mild conditions, the $\sqrt{n}$-adjusted posterior distribution of the parameters under our model converges to the same normal distribution as that of the canonical integrated posterior. Therefore, our result formally dispels a long-held belief that partial optimization of latent variables may lead to under-estimation of parameter uncertainty. We demonstrate the practical advantages of our approach under several settings, including maximum-margin classification, latent normal models, and harmonization of multiple networks.
Ocean‐Induced Melt Triggers Glacier Retreat in Northwest Greenland

In recent decades, tidewater glaciers in Northwest Greenland contributed significantly to sea level rise but exhibited a complex spatial pattern of retreat. Here we use novel observations of bathymetry and water temperature from NASA's Ocean Melting Greenland mission to quantify the role of warm, salty Atlantic Water in controlling the evolution of 37 glaciers. Modeled ocean‐induced undercutting of calving margins compared with ice advection and ice front retreat observed by satellites from 1985 to 2015 indicate that 35 glaciers retreated when cumulative anomalies in ocean‐induced undercutting rose above the range of seasonal variability of calving‐front positions, while two glaciers standing on shallow sills and colder water did not retreat. Deviations in the observed timing of retreat are explained by residual uncertainties in bathymetry, inefficient mixing of waters in shallow fjords, and the presence of small floating sections. Overall, warmer ocean temperature triggered the retreat, but calving processes dominate ablation (71%).
Phenomenology of renormalization group improved gravity from the kinematics of SPARC galaxies

Renormalization Group correction to General Relativity (RGGR) proposes a logarithmic running of the gravitational coupling $\left(G\right)$, resulting in a modified description of gravity. This has the potential to explain the observed kinematics of the galaxies, including the missing-mass problem. We, for the first time, based on the galaxy morphological types, investigate the dynamics of a diverse collection of galaxies present in the Spitzer Photometry for Accurate Rotation Curve (SPARC) catalog. We phenomenologically constrain the RGGR model parameter $\left(\bar\nu\right)$ along with the mass-to-light ratio for a sample of 100 SPARC galaxies, selected from four different morphological types, viz. early, spiral, late, and starburst. Our statistical analysis finds RGGR to fit the observed galaxy kinematics consistently. The constrained RGGR model parameter also supports the claim that it has a near-linear dependence on the galactic baryonic mass. From our morphology study, we find that the parameter $\bar\nu$ decreases from the early-type to the starburst galaxies. Finally, the renormalization group improved gravity is tested against the two established empirical relations for the SPARC catalog, viz., the Radial Acceleration Relation (RAR) and the Baryonic Tully Fisher relation (BTFR), both are found to be satisfied consistently.
Determining the bulk and surface electronic structure of $\alpha$-Sn/InSb(001) with spin- and angle-resolved photoemission spectroscopy

The surface and bulk states in topological materials have shown promise in many applications. Grey or $\alpha$-Sn, the inversion symmetric analogue to HgTe, can exhibit a variety of these phases. However there is disagreement in both calculation and experiment over the exact shape of the bulk bands and the number and origin of the surface states. Using spin- and angle-resolved photoemission we investigate the bulk and surface electronic structure of $\alpha$-Sn thin films on InSb(001) grown by molecular beam epitaxy. We find that there is no significant warping in the shapes of the bulk bands. We also observe the presence of only two surface states near the valence band maximum in both thin (13 bilayer) and thick (400 bilayer) films. In 50 bilayer films, these two surface states coexist with quantum well states. Surprisingly, both of these surface states are spin-polarized with orthogonal spin-momentum locking and opposite helicities. One of these states is the spin-polarized topological surface state and the other a spin resonance. Finally, the presence of another orthogonal spin-momentum locked topological surface state from a secondary band inversion is verified. Our work clarifies the electronic structure of $\alpha$-Sn(001) such that better control of the electronic properties can be achieved. In addition, the presence of two spin-polarized surface states near the valence band maximum has important ramifications for the use of $\alpha$-Sn in spintronics.
Simulating chiral spin liquids with fermionic Projected Entangled Paired States

Chiral Spin Liquids (CSL) based on spin-1/2 fermionic Projected Entangled Pair States (fPEPS) are considered on the square lattice. First, fPEPS approximants of Gutzwiller-projected Chern insulators (GPCI) are investigated by Variational Monte Carlo (VMC) techniques on finite size tori. We show that such fPEPS of finite bond dimension can correctly capture the topological properties of the chiral spin liquid, as the exact GPCI, with the correct topological ground state degeneracy on the torus. Further, more general fPEPS are considered and optimized (on the infinite plane) to describe the CSL phase of a chiral frustrated Heisenberg antiferromagnet. The chiral modes are computed on the edge of a semi-infinite cylinder (of finite circumference) and shown to follow the predictions from Conformal Field Theory. In contrast to their bosonic analogs the (optimized) fPEPS do not suffer from the replication of the chiral edge mode in the odd topological sector.
From spectral to scattering form factor

We propose a novel indicator for chaotic quantum scattering processes, the scattering form factor (ScFF). It is based on mapping the locations of peaks in the scattering amplitude to random matrix eigenvalues, and computing the analog of the spectral form factor (SFF). We compute the spectral and scattering form factors of several non-chaotic systems. We determine the ScFF associated with the phase shifts of the leaky torus, closely related to the distribution of the zeros of Riemann zeta function. We compute the ScFF for the decay amplitude of a highly excited string states into two tachyons. We show that it displays the universal features expected from random matrix theory - a decline, a ramp and a plateau - and is in general agreement with the Gaussian unitary ensemble. It also shows some new features, owning to the special structure of the string amplitude, including a"bump"before the ramp associated with gaps in the average eigenvalue density. The"bump"is removed for highly excited string states with an appropriate state dependent unfolding. We also discuss the SFF for the Gaussian $\beta$- ensemble, writing an interpolation between the known results of the Gaussian orthogonal, unitary, and symplectic ensembles.
Testing the presence of QGP in small systems using (multi-)strange hadron correlations with a $\phi$ meson

The study delves into the enhanced production of strange and multi-strange hadrons in proton-proton collisions at LHC. Novel observables are proposed to distinguish between models describing existing data with or without the assumption of a QGP. Two models are explored: EPOS4, based on core-corona separation between a QGP phase and a vacuum phase, and PYTHIA 8.3, based on microscopic interactions between Lund strings. The observables are based on an event selection requiring the presence of a $\phi$ meson. Correlations between the $\phi$ and (multi-)strange hadrons are shown to be an excellent discriminator between the two types of models. These observations emphasize the need for further testing in upcoming LHC Runs, leveraging improved detectors and increased data collection.
Disconnectivity graphs for visualizing combinatorial optimization problems: challenges of embedding to Ising machines

Physics-based Ising machines (IM) have risen to the challenge of solving hard combinatorial optimization problems with higher speed and better energy efficiency. Generally, such dedicated systems employ local search heuristics to traverse energy landscapes in searching for optimal solutions. Extending landscape geometry visualization tools, disconnectivity graphs, we quantify and address some of the major challenges met by IMs in the field of combinatorial optimization. Using efficient sampling methods, we visually capture landscapes of problems having diverse structure and hardness and featuring strong degeneracies, which act as entropic barriers for IMs. Furthermore, we investigate energy barriers, local minima, and configuration space clustering effects caused by locality reduction methods when embedding combinatorial problems to the Ising hardware. For this purpose, we sample disconnectivity graphs of PUBO energy landscapes and their different QUBO mappings accounting for both local minima and saddle regions. We demonstrate that QUBO energy landscape properties lead to the subpar performance of quadratic IMs and suggest directions for their improvement.
NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions

Field-Programmable Gate Array (FPGA) accelerators have proven successful in handling latency- and resource-critical deep neural network (DNN) inference tasks. Among the most computationally intensive operations in a neural network (NN) is the dot product between the feature and weight vectors. Thus, some previous FPGA acceleration works have proposed mapping neurons with quantized inputs and outputs directly to lookup tables (LUTs) for hardware implementation. In these works, the boundaries of the neurons coincide with the boundaries of the LUTs. We propose relaxing these boundaries and mapping entire sub-networks to a single LUT. As the sub-networks are absorbed within the LUT, the NN topology and precision within a partition do not affect the size of the lookup tables generated. Therefore, we utilize fully connected layers with floating-point precision inside each partition, which benefit from being universal function approximators, with rigid sparsity and quantization enforced only between partitions, where the NN topology becomes exposed to the circuit topology. Although cheap to implement, this approach can lead to very deep NNs, and so to tackle challenges like vanishing gradients, we also introduce skip connections inside the partitions. The resulting methodology can be seen as training DNNs with a specific sparsity pattern that allows them to be mapped to much shallower circuit-level networks, thereby significantly improving latency. We validate our proposed method on a known latency-critical task, jet substructure tagging, and on the classical computer vision task, the digit classification using MNIST. Our approach allows for greater function expressivity within the LUTs compared to existing work, leading to lower latency NNs for the same accuracy.
Neural Field Classifiers via Target Encoding and Classification Loss

Neural field methods have seen great progress in various long-standing tasks in computer vision and computer graphics, including novel view synthesis and geometry reconstruction. As existing neural field methods try to predict some coordinate-based continuous target values, such as RGB for Neural Radiance Field (NeRF), all of these methods are regression models and are optimized by some regression loss. However, are regression models really better than classification models for neural field methods? In this work, we try to visit this very fundamental but overlooked question for neural fields from a machine learning perspective. We successfully propose a novel Neural Field Classifier (NFC) framework which formulates existing neural field methods as classification tasks rather than regression tasks. The proposed NFC can easily transform arbitrary Neural Field Regressor (NFR) into its classification variant via employing a novel Target Encoding module and optimizing a classification loss. By encoding a continuous regression target into a high-dimensional discrete encoding, we naturally formulate a multi-label classification task. Extensive experiments demonstrate the impressive effectiveness of NFC at the nearly free extra computational costs. Moreover, NFC also shows robustness to sparse inputs, corrupted images, and dynamic scenes.
Training Computer Scientists for the Challenges of Hybrid Quantum-Classical Computing

As we enter the post-Moore era, we experience the rise of various non-von-Neumann-architectures to address the increasing computational demand for modern applications, with quantum computing being among the most prominent and promising technologies. However, this development creates a gap in current computer science curricula since most quantum computing lectures are strongly physics-oriented and have little intersection with the remaining curriculum of computer science. This fact makes designing an appealing course very difficult, in particular for non-physicists. Furthermore, in the academic community, there is consensus that quantum computers are going to be used only for specific computational tasks (e.g., in computational science), where hybrid systems - combined classical and quantum computers - facilitate the execution of an application on both quantum and classical computing resources. A hybrid system thus executes only certain suitable parts of an application on the quantum machine, while other parts are executed on the classical components of the system. To fully exploit the capabilities of hybrid systems and to meet future requirements in this emerging field, we need to prepare a new generation of computer scientists with skills in both distributed computing and quantum computing. To bridge this existing gap in standard computer science curricula, we designed a new lecture and exercise series on Hybrid Quantum-Classical Systems, where students learn how to decompose applications and implement computational tasks on a hybrid quantum-classical computational continuum. While learning the inherent concepts underlying quantum systems, students are obligated to apply techniques and methods they are already familiar with, making the entrance to the field of quantum computing comprehensive yet appealing and accessible to students of computer science.
A Modular and Robust Physics-Based Approach for Lensless Image Reconstruction

In this paper, we present a modular approach for reconstructing lensless measurements. It consists of three components: a newly-proposed pre-processor, a physics-based camera inverter to undo the multiplexing of lensless imaging, and a post-processor. The pre- and post-processors address noise and artifacts unique to lensless imaging before and after camera inversion respectively. By training the three components end-to-end, we obtain a 1.9 dB increase in PSNR and a 14% relative improvement in a perceptual image metric (LPIPS) with respect to previously proposed physics-based methods. We also demonstrate how the proposed pre-processor provides more robustness to input noise, and how an auxiliary loss can improve interpretability.
Spin-Dependent Stereochemistry: A Nonadiabatic Quantum Dynamics Case Study of S + H2 → SH + H Reaction.

We study the spin-dependent stereodynamics of the S + H2 → SH + H reaction by using full-dimensional quantum dynamics calculations with zero total nuclear angular momentum along the triplet 3A″ states and singlet 1A' states. We find that the interplay between the electronic spin direction and the molecular geometry has a measurable influence on the singlet-triplet intersystem crossing reaction probabilities. Our results show that for some incident scattering angles in the body-fixed frame, the relative difference in intersystem crossing reaction probabilities (as determined between spin up and spin down initial states) can be as large as 15%. Our findings are an ab initio demonstration of spin-dependent nonadiabatic dynamics, which we hope will shine light as far as understanding the chiral-induced spin selectivity effect.