Miracle-dz/UniFAR-FTUs huggingface dataset:
Universal approximation and model compression for radial neural networks

We introduce a class of fully-connected neural networks whose activation functions, rather than being pointwise, rescale feature vectors by a function depending only on their norm. We call such networks radial neural networks, extending previous work on rotation equivariant networks that considers rescaling activations in less generality. We prove universal approximation theorems for radial neural networks, including in the more difficult cases of bounded widths and unbounded domains. Our proof techniques are novel, distinct from those in the pointwise case. Additionally, radial neural networks exhibit a rich group of orthogonal change-of-basis symmetries on the vector space of trainable parameters. Factoring out these symmetries leads to a practical lossless model compression algorithm. Optimization of the compressed model by gradient descent is equivalent to projected gradient descent for the full model.
Memorization-Dilation: Modeling Neural Collapse Under Noise

The notion of neural collapse refers to several emergent phenomena that have been empirically observed across various canonical classiﬁcation problems. During the terminal phase of training a deep neural network, the feature embedding of all examples of the same class tend to collapse to a single representation, and the features of different classes tend to separate as much as possible. Neural collapse is often studied through a simpliﬁed model, called the unconstrained feature representation, in which the model is assumed to have ”inﬁnite expressivity” and can map each data point to any arbitrary representation. In this work, we propose a more realistic variant of the unconstrained feature representation that takes the limited expressivity of the network into account. Empirical evidence suggests that the memorization of noisy data points leads to a degradation (dilation) of the neural collapse. Using a model of the memorization-dilation (M-D) phenomenon, we show one mechanism by which different losses lead to different performances of the trained network on noisy data. Our proofs reveal why label smoothing, a modiﬁcation of cross-entropy empirically observed to produce a regularization effect, leads to improved generalization in classiﬁcation tasks.
DeepVOX: Discovering Features from Raw Audio for Speaker Recognition in Non-ideal Audio Signals

Automatic speaker recognition algorithms typically use pre-defined filterbanks, such as Mel-Frequency and Gammatone filterbanks, for characterizing speech audio. However, it has been observed that the features extracted using these filterbanks are not resilient to diverse audio degradations. In this work, we propose a deep learning-based technique to deduce the filterbank design from vast amounts of speech audio. The purpose of such a filterbank is to extract features robust to non-ideal audio conditions, such as degraded, short duration, and multi-lingual speech. To this effect, a 1D convolutional neural network is designed to learn a time-domain filterbank called DeepVOX directly from raw speech audio. Secondly, an adaptive triplet mining technique is developed to efficiently mine the data samples best suited to train the filterbank. Thirdly, a detailed ablation study of the DeepVOX filterbanks reveals the presence of both vocal source and vocal tract characteristics in the extracted features. Experimental results on VOXCeleb2, NIST SRE 2008, 2010 and 2018, and Fisher speech datasets demonstrate the efficacy of the DeepVOX features across a variety of degraded, short duration, and multi-lingual speech. The DeepVOX features also shown to improve the performance of existing speaker recognition algorithms, such as the xVector-PLDA and the iVector-PLDA.
Absolute Expressiveness of Subgraph Motif Centrality Measures

In graph-based applications, a common task is to pinpoint the most important or “central” vertex in a (directed or undirected) graph, or rank the vertices of a graph according to their importance. To this end, a plethora of so-called centrality measures have been proposed in the literature that assess which vertices in a graph are the most important ones. Riveros and Salas, in an ICDT 2020 paper, proposed a family of centrality measures based on the following intuitive principle: the importance of a vertex in a graph is relative to the number of “relevant” connected subgraphs, known as subgraph motifs, surrounding it. We refer to the measures derived from the above principle as subgraph motif measures. It has been convincingly argued that subgraph motif measures are well-suited for graph database applications. Although the ICDT paper studied several favourable properties enjoyed by subgraph motif measures, their absolute expressiveness remains largely unexplored. The goal of this workis toprecisely characterize the absoluteexpressiveness of the family of subgraph motif measures.
Grounding in social media: An approach to building a chit-chat dialogue model

Building open-domain dialogue systems capable of rich human-like conversational ability is one of the fundamental challenges in language generation. However, even with recent advancements in the field, existing open-domain generative models fail to capture and utilize external knowledge, leading to repetitive or generic responses to unseen utterances. Current work on knowledge-grounded dialogue generation primarily focuses on persona incorporation or searching a fact-based structured knowledge source such as Wikipedia. Our method takes a broader and simpler approach, which aims to improve the raw conversation ability of the system by mimicking the human response behavior through casual interactions found on social media. Utilizing a joint retriever-generator setup, the model queries a large set of filtered comment data from Reddit to act as additional context for the seq2seq generator. Automatic and human evaluations on open-domain dialogue datasets demonstrate the effectiveness of our approach.
Differentiable and Transportable Structure Learning

Directed acyclic graphs (DAGs) encode a lot of information about a particular distribution in their structure. However, compute required to infer these structures is typically super-exponential in the number of variables, as inference requires a sweep of a combinatorially large space of potential structures. That is, until recent advances made it possible to search this space using a differentiable metric, drastically reducing search time. While this technique -- named NOTEARS -- is widely considered a seminal work in DAG-discovery, it concedes an important property in favour of differentiability: transportability. To be transportable, the structures discovered on one dataset must apply to another dataset from the same domain. We introduce D-Struct which recovers transportability in the discovered structures through a novel architecture and loss function while remaining fully differentiable. Because D-Struct remains differentiable, our method can be easily adopted in existing differentiable architectures, as was previously done with NOTEARS. In our experiments, we empirically validate D-Struct with respect to edge accuracy and structural Hamming distance in a variety of settings.
Ultrafast Momentum-Resolved Hot Electron Dynamics in the Two-Dimensional Topological Insulator Bismuthene

Two-dimensional quantum spin Hall (QSH) insulators are a promising material class for spintronic applications based on topologically protected spin currents in their edges. Yet, they have not lived up to their technological potential, as experimental realizations are scarce and limited to cryogenic temperatures. These constraints have also severely restricted characterization of their dynamical properties. Here, we report on the electron dynamics of the novel room-temperature QSH candidate bismuthene after photoexcitation using time- and angle-resolved photoemission spectroscopy. We map the transiently occupied conduction band and track the full relaxation pathway of hot photocarriers. Intriguingly, we observe photocarrier lifetimes much shorter than those in conventional semiconductors. This is ascribed to the presence of topological in-gap states already established by local probes. Indeed, we find spectral signatures consistent with these earlier findings. Demonstration of the large band gap and the view into photoelectron dynamics mark a critical step toward optical control of QSH functionalities.
MLLess: Achieving Cost Efficiency in Serverless Machine Learning Training

Function-as-a-Service (FaaS) has raised a growing interest in how to"tame"serverless computing to enable domain-specific use cases such as data-intensive applications and machine learning (ML), to name a few. Recently, several systems have been implemented for training ML models. Certainly, these research articles are significant steps in the correct direction. However, they do not completely answer the nagging question of when serverless ML training can be more cost-effective compared to traditional"serverful"computing. To help in this endeavor, we propose MLLess, a FaaS-based ML training prototype built atop IBM Cloud Functions. To boost cost-efficiency, MLLess implements two innovative optimizations tailored to the traits of serverless computing: on one hand, a significance filter, to make indirect communication more effective, and on the other hand, a scale-in auto-tuner, to reduce cost by benefiting from the FaaS sub-second billing model (often per 100ms). Our results certify that MLLess can be 15X faster than serverful ML systems at a lower cost for sparse ML models that exhibit fast convergence such as sparse logistic regression and matrix factorization. Furthermore, our results show that MLLess can easily scale out to increasingly large fleets of serverless workers.
An LSTM-Aided Hybrid Random Access Scheme for Machine Type Communication with 6G Network Slicing

—In this paper, an LSTM-aided hybrid random access scheme (LSTMH-RA) is proposed to support diverse quality of service (QoS) requirements in machine-type communication (MTC) networks where massive MTC (mMTC) devices and ultra-reliable low latency communications (URLLC) devices coexist. In the proposed LSTMH-RA scheme, mMTC devices access the network via a timing advance (TA)-aided four-step procedure to meet massive access requirement, while the access procedure of the URLLC devices is completed in two steps coupled with the mMTC devices’ access procedure to reduce latency. In addition, the resource allocated to mMTC and URLLC devices are isolated to avoid interference between them by using 6G network slicing. Furthermore, we propose an attention-based LSTM prediction model to predict the number of active URLLC devices, thereby determining the parameters of the multi-user detection algorithm to guarantee the latency and reliability access requirements of URLLC devices. We analyze the successful access probability of the LSTMH-RA scheme. Numerical results show that, compared with the benchmark schemes, the proposed LSTMH-RA scheme can signiﬁcantly improve the successful access probability, and thus satisfy the diverse QoS requirements of URLLC and mMTC devices.
Predicting Gibbs-State Expectation Values with Pure Thermal Shadows

The preparation and computation of many properties of quantum Gibbs states is essential for algorithms such as quantum semidefinite programming and quantum Boltzmann machines. We propose a quantum algorithm that can predict $M$ linear functions of an arbitrary Gibbs state with only $\mathcal{O}(\log{M})$ experimental measurements. Our main insight is that for sufficiently large systems we do not need to prepare the $n$-qubit mixed Gibbs state explicitly but, instead, we can evolve a random $n$-qubit pure state in imaginary time. The result then follows by constructing classical shadows of these random pure states. We propose a quantum circuit that implements this algorithm by using quantum signal processing for the imaginary time evolution. We numerically verify the efficiency of the algorithm by simulating the circuit for a ten-spin-1/2 XXZ-Heisenberg model. In addition, we show that the algorithm can be successfully employed as a subroutine for training an eight-qubit fully connected quantum Boltzmann machine.
Algebraically closed $\sigma$-fields

The concept of a skew root of a skew polynomial is used to introduce notions of algebraic closedness for $\sigma$-fields, that is, a field equipped with an endomorphism. It is shown that every $\sigma$-field can be embedded in algebraically closed $\sigma$-fields of different types.
Strong C IV emission from star-forming galaxies: a case for high Lyman continuum photon escape

Finding reliable indicators of Lyman continuum (LyC) photon leakage from galaxies is essential in order to infer their escape fraction in the epoch of reionisation, where direct measurements of LyC ﬂux are impossible. To this end, here we investigate whether strong C iv 𝜆𝜆 1548 , 1550 emission in the rest-frame UV spectra of galaxies traces conditions ripe for ample production and escape of LyC photons. We compile a sample of 19 star-forming galaxies in the redshift range 𝑧 = 3 . 1 − 4 . 6 from the VANDELS survey that exhibit strong C iv emission, producing a stacked spectrum where all major rest-UV emission lines are clearly detected. Best-ﬁtting spectral energy distribution models containing both stellar and nebular emission suggest the need for low stellar metallicities ( 𝑍 = 0 . 1 − 0 . 2 𝑍 (cid:12) ), young stellar ages (log ( age / yr ) = 6 . 1 − 6 . 5), a high ionisation parameter (log 𝑈 = − 2) and little to no dust attenuation ( 𝐸 ( 𝐵 − 𝑉 ) = 0 . 00 − 0 . 01). However, these models are unable to fully reproduce the observed C iv and He ii line strengths. We ﬁnd that the Ly 𝛼 line in the stacked spectrum is strong and peaks close to the systemic velocity, features that are indicative of signiﬁcant LyC photon leakage along the line-of-sight. The covering fractions of low-ionisation interstellar absorption lines are also low, implying LyC escape fraction in the range ≈ 0 . 05 − 0 . 30, with signatures of outﬂowing gas. Finally, C iv/C iii] ratios of > 0 . 75 for a subset of individual galaxies with reliable detections of both lines are also consistent with physical conditions that enable signiﬁcant LyC leakage. Overall, we report that multiple spectroscopic indicators of LyC leakage are present in the stacked spectrum of strong C iv emitting galaxies, potentially making C iv an important tracer of LyC photon escape at 𝑧 > 6.
Simple Robust Estimating Method for Generalized Linear Models and its Application to Propensity Score Estimation

A generalized linear model is one of the most well-known model families in statistics. Of course, we should specify the correct model structure to estimate the unbiased result and make a valid inference, however, we would like to consider an another rational approach in this paper to make a valid inference. The proposed method is 1) preparing some candidate models, and 2) construct an estimating equation including the candidate models at once (do not have to select just one) to estimate an interested parameter. If the correct model was included in the candidate models, the parameter estimator would have the consistency. By using the idea, we will consider a robust estimating method to estimate a “valid” parameter estimator without any model/variable selection methods. As an application example, an estimation of a propensity score and a generalized propensity score will be considered.
Dynamical spontaneous scalarization in Einstein-Maxwell-scalar models in anti–de Sitter spacetime

The phenomenon of spontaneous scalarization of charged black holes has attracted a lot of attention. In this work, we study the dynamical process of the spontaneous scalarization of charged black hole in asymptotically anti-de Sitter spacetimes in Einstein-Maxwell-scalar models. Including various non-minimal couplings between the scalar ﬁeld and Maxwell ﬁeld, we observe that an initial scalar-free conﬁguration suﬀers tachyonic instability and both the scalar ﬁeld and the black hole irreducible mass grow exponentially at early times and saturate exponentially at late times. For fractional couplings, we ﬁnd that though there is negative energy distribution near the black hole horizon, the black hole horizon area never decreases. But when the parameters are large, the evolution endpoints of linearly unstable bald black holes will be spacetimes with naked singularity and the cosmic censorship is violated. The eﬀects of the black hole charge, cosmological constant and coupling strength on the dynamical scalarization process are studied in detail. We ﬁnd that large enough cosmological constant can prevent the spontaneous scalarization.
Anchor-Changing Regularized Natural Policy Gradient for Multi-Objective Reinforcement Learning

We study policy optimization for Markov decision processes (MDPs) with multiple reward value functions, which are to be jointly optimized according to given criteria such as proportional fairness (smooth concave scalarization), hard constraints (constrained MDP), and max-min trade-off. We propose an Anchor-changing Regularized Natural Policy Gradient (ARNPG) framework, which can systematically incorporate ideas from well-performing first-order methods into the design of policy optimization algorithms for multi-objective MDP problems. Theoretically, the designed algorithms based on the ARNPG framework achieve $\tilde{O}(1/T)$ global convergence with exact gradients. Empirically, the ARNPG-guided algorithms also demonstrate superior performance compared to some existing policy gradient-based approaches in both exact gradients and sample-based scenarios.
Multi-Agent Neural Rewriter for Vehicle Routing with Limited Disclosure of Costs

We interpret solving the multi-vehicle routing problem as a team Markov game with partially observable costs. For a given set of customers to serve, the playing agents (vehicles) have the common goal to determine the team-optimal agent routes with minimal total cost. Each agent thereby observes only its own cost. Our multi-agent reinforcement learning approach, the so-called multi-agent Neural Rewriter, builds on the single-agent Neural Rewriter to solve the problem by iteratively rewriting solutions. Parallel agent action execution and partial observability require new rewriting rules for the game. We propose the introduction of a so-called pool in the system which serves as a collection point for unvisited nodes. It enables agents to act simultaneously and exchange nodes in a conflict-free manner. We realize limited disclosure of agent-specific costs by only sharing them during learning. During inference, each agents acts decentrally, solely based on its own cost. First empirical results on small problem sizes demonstrate that we reach a performance close to the employed OR-Tools benchmark which operates in the perfect cost information setting.
Dislocation core structures in titanium nitride

Although titanium nitride (TiN) is among the most extensively studied and thoroughly characterized thin-film ceramic materials, detailed knowledge of relevant dislocation core structures is lacking. By high-resolution scanning transmission electron microscopy (STEM) of epitaxial single crystal (001)-oriented TiN films, we identify different dislocation types and their core structures. These include, besides the expected primary full a/2 110{110} dislocation, Shockley partial dislocations a/6 112{111} and sessile Lomer edge dislocations a/2 110{100}. Density-functional theory and classical interatomic potential simulations complement STEM observations by recovering the atomic structure of the different dislocation types, estimating Peierls stresses, and providing insights on the chemical bonding nature at the core. We underline that a variety of dislocation types should be considered in interpretations of observed nanoscale and macroscopic properties of TiN.
Exact solution for SU(2)-symmetry-breaking bosonic mixtures at strong interactions

We study the equilibrium properties of a one-dimensional mixture of two Tonks-Girardeau gases on a ring geometry in the limit of strongly-repulsive inter-species interactions. We derive the exact many-body wavefunction and compare it to the $SU(2)$ solution where intra- and inter-species interactions are also diverging but equal. We focus on the role of the $SU(2)$-symmetry breaking on the behaviour of the large- and short-distance correlations by studying the zero-momentum occupation number and the Tan's contact from the asymptotic behavior of the momentum distribution. Although the symmetry is only weakly broken, it has important consequences on spin correlations in the system as the reduction by a factor of two of the zero-momentum occupation number with respect to the $SU(2)$ case in the thermodynamic limit and the decrease of the Tan's contact.
Deep learning techniques for imaging air Cherenkov telescopes

Very High Energy (VHE) gamma rays and charged cosmic rays (CCRs) provide an observational window into the acceleration mechanisms of extreme astrophysical environments. One of the major challenges at Imaging Air Cherenkov Telescopes (IACTs) designed to look for VHE gamma rays, is the separation of air showers initiated by CCRs which form a background to gamma ray searches. Two other less well studied problems at IACTs are a) the classification of different primary nuclei among the CCR events and b) identification of anomalous events initiated by Beyond Standard Model particles that could give rise to shower signatures which differ from the standard images of either gamma rays or CCR showers. The problems of categorizing the primary particle that initiates a shower image, or the problem of tagging anomalous shower events in a model independent way, are problems that are well suited to a machine learning (ML) approach. Traditional studies that have explored gamma ray/CCR separation have used a multivariate analysis based on derived shower properties, which contains significantly reduced information about the shower. In our work, we address the problems outlined above by using ML architectures trained on full simulated shower images, as opposed to training on just a few derived shower properties. We illustrate the techniques of binary and multi-category classification using convolutional neural networks, and we also pioneer the use of autoencoders for anomaly detection at VHE gamma ray experiments. As a case study, we apply our techniques to the H.E. S.S. experiment. However, the real strength of the techniques that we broach here in the context of VHE gamma ray observatories, is that these methods can be applied broadly to any other IACT, such as the upcoming Cherenkov Telescope Array (CTA), or can even be suitably adapted to CCR experiments.
One Size Fits All: Hypernetwork for Tunable Image Restoration

We introduce a novel approach for tunable image restoration that achieves the accuracy of multiple models, each optimized for a different level of degradation, with exactly the same number of parameters as a single model. Our model can be optimized to restore as many degradation levels as required with a constant number of parameters and for various image restoration tasks. Experiments on real-world datasets show that our approach achieves state-of-the art results in de-noising, DeJPEG and super-resolution with respect to existing tunable models, allowing smoother and more accurate fitting over a wider range of degradation levels.