Miracle-dz/UniFAR-FTUs huggingface dataset:
Spectral alignment of correlated Gaussian random matrices

In this paper we analyze a simple method ($EIG1$) for the problem of matrix alignment, consisting in aligning their leading eigenvectors: given $A$ and $B$, we compute $v_1$ and $v'_1$ two leading eigenvectors of $A$ and $B$. The algorithm returns a permutation $\hat{\Pi}$ such that the rank of the coordinate $\hat{\Pi}(i)$ in $v_1$ is the rank of the coordinate $i$ in $v'_1$ (up to the sign of $v'_1$). We consider a model where $A$ belongs to the Gaussian Orthogonal Ensemble (GOE), and $B= \Pi^T (A+\sigma H) \Pi $, where $\Pi$ is a permutation matrix and $H$ is an independent copy of $A$. We show the following 0-1 law: under the condition $\sigma N^{7/6+\epsilon} \to 0$, the $EIG1$ method recovers all but a vanishing part of the underlying permutation $\Pi$. When $\sigma N^{7/6-\epsilon} \to \infty$, this algorithm cannot recover more than $o(N)$ correct matches. This result gives an understanding of the simplest and fastest spectral method for matrix alignment (or complete weighted graph alignment), and involves proof methods and techniques which could be of independent interest.
Cartesian products avoiding patterns

The pattern avoidance problem seeks to construct a set with large fractal dimension that avoids a prescribed pattern, such as three term arithmetic progressions, or more general patterns, such as finding a set whose Cartesian product avoids the zero set of a given function. Previous work on the subject has considered patterns described by polynomials, or functions satisfying certain regularity conditions. We provide an exposition of some results in this setting, as well as considering new strategies to avoid what we call `rough patterns'. This thesis contains an expanded description of a method described in a previous paper by the author and his collaborators Malabika Pramanik and Joshua Zahl, as well as new results in the rough pattern avoidance setting. There are several problems that fit into the pattern of rough pattern avoidance. For instance, we prove that for any set $X$ with lower Minkowski dimension $s$, there exists a set $Y$ with Hausdorff dimension $1-s$ such that for any rational numbers $a_1, \dots, a_N$, the set $a_1 Y + \dots + a_N Y$ is disjoint from $X$, or intersects with $X$ solely at the origin. As a second application, we construct subsets of Lipschitz curves with dimension $1/2$ not containing the vertices of any isosceles triangle.
Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning

Learning the underlying patterns in data goes beyond instance-based generalization to external knowledge represented in structured graphs or networks. Deep learning that primarily constitutes neural computing stream in AI has shown significant advances in probabilistically learning latent patterns using a multi-layered network of computational nodes (i.e., neurons/hidden units). Structured knowledge that underlies symbolic computing approaches and often supports reasoning, has also seen significant growth in recent years, in the form of broad-based (e.g., DBPedia, Yago) and domain, industry or application specific knowledge graphs. A common substrate with careful integration of the two will raise opportunities to develop neuro-symbolic learning approaches for AI, where conceptual and probabilistic representations are combined. As the incorporation of external knowledge will aid in supervising the learning of features for the model, deep infusion of representational knowledge from knowledge graphs within hidden layers will further enhance the learning process. Although much work remains, we believe that knowledge graphs will play an increasing role in developing hybrid neuro-symbolic intelligent systems (bottom-up deep learning with top-down symbolic computing) as well as in building explainable AI systems for which knowledge graphs will provide scaffolding for punctuating neural computing. In this position paper, we describe our motivation for such a neuro-symbolic approach and framework that combines knowledge graph and neural networks.
Fluid models with phase transition for kinetic equations in swarming

We concentrate on kinetic models for swarming with individuals interacting through self-propelling and friction forces, alignment and noise. We assume that the velocity of each individual relaxes to the mean velocity. In our present case, the equilibria depend on the density and the orientation of the mean velocity, whereas the mean speed is not anymore a free parameter and a phase transition occurs in the homogeneous kinetic equation. We analyze the profile of equilibria for general potentials identifying a family of potentials leading to phase transitions. Finally, we derive the fluid equations when the interaction frequency becomes very large.
Fluctuations of the Gromov-Prohorov sample model

In this paper, we study the fluctuations of observables of metric measure spaces which are random discrete approximations $X_n$ of a fixed arbitrary (complete, separable) metric measure space $X=(\mathcal{X},d,\mu)$. These observables $\Phi(X_n)$ are polynomials in the sense of Greven-Pfaffelhuber-Winter, and we show that for a generic model space $X$, they yield asymptotically normal random variables. However, if $X$ is a compact homogeneous space, then the fluctuations of the observables are much smaller, and after an adequate rescaling, they converge towards probability distributions which are not Gaussian. Conversely, we prove that if all the fluctuations of the observables $\Phi(X_n)$ are smaller than in the generic case, then the measure metric space $X$ is compact homogeneous. The proofs of these results rely on the Gromov reconstruction principle, and on an adaptation of the method of cumulants and mod-Gaussian convergence developed by Feray-Meliot-Nikeghbali.
High-temperature expansion of the grand thermodynamic potential for scalar particles in crossed electromagnetic fields

The problem of a scalar particle in a constant crossed electromagnetic field ($\mathbf{E}\perp\mathbf{H}$ and $|\mathbf{E}|=|\mathbf{H}|$) is examined. The high-temperature expansion of the grand thermodynamic potential and vacuum energy with account for non-perturbative corrections are derived. The contribution from particles and antiparticles is considered separately. It is shown that the non-perturbative corrections depend on boundary conditions but do not depend on fields.
Lower and upper bounds for unilateral coherence and applying them to the entropic uncertainty relations

In this paper, a method is provided for obtaining the entropic uncertainty relations in the presence of a quantum memory by using quantum coherence. In the method, firstly, one can use the quantum relative entropy of quantum coherence to obtain the uncertainty relations. Secondly, these relations are applied to obtain the entropic uncertainty relations in the presence of a quantum memory. In comparison with other methods this approach is much simpler. Also, for a given state, the upper bounds on the sum of the relative entropies of unilateral coherences are provided, and it is shown which one is tighter. In addition, using the upper bound obtained for unilateral coherence, the nontrivial upper bound on the sum of the entropies for different observables is derived in the presence of a quantum memory.
Biometric Recognition Using Deep Learning: A Survey

Deep learning-based models have been very successful in achieving state-of-the-art results in many of the computer vision, speech recognition, and natural language processing tasks in the last few years. These models seem a natural fit for handling the ever-increasing scale of biometric recognition problems, from cellphone authentication to airport security systems. Deep learning-based models have increasingly been leveraged to improve the accuracy of different biometric recognition systems in recent years. In this work, we provide a comprehensive survey of more than 120 promising works on biometric recognition (including face, fingerprint, iris, palmprint, ear, voice, signature, and gait recognition), which deploy deep learning models, and show their strengths and potentials in different applications. For each biometric, we first introduce the available datasets that are widely used in the literature and their characteristics. We will then talk about several promising deep learning works developed for that biometric, and show their performance on popular public benchmarks. We will also discuss some of the main challenges while using these models for biometric recognition, and possible future directions to which research in this area is headed.
Point Cloud Instance Segmentation using Probabilistic Embeddings

In this paper we propose a new framework for point cloud instance segmentation. Our framework has two steps: an embedding step and a clustering step. In the embedding step, our main contribution is to propose a probabilistic embedding space for point cloud embedding. Specifically, each point is represented as a tri-variate normal distribution. In the clustering step, we propose a novel loss function, which benefits both the semantic segmentation and the clustering. Our experimental results show important improvements to the SOTA, i.e., 3.1% increased average per-category mAP on the PartNet dataset.
QUALITY PROBLEMS OF ANTI-TUBERCULOSIS FIXED-DOSE COMBINATIONS (FDCS): A WAY FORWARD

Fixed dose combinations (FDCs) of anti-tuberculosis drugs (Rifampicin Isoniazid Pyrazinamide and Ethambutol hydrochloride) have several advantages over loose combinations of these drugs. Hence the use of FDCs in the short course chemotherapy of tuberculosis is being promoted internationally. However these products are associated with quality problems such as loss of bio-availability of Rifampicin and instability of the drugs in the combination. Recent research in Indian laboratories has helped in identification of the major reasons behind these problems which suggest that an entirely different approach is required to that currently being recommended by the world bodies. The purpose of this article is to briefly discuss the twin quality problems and to highlight the potential forward ways. (authors)
On the Delta Method for Uncertainty Approximation in Deep Learning

The Delta method is a well known procedure used to quantify uncertainty in statistical models. The method has previously been applied in the context of neural networks, but has not reached much popularity in deep learning because of the sheer size of the Hessian matrix. In this paper, we propose a low cost variant of the method based on an approximate eigendecomposition of the positive curvature subspace of the Hessian matrix. The method has a computational complexity of $O(KPN)$ time and $O(KP)$ space, where $K$ is the number of utilized Hessian eigenpairs, $P$ is the number of model parameters and $N$ is the number of training examples. Given that the model is $L_2$-regularized with rate $\lambda/2$, we provide a bound on the uncertainty approximation error given $K$. We show that when the smallest Hessian eigenvalue in the positive $K/2$-tail of the full spectrum, and the largest Hessian eigenvalue in the negative $K/2$-tail of the full spectrum are both approximately equal to $\lambda$, the error will be close to zero even when $K\ll P$ . We demonstrate the method by a TensorFlow implementation, and show that meaningful rankings of images based on prediction uncertainty can be obtained for a convolutional neural network based MNIST classifier. We also observe that false positives have higher prediction uncertainty than true positives. This suggests that there is supplementing information in the uncertainty measure not captured by the probability alone.
Attentive Modality Hopping Mechanism for Speech Emotion Recognition

In this work, we explore the impact of visual modality in addition to speech and text for improving the accuracy of the emotion detection system. The traditional approaches tackle this task by independently fusing the knowledge from the various modalities for performing emotion classification. In contrast to these approaches, we tackle the problem by introducing an attention mechanism to combine the information. In this regard, we first apply a neural network to obtain hidden representations of the modalities. Then, the attention mechanism is defined to select and aggregate important parts of the video data by conditioning on the audio and text data. Furthermore, the attention mechanism is again applied to attend the essential parts of speech and textual data by considering other modalities. Experiments are performed on the standard IEMOCAP dataset using all three modalities (audio, text, and video). The achieved results show a significant improvement of 3.65% in terms of weighted accuracy compared to the baseline system.
Adaptive divergence for rapid adversarial optimization

Adversarial Optimization provides a reliable, practical way to match two implicitly defined distributions, one of which is typically represented by a sample of real data, and the other is represented by a parameterized generator. Matching of the distributions is achieved by minimizing a divergence between these distribution, and estimation of the divergence involves a secondary optimization task, which, typically, requires training a model to discriminate between these distributions. The choice of the model has its trade-off: high-capacity models provide good estimations of the divergence, but, generally, require large sample sizes to be properly trained. In contrast, low-capacity models tend to require fewer samples for training; however, they might provide biased estimations. Computational costs of Adversarial Optimization becomes significant when sampling from the generator is expensive. One of the practical examples of such settings is fine-tuning parameters of complex computer simulations. In this work, we introduce a novel family of divergences that enables faster optimization convergence measured by the number of samples drawn from the generator. The variation of the underlying discriminator model capacity during optimization leads to a significant speed-up. The proposed divergence family suggests using low-capacity models to compare distant distributions (typically, at early optimization steps), and the capacity gradually grows as the distributions become closer to each other. Thus, it allows for a significant acceleration of the initial stages of optimization. This acceleration was demonstrated on two fine-tuning problems involving Pythia event generator and two of the most popular black-box optimization algorithms: Bayesian Optimization and Variational Optimization. Experiments show that, given the same budget, adaptive divergences yield results up to an order of magnitude closer to the optimum than Jensen-Shannon divergence. While we consider physics-related simulations, adaptive divergences can be applied to any stochastic simulation.
On the Legal Compatibility of Fairness Definitions

Past literature has been effective in demonstrating ideological gaps in machine learning (ML) fairness definitions when considering their use in complex socio-technical systems. However, we go further to demonstrate that these definitions often misunderstand the legal concepts from which they purport to be inspired, and consequently inappropriately co-opt legal language. In this paper, we demonstrate examples of this misalignment and discuss the differences in ML terminology and their legal counterparts, as well as what both the legal and ML fairness communities can learn from these tensions. We focus this paper on U.S. anti-discrimination law since the ML fairness research community regularly references terms from this body of law.
Information bottleneck through variational glasses

Information bottleneck (IB) principle [1] has become an important element in information-theoretic analysis of deep models. Many state-of-the-art generative models of both Variational Autoencoder (VAE) [2; 3] and Generative Adversarial Networks (GAN) [4] families use various bounds on mutual information terms to introduce certain regularization constraints [5; 6; 7; 8; 9; 10]. Accordingly, the main difference between these models consists in add regularization constraints and targeted objectives. In this work, we will consider the IB framework for three classes of models that include supervised, unsupervised and adversarial generative models. We will apply a variational decomposition leading a common structure and allowing easily establish connections between these models and analyze underlying assumptions. Based on these results, we focus our analysis on unsupervised setup and reconsider the VAE family. In particular, we present a new interpretation of VAE family based on the IB framework using a direct decomposition of mutual information terms and show some interesting connections to existing methods such as VAE [2; 3], beta-VAE [11], AAE [12], InfoVAE [5] and VAE/GAN [13]. Instead of adding regularization constraints to an evidence lower bound (ELBO) [2; 3], which itself is a lower bound, we show that many known methods can be considered as a product of variational decomposition of mutual information terms in the IB framework. The proposed decomposition might also contribute to the interpretability of generative models of both VAE and GAN families and create a new insights to a generative compression [14; 15; 16; 17]. It can also be of interest for the analysis of novelty detection based on one-class classifiers [18] with the IB based discriminators.
Low Rank Factorization for Compact Multi-Head Self-Attention

Effective representation learning from text has been an active area of research in the fields of NLP and text mining. Attention mechanisms have been at the forefront in order to learn contextual sentence representations. Current state-of-the-art approaches for many NLP tasks use large pre-trained language models such as BERT, XLNet and so on for learning representations. These models are based on the Transformer architecture that involves recurrent blocks of computation consisting of multi-head self-attention and feedforward networks. One of the major bottlenecks largely contributing to the computational complexity of the Transformer models is the self-attention layer, that is both computationally expensive and parameter intensive. In this work, we introduce a novel multi-head self-attention mechanism operating on GRUs that is shown to be computationally cheaper and more parameter efficient than self-attention mechanism proposed in Transformers for text classification tasks. The efficiency of our approach mainly stems from two optimizations; 1) we use low-rank matrix factorization of the affinity matrix to efficiently get multiple attention distributions instead of having separate parameters for each head 2) attention scores are obtained by querying a global context vector instead of densely querying all the words in the sentence. We evaluate the performance of the proposed model on tasks such as sentiment analysis from movie reviews, predicting business ratings from reviews and classifying news articles into topics. We find that the proposed approach matches or outperforms a series of strong baselines and is more parameter efficient than comparable multi-head approaches. We also perform qualitative analyses to verify that the proposed approach is interpretable and captures context-dependent word importance.
EduBERT: Pretrained Deep Language Models for Learning Analytics

The use of large pretrained neural networks to create contextualized word embeddings has drastically improved performance on several natural language processing (NLP) tasks. These computationally expensive models have begun to be applied to domain-specific NLP tasks such as re-hospitalization prediction from clinical notes. This paper demonstrates that using large pretrained models produces excellent results on common learning analytics tasks. Pre-training deep language models using student forum data from a wide array of online courses improves performance beyond the state of the art on three text classification tasks. We also show that a smaller, distilled version of our model produces the best results on two of the three tasks while limiting computational cost. We make both models available to the research community at large.
Automatic Generation of Headlines for Online Math Questions

Mathematical equations are an important part of dissemination and communication of scientific information. Students, however, often feel challenged in reading and understanding math content and equations. With the development of the Web, students are posting their math questions online. Nevertheless, constructing a concise math headline that gives a good description of the posted detailed math question is nontrivial. In this study, we explore a novel summarization task denoted as geNerating A concise Math hEadline from a detailed math question (NAME). Compared to conventional summarization tasks, this task has two extra and essential constraints: 1) Detailed math questions consist of text and math equations which require a unified framework to jointly model textual and mathematical information; 2) Unlike text, math equations contain semantic and structural features, and both of them should be captured together. To address these issues, we propose MathSum, a novel summarization model which utilizes a pointer mechanism combined with a multi-head attention mechanism for mathematical representation augmentation. The pointer mechanism can either copy textual tokens or math tokens from source questions in order to generate math headlines. The multi-head attention mechanism is designed to enrich the representation of math equations by modeling and integrating both its semantic and structural features. For evaluation, we collect and make available two sets of real-world detailed math questions along with human-written math headlines, namely EXEQ-300k and OFEQ-10k. Experimental results demonstrate that our model (MathSum) significantly outperforms state-of-the-art models for both the EXEQ-300k and OFEQ-10k datasets.
Optical transmission matrix measurement sampled on a dense hexagonal lattice

The optical transmission matrix (TM) characterizes the transmission properties of a sample. We show a novel experimental procedure for measuring the TM of light waves in a slab geometry based on sampling the light field on a hexagonal lattice at the Rayleigh criterion. Our method enables the efficient measurement of a large fraction of the complete TM without oversampling while minimizing sampling crosstalk and the associated distortion of the statistics of the matrix elements. The procedure and analysis described here is demonstrated on a clear sample which serves as an important reference for other systems and geometries such as dense scattering media.
Two mass scales for the Higgs field

In the original version of the theory, the driving mechanism for spontaneous symmetry breaking was identified in the pure scalar sector. However, this old idea requires a heavy Higgs particle that, after the discovery of the 125 GeV resonance, seems to be ruled out. We argue that this is not necessarily true. If the phase transition is weakly first order, as indicated by most recent lattice simulations, one should consider those approximation schemes that are in agreement with this scenario. Then, even in a simple one-component theory, it becomes natural to introduce two mass scales, say $M_h$ and $m_h$ with $m_h \ll M_h$. This resembles the coexistence of phonons and rotons in superfluid helium-4, which is the non-relativistic analogue of the scalar condensate, and is potentially relevant for the Standard Model. In fact, vacuum stability would depend on $M_h$ and not on $m_h$ and be nearly insensitive to the other parameters of the theory (e.g. the top quark mass). By identifying $m_h=125$ GeV, and with our previous estimate from lattice simulations $M_h= 754 \pm 20 ~\rm{(stat)} \pm 20 ~\rm{(syst)}$ GeV, we thus get in touch with a recent, independent analysis of the ATLAS + CMS data which claims experimental evidence for a scalar resonance around $700$ GeV.