Miracle-dz/UniFAR-FTUs huggingface dataset:
Explainability in Machine Learning: a Pedagogical Perspective

Given the importance of integrating of explainability into machine learning, at present, there are a lack of pedagogical resources exploring this. Specifically, we have found a need for resources in explaining how one can teach the advantages of explainability in machine learning. Often pedagogical approaches in the field of machine learning focus on getting students prepared to apply various models in the real world setting, but much less attention is given to teaching students the various techniques one could employ to explain a model’s decision-making process. Furthermore, explainability can benefit from a narrative structure that aids one in understanding which techniques are governed by which questions about the data. We provide a pedagogical perspective on how to structure the learning process to better impart knowledge to students and researchers in machine learning, when and how to implement various explainability techniques as well as how to interpret the results. We discuss a system of teaching explainability in machine learning, by exploring the advantages and disadvantages of various opaque and transparent machine learning models, as well as when to utilize specific explainability techniques and the various frameworks used to structure the tools for explainability. Among discussing concrete assignments, we will also discuss ways to structure potential assignments to best help students learn to use explainability as a tool alongside any given machine learning application. Data science professionals completing the course will have a birds-eye view of a rapidly developing area and will be confident to deploy machine learning more widely. A preliminary analysis on the effectiveness of a recently delivered course following the structure presented here is included as evidence supporting our pedagogical approach.
Efficient Cross-Modal Retrieval via Deep Binary Hashing and Quantization

Cross-modal retrieval aims to search for data with similar semantic meanings across different content modalities. However, cross-modal retrieval requires huge amounts of storage and retrieval time since it needs to process data in multiple modalities. Existing works focused on learning single-source compact features such as binary hash codes that preserve similarities between different modalities. In this work, we propose a jointly learned deep hashing and quantization network (HQ) for cross-modal retrieval. We simultaneously learn binary hash codes and quantization codes to preserve semantic information in multiple modalities by an end-to-end deep learning architecture. At the retrieval step, binary hashing is used to retrieve a subset of items from the search space, then quantization is used to re-rank the retrieved items. We theoretically and empirically show that this two-stage retrieval approach provides faster retrieval results while preserving accuracy. Experimental results on the NUS-WIDE, MIR-Flickr, and Amazon datasets demonstrate that HQ achieves boosts of more than 7% in precision compared to supervised neural network-based compact coding models.
Optimizing Recursive Queries with Progam Synthesis

Most work on query optimization has concentrated on loop-free queries. However, data science and machine learning workloads today typically involve recursive or iterative computation. In this work, we propose a novel framework for optimizing recursive queries using methods from program synthesis. In particular, we introduce a simple yet powerful optimization rule called the "FGH-rule" which aims to find a faster way to evaluate a recursive program. The solution is found by making use of powerful tools, such as a program synthesizer, an SMT-solver, and an equality saturation system. We demonstrate the strength of the optimization by showing that the FGH-rule can lead to speedups up to 4 orders of magnitude on three, already optimized Datalog systems.
Long Run Risk in Stationary Structural Vector Autoregressive Models

This paper introduces a local-to-unity/small sigma process for a stationary time series with strong persistence and non-negligible long run risk. This process represents the stationary long run component in an unobserved shortand long-run components model involving different time scales. More specifically, the short run component evolves in the calendar time and the long run component evolves in an ultra long time scale. We develop the methods of estimation and long run prediction for the univariate and multivariate Structural VAR (SVAR) models with unobserved components and reveal the impossibility to consistently estimate some of the long run parameters. The approach is illustrated by a Monte-Carlo study and an application to macroeconomic data.
Van der Waals equation of state for asymmetric nuclear matter

The application of the van der Waals equation of state to the asymmetric nuclear matter is considered in a critical state region. The corrections to the van der Waals pressure and free energy due to the Fermi statistics are obtained starting from the Thomas - Fermi entropy expression which ensures the fulfilment of the Nernst theorem. The derived corrections account for the effective nucleon mass and neutron-proton isotopic asymmetry. The parameters of the van der Waals equation of state are deduced by taking the experimental value of critical temperature for symmetric nuclear matter and testing the model of van der Waals with statistics corrections included against the theory of Skyrme energy density functional. A critical line in pressure-temperature-composition space is considered. The incompressibility coefficient is determined along the critical line as a function of nuclear matter composition. A jump in the value of specific heat upon crossing a critical line is discussed.
Toric surfaces with symmetries by reflections

Let W be a reflection group in a plane and P a rational polygon that is invariant under the W -action. The action of W on P induces a W action on the toric variety XP associated with P . In this paper, we study the W -representation on the cohomology H ̊pXP q and show that the invariant subring H ̊pXP q W is isomorphic to the cohomology ring of the toric variety associated with the fundamental region P {W . As an example, we provide an explicit description of the main result for the case of the toric variety associated with the fan of Weyl chambers of type G2.
It's Raw! Audio Generation with State-Space Models

Developing architectures suitable for modeling raw audio is a challenging problem due to the high sampling rates of audio waveforms. Standard sequence modeling approaches like RNNs and CNNs have previously been tailored to fit the demands of audio, but the resultant architectures make undesirable computational tradeoffs and struggle to model waveforms effectively. We propose SaShiMi, a new multi-scale architecture for waveform modeling built around the recently introduced S4 model for long sequence modeling. We identify that S4 can be unstable during autoregressive generation, and provide a simple improvement to its parameterization by drawing connections to Hurwitz matrices. SaShiMi yields state-of-the-art performance for unconditional waveform generation in the autoregressive setting. Additionally, SaShiMi improves non-autoregressive generation performance when used as the backbone architecture for a diffusion model. Compared to prior architectures in the autoregressive generation setting, SaShiMi generates piano and speech waveforms which humans find more musical and coherent respectively, e.g. 2x better mean opinion scores than WaveNet on an unconditional speech generation task. On a music generation task, SaShiMi outperforms WaveNet on density estimation and speed at both training and inference even when using 3x fewer parameters. Code can be found at https://github.com/HazyResearch/state-spaces and samples at https://hazyresearch.stanford. edu/sashimi-examples.
Seeing the advantage: visually grounding word embeddings to better capture human semantic knowledge

Distributional semantic models capture word-level meaning that is useful in many natural language processing tasks and have even been shown to capture cognitive aspects of word meaning. The majority of these models are purely text based, even though the human sensory experience is much richer. In this paper we create visually grounded word embeddings by combining English text and images and compare them to popular text-based methods, to see if visual information allows our model to better capture cognitive aspects of word meaning. Our analysis shows that visually grounded embedding similarities are more predictive of the human reaction times in a large priming experiment than the purely text-based embeddings. The visually grounded embeddings also correlate well with human word similarity ratings. Importantly, in both experiments we show that the grounded embeddings account for a unique portion of explained variance, even when we include text-based embeddings trained on huge corpora. This shows that visual grounding allows our model to capture information that cannot be extracted using text as the only source of information.
Spatio-Temporal EEG Representation Learning on Riemannian Manifold and Euclidean Space

We present a novel deep neural architecture for learning electroencephalogram (EEG). To learn the spatial information, our model first obtains the Riemannian mean and distance from spatial covariance matrices (SCMs) on a Riemannian manifold. We then project the spatial information onto a Euclidean space via tangent space learning. Following, two fully connected layers are used to learn the spatial information embeddings. Moreover, our proposed method learns the temporal information via differential entropy and logarithm power spectrum density features extracted from EEG signals in a Euclidean space using a deep long short-term memory network with a soft attention mechanism. To combine the spatial and temporal information, we use an effective fusion strategy, which learns attention weights applied to embedding-specific features for decision making. We evaluate our proposed framework on four public datasets across three popular EEG-related tasks, notably emotion recognition, vigilance estimation, and motor imagery classification, containing various types of tasks such as binary classification, multi-class classification, and regression. Our proposed architecture outperforms other methods on SEED-VIG, and approaches the state-of-the-art on the other three datasets (SEED, BCI-IV 2 A, and BCI-IV 2B), showing the robustness of our framework in EEG representation learning.
PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior

Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density. The framework defines the prior noise as a standard Gaussian distribution, whereas the corresponding data distribution may be more complicated than the standard Gaussian distribution, which potentially introduces inefficiency in denoising the prior noise into the data sample because of the discrepancy between the data and the prior. In this paper, we propose PriorGrad to improve the efficiency of the conditional diffusion model for speech synthesis (for example, a vocoder using a mel-spectrogram as the condition) by applying an adaptive prior derived from the data statistics based on the conditional information. We formulate the training and sampling procedures of PriorGrad and demonstrate the advantages of an adaptive prior through a theoretical analysis. Focusing on the speech synthesis domain, we consider the recently proposed diffusion-based speech generative models based on both the spectral and time domains and show that PriorGrad achieves faster convergence and inference with superior performance, leading to an improved perceptual quality and robustness to a smaller network capacity, and thereby demonstrating the efficiency of a data-dependent adaptive prior.
Simulation of the bi-directional flow effects on the indoor walkway

Bi-directional flow is a common phenomenon in indoor walkways of public venues. Unreasonable design of the walkway can incur crowd blocks and even result in trample disaster. How to determine the maximum flow of the walkway is a key issue in its design. The paper studied the relationship between the walking speed and maximum flow rates under different flow ratios by simulation. The maximum flow rates of different flow ratios, the free walking speed of low density as well as the walking speed of maximum flow rates are all derived from simulations. The function between the relationship of flow ratio and maximum flow was deduced and standardized according to the results of simulation. It was found that the ratio of the bi-directional flow affected the maximum flow and the maximum walking speed greatly, but hardly affected the free walking speed. The results of the research can benefit the design of the walkway and help evaluate the performance of pedestrian facilities.
Transformer Quality in Linear Time

We revisit the design choices in Transformers, and propose methods to address their weaknesses in handling long sequences. First, we propose a simple layer named gated attention unit, which allows the use of a weaker single-head attention with minimal quality loss. We then propose a linear approximation method complementary to this new layer, which is accelerator-friendly and highly competitive in quality. The resulting model, named FLASH, matches the perplexity of improved Transformers over both short (512) and long (8K) context lengths, achieving training speedups of up to 4.9$\times$ on Wiki-40B and 12.1$\times$ on PG-19 for auto-regressive language modeling, and 4.8$\times$ on C4 for masked language modeling.
A Variance-Reduced Stochastic Accelerated Primal Dual Algorithm

In this work, we consider strongly convex strongly concave (SCSC) saddle point (SP) problems $\min_{x\in\mathbb{R}^{d_x}}\max_{y\in\mathbb{R}^{d_y}}f(x,y)$ where $f$ is $L$-smooth, $f(.,y)$ is $\mu$-strongly convex for every $y$, and $f(x,.)$ is $\mu$-strongly concave for every $x$. Such problems arise frequently in machine learning in the context of robust empirical risk minimization (ERM), e.g. $\textit{distributionally robust}$ ERM, where partial gradients are estimated using mini-batches of data points. Assuming we have access to an unbiased stochastic first-order oracle we consider the stochastic accelerated primal dual (SAPD) algorithm recently introduced in Zhang et al. [2021] for SCSC SP problems as a robust method against gradient noise. In particular, SAPD recovers the well-known stochastic gradient descent ascent (SGDA) as a special case when the momentum parameter is set to zero and can achieve an accelerated rate when the momentum parameter is properly tuned, i.e., improving the $\kappa \triangleq L/\mu$ dependence from $\kappa^2$ for SGDA to $\kappa$. We propose efficient variance-reduction strategies for SAPD based on Richardson-Romberg extrapolation and show that our method improves upon SAPD both in practice and in theory.
On the Best Lattice Quantizers

A lattice quantizer approximates an arbitrary real-valued source vector with a vector taken from a specific discrete lattice. The quantization error is the difference between the source vector and the lattice vector. In a classic 1996 paper, Zamir and Feder show that the globally optimal lattice quantizer (which minimizes the mean square error) has white quantization error: for a uniformly distributed source, the covariance of the error is the identity matrix, multiplied by a positive real factor. We generalize the theorem, showing that the same property holds (i) for any lattice whose mean square error cannot be decreased by a small perturbation of the generator matrix, and (ii) for an optimal product of lattices that are themselves locally optimal in the sense of (i). We derive an upper bound on the normalized second moment (NSM) of the optimal lattice in any dimension, by proving that any lower- or upper-triangular modification to the generator matrix of a product lattice reduces the NSM. Using these tools and employing the best currently known lattice quantizers to build product lattices, we construct improved lattice quantizers in dimensions 13 to 15, 17 to 23, and 25 to 48. In some dimensions, these are the first reported lattices with normalized second moments below the best known upper bound.
A Lightweight Dual-Domain Attention Framework for Sparse-View CT Reconstruction

Computed tomography (CT) plays an essential role in clinical diagnosis. Sparse sampling is an effective way to reduce the radiation on patients, but it will lead to severe artifacts on the reconstructed CT images, and artifacts may interfere with the diagnosis. Therefore, it is significant to ensure the quality of the region of interest, with extra parameters as few as possible. In this paper, we design a novel lightweight network called CAGAN and propose a dual-domain reconstruction pipeline for parallel beam sparse-view CT. CAGAN is an adversarial auto-encoder with the coordinate attention (CA), which preserves the spatial information of features and learns to focus on the textures. Also, the application of lightweight convolutional blocks reduces the parameters by a quarter without performance loss. Taking 180 views as full-view, our model reaches a PSNR of 41.975 given 45 views and a PSNR of 34.546 given 10 views, which is extremely sparse.
Global convergence and acceleration of ﬁxed point iterations of union upper semicontinuous operators: proximal algorithms, alternating and averaged nonconvex projections, and linear complementarity problems

We propose a uniﬁed framework to analyze ﬁxed point iterations of a set-valued operator that is the union of a ﬁnite number of upper semicontinuous maps, each with a nonempty closed domain and compact values. We discuss global convergence, local linear convergence under a calmness condition, and component identiﬁcation, and further propose acceleration strategies that drastically improve the convergence speed. Our framework is applied to analyze a class of proximal algorithms for minimizing the sum of a piecewise smooth function and the diﬀerence between pointwise minimum of ﬁnitely many weakly convex functions and a piecewise smooth convex function. When realized on two-set feasibility problems, this algorithm class recovers alternating projections and averaged projections as special cases, and our framework thus equips these classical methods with global convergence and possibilities for acceleration on a broad class of nonconvex feasibility problems. By specializing the framework to a nonconvex feasibility problem reformulation of the linear complementarity problem, we show global convergence to a solution from any initial point, with a local linear rate, of the alternating projection as well as the averaged projection methods, which is diﬃcult to obtain on nonconvex problems. Numerical results further exemplify that the proposed acceleration algorithms signiﬁcantly improve upon their non-accelerated counterparts in eﬃciency.
Poisoning Attacks and Defenses on Artificial Intelligence: A Survey

Machine learning models have been widely adopted in several fields. However, most recent studies have shown several vulnerabilities from attacks with a potential to jeopardize the integrity of the model, presenting a new window of research opportunity in terms of cyber-security. This survey is conducted with a main intention of highlighting the most relevant information related to security vulnerabilities in the context of machine learning (ML) classifiers; more specifically, directed towards training procedures against data poisoning attacks, representing a type of attack that consists of tampering the data samples fed to the model during the training phase, leading to a degradation in the models accuracy during the inference phase. This work compiles the most relevant insights and findings found in the latest existing literatures addressing this type of attacks. Moreover, this paper also covers several defense techniques that promise feasible detection and mitigation mechanisms, capable of conferring a certain level of robustness to a target model against an attacker. A thorough assessment is performed on the reviewed works, comparing the effects of data poisoning on a wide range of ML models in real-world conditions, performing quantitative and qualitative analyses. This paper analyzes the main characteristics for each approach including performance success metrics, required hyperparameters, and deployment complexity. Moreover, this paper emphasizes the underlying assumptions and limitations considered by both attackers and defenders along with their intrinsic properties such as: availability, reliability, privacy, accountability, interpretability, etc. Finally, this paper concludes by making references of some of main existing research trends that provide pathways towards future research directions in the field of cyber-security.
Some results for semi-stable radial solutions of k-Hessian equations with weight on ℝn

We devote this paper to study semi-stable nonconstant radial solutions of $S_k(D^2u)=w(\left \vert x \right \vert )g(u)$ on the Euclidean space $\mathbb {R}^n$. We establish pointwise estimates and necessary conditions for the existence of such solutions (not necessarily bounded) for this equation. For bounded solutions we estimate their asymptotic behaviour at infinity. All the estimates are given in terms of the spatial dimension $n$, the values of $k$ and the behaviour at infinity of the growth rate function of $w$.
Coverage-guided tensor compiler fuzzing with joint IR-pass mutation

In the past decade, Deep Learning (DL) systems have been widely deployed in various application domains to facilitate our daily life, e.g., natural language processing, healthcare, activity recognition, and autonomous driving. Meanwhile, it is extremely challenging to ensure the correctness of DL systems (e.g., due to their intrinsic nondeterminism), and bugs in DL systems can cause serious consequences and may even threaten human lives. In the literature, researchers have explored various techniques to test, analyze, and verify DL models, since their quality directly affects the corresponding system behaviors. Recently, researchers have also proposed novel techniques for testing the underlying operator-level DL libraries (such as TensorFlow and PyTorch), which provide general binary implementations for each high-level DL operator and are the foundation for running DL models on different hardware platforms. However, there is still limited work targeting the reliability of the emerging tensor compilers (also known as DL compilers), which aim to automatically compile high-level tensor computation graphs directly into high-performance binaries for better efficiency, portability, and scalability than traditional operator-level libraries. Therefore, in this paper, we target the important problem of tensor compiler testing, and have proposed Tzer, a practical fuzzing technique for the widely used TVM tensor compiler. Tzer focuses on mutating the low-level Intermediate Representation (IR) for TVM due to the limited mutation space for the high-level IR. More specifically, Tzer leverages both general-purpose and tensor-compiler-specific mutators guided by coverage feedback for diverse and evolutionary IR mutation; furthermore, since tensor compilers provide various passes (i.e., transformations) for IR optimization, Tzer also performs pass mutation in tandem with IR mutation for more effective fuzzing. Our experimental results show that Tzer substantially outperforms existing fuzzing techniques on tensor compiler testing, with 75% higher coverage and 50% more valuable tests than the 2nd-best technique. Also, different components of Tzer have been validated via ablation study. To date, Tzer has detected 49 previously unknown bugs for TVM, with 37 bugs confirmed and 25 bugs fixed (PR merged).