Miracle-dz/UniFAR-FTUs huggingface dataset:
Spectral Smoothing Unveils Phase Transitions in Hierarchical Variational Autoencoders

Variational autoencoders with deep stochastic hierarchies are known to suffer from the problem of posterior collapse, where the top layers fall back to the prior and become independent of input. We suggest that the hierarchical VAE objective explicitly includes the variance of the function pa-rameterizing the mean and variance of the latent Gaussian distribution which itself is often a high variance function. Building on this we generalize VAE neural networks by incorporating a smoothing parameter motivated by Gaussian analysis to reduce higher frequency components and consequently the variance in parameterizing functions. We show this helps to solve the problem of posterior collapse. We further show that under such smoothing the VAE loss exhibits a phase transition, where the top layer KL divergence sharply drops to zero at a critical value of the smoothing parameter that is similar for the same model across datasets. We validate the phenomenon across model conﬁgurations and datasets.
Self-Governance in Generalized Exchange. A Laboratory Experiment on the Structural Embeddedness of Peer Punishment

Peer punishment is widely lauded as a decentralized solution to the problem of social cooperation. However, experimental evidence of its effectiveness primarily stems from public good structures. This paper explores peer punishment in another structural setting: a system of generalized exchange. In a laboratory experiment, a repeated four-player prisoner’s dilemma is arranged either in a public good structure or in a circular network of generalized exchange. The experimental results demonstrate that the merits of peer punishment do not extend to generalized exchange. In the public good, peer punishment was primarily altruistic, was sensitive to costs, and promoted cooperation. In generalized exchange, peer punishment was also altruistic and relatively frequent, but did not increase cooperation. While the dense punishment network underlying the public good facilitates norm enforcement, generalized exchange decreases control over norm violators and reduces the capacity of peer punishment. I conclude that generalized exchange systems require stronger forms of punishment to sustain social cooperation.
An Analysis of the Impact of Spectral Contrast Feature in Speech Emotion Recognition

Feature extraction is an integral part in speech emotion recognition. Some emotions become indistinguishable from others due to high resemblance in their features, which results in low prediction accuracy. This paper analyses the impact of spectral contrast feature in increasing the accuracy for such emotions. The RAVDESS dataset has been chosen for this study. The SAVEE dataset, CREMA-D dataset and JL corpus dataset were also used to test its performance over different English accents. In addition to that, EmoDB dataset has been used to study its performance in the German language. The use of spectral contrast feature has increased the prediction accuracy in speech emotion recognition systems to a good degree as it performs well in distinguishing emotions with significant differences in arousal levels, and it has been discussed in detail.
Machine Learning Based Approach for the Automated Mapping of Discovered Vulnerabilities to Adversial Tactics

To defend networks against security attacks, cyber defenders have to identify vulnerabilities that could be exploited by an attacker and fix them. However, vulnerabilities are constantly evolving and their number is rising. In addition, the resources required (i.e., time and cost) to patch all the identified vulnerabilities and update the affected assets are not always affordable. For these reasons, the defender needs to have a set of metrics that could be used to automatically map new discovered vulnerabilities to potential attack tactics. Using such a mapping to attack tactics, will allow security solutions to better respond inline to any vulnerabilities exploitation tentatives, by selecting and prioritizing suitable response strategy. In this work, we provide a multilabel classification approach to automatically map a detected vulnerability to the MITRE Adversarial Tactics that could be used by the attacker. The proposed approach will help cyber defenders to prioritize their defense strategies, ensure a rapid and efficient investigation process, and well manage new detected vulnerabilities. We evaluate a set of machine learning algorithms (BinaryRelevance, LabelPowerset, ClassifierChains, MLKNN, BRKNN, RAkELd, NLSP, and Neural Networks) and found out that ClassifierChains with RandomForest classifier is the best method in our experiment.
Streaming and Distributed Algorithms for Robust Column Subset Selection

We give the first single-pass streaming algorithm for Column Subset Selection with respect to the entrywise `p-norm with 1 ≤ p < 2. We study the `p norm loss since it is often considered more robust to noise than the standard Frobenius norm. Given an input matrix A ∈ Rd×n (n d), our algorithm achieves a multiplicative k 1 p− 1 2 poly(log nd)-approximation to the error with respect to the best possible column subset of size k. Furthermore, the space complexity of the streaming algorithm is optimal up to a logarithmic factor. Our streaming algorithm also extends naturally to a 1-round distributed protocol with nearly optimal communication cost. A key ingredient in our algorithms is a reduction to column subset selection in the `p,2-norm, which corresponds to the p-norm of the vector of Euclidean norms of each of the columns of A. This enables us to leverage strong coreset constructions for the Euclidean norm, which previously had not been applied in this context. We also give the first provable guarantees for greedy column subset selection in the `1,2 norm, which can be used as an alternative, practical subroutine in our algorithms. Finally, we show that our algorithms give significant practical advantages on real-world data analysis tasks.
Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrations

We consider large-scale Markov decision processes with an unknown cost function and address the problem of learning a policy from a finite set of expert demonstrations. We assume that the learner is not allowed to interact with the expert and has no access to reinforcement signal of any kind. Existing inverse reinforcement learning methods come with strong theoretical guarantees, but are computationally expensive, while state-of-the-art policy optimization algorithms achieve significant empirical success, but are hampered by limited theoretical understanding. To bridge the gap between theory and practice, we introduce a novel bilinear saddle-point framework using Lagrangian duality. The proposed primal-dual viewpoint allows us to develop a model-free provably efficient algorithm through the lens of stochastic convex optimization. The method enjoys the advantages of simplicity of implementation, low memory requirements, and computational and sample complexities independent of the number of states. We further present an equivalent no-regret online-learning interpretation.
Model Performance Scaling with Multiple Data Sources

Real-world machine learning systems are often trained using a mix of data sources with varying cost and quality. Understanding how the size and composition of a training dataset affect model performance is critical for advancing our understanding of generalization, as well as designing more effective data collection policies. We show that there is a simple scaling law that predicts the loss incurred by a model even under varying dataset composition. Our work expands recent observations of scaling laws for log-linear generalization error in the i.i. d setting and uses this to cast model performance prediction as a learning problem. Using the theory of optimal experimental design, we derive a simple rational function approximation to generalization error that can be ﬁtted using a few model training runs. Our approach can achieve highly accurate ( r 2 ≈ . 9 ) predictions of model performance under substantial extrapolation in two different standard supervised learning tasks and is accurate ( r 2 ≈ . 83 ) on more challenging machine translation and question answering tasks where many baselines achieve worse-than-random performance.
Hierarchical Clustering of Data Streams: Scalable Algorithms and Approximation Guarantees

We investigate the problem of hierarchically clustering data streams containing metric data in R d . We introduce a desirable invariance property for such algorithms, describe a general family of hyperplane-based methods enjoying this property, and analyze two scalable instances of this general family against recently popularized similarity/dissimilarity-based metrics for hierarchical clustering. We prove a number of new results related to the approximation ratios of these algorithms, improving in various ways over the literature on this subject. Finally, since our algorithms are principled but also very practical, we carry out an experimental comparison on both synthetic and real-world datasets showing competitive results against known baselines.
Consistent Nonparametric Methods for Network Assisted Covariate Estimation

Networks with node covariates are commonplace: for example, people in a social network have interests, or product preferences, etc. If we know the covariates for some nodes, can we infer them for the remaining nodes? In this paper we propose a new similarity measure between two nodes based on the patterns of their 2-hop neighborhoods. We show that a simple algorithm (CN-VEC) like nearest neighbor regression with this metric is consistent for a wide range of models when the degree grows faster than n 1 / 3 up-to logarithmic factors, where n is the number of nodes. For “low-rank” latent variable models, the natural contender will be to estimate the latent variables using SVD and use them for non-parametric regression. While we show consistency of this method under less stringent sparsity conditions, our experimental results suggest that the simple local CN-VEC method either outperforms the global SVD-RBF method, or has comparable performance for low rank models. We also present simulated and real data experiments to show the effectiveness of our algorithms compared to the state of the art.
Model Fusion for Personalized Learning

Production systems operating on a growing domain of analytic services often require generating warm-start solution models for emerging tasks with limited data. One potential approach to address this challenge is to adopt meta learning to generate a base model that can be adapted to solve unseen tasks with minimal ﬁne-tuning. This however requires the training processes of previous solution models of existing tasks to be synchronized. This is not possible if these models were pre-trained separately on private data owned by different entities and cannot be synchronously re-trained. To accommodate for such scenarios, we develop a new personalized learning framework that synthesizes customized models for un-seen tasks via fusion of independently pre-trained models of related tasks. We also establish performance guarantee for the proposed framework and demonstrate its effectiveness empirically.
Knowledge and Innovation Management Model in the Organizational Environment

The objective of this research is to develop a methodology that allows the construction of a standard of knowledge management and technological innovation applicable to the field of the universities in Mexico, the use of which results in better use of their resources and the improvement of their productivity. The analysis is carried out based on the systematic integration of the knowledge management and innovation models published in recent years, as well as the results of the experiences that propose standards for knowledge management and innovation. The results found are that knowledge management favors and facilitates innovation in organizations and allows them to offer better products and services. These findings are with the application of a methodology and an instrument to measure the maturity of the organizational structure. The application of the evaluation instrument was applied in a single organization. However, it can be considered a case study that empirically contributes to the state of the art of knowledge management. Organizations need to become aware of the high value of the knowledge and experience in various fields of the people who participate in them. It has been observed that the participation of experts facilitates the achievement of organizational objectives and favors knowledge management and innovation. This study provided information and experiences of an administrative nature to a higher education institution, which allowed an improvement in strategic decision-making, the promotion of technological developments, and the improvement of internal processes that lead to greater effectiveness of the academic and research processes.
Fixed-Parameter and Approximation Algorithms for PCA with Outliers

PCA WITH O UTLIERS is the fundamental problem of identifying an underlying low-dimensional subspace in a data set corrupted with outliers. A large body of work is devoted to the information-theoretic aspects of this problem. However, from the computational perspective, its complexity is still not well-understood. We study this problem from the perspective of parameterized complexity by investigating how parameters like the di-mension of the data, the subspace dimension, the number of outliers and their structure, and approximation error, inﬂuence the computational complexity of the problem. Our algorithmic meth-ods are based on techniques of randomized linear algebra and algebraic geometry.
KO codes: inventing nonlinear encoding and decoding for reliable wireless communication via deep-learning

Landmark codes underpin reliable physical layer communication, e.g., Reed-Muller, BCH, Convolution, Turbo, LDPC and Polar codes: each is a linear code and represents a mathematical breakthrough. The impact on humanity is huge: each of these codes has been used in global wireless communication standards (satellite, WiFi, cellular). Reliability of communication over the classical additive white Gaussian noise (AWGN) channel enables benchmarking and ranking of the different codes. In this paper, we construct KO codes, a computationaly efficient family of deep-learning driven (encoder, decoder) pairs that outperform the state-of-the-art reliability performance on the standardized AWGN channel. KO codes beat state-of-the-art Reed-Muller and Polar codes, under the low-complexity successive cancellation decoding, in the challenging short-to-medium block length regime on the AWGN channel. We show that the gains of KO codes are primarily due to the nonlinear mapping of information bits directly to transmit real symbols (bypassing modulation) and yet possess an efficient, high performance decoder. The key technical innovation that renders this possible is design of a novel family of neural architectures inspired by the computation tree of the {\bf K}ronecker {\bf O}peration (KO) central to Reed-Muller and Polar codes. These architectures pave way for the discovery of a much richer class of hitherto unexplored nonlinear algebraic structures. The code is available at \href{https://github.com/deepcomm/KOcodes}{https://github.com/deepcomm/KOcodes}
Reward Identification in Inverse Reinforcement Learning

We study the problem of reward identiﬁability in the context of Inverse Reinforcement Learning (IRL). The reward identiﬁability question is critical to answer when reasoning about the effectiveness of using Markov Decision Processes (MDPs) as computational models of real world decision makers in order to understand complex decision making behavior and perform counter-factual reasoning. While identiﬁability has been acknowledged as a fundamental theoretical question in IRL, little is known about the types of MDPs for which rewards are identiﬁable, or even if there exist such MDPs. In this work, we formalize the reward identiﬁcation problem in IRL and study how identiﬁability relates to properties of the MDP model. For deterministic MDP models with the MaxEntRL objective, we prove necessary and sufﬁcient conditions for identiﬁability. Building on these results, we present efﬁcient algorithms for testing whether or not an MDP model is identiﬁable.
When Does Data Augmentation Help With Membership Inference Attacks?

Deep learning models often raise privacy concerns as they leak information about their training data. This leakage enables membership inference attacks (MIA) that can identify whether a data point was in a model’s training set. Research shows that some data augmentation mechanisms may reduce the risk by combatting a key factor increasing the leakage, overﬁtting. While many mechanisms exist, their effectiveness against MIAs and privacy properties have not been studied systematically. Employing two recent MIAs, we explore the lower bound on the risk in the absence of formal upper bounds. First, we evaluate 7 mechanisms and differential privacy, on three image classiﬁcation tasks. We ﬁnd that applying augmentation to increase the model’s utility does not mitigate the risk and protection comes with a utility penalty. Further, we also investigate why popular label smoothing mechanism consistently ampliﬁes the risk. Finally, we propose loss-rank-correlation (LRC) metric to assess how similar the effects of different mechanisms are. This, for example, reveals the similarity of applying high-intensity augmentation against MIAs to simply reducing the training time. Our ﬁndings emphasize the utility-privacy trade-off and provide practical guidelines on using augmentation to manage the trade-off.
Parametric Graph for Unimodal Ranking Bandit

We tackle the online ranking problem of assigning L items to K positions on a web page in order to maximize the number of user clicks. We propose an original algorithm, easy to implement and with strong theoretical guarantees to tackle this problem in the Position-Based Model (PBM) setting, well suited for applications where items are displayed on a grid. Besides learning to rank, our algorithm, GRAB (for parametric Graph for uni-modal RAnking Bandit ), also learns the parameter of a compact graph over permutations of K items among L . The logarithmic regret bound of this algorithm is a direct consequence of the unimodal-ity property of the bandit setting with respect to the learned graph. Experiments against state-of-the-art learning algorithms which also tackle the PBM setting, show that our method is more efﬁ-cient while giving regret performance on par with the best known algorithms on simulated and real life datasets.
BasisDeVAE: Interpretable Simultaneous Dimensionality Reduction and Feature-Level Clustering with Derivative-Based Variational Autoencoders

The Variational Autoencoder (VAE) performs effective nonlinear dimensionality reduction in a variety of problem settings. However, the black-box neural network decoder function typically employed limits the ability of the decoder function to be constrained and interpreted, making the use of VAEs problematic in settings where prior knowledge should be embedded within the decoder. We present DeVAE, a novel VAE-based model with a derivative-based forward mapping, allowing for greater control over decoder behaviour via speciﬁ-cation of the decoder function in derivative space. Additionally, we show how DeVAE can be paired with a sparse clustering prior to create BasisDe-VAE and perform interpretable simultaneous dimensionality reduction and feature-level clustering. We demonstrate the performance and scalability of the DeVAE and BasisDeVAE models on synthetic and real-world data and present how the derivative-based approach allows for expressive yet interpretable forward models which respect prior knowledge.
Optimal Streaming Algorithms for Multi-Armed Bandits

This paper studies two variants of the best arm identiﬁcation (BAI) problem under the streaming model, where we have a stream of n arms with reward distributions supported on [0 , 1] with unknown means. The arms in the stream are arriving one by one, and the algorithm cannot access an arm unless it is stored in a limited size memory. We ﬁrst study the streaming ε - top - k arms identiﬁ-cation problem, which asks for k arms whose reward means are lower than that of the k -th best arm by at most ε with probability at least 1 − δ . For general ε ∈ (0 , 1) , the existing solu-tion for this problem assumes k = 1 and achieves the optimal sample complexity O ( nε 2 log 1 δ ) using O (log ∗ ( n )) 1 memory and a single pass of the stream. We propose an algorithm that works for any k and achieves the optimal sample complexity O ( nε 2 log kδ ) using a single-arm memory and a single pass of the stream. Second, we study the streaming BAI problem, where the objective is to identify the arm with the maximum reward mean with at least 1 − δ probability, using a single-arm memory and as few passes of the input stream as possible. We present a single-arm-memory algorithm that achieves a near instance-dependent optimal sample complexity within O (log ∆ − 1 2 ) passes, where ∆ 2 is the gap between the mean of the best arm and that of the second best arm.
On the Convergence of Hamiltonian Monte Carlo with Stochastic Gradients

Hamiltonian Monte Carlo (HMC), built based on the Hamilton’s equation, has been witnessed great success in sampling from high-dimensional posterior distributions. However, it also suffers from computational inefﬁciency, especially for large training datasets. One common idea to overcome this computational bottleneck is using stochastic gradients, which only queries a mini-batch of training data in each iteration. However, unlike the extensive studies on the convergence analysis of HMC using full gradients, few works focus on establishing the convergence guarantees of stochastic gradient HMC algorithms. In this paper, we propose a general framework for proving the convergence rate of HMC with stochastic gradient estimators, for sampling from strongly log-concave and log-smooth target distributions. We show that the convergence to the target distribution in 2 -Wasserstein distance can be guaranteed as long as the stochastic gradient estimator is unbiased and its variance is upper bounded along the algorithm trajectory. We further apply the proposed framework to analyze the convergence rates of HMC with four standard stochastic gradient estimators: mini-batch stochastic gradient (SG), stochastic variance reduced gradient (SVRG), stochastic average gradient (SAGA), and control variate gradient (CVG). Theoretical re-sults explain the inefﬁciency of mini-batch SG, and suggest that SVRG and SAGA perform better in the tasks with high-precision requirements, while CVG performs better for large dataset. Experiment results verify our theoretical ﬁndings.
On Estimation in Latent Variable Models

Latent variable models have been playing a central role in statistics, econometrics, machine learning with applications to repeated observation study, panel data inference, user behavior analysis, etc. In many modern applications, the inference based on latent variable models involves one or several of the following features: the presence of complex latent structure, the observed and latent variables being continuous or discrete, constraints on parameters, and data size being large. Therefore, solving an estimation problem for general latent variable models is highly non-trivial. In this paper, we consider a gradient based method via using variance reduction technique to accelerate estimation procedure. Theoretically, we show the convergence results for the proposed method under general and mild model assumptions. The algorithm has better computational complexity compared with the classical gradient methods and maintains nice statistical properties. Various numerical results corroborate our theory.