Miracle-dz/UniFAR-FTUs huggingface dataset:
Quantization-Based Regularization for Autoencoders

Autoencoders and their variations provide unsupervised models for learning low-dimensional representations for downstream tasks. Without proper regularization, autoencoder models are susceptible to the overfitting problem and the so-called posterior collapse phenomenon. In this paper, we introduce a quantization-based regularizer in the bottleneck stage of autoencoder models to learn meaningful latent representations. We combine both perspectives of Vector Quantized-Variational AutoEncoders (VQ-VAE) and classical denoising regularization methods of neural networks. We interpret quantizers as regularizers that constrain latent representations while fostering a similarity-preserving mapping at the encoder. Before quantization, we impose noise on the latent codes and use a Bayesian estimator to optimize the quantizer-based representation. The introduced bottleneck Bayesian estimator outputs the posterior mean of the centroids to the decoder, and thus, is performing soft quantization of the noisy latent codes. We show that our proposed regularization method results in improved latent representations for both supervised learning and clustering downstream tasks when compared to autoencoders using other bottleneck structures.
Perfect Zero Knowledge for Quantum Multiprover Interactive Proofs

In this work we consider the interplay between multiprover interactive proofs, quantum entanglement, and zero knowledge proofs - notions that are central pillars of complexity theory, quantum information and cryptography. In particular, we study the relationship between the complexity class MIP*, the set of languages decidable by multiprover interactive proofs with quantumly entangled provers, and the class PZK-MIP*, which is the set of languages decidable by MIP* protocols that furthermore possess the perfect zero knowledge property. Our main result is that the two classes are equal, i.e., MIP* = PZK-MIP*. This result provides a quantum analogue of the celebrated result of Ben-Or, Goldwasser, Kilian, and Wigderson (STOC 1988) who show that MIP = PZK-MIP (in other words, all classical multiprover interactive protocols can be made zero knowledge). We prove our result by showing that every MIP* protocol can be efficiently transformed into an equivalent zero knowledge MIP* protocol in a manner that preserves the completeness-soundness gap. Combining our transformation with previous results, we obtain the corollaries that i) all languages that can be solved in non-deterministic double exponential time have zero knowledge MIP* protocols and ii) all co-recursively enumerable languages (which include undecidable problems as well as all decidable problems) have zero knowledge MIP* protocols with vanishing promise gap.
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence

Perhaps the most ambitious scientific quest in human history is the creation of general artificial intelligence, which roughly means AI that is as smart or smarter than humans. The dominant approach in the machine learning community is to attempt to discover each of the pieces required for intelligence, with the implicit assumption that some future group will complete the Herculean task of figuring out how to combine all of those pieces into a complex thinking machine. I call this the ``manual AI approach. '' This paper describes another exciting path that ultimately may be more successful at producing general AI. It is based on the clear trend in machine learning that hand-designed solutions eventually are replaced by more effective, learned solutions. The idea is to create an AI-generating algorithm (AI-GA), which automatically learns how to produce general AI. Three Pillars are essential for the approach: (1) meta-learning architectures, (2) meta-learning the learning algorithms themselves, and (3) generating effective learning environments. I argue that either approach could produce general AI first, and both are scientifically worthwhile irrespective of which is the fastest path. Because both are promising, yet the ML community is currently committed to the manual approach, I argue that our community should increase its research investment in the AI-GA approach. To encourage such research, I describe promising work in each of the Three Pillars. I also discuss AI-GA-specific safety and ethical considerations. Because it it may be the fastest path to general AI and because it is inherently scientifically interesting to understand the conditions in which a simple algorithm can produce general AI (as happened on Earth where Darwinian evolution produced human intelligence), I argue that the pursuit of AI-GAs should be considered a new grand challenge of computer science research.
A collaborative filtering model with heterogeneous neural networks for recommender systems

In recent years, deep neural network is introduced in recommender systems to solve the collaborative filtering problem, which has achieved immense success on computer vision, speech recognition and natural language processing. On one hand, deep neural network can be used to model the auxiliary information in recommender systems. On the other hand, it is also capable of modeling nonlinear relationships between users and items. One advantage of deep neural network is that the performance of the algorithm can be easily enhanced by augmenting the depth of the neural network. However, two potential problems may emerge when the deep neural work is exploited to model relationships between users and items. The fundamental problem is that the complexity of the algorithm grows significantly with the increment in the depth of the neural network. The second one is that a deeper neural network may undermine the accuracy of the algorithm. In order to alleviate these problems, we propose a hybrid neural network that combines heterogeneous neural networks with different structures. The experimental results on real datasets reveal that our method is superior to the state-of-the-art methods in terms of the item ranking.
Magnetoresistive RAM for error resilient XNOR-Nets

We trained three Binarized Convolutional Neural Network architectures (LeNet-4, Network-In-Network, AlexNet) on a variety of datasets (MNIST, CIFAR-10, CIFAR-100, extended SVHN, ImageNet) using error-prone activations and tested them without errors to study the resilience of the training process. With the exception of the AlexNet when trained on the ImageNet dataset, we found that Bit Error Rates of a few percent during training do not degrade the test accuracy. Furthermore, by training the AlexNet on progressively smaller subsets of ImageNet classes, we observed increasing tolerance to activation errors. The ability to operate with high BERs is critical for reducing power consumption in existing hardware and for facilitating emerging memory technologies. We discuss how operating at moderate BER can enable Magnetoresistive RAM with higher endurance, speed and density.
Semi-supervised Learning with Contrastive Predicative Coding

Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, many of them have thus far been either inflexible, inefficient or non-scalable. This paper explores recently developed contrastive predictive coding technique to improve discriminative power of deep learning models when a large portion of labels are absent. Two models, cpc-SSL and a class conditional variant~(ccpc-SSL) are presented. They effectively exploit the unlabeled data by extracting shared information between different parts of the (high-dimensional) data. The proposed approaches are inductive, and scale well to very large datasets like ImageNet, making them good candidates in real-world large scale applications.
Regenerative properties of the linear hawkes process with unbounded memory

We prove regenerative properties for the linear Hawkes process under minimal assumptions on the transfer function, which may have unbounded support. These results are applicable to sliding window statistical estimators. We exploit independence in the Poisson cluster point process decomposition, and the regeneration times are not stopping times for the Hawkes process. The regeneration time is interpreted as the renewal time at zero of a M/G/infinity queue, which yields a formula for its Laplace transform. When the transfer function admits some exponential moments, we stochastically dominate the cluster length by exponential random variables with parameters expressed in terms of these moments. This yields explicit bounds on the Laplace transform of the regeneration time in terms of simple integrals or special functions yielding an explicit negative upper-bound on its abscissa of convergence. These regenerative results allow, e.g., to systematically derive long-time asymptotic results in view of statistical applications. This is illustrated on a concentration inequality previously obtained with coauthors.
Lightlike Neuromanifolds, Occam's Razor and Deep Learning

How do deep neural networks benefit from a very high dimensional parameter space? Their high complexity vs stunning generalization performance forms an intriguing paradox. We took an information-theoretic approach. We find that the locally varying dimensionality of the parameter space can be studied by the discipline of singular semi-Riemannian geometry. We adapt Fisher information to this singular neuromanifold. We use a new prior to interpolate between Jeffreys' prior and the Gaussian prior. We derive a minimum description length of a deep learning model, where the spectrum of the Fisher information matrix plays a key role to reduce the model complexity.
Robust probabilistic modeling of photoplethysmography signals with application to the classification of premature beats

In this paper we propose a robust approach to model photoplethysmography (PPG) signals. After decomposing the signal into two components, we focus the analysis on the pulsatile part, related to cardiac information. The goal is to enable a deeper understanding of the information contained in the pulse shape, together with that derived from the rhythm. Our approach combines functional data analysis with a state space representation and guarantees fitting robustness and flexibility on stationary signals, without imposing a priori information on the waveform and heart rhythm. With a Bayesian approach, we learn the distribution of the parameters, used for understanding and monitoring PPG signals. The model can be used for data compression, for inferring medical parameters and to understand condition-related waveform characteristics. In particular, we detail a procedure for the detection of premature contractions based on the residuals of the fit. This method can handle both atrial and ventricular premature contractions, and classify the type by only using information from the model fit.
PTrix: Efficient Hardware-Assisted Fuzzing for COTS Binary

Despite its effectiveness in uncovering software defects, American Fuzzy Lop (AFL), one of the best grey-box fuzzers, is inefficient when fuzz-testing source-unavailable programs. AFL's binary-only fuzzing mode, QEMU-AFL, is typically 2-5× slower than its source- available fuzzing mode. The slowdown is largely caused by the heavy dynamic instrumentation. Recent fuzzing techniques use Intel Processor Tracing (PT), a light-weight tracing feature supported by recent Intel CPUs, to re- move the need of dynamic instrumentation. However, we found that these PT-based fuzzing techniques are even slower than QEMU-AFL when fuzzing real-world programs, making them less effective than QEMU-AFL. This poor performance is caused by the slow extraction of code coverage information from highly compressed PT traces. In this work, we present the design and implementation of PTrix, which fully unleashes the benefits of PT for fuzzing via three novel techniques. First, PTrix introduces a scheme to highly parallel the processing of PT trace and target program execution. Second, it directly takes decoded PT trace as feedback for fuzzing, avoiding the expensive reconstruction of code coverage information. Third, PTrix maintains the new feedback with stronger feedback than edge-based code coverage, which helps reach new code space and defects that AFL may not. We evaluated PTrix by comparing its performance with the state- of-the-art fuzzers. Our results show that, given the same amount of time, PTrix achieves a significantly higher fuzzing speed and reaches into code regions missed by the other fuzzers. In addition, PTrix identifies 35 new vulnerabilities in a set of previously well- fuzzed binaries, showing its ability to complement existing fuzzers.
UWB-NTIS Speaker Diarization System for the DIHARD II 2019 Challenge

In this paper, we present our system developed by the team from the New Technologies for the Information Society (NTIS) research center of the University of West Bohemia in Pilsen, for the Second DIHARD Speech Diarization Challenge. The base of our system follows the currently-standard approach of segmentation, i/x-vector extraction, clustering, and resegmentation. The hyperparameters for each of the subsystems were selected according to the domain classifier trained on the development set of DIHARD II. We compared our system with results from the Kaldi diarization (with i/x-vectors) and combined these systems. At the time of writing of this abstract, our best submission achieved a DER of 23.47% and a JER of 48.99% on the evaluation set (in Track 1 using reference SAD).
Hydrogen bond analysis of confined water in mesoporous silica using the reactive force field

ABSTRACT The structural and dynamical properties of water confined in nanoporous silica with a pore diameter of 2.7 nm were investigated by performing large-scale molecular dynamics simulations using the reactive force field. The radial distribution function and diffusion coefficient of water were calculated, and the values at the centre of the pore agreed well with experimental values for real water. In addition, the pore was divided into thin coaxial layers, and the average number of hydrogen bonds, hydrogen bond lifetime and hydrogen bond strength were calculated as a function of the radial distance from the pore central axis. The analysis showed that hydrogen bonds involving silanol (Si–OH) have a longer lifetime, although the average number of hydrogen bonds per atom does not change from that at the pore centre. The longer lifetime, as well as smaller diffusion coefficient, of these hydrogen bonds is attributed to their greater strength.
An Extended Catalog of Galaxy–Galaxy Strong Gravitational Lenses Discovered in DES Using Convolutional Neural Networks

We search Dark Energy Survey (DES) Year 3 imaging for galaxy–galaxy strong gravitational lenses using convolutional neural networks, extending previous work with new training sets and covering a wider range of redshifts and colors. We train two neural networks using images of simulated lenses, then use them to score postage-stamp images of 7.9 million sources from DES chosen to have plausible lens colors based on simulations. We examine 1175 of the highest-scored candidates and identify 152 probable or definite lenses. Examining an additional 20,000 images with lower scores, we identify a further 247 probable or definite candidates. After including 86 candidates discovered in earlier searches using neural networks and 26 candidates discovered through visual inspection of blue-near-red objects in the DES catalog, we present a catalog of 511 lens candidates.
Sparse Gaussian Process Modulated Hawkes Process

The Hawkes process has been widely applied to modeling self-exciting events, including neuron spikes, earthquakes and tweets. To avoid designing parametric kernel functions and to be able to quantify the prediction confidence, non-parametric Bayesian Hawkes processes have been proposed. However the inference of such models suffers from unscalability or slow convergence. In this paper, we first propose a new non-parametric Bayesian Hawkes process whose triggering kernel is modeled as a squared sparse Gaussian process. Second, we present the variational inference scheme for the model optimization, which has the advantage of linear time complexity by leveraging the stationarity of the triggering kernel. Third, we contribute a tighter lower bound than the evidence lower bound of the marginal likelihood for the model selection. Finally, we exploit synthetic data and large-scale social media data to validate the efficiency of our method and the practical utility of our approximate marginal likelihood. We show that our approach outperforms state-of-the-art non-parametric Bayesian and non-Bayesian methods.
3D Trajectory Optimization for Secure UAV Communication with CoMP Reception

This paper studies a secrecy unmanned aerial vehicle (UAV) communication system with coordinated multi- point (CoMP) reception, in which one UAV sends confidential messages to a set of distributed ground nodes (GNs) that can cooperate in signal detection, in the presence of several colluding suspicious eavesdroppers. Different from prior works considering the two-dimensional (2D) horizontal trajectory design in the non-CoMP scenario, this paper additionally exploits the UAV's vertical trajectory (or altitude) control for further improving the secrecy communication performance with CoMP. In particular, we jointly optimize the three dimensional (3D) trajectory and transmit power allocation of the UAV to maximize the average secrecy rate at GNs over a particular flight period, subject to the UAV's maximum flight speed and maximum transmit power constraints. To solve the non-convex optimization problem, we propose an alternating-optimization-based approach, which optimizes the transmit power allocation and trajectory design in an alternating manner, by convex optimization and successive convex approximation (SCA), respectively. Numerical results show that in the scenario with CoMP reception, our proposed 3D trajectory optimization significantly outperforms the conventional 2D horizontal trajectory design, by exploiting the additional degree of freedom in vertical trajectory.
MCNE: An End-to-End Framework for Learning Multiple Conditional Network Representations of Social Network

Recently, the Network Representation Learning (NRL) techniques, which represent graph structure via low-dimension vectors to support social-oriented application, have attracted wide attention. Though large efforts have been made, they may fail to describe the multiple aspects of similarity between social users, as only a single vector for one unique aspect has been represented for each node. To that end, in this paper, we propose a novel end-to-end framework named MCNE to learn multiple conditional network representations, so that various preferences for multiple behaviors could be fully captured. Specifically, we first design a binary mask layer to divide the single vector as conditional embeddings for multiple behaviors. Then, we introduce the attention network to model interaction relationship among multiple preferences, and further utilize the adapted message sending and receiving operation of graph neural network, so that multi-aspect preference information from high-order neighbors will be captured. Finally, we utilize Bayesian Personalized Ranking loss function to learn the preference similarity on each behavior, and jointly learn multiple conditional node embeddings via multi-task learning framework. Extensive experiments on public datasets validate that our MCNE framework could significantly outperform several state-of-the-art baselines, and further support the visualization and transfer learning tasks with excellent interpretability and robustness.
Induction of Non-Monotonic Rules From Statistical Learning Models Using High-Utility Itemset Mining

We present a fast and scalable algorithm to induce non-monotonic logic programs from statistical learning models. We reduce the problem of search for best clauses to instances of the High-Utility Itemset Mining (HUIM) problem. In the HUIM problem, feature values and their importance are treated as transactions and utilities respectively. We make use of TreeExplainer, a fast and scalable implementation of the Explainable AI tool SHAP, to extract locally important features and their weights from ensemble tree models. Our experiments with UCI standard benchmarks suggest a significant improvement in terms of classification evaluation metrics and running time of the training algorithm compared to ALEPH, a state-of-the-art Inductive Logic Programming (ILP) system.
Identifying Serum Biomarkers for Ovarian Cancer by Screening With Surface-Enhanced Laser Desorption/Ionization Mass Spectrometry and the Artificial Neural Network

Objective The purpose of this study was to screen potential serum tumor biomarkers for the diagnosis of ovarian cancer. Methods The study includes 3 sets. The first set of patients included 37 ovarian cancers and 31 healthy women (healthy controls). The second set included 42 ovarian cancers, 33 patients with benign ovarian tumor, and 29 healthy women (noncancer controls). The third set included 39 ovarian cancers and 35 patients with benign ovarian tumor (benign controls). Serum samples from ovarian cancers, healthy controls, noncancer controls, and benign controls were analyzed by surface-enhanced laser desorption/ionization time-of-flight mass spectrometry. Results A 3-peak model (peaks of mass-to-charge ratio values at 5766.379 d, 5912.586 d, and 11695.56 d) was established in the training set that discriminated cancer from noncancer with high sensitivity (10/11, 90.90%) and specificity (19/20, 95.00%). The peaks corresponding to 3 potential biomarkers increased significantly with the degree of malignancy. Conclusions The proteins represented by these 3 peaks are biomarker candidates for ovarian cancer diagnosis and/or monitoring treatment response.
A 3+1 Decomposition of the Minimal Standard-Model Extension Gravitational Sector

The 3+1 (ADM) formulation of General Relativity is used in, for example, canonical quantum gravity and numerical relativity. Here we present a 3+1 decomposition of the minimal Standard-Model Extension gravity Lagrangian. By choosing the leaves of foliation to lie along a timelike vector field we write the theory in a form which will allow for comparison and matching to other gravity models.
Fast and Accurate Gaussian Kernel Ridge Regression Using Matrix Decompositions for Preconditioning

This paper presents a method for building a preconditioner for a kernel ridge regression problem, where the preconditioner is not only effective in its ability to reduce the condition number substantially, but also efficient in its application in terms of computational cost and memory consumption. The suggested approach is based on randomized matrix decomposition methods, combined with the fast multipole method to achieve an algorithm that can process large datasets in complexity linear to the number of data points. In addition, a detailed theoretical analysis is provided, including an upper bound to the condition number. Finally, for Gaussian kernels, the analysis shows that the required rank for a desired condition number can be determined directly from the dataset itself without performing any analysis on the kernel matrix.