Miracle-dz/UniFAR-FTUs huggingface dataset:
Adversarial Reinforcement Learning under Partial Observability in Software-Defined Networking

Recent studies have demonstrated that reinforcement learning (RL) agents are susceptible to adversarial manipulation, similar to vulnerabilities previously demonstrated in the supervised setting. Accordingly focus has remained with computer vision, and full observability. This paper focuses on reinforcement learning in the context of autonomous defence in Software-Defined Networking (SDN). We demonstrate that causative attacks---attacks that target the training process---can poison RL agents even if the attacker only has partial observability of the environment. In addition, we propose an inversion defence method that aims to apply the opposite perturbation to that which an attacker might use to generate their adversarial samples. Our experimental results illustrate that the countermeasure can effectively reduce the impact of the causative attack, while not significantly affecting the training process in non-attack scenarios.
Context Vectors are Reflections of Word Vectors in Half the Dimensions

This paper takes a step towards the theoretical analysis of the relationship between word embeddings and context embeddings in models such as word2vec. We start from basic probabilistic assumptions on the nature of word vectors, context vectors, and text generation. These assumptions are supported either empirically or theoretically by the existing literature. Next, we show that under these assumptions the widely-used word-word PMI matrix is approximately a random symmetric Gaussian ensemble. This, in turn, implies that context vectors are reflections of word vectors in approximately half the dimensions. As a direct application of our result, we suggest a theoretically grounded way of tying weights in the SGNS model.
Semi-supervised Learning for Quantification of Pulmonary Edema in Chest X-Ray Images

We propose and demonstrate machine learning algorithms to assess the severity of pulmonary edema in chest x-ray images of congestive heart failure patients. Accurate assessment of pulmonary edema in heart failure is critical when making treatment and disposition decisions. Our work is grounded in a large-scale clinical dataset of over 300,000 x-ray images with associated radiology reports. While edema severity labels can be extracted unambiguously from a small fraction of the radiology reports, accurate annotation is challenging in most cases. To take advantage of the unlabeled images, we develop a Bayesian model that includes a variational auto-encoder for learning a latent representation from the entire image set trained jointly with a regressor that employs this representation for predicting pulmonary edema severity. Our experimental results suggest that modeling the distribution of images jointly with the limited labels improves the accuracy of pulmonary edema scoring compared to a strictly supervised approach. To the best of our knowledge, this is the first attempt to employ machine learning algorithms to automatically and quantitatively assess the severity of pulmonary edema in chest x-ray images.
Parvalbumin+ interneurons enable efficient pattern separation in hippocampal microcircuits

Distinguishing between similar experiences is achieved by the brain in a process called pattern separation. In the hippocampus, pattern separation reduces the interference of memories and increases the storage capacity by decorrelating similar inputs patterns of neuronal activity into non-overlapping output firing patterns. Winners-take-all (WTA) mechanism is a theoretical model for pattern separation in which a "winner" cell suppresses the activity of the neighboring neurons through feedback inhibition. However, if the network properties of the dentate gyrus support WTA as a biologically conceivable model remains unknown. Here, we showed that the connectivity rules of PV+interneurons and their synaptic properties are optimizedfor efficient pattern separation. We found using multiple whole-cell in vitrorecordings that PV+interneurons mainly connect to granule cells (GC) through lateral inhibition, a form of feedback inhibition in which a GC inhibits other GCs but not itself through the activation of PV+interneurons. Thus, lateral inhibition between GC–PV+interneurons was ~10 times more abundant than recurrent connections. Furthermore, the GC–PV+interneuron connectivity was more spatially confined but less abundant than PV+interneurons–GC connectivity, leading to an asymmetrical distribution of excitatory and inhibitory connectivity. Our network model of the dentate gyrus with incorporated real connectivity rules efficiently decorrelates neuronal activity patterns using WTA as the primary mechanism. This process relied on lateral inhibition, fast-signaling properties of PV+interneurons and the asymmetrical distribution of excitatory and inhibitory connectivity. Finally, we found that silencing the activity of PV+interneurons in vivoleads to acute deficits in discrimination between similar environments, suggesting that PV+interneuron networks are necessary for behavioral relevant computations. Our results demonstrate that PV+interneurons possess unique connectivity and fast signaling properties that confer to the dentate gyrus network properties that allow the emergence of pattern separation. Thus, our results contribute to the knowledge of how specific forms of network organization underlie sophisticated types of information processing.
A Variational Dirichlet Framework for Out-of-Distribution Detection

With the recently rapid development in deep learning, deep neural networks have been widely adopted in many real-life applications. However, deep neural networks are also known to have very little control over its uncertainty for unseen examples, which potentially causes very harmful and annoying consequences in practical scenarios. In this paper, we are particularly interested in designing a higher-order uncertainty metric for deep neural networks and investigate its effectiveness under the out-of-distribution detection task proposed by~\cite{hendrycks2016baseline}. Our method first assumes there exists an underlying higher-order distribution $\mathbb{P}(z)$, which controls label-wise categorical distribution $\mathbb{P}(y)$ over classes on the K-dimension simplex, and then approximate such higher-order distribution via parameterized posterior function $p_{\theta}(z|x)$ under variational inference framework, finally we use the entropy of learned posterior distribution $p_{\theta}(z|x)$ as uncertainty measure to detect out-of-distribution examples. Further, we propose an auxiliary objective function to discriminate against synthesized adversarial examples to further increase the robustness of the proposed uncertainty measure. Through comprehensive experiments on various datasets, our proposed framework is demonstrated to consistently outperform competing algorithms.
Equi-normalization of Neural Networks

Modern neural networks are over-parametrized. In particular, each rectified linear hidden unit can be modified by a multiplicative factor by adjusting input and output weights, without changing the rest of the network. Inspired by the Sinkhorn-Knopp algorithm, we introduce a fast iterative method for minimizing the L2 norm of the weights, equivalently the weight decay regularizer. It provably converges to a unique solution. Interleaving our algorithm with SGD during training improves the test accuracy. For small batches, our approach offers an alternative to batch-and group-normalization on CIFAR-10 and ImageNet with a ResNet-18.
A detailed comparative study of open source deep learning frameworks

Deep Learning (DL) is one of the hottest trends in machine learning as DL approaches produced results superior to the state-of-the-art in problematic areas such as image processing and natural language processing (NLP). To foster the growth of DL, several open source frameworks appeared providing implementations of the most common DL algorithms. These frameworks vary in the algorithms they support and in the quality of their implementations. The purpose of this work is to provide a qualitative and quantitative comparison among three of the most popular and most comprehensive DL frameworks (namely Google's TensorFlow, University of Montreal's Theano and Microsoft's CNTK). The ultimate goal of this work is to help end users make an informed decision about the best DL framework that suits their needs and resources. To ensure that our study is as comprehensive as possible, we conduct several experiments using multiple benchmark datasets from different fields (image processing, NLP, etc.) and measure the performance of the frameworks' implementations of different DL algorithms. For most of our experiments, we find out that CNTK's implementations are superior to the other ones under consideration.
GANSynth: Adversarial Neural Audio Synthesis

Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence. Autoregressive models, such as WaveNet, model local structure at the expense of global latent structure and slow iterative sampling, while Generative Adversarial Networks (GANs), have global latent conditioning and efficient parallel sampling, but struggle to generate locally-coherent audio waveforms. Herein, we demonstrate that GANs can in fact generate high-fidelity and locally-coherent audio by modeling log magnitudes and instantaneous frequencies with sufficient frequency resolution in the spectral domain. Through extensive empirical investigations on the NSynth dataset, we demonstrate that GANs are able to outperform strong WaveNet baselines on automated and human evaluation metrics, and efficiently generate audio several orders of magnitude faster than their autoregressive counterparts.
Nonconvex sampling with the Metropolis-adjusted Langevin algorithm

The Langevin Markov chain algorithms are widely deployed methods to sample from distributions in challenging high-dimensional and non-convex statistics and machine learning applications. Despite this, current bounds for the Langevin algorithms are slower than those of competing algorithms in many important situations, for instance when sampling from weakly log-concave distributions, or when sampling or optimizing non-convex log-densities. In this paper, we obtain improved bounds in many of these situations, showing that the Metropolis-adjusted Langevin algorithm (MALA) is faster than the best bounds for its competitor algorithms when the target distribution satisfies weak third- and fourth- order regularity properties associated with the input data. In many settings, our regularity conditions are weaker than the usual Euclidean operator norm regularity properties, allowing us to show faster bounds for a much larger class of distributions than would be possible with the usual Euclidean operator norm approach, including in statistics and machine learning applications where the data satisfy a certain incoherence condition. In particular, we show that using our regularity conditions one can obtain faster bounds for applications which include sampling problems in Bayesian logistic regression with weakly convex priors, and the nonconvex optimization problem of learning linear classifiers with zero-one loss functions. Our main technical contribution in this paper is our analysis of the Metropolis acceptance probability of MALA in terms of its "energy-conservation error," and our bound for this error in terms of third- and fourth- order regularity conditions. Our combination of this higher-order analysis of the energy conservation error with the conductance method is key to obtaining bounds which have a sub-linear dependence on the dimension $d$ in the non-strongly logconcave setting.
Robust Re-identification of Manta Rays from Natural Markings by Learning Pose Invariant Embeddings

Visual re-identification of individual animals that bear unique natural body markings is an essential task in wildlife conservation. The photo databases of animal markings grow with each new observation and identifying an individual means matching against thousands of images. We focus on the re-identification of manta rays because the existing process is time-consuming and only semi-automatic. The current solution Manta Matcher requires images of high quality with the pattern of interest in a near frontal view limiting the use of photos sourced from citizen scientists. This paper presents a novel application of a deep convolutional neural network (CNN) for visual re-identification based on natural markings. Our contribution is an experimental demonstration of the superiority of CNNs in learning embeddings for patterns under viewpoint changes on a novel and challenging dataset. We show that our system can handle more variations in viewing angle, occlusions and illumination compared to the current solution. Our system achieves top-10 accuracy of 98% with only 2 matching examples in the database which makes it of practical value and ready for adoption by marine biologists. We also evaluate our system on a dataset of humpback whale flukes to demonstrate that the approach is generic and not species-specific.
Whitening and Coloring Batch Transform for GANs

Batch Normalization (BN) is a common technique used to speed-up and stabilize training. On the other hand, the learnable parameters of BN are commonly used in conditional Generative Adversarial Networks (cGANs) for representing class-specific information using conditional Batch Normalization (cBN). In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization. We show that our conditional Coloring can represent categorical conditioning information which largely helps the cGAN qualitative results. Moreover, we show that full-feature whitening is important in a general GAN scenario in which the training process is known to be highly unstable. We test our approach on different datasets and using different GAN networks and training protocols, showing a consistent improvement in all the tested frameworks. Our CIFAR-10 conditioned results are higher than all previous works on this dataset.
Alternating Synthetic and Real Gradients for Neural Language Modeling

Training recurrent neural networks (RNNs) with backpropagation through time (BPTT) has known drawbacks such as being difficult to capture longterm dependencies in sequences. Successful alternatives to BPTT have not yet been discovered. Recently, BP with synthetic gradients by a decoupled neural interface module has been proposed to replace BPTT for training RNNs. On the other hand, it has been shown that the representations learned with synthetic and real gradients are different though they are functionally identical. In this project, we explore ways of combining synthetic and real gradients with application to neural language modeling tasks. Empirically, we demonstrate the effectiveness of alternating training with synthetic and real gradients after periodic warm restarts on language modeling tasks.
Residual-CNDS for Grand Challenge Scene Dataset

Increasing depth of convolutional neural networks (CNNs) is a highly promising method of increasing the accuracy of the (CNNs). Increased CNN depth will also result in increased layer count (parameters), leading to a slow backpropagation convergence prone to overfitting. We trained our model (Residual-CNDS) to classify very large-scale scene datasets MIT Places 205, and MIT Places 365-Standard. The outcome result from the two datasets proved our proposed model (Residual-CNDS) effectively handled the slow convergence, overfitting, and degradation. CNNs that include deep supervision (CNDS) add supplementary branches to the deep convolutional neural network in specified layers by calculating vanishing, effectively addressing delayed convergence and overfitting. Nevertheless, (CNDS) does not resolve degradation; hence, we add residual learning to the (CNDS) in certain layers after studying the best place in which to add it. With this approach we overcome degradation in the very deep network. We have built two models (Residual-CNDS 8), and (Residual-CNDS 10). Moreover, we tested our models on two large-scale datasets, and we compared our results with other recently introduced cutting-edge networks in the domain of top-1 and top-5 classification accuracy. As a result, both of models have shown good improvement, which supports the assertion that the addition of residual connections enhances network CNDS accuracy without adding any computation complexity.
Model-Based Reinforcement Learning for Atari

Model-free reinforcement learning (RL) can be used to learn effective policies for complex tasks, such as Atari games, even from image observations. However, this typically requires very large amounts of interaction -- substantially more, in fact, than a human would need to learn the same games. How can people learn so quickly? Part of the answer may be that people can learn how the game works and predict which actions will lead to desirable outcomes. In this paper, we explore how video prediction models can similarly enable agents to solve Atari games with fewer interactions than model-free methods. We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting. Our experiments evaluate SimPLe on a range of Atari games in low data regime of 100k interactions between the agent and the environment, which corresponds to two hours of real-time play. In most games SimPLe outperforms state-of-the-art model-free algorithms, in some games by over an order of magnitude.
A Survey on Applications of Game Theory in Blockchain

In the past decades, the blockchain technology has attracted tremendous attention from both academia and industry. The popularity of blockchain networks was originated from a crypto-currency to serve as a decentralized and tamperproof transaction data ledger. Nowadays, blockchain as the key framework in the decentralized public data-ledger, has been applied to a wide range of scenarios far beyond crypto-currencies, such as Internet of Things (IoT), healthcare, and insurance. This survey aims to fill the gap between the large number of studies on blockchain network, where game theory emerges as an analytical tool, and the lack of a comprehensive survey on the game theoretical approaches applied in blockchain related issues. In this paper, we review game models proposed to address common issues in the blockchain network. The issues include security issues, e.g., selfish mining, majority attack and Denial of Service (DoS) attack, issues regard mining management, e.g., computational power allocation, reward allocation, and pool selection, as well as issues regarding blockchain economic and energy trading. Additionally, we discuss advantages and disadvantages of these selected game models and solutions. Finally, we highlight important challenges and future research directions of applying game theoretical approaches to incentive mechanism design, and the combination of blockchain with other technologies.
A Rewriting System for Convex Optimization Problems

We describe a modular rewriting system for translating optimization problems written in a domain-specific language to forms compatible with low-level solver interfaces. Translation is facilitated by reductions, which accept a category of problems and transform instances of that category to equivalent instances of another category. Our system proceeds in two key phases: analysis, in which we attempt to find a suitable solver for a supplied problem, and canonicalization, in which we rewrite the problem in the selected solver's standard form. We implement the described system in version 1.0 of CVXPY, a domain-specific language for mathematical and especially convex optimization. By treating reductions as first-class objects, our method makes it easy to match problems to solvers well-suited for them and to support solvers with a wide variety of standard forms.
Beam Squint and Channel Estimation for Wideband mmWave Massive MIMO-OFDM Systems

With the increasing scale of antenna arrays in wideband millimeter-wave (mmWave) communications, the physical propagation delays of electromagnetic waves traveling across the whole array will become large and comparable to the time-domain sample period, which is known as the spatial-wideband effect. In this case, different subcarriers in an orthogonal frequency division multiplexing (OFDM) system will “see” distinct angles of arrival (AoAs) for the same path. This effect is known as beam squint, resulting from the spatial-wideband effect, and makes the approaches based on the conventional multiple-input multiple-output (MIMO) model, such as channel estimation and precoding, inapplicable. After discussing the relationship between beam squint and the spatial-wideband effect, we propose a channel estimation scheme for frequency-division duplex (FDD) mmWave massive MIMO-OFDM systems with hybrid analog/digital precoding, which takes the beam squint effect into consideration. A compressive sensing-based approach is developed to extract the frequency-insensitive parameters of each uplink channel path, i.e., the AoA and the time delay, and the frequency-sensitive parameter, i.e., the complex channel gain. With the help of the reciprocity of these frequency-insensitive parameters in FDD systems, the downlink channel estimation can be greatly simplified, where only limited pilots are needed to obtain downlink complex gains and reconstruct downlink channels. Furthermore, the uplink and downlink channel covariance matrices can be constructed from these frequency-insensitive channel parameters rather than through a long-term average, which enables the minimum mean-squared error (MMSE) channel estimation to further enhance performance. Numerical results demonstrate the superiority of the proposed scheme over the conventional methods under general system configurations in mmWave communications.
Multilingual Neural Machine Translation with Knowledge Distillation

Multilingual machine translation, which translates multiple languages with a single model, has attracted much attention due to its efficiency of offline training and online serving. However, traditional multilingual translation usually yields inferior accuracy compared with the counterpart using individual models for each language pair, due to language diversity and model capacity limitations. In this paper, we propose a distillation-based approach to boost the accuracy of multilingual machine translation. Specifically, individual models are first trained and regarded as teachers, and then the multilingual model is trained to fit the training data and match the outputs of individual models simultaneously through knowledge distillation. Experiments on IWSLT, WMT and Ted talk translation datasets demonstrate the effectiveness of our method. Particularly, we show that one model is enough to handle multiple languages (up to 44 languages in our experiment), with comparable or even better accuracy than individual models.
A Degeneracy Framework for Scalable Graph Autoencoders

In this paper, we present a general framework to scale graph autoencoders (AE) and graph variational autoencoders (VAE). This framework leverages graph degeneracy concepts to train models only from a dense subset of nodes instead of using the entire graph. Together with a simple yet effective propagation mechanism, our approach significantly improves scalability and training speed while preserving performance. We evaluate and discuss our method on several variants of existing graph AE and VAE, providing the first application of these models to large graphs with up to millions of nodes and edges. We achieve empirically competitive results w.r. t. several popular scalable node embedding methods, which emphasizes the relevance of pursuing further research towards more scalable graph AE and VAE.