OpenRaiser/Intern-Atlas huggingface dataset:
BOAH: A Tool Suite for Multi-Fidelity Bayesian Optimization & Analysis of Hyperparameters
Venue: arXiv cs.LG 2019
Year: 2019

Hyperparameter optimization and neural architecture search can become prohibitively expensive for regular black-box Bayesian optimization because the training and evaluation of a single model can easily take several hours. To overcome this, we introduce a comprehensive tool suite for effective multi-fidelity Bayesian optimization and the analysis of its runs. The suite, written in Python, provides a simple way to specify complex design spaces, a robust and efficient combination of Bayesian optimization and HyperBand, and a comprehensive analysis of the optimization process and its outcomes.
Iterative Update and Unified Representation for Multi-Agent Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Multi-agent systems have a wide range of applications in cooperative and competitive tasks. As the number of agents increases, nonstationarity gets more serious in multi-agent reinforcement learning (MARL), which brings great difficulties to the learning process. Besides, current mainstream algorithms configure each agent an independent network,so that the memory usage increases linearly with the number of agents which greatly slows down the interaction with the environment. Inspired by Generative Adversarial Networks (GAN), this paper proposes an iterative update method (IU) to stabilize the nonstationary environment. Further, we add first-person perspective and represent all agents by only one network which can change agents' policies from sequential compute to batch compute. Similar to continual lifelong learning, we realize the iterative update method in this unified representative network (IUUR). In this method, iterative update can greatly alleviate the nonstationarity of the environment, unified representation can speed up the interaction with environment and avoid the linear growth of memory usage. Besides, this method does not bother decentralized execution and distributed deployment. Experiments show that compared with MADDPG, our algorithm achieves state-of-the-art performance and saves wall-clock time by a large margin especially with more agents.
Efficient Discovery of Expressive Multi-label Rules using Relaxed Pruning
Venue: arXiv cs.LG 2019
Year: 2019

Being able to model correlations between labels is considered crucial in multi-label classification. Rule-based models enable to expose such dependencies, e.g., implications, subsumptions, or exclusions, in an interpretable and human-comprehensible manner. Albeit the number of possible label combinations increases exponentially with the number of available labels, it has been shown that rules with multiple labels in their heads, which are a natural form to model local label dependencies, can be induced efficiently by exploiting certain properties of rule evaluation measures and pruning the label search space accordingly. However, experiments have revealed that multi-label heads are unlikely to be learned by existing methods due to their restrictiveness. To overcome this limitation, we propose a plug-in approach that relaxes the search space pruning used by existing methods in order to introduce a bias towards larger multi-label heads resulting in more expressive rules. We further demonstrate the effectiveness of our approach empirically and show that it does not come with drawbacks in terms of training time or predictive performance.
Decentralising power: how we are trying to keep CALLector ethical
Venue: arXiv cs.HC 2019
Year: 2019

We present a brief overview of the CALLector project, and consider ethical questions arising from its overall goal of creating a social network to support creation and use of online CALL resources. We argue that these questions are best addressed in a decentralised, pluralistic open source architecture.
Weil descent and cryptographic trilinear maps
Venue: arXiv cs.CR 2019
Year: 2019

It has recently been shown that cryptographic trilinear maps are sufficient for achieving indistinguishability obfuscation. In this paper we develop a method for constructing such maps on the Weil descent (restriction) of abelian varieties over finite fields, including the Jacobian varieties of hyperelliptic curves and elliptic curves. The security of these candidate cryptographic trilinear maps raises several interesting questions, including the computational complexity of a trapdoor discrete logarithm problem.
Encoder-Agnostic Adaptation for Conditional Language Generation
Venue: arXiv cs.CL 2019
Year: 2019

Large pretrained language models have changed the way researchers approach discriminative natural language understanding tasks, leading to the dominance of approaches that adapt a pretrained model for arbitrary downstream tasks. However it is an open-question how to use similar techniques for language generation. Early results in the encoder-agnostic setting have been mostly negative. In this work we explore methods for adapting a pretrained language model to arbitrary conditional input. We observe that pretrained transformer models are sensitive to large parameter changes during tuning. We therefore propose an adaptation that directly injects arbitrary conditioning into self attention, an approach we call pseudo self attention. Through experiments on four diverse conditional text generation tasks we show that this encoder-agnostic technique outperforms strong baselines, produces coherent generations, and is data efficient.
PPT: New Low Complexity Deterministic Primality Tests Leveraging Explicit and Implicit Non-Residues. A Set of Three Companion Manuscripts
Venue: arXiv cs.CR 2019
Year: 2019

In this set of three companion manuscripts/articles, we unveil our new results on primality testing and reveal new primality testing algorithms enabled by those results. The results have been classified (and referred to) as lemmas/corollaries/claims whenever we have complete analytic proof(s); otherwise the results are introduced as conjectures. In Part/Article 1, we start with the Baseline Primality Conjecture~(PBPC) which enables deterministic primality detection with a low complexity = O((log N)^2) ; when an explicit value of a Quadratic Non Residue (QNR) modulo-N is available (which happens to be the case for an overwhelming majority = 11/12 = 91.67% of all odd integers). We then demonstrate Primality Lemma PL-1, which reveals close connections between the state-of-the-art Miller-Rabin method and the renowned Euler-Criterion. This Lemma, together with the Baseline Primality Conjecture enables a synergistic fusion of Miller-Rabin iterations and our method(s), resulting in hybrid algorithms that are substantially better than their components. Next, we illustrate how the requirement of an explicit value of a QNR can be circumvented by using relations of the form: Polynomial(x) mod N = 0 ; whose solutions implicitly specify Non Residues modulo-N. We then develop a method to derive low-degree canonical polynomials that together guarantee implicit Non Residues modulo-N ; which along with the Generalized Primality Conjectures enable algorithms that achieve a worst case deterministic polynomial complexity = O( (log N)^3 polylog(log N)) ; unconditionally ; for any/all values of N. In Part/Article 2 , we present substantial experimental data that corroborate all the conjectures. No counter example has been found. Finally in Part/Article 3, we present analytic proof(s) of the Baseline Primality Conjecture that we have been able to complete for some special cases.
Reinforcement Learning Applications
Venue: arXiv cs.LG 2019
Year: 2019

We start with a brief introduction to reinforcement learning (RL), about its successful stories, basics, an example, issues, the ICML 2019 Workshop on RL for Real Life, how to use it, study material and an outlook. Then we discuss a selection of RL applications, including recommender systems, computer systems, energy, finance, healthcare, robotics, and transportation.
Landmark Map: An Extension of the Self-Organizing Map for a User-Intended Nonlinear Projection
Venue: arXiv cs.NE 2019
Year: 2019

The self-organizing map (SOM) is an unsupervised artificial neural network that is widely used in, e.g., data mining and visualization. Supervised and semi-supervised learning methods have been proposed for the SOM. However, their teacher labels do not describe the relationship between the data and the location of nodes. This study proposes a landmark map (LAMA), which is an extension of the SOM that utilizes several landmarks, e.g., pairs of nodes and data points. LAMA is designed to obtain a user-intended nonlinear projection to achieve, e.g., the landmark-oriented data visualization. To reveal the learning properties of LAMA, the Zoo dataset from the UCI Machine Learning Repository and an artificial formant dataset were analyzed. The analysis results of the Zoo dataset indicated that LAMA could provide a new data view such as the landmark-centered data visualization. Furthermore, the artificial formant data analysis revealed that LAMA successfully provided the intended nonlinear projection associating articular movement with vertical and horizontal movement of a computer cursor. Potential applications of LAMA include data mining, recommendation systems, and human-computer interaction.
Discrete and Fast Fourier Transform Made Clear
Venue: arXiv cs.DS 2019
Year: 2019

Fast Fourier transform was included in the Top 10 Algorithms of 20th Century by Computing in Science & Engineering. In this paper, we provide a new simple derivation of both the discrete Fourier transform and fast Fourier transform by means of elementary linear algebra. We start the exposition by introducing the convolution product of vectors, represented by a circulant matrix, and derive the discrete Fourier transform as the change of basis matrix that diagonalizes the circulant matrix. We also generalize our approach to derive the Fourier transform on any finite abelian group, where the case of Fourier transform on the Boolean cube is especially important for many applications in theoretical computer science.
Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
Venue: arXiv cs.CL 2019
Year: 2019

Although neural machine translation models reached high translation quality, the autoregressive nature makes inference difficult to parallelize and leads to high translation latency. Inspired by recent refinement-based approaches, we propose LaNMT, a latent-variable non-autoregressive model with continuous latent variables and deterministic inference procedure. In contrast to existing approaches, we use a deterministic inference algorithm to find the target sequence that maximizes the lowerbound to the log-probability. During inference, the length of translation automatically adapts itself. Our experiments show that the lowerbound can be greatly increased by running the inference algorithm, resulting in significantly improved translation quality. Our proposed model closes the performance gap between non-autoregressive and autoregressive approaches on ASPEC Ja-En dataset with 8.6x faster decoding. On WMT'14 En-De dataset, our model narrows the gap with autoregressive baseline to 2.0 BLEU points with 12.5x speedup. By decoding multiple initial latent variables in parallel and rescore using a teacher model, the proposed model further brings the gap down to 1.0 BLEU point on WMT'14 En-De task with 6.8x speedup.
Counterfactual Distribution Regression for Structured Inference
Venue: arXiv cs.LG 2019
Year: 2019

We consider problems in which a system receives external \emph{perturbations} from time to time. For instance, the system can be a train network in which particular lines are repeatedly disrupted without warning, having an effect on passenger behavior. The goal is to predict changes in the behavior of the system at particular points of interest, such as passenger traffic around stations at the affected rails. We assume that the data available provides records of the system functioning at its "natural regime" (e.g., the train network without disruptions) and data on cases where perturbations took place. The inference problem is how information concerning perturbations, with particular covariates such as location and time, can be generalized to predict the effect of novel perturbations. We approach this problem from the point of view of a mapping from the counterfactual distribution of the system behavior without disruptions to the distribution of the disrupted system. A variant on \emph{distribution regression} is developed for this setup.
Indoor Navigation Using Information From A Map And A Rangefinder
Venue: arXiv cs.RO 2019
Year: 2019

The problem of indoor navigation of mobile objects, using a map and measurements of distances to the walls is considered. A nonlinear filtering problem aimed at calculating the optimal, in the root-mean-square sense, of the sought parameters is formulated in the context of the Bayesian approach. The algorithm for its solution based on the point-mass method is described. The simulation results illustrating the advantages of the proposed problem statement and the resultant algorithm are discussed.
Evacuation of equilateral triangles by mobile agents of limited communication range
Venue: arXiv cs.DC 2019
Year: 2019

We consider the problem of evacuating $k \geq 2$ mobile agents from a unit-sided equilateral triangle through an exit located at an unknown location on the perimeter of the triangle. The agents are initially located at the centroid of the triangle and they can communicate with other agents at distance at most $r$ with $0\leq r \leq 1$. An agent can move at speed at most one, and finds the exit only when it reaches the point where the exit is located. The agents can collaborate in the search for the exit. The goal of the {\em evacuation problem} is to minimize the evacuation time, defined as the worst-case time for {\em all} the agents to reach the exit. We propose and analyze several algorithms for the problem of evacuation by $k \geq 2$ agents; our results indicate that the best strategy to be used varies depending on the values of $r$ and $k$. For two agents, we give three algorithms, each of which achieves the best performance for different sub-ranges of $r$ in the range $0 \leq r \leq 1$. Finally, we show that for any $r$, evacuation of $k=6 +2\lceil(\frac{1}{r}-1)\rceil$ agents can be done in time $1+\sqrt{3}/3$, which is optimal in terms of time, and asymptotically optimal in terms of the number of agents.
An algorithm for destroying claws and diamonds
Venue: arXiv cs.DS 2019
Year: 2019

In the {Claw,Diamond}-Free Edge Deletion problem the input is a graph $G$ and an integer $k$, and the goal is to decide whether there is a set of edges of size at most $k$ such that removing the edges of the set from $G$ results a graph that does not contain an induced claw or diamond. In this paper we give an algorithm for this problem whose running time is $O^*(3.562^k)$.
Fairness Issues in AI Systems that Augment Sensory Abilities
Venue: arXiv cs.CY 2019
Year: 2019

Systems that augment sensory abilities are increasingly employing AI and machine learning (ML) approaches, with applications ranging from object recognition and scene description tools for blind users to sound awareness tools for d/Deaf users. However, unlike many other AI-enabled technologies, these systems provide information that is already available to non-disabled people. In this paper, we discuss unique AI fairness challenges that arise in this context, including accessibility issues with data and models, ethical implications in deciding what sensory information to convey to the user, and privacy concerns both for the primary user and for others.
Learning-Driven Wireless Communications, towards 6G
Venue: arXiv cs.NI 2019
Year: 2019

The fifth generation (5G) of wireless communication is in its infancy, and its evolving versions will be launched over the coming years. However, according to exposing the inherent constraints of 5G and the emerging applications and services with stringent requirements e.g. latency, energy/bit, traffic capacity, peak data rate, and reliability, telecom researchers are turning their attention to conceptualize the next generation of wireless communications, i.e. 6G. In this paper, we investigate 6G challenges, requirements, and trends. Furthermore, we discuss how artificial intelligence (AI) techniques can contribute to 6G. Based on the requirements and solutions, we identify some new fascinating services and use-cases of 6G, which can not be supported by 5G appropriately. Moreover, we explain some research directions that lead to the successful conceptualization and implementation of 6G.
Knowledge distillation for semi-supervised domain adaptation
Venue: arXiv cs.LG 2019
Year: 2019

In the absence of sufficient data variation (e.g., scanner and protocol variability) in annotated data, deep neural networks (DNNs) tend to overfit during training. As a result, their performance is significantly lower on data from unseen sources compared to the performance on data from the same source as the training data. Semi-supervised domain adaptation methods can alleviate this problem by tuning networks to new target domains without the need for annotated data from these domains. Adversarial domain adaptation (ADA) methods are a popular choice that aim to train networks in such a way that the features generated are domain agnostic. However, these methods require careful dataset-specific selection of hyperparameters such as the complexity of the discriminator in order to achieve a reasonable performance. We propose to use knowledge distillation (KD) -- an efficient way of transferring knowledge between different DNNs -- for semi-supervised domain adaption of DNNs. It does not require dataset-specific hyperparameter tuning, making it generally applicable. The proposed method is compared to ADA for segmentation of white matter hyperintensities (WMH) in magnetic resonance imaging (MRI) scans generated by scanners that are not a part of the training set. Compared with both the baseline DNN (trained on source domain only and without any adaption to target domain) and with using ADA for semi-supervised domain adaptation, the proposed method achieves significantly higher WMH dice scores.
Primitive Idempotents and Constacyclic Codes over Finite Chain Rings
Venue: arXiv cs.IT 2019
Year: 2019

Let $R$ be a commutative local finite ring. In this paper, we construct the complete set of pairwise orthogonal primitive idempotents of $R[X]/<g>$ where $g$ is a regular polynomial in $R[X]$. We use this set to decompose the ring $R[X]/<g>$ and to give the structure of constacyclic codes over finite chain rings. This allows us to describe generators of the dual code $\mathcal{C}^\bot$ of a constacyclic code $\mathcal{C}$ and to characterize non-trivial self-dual constacyclic codes over finite chain rings.
The Rise of Blockchain Technology in Agriculture and Food Supply Chains
Venue: arXiv cs.CY 2019
Year: 2019

Blockchain is an emerging digital technology allowing ubiquitous financial transactions among distributed untrusted parties, without the need of intermediaries such as banks. This article examines the impact of blockchain technology in agriculture and food supply chain, presents existing ongoing projects and initiatives, and discusses overall implications, challenges and potential, with a critical view over the maturity of these projects. Our findings indicate that blockchain is a promising technology towards a transparent supply chain of food, with many ongoing initiatives in various food products and food-related issues, but many barriers and challenges still exist, which hinder its wider popularity among farmers and systems. These challenges involve technical aspects, education, policies and regulatory frameworks.
Blind Image Deconvolution using Pretrained Generative Priors
Venue: arXiv cs.CV 2019
Year: 2019

This paper proposes a novel approach to regularize the ill-posed blind image deconvolution (blind image deblurring) problem using deep generative networks. We employ two separate deep generative models - one trained to produce sharp images while the other trained to generate blur kernels from lower dimensional parameters. To deblur, we propose an alternating gradient descent scheme operating in the latent lower-dimensional space of each of the pretrained generative models. Our experiments show excellent deblurring results even under large blurs and heavy noise. To improve the performance on rich image datasets not well learned by the generative networks, we present a modification of the proposed scheme that governs the deblurring process under both generative and classical priors.
On the Differential-Linear Connectivity Table of Vectorial Boolean Functions
Venue: arXiv cs.IT 2019
Year: 2019

Vectorial Boolean functions are crucial building-blocks in symmetric ciphers. Different known attacks on block ciphers have resulted in diverse cryptographic criteria for vectorial Boolean functions, such as differential uniformity and nonlinearity. Very recently, Bar-On et al. introduced at Eurocrypt'19 a new tool, called the differential-linear connectivity table (DLCT), which allows for taking into account the dependency between the two subciphers $E_0$ and $E_1$ involved in differential-linear attacks. This new notion leads to significant improvements of differential-linear attacks on several ciphers. This paper presents a theoretical characterization of the DLCT of vectorial Boolean functions and also investigates this new criterion for some families of functions with specific forms. More precisely, we firstly reveal the connection between the DLCT and the autocorrelation of vectorial Boolean functions, we characterize properties of the DLCT by means of the Walsh transform of the function and of its differential distribution table, and we present generic bounds on the highest magnitude occurring in the DLCT of vectorial Boolean functions, which coincides (up to a factor~\(2\)) with the well-established notion of absolute indicator. Next, we investigate the invariance property of the DLCT of vectorial Boolean functions under the affine, extended-affine, and Carlet-Charpin-Zinoviev (CCZ) equivalence and exhaust the DLCT spectra of optimal $4$-bit S-boxes under affine equivalence. Furthermore, we study the DLCT of APN, plateaued and AB functions and establish its connection with other cryptographic criteria. Finally, we investigate the DLCT and the absolute indicator of some specific polynomials with optimal or low differential uniformity, including monomials, cubic functions, quadratic functions and inverses of quadratic permutations.
Controversy in Context
Venue: arXiv cs.CL 2019
Year: 2019

With the growing interest in social applications of Natural Language Processing and Computational Argumentation, a natural question is how controversial a given concept is. Prior works relied on Wikipedia's metadata and on content analysis of the articles pertaining to a concept in question. Here we show that the immediate textual context of a concept is strongly indicative of this property, and, using simple and language-independent machine-learning tools, we leverage this observation to achieve state-of-the-art results in controversiality prediction. In addition, we analyze and make available a new dataset of concepts labeled for controversiality. It is significantly larger than existing datasets, and grades concepts on a 0-10 scale, rather than treating controversiality as a binary label.