OpenRaiser/Intern-Atlas huggingface dataset:
Acoustic Features Fusion using Attentive Multi-channel Deep Architecture
Venue: arXiv cs.SD 2018
Year: 2018

In this paper, we present a novel deep fusion architecture for audio classification tasks. The multi-channel model presented is formed using deep convolution layers where different acoustic features are passed through each channel. To enable dissemination of information across the channels, we introduce attention feature maps that aid in the alignment of frames. The output of each channel is merged using interaction parameters that non-linearly aggregate the representative features. Finally, we evaluate the performance of the proposed architecture on three benchmark datasets:- DCASE-2016 and LITIS Rouen (acoustic scene recognition), and CHiME-Home (tagging). Our experimental results suggest that the architecture presented outperforms the standard baselines and achieves outstanding performance on the task of acoustic scene recognition and audio tagging.
Near-Linear Time Algorithm for n-fold ILPs via Color Coding
Venue: arXiv cs.DS 2018
Year: 2018

We study an important case of ILPs $\max\{c^Tx \ \vert\ \mathcal Ax = b, l \leq x \leq u,\, x \in \mathbb{Z}^{n t} \} $ with $n\cdot t$ variables and lower and upper bounds $\ell, u\in\mathbb Z^{nt}$. In $n$-fold ILPs non-zero entries only appear in the first $r$ rows of the matrix $\mathcal A$ and in small blocks of size $s\times t$ along the diagonal underneath. Despite this restriction many optimization problems can be expressed in this form. It is known that $n$-fold ILPs can be solved in FPT time regarding the parameters $s, r,$ and $Δ$, where $Δ$ is the greatest absolute value of an entry in $\mathcal A$. The state-of-the-art technique is a local search algorithm that subsequently moves in an improving direction. Both, the number of iterations and the search for such an improving direction take time $Ω(n)$, leading to a quadratic running time in $n$. We introduce a technique based on Color Coding, which allows us to compute these improving directions in logarithmic time after a single initialization step. This leads to the first algorithm for $n$-fold ILPs with a running time that is near-linear in the number $nt$ of variables, namely $(rsΔ)^{O(r^2s + s^2)} L^2 \cdot nt \log^{O(1)}(nt)$, where $L$ is the encoding length of the largest integer in the input. In contrast to the algorithms in recent literature, we do not need to solve the LP relaxation in order to handle unbounded variables. Instead, we give a structural lemma to introduce appropriate bounds. If, on the other hand, we are given such an LP solution, the running time can be decreased by a factor of $L$.
Local search breaks 1.75 for Graph Balancing
Venue: arXiv cs.DS 2018
Year: 2018

Graph Balancing is the problem of orienting the edges of a weighted multigraph so as to minimize the maximum weighted in-degree. Since the introduction of the problem the best algorithm known achieves an approximation ratio of $1.75$ and it is based on rounding a linear program with this exact integrality gap. It is also known that there is no $(1.5 - ε)$-approximation algorithm, unless $\mathrm{P}=\mathrm{NP}$. Can we do better than $1.75$? We prove that a different LP formulation, the configuration LP, has a strictly smaller integrality gap. Graph Balancing was the last one in a group of related problems from literature, for which it was open whether the configuration LP is stronger than previous, simple LP relaxations. We base our proof on a local search approach that has been applied successfully to the more general Restricted Assignment problem, which in turn is a prominent special case of makespan minimization on unrelated machines. With a number of technical novelties we are able to obtain a bound of $1.749$ for the case of Graph Balancing. It is not clear whether the local search algorithm we present terminates in polynomial time, which means that the bound is non-constructive. However, it is a strong evidence that a better approximation algorithm is possible using the configuration LP and it allows the optimum to be estimated within a factor better than $1.75$. A particularly interesting aspect of our techniques is the way we handle small edges in the local search. We manage to exploit the configuration constraints enforced on small edges in the LP. This may be of interest to other problems such as Restricted Assignment as well.
Improving the Coverage and the Generalization Ability of Neural Word Sense Disambiguation through Hypernymy and Hyponymy Relationships
Venue: arXiv cs.CL 2018
Year: 2018

In Word Sense Disambiguation (WSD), the predominant approach generally involves a supervised system trained on sense annotated corpora. The limited quantity of such corpora however restricts the coverage and the performance of these systems. In this article, we propose a new method that solves these issues by taking advantage of the knowledge present in WordNet, and especially the hypernymy and hyponymy relationships between synsets, in order to reduce the number of different sense tags that are necessary to disambiguate all words of the lexical database. Our method leads to state of the art results on most WSD evaluation tasks, while improving the coverage of supervised systems, reducing the training time and the size of the models, without additional training data. In addition, we exhibit results that significantly outperform the state of the art when our method is combined with an ensembling technique and the addition of the WordNet Gloss Tagged as training corpus.
Clustering and Learning from Imbalanced Data
Venue: arXiv cs.LG 2018
Year: 2018

A learning classifier must outperform a trivial solution, in case of imbalanced data, this condition usually does not hold true. To overcome this problem, we propose a novel data level resampling method - Clustering Based Oversampling for improved learning from class imbalanced datasets. The essential idea behind the proposed method is to use the distance between a minority class sample and its respective cluster centroid to infer the number of new sample points to be generated for that minority class sample. The proposed algorithm has very less dependence on the technique used for finding cluster centroids and does not effect the majority class learning in any way. It also improves learning from imbalanced data by incorporating the distribution structure of minority class samples in generation of new data samples. The newly generated minority class data is handled in a way as to prevent outlier production and overfitting. Implementation analysis on different datasets using deep neural networks as the learning classifier shows the effectiveness of this method as compared to other synthetic data resampling techniques across several evaluation metrics.
Anomaly Detection for imbalanced datasets with Deep Generative Models
Venue: arXiv cs.LG 2018
Year: 2018

Many important data analysis applications present with severely imbalanced datasets with respect to the target variable. A typical example is medical image analysis, where positive samples are scarce, while performance is commonly estimated against the correct detection of these positive examples. We approach this challenge by formulating the problem as anomaly detection with generative models. We train a generative model without supervision on the `negative' (common) datapoints and use this model to estimate the likelihood of unseen data. A successful model allows us to detect the `positive' case as low likelihood datapoints. In this position paper, we present the use of state-of-the-art deep generative models (GAN and VAE) for the estimation of a likelihood of the data. Our results show that on the one hand both GANs and VAEs are able to separate the `positive' and `negative' samples in the MNIST case. On the other hand, for the NLST case, neither GANs nor VAEs were able to capture the complexity of the data and discriminate anomalies at the level that this task requires. These results show that even though there are a number of successes presented in the literature for using generative models in similar applications, there remain further challenges for broad successful implementation.
Analysing Dropout and Compounding Errors in Neural Language Models
Venue: arXiv cs.CL 2018
Year: 2018

This paper carries out an empirical analysis of various dropout techniques for language modelling, such as Bernoulli dropout, Gaussian dropout, Curriculum Dropout, Variational Dropout and Concrete Dropout. Moreover, we propose an extension of variational dropout to concrete dropout and curriculum dropout with varying schedules. We find these extensions to perform well when compared to standard dropout approaches, particularly variational curriculum dropout with a linear schedule. Largest performance increases are made when applying dropout on the decoder layer. Lastly, we analyze where most of the errors occur at test time as a post-analysis step to determine if the well-known problem of compounding errors is apparent and to what end do the proposed methods mitigate this issue for each dataset. We report results on a 2-hidden layer LSTM, GRU and Highway network with embedding dropout, dropout on the gated hidden layers and the output projection layer for each model. We report our results on Penn-TreeBank and WikiText-2 word-level language modelling datasets, where the former reduces the long-tail distribution through preprocessing and one which preserves rare words in the training and test set.
On Evaluating the Generalization of LSTM Models in Formal Languages
Venue: arXiv cs.CL 2018
Year: 2018

Recurrent Neural Networks (RNNs) are theoretically Turing-complete and established themselves as a dominant model for language processing. Yet, there still remains an uncertainty regarding their language learning capabilities. In this paper, we empirically evaluate the inductive learning capabilities of Long Short-Term Memory networks, a popular extension of simple RNNs, to learn simple formal languages, in particular $a^nb^n$, $a^nb^nc^n$, and $a^nb^nc^nd^n$. We investigate the influence of various aspects of learning, such as training data regimes and model capacity, on the generalization to unobserved samples. We find striking differences in model performances under different training settings and highlight the need for careful analysis and assessment when making claims about the learning capabilities of neural network models.
Unsupervised Learning of Interpretable Dialog Models
Venue: arXiv cs.AI 2018
Year: 2018

Recently several deep learning based models have been proposed for end-to-end learning of dialogs. While these models can be trained from data without the need for any additional annotations, it is hard to interpret them. On the other hand, there exist traditional state based dialog systems, where the states of the dialog are discrete and hence easy to interpret. However these states need to be handcrafted and annotated in the data. To achieve the best of both worlds, we propose Latent State Tracking Network (LSTN) using which we learn an interpretable model in unsupervised manner. The model defines a discrete latent variable at each turn of the conversation which can take a finite set of values. Since these discrete variables are not present in the training data, we use EM algorithm to train our model in unsupervised manner. In the experiments, we show that LSTN can help achieve interpretability in dialog models without much decrease in performance compared to end-to-end approaches.
Machine learning architectures to predict motion sickness using a Virtual Reality rollercoaster simulation tool
Venue: arXiv cs.HC 2018
Year: 2018

Virtual Reality (VR) can cause an unprecedented immersion and feeling of presence yet a lot of users experience motion sickness when moving through a virtual environment. Rollercoaster rides are popular in Virtual Reality but have to be well designed to limit the amount of nausea the user may feel. This paper describes a novel framework to get automated ratings on motion sickness using Neural Networks. An application that lets users create rollercoasters directly in VR, share them with other users and ride and rate them is used to gather real-time data related to the in-game behaviour of the player, the track itself and users' ratings based on a Simulator Sickness Questionnaire (SSQ) integrated into the application. Machine learning architectures based on deep neural networks are trained using this data aiming to predict motion sickness levels. While this paper focuses on rollercoasters this framework could help to rate any VR application on motion sickness and intensity that involves camera movement. A new well defined dataset is provided in this paper and the performance of the proposed architectures are evaluated in a comparative study.
Neural Task Representations as Weak Supervision for Model Agnostic Cross-Lingual Transfer
Venue: arXiv cs.CL 2018
Year: 2018

Natural language processing is heavily Anglo-centric, while the demand for models that work in languages other than English is greater than ever. Yet, the task of transferring a model from one language to another can be expensive in terms of annotation costs, engineering time and effort. In this paper, we present a general framework for easily and effectively transferring neural models from English to other languages. The framework, which relies on task representations as a form of weak supervision, is model and task agnostic, meaning that many existing neural architectures can be ported to other languages with minimal effort. The only requirement is unlabeled parallel data, and a loss defined over task representations. We evaluate our framework by transferring an English sentiment classifier to three different languages. On a battery of tests, we show that our models outperform a number of strong baselines and rival state-of-the-art results, which rely on more complex approaches and significantly more resources and data. Additionally, we find that the framework proposed in this paper is able to capture semantically rich and meaningful representations across languages, despite the lack of direct supervision.
Optimal Power Flow: An Introduction to Predictive, Distributed and Stochastic Control Challenges
Venue: arXiv cs.SY 2018
Year: 2018

The Energiewende is a paradigm change that can be witnessed at latest since the political decision to step out of nuclear energy. Moreover, despite common roots in Electrical Engineering, the control community and the power systems community face a lack of common vocabulary. In this context, this paper aims at providing a systems-and-control specific introduction to optimal power flow problems which are pivotal in the operation of energy systems. Based on a concise problem statement, we introduce a common description of optimal power flow variants including multi-stage-problems and predictive control, stochastic uncertainties, and issues of distributed optimization. Moreover, we sketch open questions that might be of interest for the systems and control community.
Radius-margin bounds for deep neural networks
Venue: arXiv cs.LG 2018
Year: 2018

Explaining the unreasonable effectiveness of deep learning has eluded researchers around the globe. Various authors have described multiple metrics to evaluate the capacity of deep architectures. In this paper, we allude to the radius margin bounds described for a support vector machine (SVM) with hinge loss, apply the same to the deep feed-forward architectures and derive the Vapnik-Chervonenkis (VC) bounds which are different from the earlier bounds proposed in terms of number of weights of the network. In doing so, we also relate the effectiveness of techniques like Dropout and Dropconnect in bringing down the capacity of the network. Finally, we describe the effect of maximizing the input as well as the output margin to achieve an input noise-robust deep architecture.
Adversarial Gain
Venue: arXiv cs.LG 2018
Year: 2018

Adversarial examples can be defined as inputs to a model which induce a mistake - where the model output is different than that of an oracle, perhaps in surprising or malicious ways. Original models of adversarial attacks are primarily studied in the context of classification and computer vision tasks. While several attacks have been proposed in natural language processing (NLP) settings, they often vary in defining the parameters of an attack and what a successful attack would look like. The goal of this work is to propose a unifying model of adversarial examples suitable for NLP tasks in both generative and classification settings. We define the notion of adversarial gain: based in control theory, it is a measure of the change in the output of a system relative to the perturbation of the input (caused by the so-called adversary) presented to the learner. This definition, as we show, can be used under different feature spaces and distance conditions to determine attack or defense effectiveness across different intuitive manifolds. This notion of adversarial gain not only provides a useful way for evaluating adversaries and defenses, but can act as a building block for future work in robustness under adversaries due to its rooted nature in stability and manifold theory.
ColNet: Embedding the Semantics of Web Tables for Column Type Prediction
Venue: arXiv cs.CL 2018
Year: 2018

Automatically annotating column types with knowledge base (KB) concepts is a critical task to gain a basic understanding of web tables. Current methods rely on either table metadata like column name or entity correspondences of cells in the KB, and may fail to deal with growing web tables with incomplete meta information. In this paper we propose a neural network based column type annotation framework named ColNet which is able to integrate KB reasoning and lookup with machine learning and can automatically train Convolutional Neural Networks for prediction. The prediction model not only considers the contextual semantics within a cell using word representation, but also embeds the semantics of a column by learning locality features from multiple cells. The method is evaluated with DBPedia and two different web table datasets, T2Dv2 from the general Web and Limaye from Wikipedia pages, and achieves higher performance than the state-of-the-art approaches.
A Batched Scalable Multi-Objective Bayesian Optimization Algorithm
Venue: arXiv cs.NE 2018
Year: 2018

The surrogate-assisted optimization algorithm is a promising approach for solving expensive multi-objective optimization problems. However, most existing surrogate-assisted multi-objective optimization algorithms have three main drawbacks: 1) cannot scale well for solving problems with high dimensional decision space, 2) cannot incorporate available gradient information, and 3) do not support batch optimization. These drawbacks prevent their use for solving many real-world large scale optimization problems. This paper proposes a batched scalable multi-objective Bayesian optimization algorithm to tackle these issues. The proposed algorithm uses the Bayesian neural network as the scalable surrogate model. Powered with Monte Carlo dropout and Sobolov training, the model can be easily trained and can incorporate available gradient information. We also propose a novel batch hypervolume upper confidence bound acquisition function to support batch optimization. Experimental results on various benchmark problems and a real-world application demonstrate the efficiency of the proposed algorithm.
Testing Reviewer Suggestions Derived from Bibliometric Specialty Approximations in Real Research Evaluations
Venue: arXiv cs.DL 2018
Year: 2018

Many contemporary research funding instruments and research policies aim for excellence at the level of individual scientists, teams or research programmes. Good bibliometric approximations of related specialties could be useful for instance to help assign reviewers to applications. This paper reports findings on the usability of reviewer suggestions derived from a recently developed specialty approximation method combining key sources, title words, authors and references (Rons, 2018). Reviewer suggestions for applications for Senior Research Fellowships were made available to the evaluation coordinators. Those who were invited to review an application showed a normal acceptance rate, and responses from experts and coordinators contained no indications of mismatched scientific focus. The results confirm earlier indications that this specialty approximation method can successfully support tasks in research management.
Multiuser Wirelessly Powered Backscatter Communications: Nonlinearity, Waveform Design and SINR-Energy Tradeoff
Venue: arXiv cs.IT 2018
Year: 2018

Wireless power transfer and backscatter communications have emerged as promising solutions for energizing and communicating with power limited devices. Despite some progress in wirelessly powered backscatter communications, the focus has been concentrated on backscatter and energy harvester. Recently, significant progress has been made on the design of transmit multisine waveform, adaptive to the Channel State Information at the transmitter (CSIT), in point-to-point backscatter system. In this paper, we leverage the work and study the design of transmit multisine waveform in a multiuser backscatter system, made of one transmitter, one reader and multiple tags active simultaneously. We derive an efficient algorithm to optimize the transmit waveform so as to identify the tradeoff between the amount of energy harvested at the tags and the reliability of the communication, measured in terms of Signal-to-Interference-plus-Noise Ratio (SINR) at the reader. Performance with the optimized waveform based on linear and nonlinear energy harvester (EH) models are studied. Numerical results demonstrate the benefits of accounting for the energy harvester nonlinearity, multiuser diversity, frequency diversity and multisine waveform adaptive to the CSIT to enlarge the SINR-energy region.
Web Security Investigation through Penetration Tests: A Case study of an Educational Institution Portal
Venue: arXiv cs.CR 2018
Year: 2018

Web security has become an important subject; many companies and organizations are becoming more security conscious as they build web applications to render online services and increase web presence. Unfortunately, many of these web applications are still susceptible to threats as they lack strong immunity to malicious attacks. This poses potential danger to the users of the sites and could also affect operations of the organizations or companies concerned. Educational institutions are not left out, their portals and websites hold vital information whose integrity is of utmost importance. Taking Carnegie Mellon University Africa's internship portal as case study, we carried out penetration tests to investigate web vulnerabilities and proffered possible remedies to the discovered vulnerabilities. Our result will inform educational institutions on better website security practices, especially in the African domain.
Fast Face Image Synthesis with Minimal Training
Venue: arXiv cs.CV 2018
Year: 2018

We propose an algorithm to generate realistic face images of both real and synthetic identities (people who do not exist) with different facial yaw, shape and resolution.The synthesized images can be used to augment datasets to train CNNs or as massive distractor sets for biometric verification experiments without any privacy concerns. Additionally, law enforcement can make use of this technique to train forensic experts to recognize faces. Our method samples face components from a pool of multiple face images of real identities to generate the synthetic texture. Then, a real 3D head model compatible to the generated texture is used to render it under different facial yaw transformations. We perform multiple quantitative experiments to assess the effectiveness of our synthesis procedure in CNN training and its potential use to generate distractor face images. Additionally, we compare our method with popular GAN models in terms of visual quality and execution time.