OpenRaiser/Intern-Atlas huggingface dataset:
Mitigating Confirmation Bias on Twitter by Recommending Opposing Views
Venue: arXiv cs.IR 2018
Year: 2018

In this work, we propose a content-based recommendation approach to increase exposure to opposing beliefs and opinions. Our aim is to help provide users with more diverse viewpoints on issues, which are discussed in partisan groups from different perspectives. Since due to the backfire effect, people's original beliefs tend to strengthen when challenged with counter evidence, we need to expose them to opposing viewpoints at the right time. The preliminary work presented here describes our first step into this direction. As illustrative showcase, we take the political debate on Twitter around the presidency of Donald Trump.
Simple single-scale microstructures based on optimal rank-3 laminates
Venue: arXiv cs.CE 2018
Year: 2018

With the goal of identifying optimal elastic single-scale microstructures for multiple loading situations, the paper shows that qualified starting guesses, based on knowledge of optimal rank-3 laminates, significantly improves chances of convergence to near optimal designs. Rank-3 laminates, optimal for a given set of anisotropic loading conditions, are approximated on a single scale using a simple mapping approach. We demonstrate that these mapped microstructures perform relatively close to theoretical energy bounds. Microstructures with performance even closer to the bounds can be obtained by using the approximated rank-3 structures in a further step as starting guesses for inverse homogenization problems. Due to the non-convex nature of inverse homogenization problems, the starting guesses based on rank-3 laminates outperform classical starting guesses with homogeneous or random material distributions. Furthermore, the obtained single-scale microstructures are relatively simple, which enhances manufacturability. Results, obtained for a wide range of loading cases, indicate that microstructures with performance within 5-8% of the theoretical optima can be guarantied, as long as feature sizes are not limited by minimium size constraints.
On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translation
Venue: arXiv cs.CL 2018
Year: 2018

This work investigates the alignment problem in state-of-the-art multi-head attention models based on the transformer architecture. We demonstrate that alignment extraction in transformer models can be improved by augmenting an additional alignment head to the multi-head source-to-target attention component. This is used to compute sharper attention weights. We describe how to use the alignment head to achieve competitive performance. To study the effect of adding the alignment head, we simulate a dictionary-guided translation task, where the user wants to guide translation using pre-defined dictionary entries. Using the proposed approach, we achieve up to $3.8$ % BLEU improvement when using the dictionary, in comparison to $2.4$ % BLEU in the baseline case. We also propose alignment pruning to speed up decoding in alignment-based neural machine translation (ANMT), which speeds up translation by a factor of $1.8$ without loss in translation performance. We carry out experiments on the shared WMT 2016 English$\to$Romanian news task and the BOLT Chinese$\to$English discussion forum task.
Efficient Road Lane Marking Detection with Deep Learning
Venue: arXiv cs.CV 2018
Year: 2018

Lane mark detection is an important element in the road scene analysis for Advanced Driver Assistant System (ADAS). Limited by the onboard computing power, it is still a challenge to reduce system complexity and maintain high accuracy at the same time. In this paper, we propose a Lane Marking Detector (LMD) using a deep convolutional neural network to extract robust lane marking features. To improve its performance with a target of lower complexity, the dilated convolution is adopted. A shallower and thinner structure is designed to decrease the computational cost. Moreover, we also design post-processing algorithms to construct 3rd-order polynomial models to fit into the curved lanes. Our system shows promising results on the captured road scenes.
Camouflaged with Size: A Case Study of Espionage using Acquirable Single-Board Computers
Venue: arXiv cs.CR 2018
Year: 2018

Single-Board Computers (SBC) refer to pocket-sized computers built on a single circuit board. A number of studies have explored the use of these highly popular devices in a variety of domains, including military, agriculture, healthcare, and more. However, no attempt was made to signify possible security risks that misuse of these devices may bring to organizations. In this study, we perform a series of experiments to validate the possibility of using SBCs as an espionage gadget. We show how an attacker can turn a Raspberry Pi device to an attacking gadget and benefit from short-term physical access to attach the gadget to the network in order to access unauthorized data or perform other malicious activities. We then provide experimental results of placing such tools in two real-world networks. Given the small size of SBCs, traditional physical security measures deployed in organizations may not be sufficient to detect and restrict the entrance of SBCs to their premises. Therefore, we reiterate possible directions for network administrators to deploy defensive mechanisms for detecting and preventing such attacks.
Cartesian Neural Network Constitutive Models for Data-driven Elasticity Imaging
Venue: arXiv cs.LG 2018
Year: 2018

Elasticity images map biomechanical properties of soft tissues to aid in the detection and diagnosis of pathological states. In particular, quasi-static ultrasonic (US) elastography techniques use force-displacement measurements acquired during an US scan to parameterize the spatio-temporal stress-strain behavior. Current methods use a model-based inverse approach to estimate the parameters associated with a chosen constitutive model. However, model-based methods rely on simplifying assumptions of tissue biomechanical properties, often limiting elastography to imaging one or two linear-elastic parameters. We previously described a data-driven method for building neural network constitutive models (NNCMs) that learn stress-strain relationships from force-displacement data. Using measurements acquired on gelatin phantoms, we demonstrated the ability of NNCMs to characterize linear-elastic mechanical properties without an initial model assumption and thus circumvent the mathematical constraints typically encountered in classic model-based approaches to the inverse problem. While successful, we were required to use a priori knowledge of the internal object shape to define the spatial distribution of regions exhibiting different material properties. Here, we introduce Cartesian neural network constitutive models (CaNNCMs) that are capable of using data to model both linear-elastic mechanical properties and their distribution in space. We demonstrate the ability of CaNNCMs to capture arbitrary material property distributions using stress-strain data from simulated phantoms. Furthermore, we show that a trained CaNNCM can be used to reconstruct a Young's modulus image. CaNNCMs are an important step toward data-driven modeling and imaging the complex mechanical properties of soft tissues.
Faster Variational Execution with Transparent Bytecode Transformation
Venue: arXiv cs.PL 2018
Year: 2018

Variational execution is a novel dynamic analysis technique for exploring highly configurable systems and accurately tracking information flow. It is able to efficiently analyze many configurations by aggressively sharing redundancies of program executions. The idea of variational execution has been demonstrated to be effective in exploring variations in the program, especially when the configuration space grows out of control. Existing implementations of variational execution often require heavy lifting of the runtime interpreter, which is painstaking and error-prone. Furthermore, the performance of this approach is suboptimal. For example, the state-of-the-art variational execution interpreter for Java, VarexJ, slows down executions by 100 to 800 times over a single execution for small to medium size Java programs. Instead of modifying existing JVMs, we propose to transform existing bytecode to make it variational, so it can be executed on an unmodified commodity JVM. Our evaluation shows a dramatic improvement on performance over the state-of-the-art, with a speedup of 2 to 46 times, and high efficiency in sharing computations.
Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals
Venue: arXiv cs.LG 2018
Year: 2018

We show that many machine learning goals, such as improved fairness metrics, can be expressed as constraints on the model's predictions, which we call rate constraints. We study the problem of training non-convex models subject to these rate constraints (or any non-convex and non-differentiable constraints). In the non-convex setting, the standard approach of Lagrange multipliers may fail. Furthermore, if the constraints are non-differentiable, then one cannot optimize the Lagrangian with gradient-based methods. To solve these issues, we introduce the proxy-Lagrangian formulation. This new formulation leads to an algorithm that produces a stochastic classifier by playing a two-player non-zero-sum game solving for what we call a semi-coarse correlated equilibrium, which in turn corresponds to an approximately optimal and feasible solution to the constrained optimization problem. We then give a procedure which shrinks the randomized solution down to one that is a mixture of at most $m+1$ deterministic solutions, given $m$ constraints. This culminates in algorithms that can solve non-convex constrained optimization problems with possibly non-differentiable and non-convex constraints with theoretical guarantees. We provide extensive experimental results enforcing a wide range of policy goals including different fairness metrics, and other goals on accuracy, coverage, recall, and churn.
Synthetic Attribute Data for Evaluating Consumer-side Fairness
Venue: arXiv cs.CY 2018
Year: 2018

When evaluating recommender systems for their fairness, it may be necessary to make use of demographic attributes, which are personally sensitive and usually excluded from publicly-available data sets. In addition, these attributes are fixed and therefore it is not possible to experiment with different distributions using the same data. In this paper, we describe the Frequency-Linked Attribute Generation (FLAG) algorithm, and show its applicability for assigning synthetic demographic attributes to recommendation data sets.
Safe Exploration in Markov Decision Processes with Time-Variant Safety using Spatio-Temporal Gaussian Process
Venue: arXiv cs.AI 2018
Year: 2018

In many real-world applications (e.g., planetary exploration, robot navigation), an autonomous agent must be able to explore a space with guaranteed safety. Most safe exploration algorithms in the field of reinforcement learning and robotics have been based on the assumption that the safety features are a priori known and time-invariant. This paper presents a learning algorithm called ST-SafeMDP for exploring Markov decision processes (MDPs) that is based on the assumption that the safety features are a priori unknown and time-variant. In this setting, the agent explores MDPs while constraining the probability of entering unsafe states defined by a safety function being below a threshold. The unknown and time-variant safety values are modeled using a spatio-temporal Gaussian process. However, there remains an issue that an agent may have no viable action in a shrinking true safe space. To address this issue, we formulate a problem maximizing the cumulative number of safe states in the worst case scenario with respect to future observations. The effectiveness of this approach was demonstrated in two simulation settings, including one using real lunar terrain data.
Music Transformer
Venue: arXiv cs.LG 2018
Year: 2018

Music relies heavily on repetition to build structure and meaning. Self-reference occurs on multiple timescales, from motifs to phrases to reusing of entire sections of music, such as in pieces with ABA structure. The Transformer (Vaswani et al., 2017), a sequence model based on self-attention, has achieved compelling results in many generation tasks that require maintaining long-range coherence. This suggests that self-attention might also be well-suited to modeling music. In musical composition and performance, however, relative timing is critically important. Existing approaches for representing relative positional information in the Transformer modulate attention based on pairwise distance (Shaw et al., 2018). This is impractical for long sequences such as musical compositions since their memory complexity for intermediate relative information is quadratic in the sequence length. We propose an algorithm that reduces their intermediate memory requirement to linear in the sequence length. This enables us to demonstrate that a Transformer with our modified relative attention mechanism can generate minute-long compositions (thousands of steps, four times the length modeled in Oore et al., 2018) with compelling structure, generate continuations that coherently elaborate on a given motif, and in a seq2seq setup generate accompaniments conditioned on melodies. We evaluate the Transformer with our relative attention mechanism on two datasets, JSB Chorales and Piano-e-Competition, and obtain state-of-the-art results on the latter.
Data-driven repetitive control: Wind tunnel experiments under turbulent conditions
Venue: arXiv cs.SY 2018
Year: 2018

A commonly applied method to reduce the cost of wind energy, is alleviating the periodic loads on turbine blades using Individual Pitch Control (IPC). In this paper, a data-driven IPC methodology called Subspace Predictive Repetitive Control (SPRC) is employed. The effectiveness of SPRC will be demonstrated on a scaled 2-bladed wind turbine. An open-jet wind tunnel with an innovative active grid is employed to generate reproducible turbulent wind conditions. A significant load reduction with limited actuator duty is achieved even under these high turbulent conditions. Furthermore, it will be demonstrated that SPRC is able to adapt to changing operating conditions.
Thermal Features for Presentation Attack Detection in Hand Biometrics
Venue: arXiv cs.CV 2018
Year: 2018

This paper proposes a method for utilizing thermal features of the hand for the purpose of presentation attack detection (PAD) that can be employed in a hand biometrics system's pipeline. By envisaging two different operational modes of our system, and by employing a DCNN-based classifiers fine-tuned with a dataset of real and fake hand representations captured in both visible and ther- mal spectrum, we were able to bring two important deliverables. First, a PAD method operating in an open-set mode, capable of correctly discerning 100% of fake thermal samples, achieving Attack Presentation Classification Error Rate (APCER) and Bona-Fide Presentation Classification Error Rate (BPCER) equal to 0%, which can be easily implemented into any existing system as a separate component. Second, a hand biometrics system operating in a closed-set mode, that has PAD built right into the recognition pipeline, and operating simultaneously with the user-wise classification, achieving rank-1 recognition accuracy of up to 99.75%. We also show that thermal images of the human hand, in addition to liveness features they carry, can also improve classification accuracy of a biometric system, when coupled with visible light images. To follow the reproducibility guidelines and to stimulate further research in this area, we share the trained model weights, source codes, and a newly created dataset of fake hand representations with interested researchers.
A Framework for Approval-based Budgeting Methods
Venue: arXiv cs.AI 2018
Year: 2018

We define and study a general framework for approval-based budgeting methods and compare certain methods within this framework by their axiomatic and computational properties. Furthermore, we visualize their behavior on certain Euclidean distributions and analyze them experimentally.
A Curry-Howard Correspondence for the Minimal Fragment of Łukasiewicz Logic
Venue: arXiv cs.LO 2018
Year: 2018

In this paper we introduce a term calculus ${\cal B}$ which adds to the affine $λ$-calculus with pairing a new construct allowing for a restricted form of contraction. We obtain a Curry-Howard correspondence between ${\cal B}$ and the sub-structural logical system which we call "minimal Łukasiewicz logic", also known in the literature as the logic of hoops (a generalisation of MV-algebras). This logic lies strictly in between affine minimal logic and standard minimal logic. We prove that ${\cal B}$ is strongly normalising and has the Church-Rosser property. We also give examples of terms in ${\cal B}$ corresponding to some important derivations from our work and the literature. Finally, we discuss the relation between normalisation in ${\cal B}$ and cut-elimination for a Gentzen-style formulation of minimal Łukasiewicz logic.
Emo2Vec: Learning Generalized Emotion Representation by Multi-task Training
Venue: arXiv cs.CL 2018
Year: 2018

In this paper, we propose Emo2Vec which encodes emotional semantics into vectors. We train Emo2Vec by multi-task learning six different emotion-related tasks, including emotion/sentiment analysis, sarcasm classification, stress detection, abusive language classification, insult detection, and personality recognition. Our evaluation of Emo2Vec shows that it outperforms existing affect-related representations, such as Sentiment-Specific Word Embedding and DeepMoji embeddings with much smaller training corpora. When concatenated with GloVe, Emo2Vec achieves competitive performances to state-of-the-art results on several tasks using a simple logistic regression classifier.
Image contrast enhancement using fuzzy logic
Venue: arXiv cs.CV 2018
Year: 2018

Image enhancement is a method of improving the quality of an image and contrast is a major aspect. Traditional methods of contrast enhancement like histogram equalization results in over/under enhancement of the image especially a lower resolution one. This paper aims at developing a new Fuzzy Inference System to enhance the contrast of the low resolution images overcoming the shortcomings of the traditional methods. Results obtained using both the approaches are compared.
The Inductive Bias of Restricted f-GANs
Venue: arXiv cs.LG 2018
Year: 2018

Generative adversarial networks are a novel method for statistical inference that have achieved much empirical success; however, the factors contributing to this success remain ill-understood. In this work, we attempt to analyze generative adversarial learning -- that is, statistical inference as the result of a game between a generator and a discriminator -- with the view of understanding how it differs from classical statistical inference solutions such as maximum likelihood inference and the method of moments. Specifically, we provide a theoretical characterization of the distribution inferred by a simple form of generative adversarial learning called restricted f-GANs -- where the discriminator is a function in a given function class, the distribution induced by the generator is restricted to lie in a pre-specified distribution class and the objective is similar to a variational form of the f-divergence. A consequence of our result is that for linear KL-GANs -- that is, when the discriminator is a linear function over some feature space and f corresponds to the KL-divergence -- the distribution induced by the optimal generator is neither the maximum likelihood nor the method of moments solution, but an interesting combination of both.
A Unified Batch Online Learning Framework for Click Prediction
Venue: arXiv cs.LG 2018
Year: 2018

We present a unified framework for Batch Online Learning (OL) for Click Prediction in Search Advertisement. Machine Learning models once deployed, show non-trivial accuracy and calibration degradation over time due to model staleness. It is therefore necessary to regularly update models, and do so automatically. This paper presents two paradigms of Batch Online Learning, one which incrementally updates the model parameters via an early stopping mechanism, and another which does so through a proximal regularization. We argue how both these schemes naturally trade-off between old and new data. We then theoretically and empirically show that these two seemingly different schemes are closely related. Through extensive experiments, we demonstrate the utility of of our OL framework; how the two OL schemes relate to each other and how they trade-off between the new and historical data. We then compare batch OL to full model retrains, and show how online learning is more robust to data issues. We also demonstrate the long term impact of Online Learning, the role of the initial Models in OL, the impact of delays in the update, and finally conclude with some implementation details and challenges in deploying a real world online learning system in production. While this paper mostly focuses on application of click prediction for search advertisement, we hope that the lessons learned here can be carried over to other problem domains.
Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation
Venue: arXiv cs.CL 2018
Year: 2018

Transferring representations from large supervised tasks to downstream tasks has shown promising results in AI fields such as Computer Vision and Natural Language Processing (NLP). In parallel, the recent progress in Machine Translation (MT) has enabled one to train multilingual Neural MT (NMT) systems that can translate between multiple languages and are also capable of performing zero-shot translation. However, little attention has been paid to leveraging representations learned by a multilingual NMT system to enable zero-shot multilinguality in other NLP tasks. In this paper, we demonstrate a simple framework, a multilingual Encoder-Classifier, for cross-lingual transfer learning by reusing the encoder from a multilingual NMT system and stitching it with a task-specific classifier component. Our proposed model achieves significant improvements in the English setup on three benchmark tasks - Amazon Reviews, SST and SNLI. Further, our system can perform classification in a new language for which no classification data was seen during training, showing that zero-shot classification is possible and remarkably competitive. In order to understand the underlying factors contributing to this finding, we conducted a series of analyses on the effect of the shared vocabulary, the training data type for NMT, classifier complexity, encoder representation power, and model generalization on zero-shot performance. Our results provide strong evidence that the representations learned from multilingual NMT systems are widely applicable across languages and tasks.
Semantically Enhanced Models for Commonsense Knowledge Acquisition
Venue: arXiv cs.AI 2018
Year: 2018

Commonsense knowledge is paramount to enable intelligent systems. Typically, it is characterized as being implicit and ambiguous, hindering thereby the automation of its acquisition. To address these challenges, this paper presents semantically enhanced models to enable reasoning through resolving part of commonsense ambiguity. The proposed models enhance in a knowledge graph embedding (KGE) framework for knowledge base completion. Experimental results show the effectiveness of the new semantic models in commonsense reasoning.