OpenRaiser/Intern-Atlas huggingface dataset:
Multimodal Dialogue Management for Multiparty Interaction with Infants
Venue: arXiv cs.HC 2018
Year: 2018

We present dialogue management routines for a system to engage in multiparty agent-infant interaction. The ultimate purpose of this research is to help infants learn a visual sign language by engaging them in naturalistic and socially contingent conversations during an early-life critical period for language development (ages 6 to 12 months) as initiated by an artificial agent. As a first step, we focus on creating and maintaining agent-infant engagement that elicits appropriate and socially contingent responses from the baby. Our system includes two agents, a physical robot and an animated virtual human. The system's multimodal perception includes an eye-tracker (measures attention) and a thermal infrared imaging camera (measures patterns of emotional arousal). A dialogue policy is presented that selects individual actions and planned multiparty sequences based on perceptual inputs about the baby's internal changing states of emotional engagement. The present version of the system was evaluated in interaction with 8 babies. All babies demonstrated spontaneous and sustained engagement with the agents for several minutes, with patterns of conversationally relevant and socially contingent behaviors. We further performed a detailed case-study analysis with annotation of all agent and baby behaviors. Results show that the baby's behaviors were generally relevant to agent conversations and contained direct evidence for socially contingent responses by the baby to specific linguistic samples produced by the avatar. This work demonstrates the potential for language learning from agents in very young babies and has especially broad implications regarding the use of artificial agents with babies who have minimal language exposure in early life.
Efficient Egocentric Visual Perception Combining Eye-tracking, a Software Retina and Deep Learning
Venue: arXiv cs.CV 2018
Year: 2018

We present ongoing work to harness biological approaches to achieving highly efficient egocentric perception by combining the space-variant imaging architecture of the mammalian retina with Deep Learning methods. By pre-processing images collected by means of eye-tracking glasses to control the fixation locations of a software retina model, we demonstrate that we can reduce the input to a DCNN by a factor of 3, reduce the required number of training epochs and obtain over 98% classification rates when training and validating the system on a database of over 26,000 images of 9 object classes.
Measures of Cluster Informativeness for Medical Evidence Aggregation and Dissemination
Venue: arXiv cs.IR 2018
Year: 2018

The largest collection of medical evidence in the world is PubMed. However, the significant barrier in accessing and extracting information is information organization. A factor that contributes towards this barrier is managing medical controlled vocabularies that allow us to systematically and consistently organize, index, and search biomedical literature. Additionally, from users' perspective, to ultimately improve access, visualization is likely to play a powerful role. There is a strong link between information organization and information visualization, as many powerful visualizations depend on clustering methods. To improve visualization, therefore, one has to develop concrete and scalable measures for vocabularies used in indexing and their impact on document clustering. The focus of this study is on the development and evaluation of clustering methods. The paper concludes with demonstration of downstream network visualizations and their impact on discovering potentially valuable and latent genetic and molecular associations.
Bridging machine learning and cryptography in defence against adversarial attacks
Venue: arXiv cs.CR 2018
Year: 2018

In the last decade, deep learning algorithms have become very popular thanks to the achieved performance in many machine learning and computer vision tasks. However, most of the deep learning architectures are vulnerable to so called adversarial examples. This questions the security of deep neural networks (DNN) for many security- and trust-sensitive domains. The majority of the proposed existing adversarial attacks are based on the differentiability of the DNN cost function.Defence strategies are mostly based on machine learning and signal processing principles that either try to detect-reject or filter out the adversarial perturbations and completely neglect the classical cryptographic component in the defence. In this work, we propose a new defence mechanism based on the second Kerckhoffs's cryptographic principle which states that the defence and classification algorithm are supposed to be known, but not the key. To be compliant with the assumption that the attacker does not have access to the secret key, we will primarily focus on a gray-box scenario and do not address a white-box one. More particularly, we assume that the attacker does not have direct access to the secret block, but (a) he completely knows the system architecture, (b) he has access to the data used for training and testing and (c) he can observe the output of the classifier for each given input. We show empirically that our system is efficient against most famous state-of-the-art attacks in black-box and gray-box scenarios.
Extending Mandelbox Fractals with Shape Inversions
Venue: arXiv cs.GR 2018
Year: 2018

The Mandelbox is a recently discovered class of escape-time fractals which use a conditional combination of reflection, spherical inversion, scaling, and translation to transform points under iteration. In this paper we introduce a new extension to Mandelbox fractals which replaces spherical inversion with a more generalized shape inversion. We then explore how this technique can be used to generate new fractals in 2D, 3D, and 4D.
Token Curated Registries - A Game Theoretic Approach
Venue: arXiv cs.GT 2018
Year: 2018

Token curated registries (TCRs) have been proposed recently as an approach to create and maintain high quality lists of resources or recommendations in a decentralized manner. Applications range from maintaining registries of web domains for advertising purposes (e.g., adChain) or restaurants, consumer products, etc. The registry is maintained through a combination of candidate applications requiring a token deposit, challenges based on token staking and token-weighted votes with a redistribution of tokens occurring as a consequence of the vote. We present a simplified mathematical model of a TCR and its challenge and voting process analyze it from a game-theoretic perspective. We derive some insights into conditions with respect to the quality of a candidate under which challenges occur, and under which the outcome is reject or accept. We also show that there are conditions under which the outcome may not be entirely predictable in the sense that everyone voting for accept and everyone voting for reject could both be Nash Equilibria outcomes. For such conditions, we also explore when a particular strategy profile may be payoff dominant. We identify ways in which our modeling can be extended and also some implications of our model with respect to the composition of TCRs.
Multi-view Factorization AutoEncoder with Network Constraints for Multi-omic Integrative Analysis
Venue: arXiv cs.LG 2018
Year: 2018

Multi-omic data provides multiple views of the same patients. Integrative analysis of multi-omic data is crucial to elucidate the molecular underpinning of disease etiology. However, multi-omic data has the "big p, small N" problem (the number of features is large, but the number of samples is small), it is challenging to train a complicated machine learning model from the multi-omic data alone and make it generalize well. Here we propose a framework termed Multi-view Factorization AutoEncoder with network constraints to integrate multi-omic data with domain knowledge (biological interactions networks). Our framework employs deep representation learning to learn feature embeddings and patient embeddings simultaneously, enabling us to integrate feature interaction network and patient view similarity network constraints into the training objective. The whole framework is end-to-end differentiable. We applied our approach to the TCGA Pan-cancer dataset and achieved satisfactory results to predict disease progression-free interval (PFI) and patient overall survival (OS) events. Code will be made publicly available.
Driving Experience Transfer Method for End-to-End Control of Self-Driving Cars
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we present a transfer learning method for the end-to-end control of self-driving cars, which enables a convolutional neural network (CNN) trained on a source domain to be utilized for the same task in a different target domain. A conventional CNN for the end-to-end control is designed to map a single front-facing camera image to a steering command. To enable the transfer learning, we let the CNN produce not only a steering command but also a lane departure level (LDL) by adding a new task module, which takes the output of the last convolutional layer as input. The CNN trained on the source domain, called source network, is then utilized to train another task module called target network, which also takes the output of the last convolutional layer of the source network and is trained to produce a steering command for the target domain. The steering commands from the source and target network are finally merged according to the LDL and the merged command is utilized for controlling a car in the target domain. To demonstrate the effectiveness of the proposed method, we utilized two simulators, TORCS and GTAV, for the source and the target domains, respectively. Experimental results show that the proposed method outperforms other baseline methods in terms of stable and safe control of cars.
Robust Signaling for Bursty Interference
Venue: arXiv cs.IT 2018
Year: 2018

This paper studies a bursty interference channel, where the presence/absence of interference is modeled by a block-i.i.d.\ Bernoulli process that stays constant for a duration of $T$ symbols (referred to as coherence block) and then changes independently to a new state. We consider both a quasi-static setup, where the interference state remains constant during the whole transmission of the codeword, and an ergodic setup, where a codeword spans several coherence blocks. For the quasi-static setup, we study the largest rate of a coding strategy that provides reliable communication at a basic rate and allows an increased (opportunistic) rate when there is no interference. For the ergodic setup, we study the largest achievable rate. We study how non-causal knowledge of the interference state, referred to as channel-state information (CSI), affects the achievable rates. We derive converse and achievability bounds for (i) local CSI at the receiver-side only; (ii) local CSI at the transmitter- and receiver-side, and (iii) global CSI at all nodes. Our bounds allow us to identify when interference burstiness is beneficial and in which scenarios global CSI outperforms local CSI. The joint treatment of the quasi-static and ergodic setup further allows for a thorough comparison of these two setups.
Exploring Graph-structured Passage Representation for Multi-hop Reading Comprehension with Graph Neural Networks
Venue: arXiv cs.CL 2018
Year: 2018

Multi-hop reading comprehension focuses on one type of factoid question, where a system needs to properly integrate multiple pieces of evidence to correctly answer a question. Previous work approximates global evidence with local coreference information, encoding coreference chains with DAG-styled GRU layers within a gated-attention reader. However, coreference is limited in providing information for rich inference. We introduce a new method for better connecting global evidence, which forms more complex graphs compared to DAGs. To perform evidence integration on our graphs, we investigate two recent graph neural networks, namely graph convolutional network (GCN) and graph recurrent network (GRN). Experiments on two standard datasets show that richer global information leads to better answers. Our method performs better than all published results on these datasets.
Multiple Object Tracking in Urban Traffic Scenes with a Multiclass Object Detector
Venue: arXiv cs.CV 2018
Year: 2018

Multiple object tracking (MOT) in urban traffic aims to produce the trajectories of the different road users that move across the field of view with different directions and speeds and that can have varying appearances and sizes. Occlusions and interactions among the different objects are expected and common due to the nature of urban road traffic. In this work, a tracking framework employing classification label information from a deep learning detection approach is used for associating the different objects, in addition to object position and appearances. We want to investigate the performance of a modern multiclass object detector for the MOT task in traffic scenes. Results show that the object labels improve tracking performance, but that the output of object detectors are not always reliable.
Agile meets Assessments: Case Study on how to do Agile Process Improvement in a Very Small Enterprise
Venue: arXiv cs.SE 2018
Year: 2018

Smaller software companies, such as start-ups do not often follow an explicit process, but rather develop in a more or less unstructured way. Especially when they grow or customer involvement increases. This development without any structured process results in problems. Thus, our objective was the improvement of the current development process of one software start-up by introducing appropriate agile practices and eliciting their effects. For this reason, we per-formed a pre and post process assessment using interviews. Based on the initial assessment, agile practices were selected and implemented. Finally, the post assessment and additional code metrics served as controlling mechanism to check whether weak points are addressed. The comparison of the two assessments showed that 13 ISO29110 base practices have been improved by the introduced eight agile practices. Thus, even more aspects have casually been improved than initially planned. Finally, the additional retrospective with company employees showed how the introduced agile practices positively influenced their work.
Five lessons from building a deep neural network recommender
Venue: arXiv cs.IR 2018
Year: 2018

Recommendation algorithms are widely adopted in marketplaces to help users find the items they are looking for. The sparsity of the items by user matrix and the cold-start issue in marketplaces pose challenges for the off-the-shelf matrix factorization based recommender systems. To understand user intent and tailor recommendations to their needs, we use deep learning to explore various heterogeneous data available in marketplaces. This paper summarizes five lessons we learned from experimenting with state-of-the-art deep learning recommenders at the leading Norwegian marketplace FINN.no. We design a hybrid recommender system that takes the user-generated contents of a marketplace (including text, images and meta attributes) and combines them with user behavior data such as page views and messages to provide recommendations for marketplace items. Among various tactics we experimented with, the following five show the best impact: staged training instead of end-to-end training, leveraging rich user behaviors beyond page views, using user behaviors as noisy labels to train embeddings, using transfer learning to solve the unbalanced data problem, and using attention mechanisms in the hybrid model. This system is currently running with around 20% click-through-rate in production at FINN.no and serves over one million visitors everyday.
End-to-End Analysis of In-Browser Cryptojacking
Venue: arXiv cs.CR 2018
Year: 2018

In-browser cryptojacking involves hijacking the CPU power of a website's visitor to perform CPU-intensive cryptocurrency mining, and has been on the rise, with 8500% growth during 2017. While some websites advocate cryptojacking as a replacement for online advertisement, web attackers exploit it to generate revenue by embedding malicious cryptojacking code in highly ranked websites. Motivated by the rise of cryptojacking and the lack of any prior systematic work, we set out to analyze malicious cryptojacking statically and dynamically, and examine the economical basis of cryptojacking as an alternative to advertisement. For our static analysis, we perform content-, currency-, and code-based analyses. Through the content-based analysis, we unveil that cryptojacking is a wide-spread threat targeting a variety of website types. Through a currency-based analysis we highlight affinities between mining platforms and currencies: the majority of cryptojacking websites use Coinhive to mine Monero. Through code-based analysis, we highlight unique code complexity features of cryptojacking scripts, and use them to detect cryptojacking code among benign and other malicious JavaScript code, with an accuracy of 96.4%. Through dynamic analysis, we highlight the impact of cryptojacking on system resources, such as CPU and battery consumption (in battery-powered devices); we use the latter to build an analytical model that examines the feasibility of cryptojacking as an alternative to online advertisement, and show a huge negative profit/loss gap, suggesting that the model is impractical. By surveying existing countermeasures and their limitations, we conclude with long-term countermeasures using insights from our analysis.
2PFPCE: Two-Phase Filter Pruning Based on Conditional Entropy
Venue: arXiv cs.CV 2018
Year: 2018

Deep Convolutional Neural Networks~(CNNs) offer remarkable performance of classifications and regressions in many high-dimensional problems and have been widely utilized in real-word cognitive applications. However, high computational cost of CNNs greatly hinder their deployment in resource-constrained applications, real-time systems and edge computing platforms. To overcome this challenge, we propose a novel filter-pruning framework, two-phase filter pruning based on conditional entropy, namely \textit{2PFPCE}, to compress the CNN models and reduce the inference time with marginal performance degradation. In our proposed method, we formulate filter pruning process as an optimization problem and propose a novel filter selection criteria measured by conditional entropy. Based on the assumption that the representation of neurons shall be evenly distributed, we also develop a maximum-entropy filter freeze technique that can reduce over fitting. Two filter pruning strategies -- global and layer-wise strategies, are compared. Our experiment result shows that combining these two strategies can achieve a higher neural network compression ratio than applying only one of them under the same accuracy drop threshold. Two-phase pruning, that is, combining both global and layer-wise strategies, achieves 10 X FLOPs reduction and 46% inference time reduction on VGG-16, with 2% accuracy drop.
Automated Game Design via Conceptual Expansion
Venue: arXiv cs.AI 2018
Year: 2018

Automated game design has remained a key challenge within the field of Game AI. In this paper, we introduce a method for recombining existing games to create new games through a process called conceptual expansion. Prior automated game design approaches have relied on hand-authored or crowd-sourced knowledge, which limits the scope and applications of such systems. Our approach instead relies on machine learning to learn approximate representations of games. Our approach recombines knowledge from these learned representations to create new games via conceptual expansion. We evaluate this approach by demonstrating the ability for the system to recreate existing games. To the best of our knowledge, this represents the first machine learning-based automated game design system.
Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations
Venue: arXiv cs.CL 2018
Year: 2018

Most existing recursive neural network (RvNN) architectures utilize only the structure of parse trees, ignoring syntactic tags which are provided as by-products of parsing. We present a novel RvNN architecture that can provide dynamic compositionality by considering comprehensive syntactic information derived from both the structure and linguistic tags. Specifically, we introduce a structure-aware tag representation constructed by a separate tag-level tree-LSTM. With this, we can control the composition function of the existing word-level tree-LSTM by augmenting the representation as a supplementary input to the gate functions of the tree-LSTM. In extensive experiments, we show that models built upon the proposed architecture obtain superior or competitive performance on several sentence-level tasks such as sentiment analysis and natural language inference when compared against previous tree-structured models and other sophisticated neural models.
Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion
Venue: arXiv cs.LG 2018
Year: 2018

In tensor completion tasks, the traditional low-rank tensor decomposition models suffer from the laborious model selection problem due to their high model sensitivity. In particular, for tensor ring (TR) decomposition, the number of model possibilities grows exponentially with the tensor order, which makes it rather challenging to find the optimal TR decomposition. In this paper, by exploiting the low-rank structure of the TR latent space, we propose a novel tensor completion method which is robust to model selection. In contrast to imposing the low-rank constraint on the data space, we introduce nuclear norm regularization on the latent TR factors, resulting in the optimization step using singular value decomposition (SVD) being performed at a much smaller scale. By leveraging the alternating direction method of multipliers (ADMM) scheme, the latent TR factors with optimal rank and the recovered tensor can be obtained simultaneously. Our proposed algorithm is shown to effectively alleviate the burden of TR-rank selection, thereby greatly reducing the computational cost. The extensive experimental results on both synthetic and real-world data demonstrate the superior performance and efficiency of the proposed approach against the state-of-the-art algorithms.