OpenRaiser/Intern-Atlas huggingface dataset:
NGEMM: Optimizing GEMM for Deep Learning via Compiler-based Techniques
Venue: arXiv cs.LG 2019
Year: 2019

Quantization has emerged to be an effective way to significantly boost the performance of deep neural networks (DNNs) by utilizing low-bit computations. Despite having lower numerical precision, quantized DNNs are able to reduce both memory bandwidth and computation cycles with little losses of accuracy. Integer GEMM (General Matrix Multiplication) is critical to running quantized DNN models efficiently, as GEMM operations often dominate the computations in these models. Various approaches have been developed by leveraging techniques such as vectorization and memory layout to improve the performance of integer GEMM. However, these existing approaches are not fast enough in certain scenarios. We developed NGEMM, a compiler-based GEMM implementation for accelerating lower-precision training and inference. NGEMM has better use of the vector units by avoiding unnecessary vector computation that is introduced during tree reduction. We compared NGEMM's performance with the state-of-art BLAS libraries such as MKL. Our experimental results showed that NGEMM outperformed MKL non-pack and pack version by an average of 1.86x and 1.16x, respectively. We have applied NGEMM to a number of production services in Microsoft.
Beyond Textual Issues: Understanding the Usage and Impact of GitHub Reactions
Venue: arXiv cs.SE 2019
Year: 2019

Recently, GitHub introduced a new social feature, named reactions, which are "pictorial characters" similar to emoji symbols widely used nowadays in text-based communications. Particularly, GitHub users can use a pre-defined set of such symbols to react to issues and pull requests. However, little is known about the real usage and impact of GitHub reactions. In this paper, we analyze the reactions provided by developers to more than 2.5 million issues and 9.7 million issue comments, in order to answer an extensive list of nine research questions about the usage and adoption of reactions. We show that reactions are being increasingly used by open source developers. Moreover, we also found that issues with reactions usually take more time to be handled and have longer discussions.
Blending Diverse Physical Priors with Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Machine learning in context of physical systems merits a re-examination of the learning strategy. In addition to data, one can leverage a vast library of physical prior models (e.g. kinematics, fluid flow, etc) to perform more robust inference. The nascent sub-field of \emph{physics-based learning} (PBL) studies the blending of neural networks with physical priors. While previous PBL algorithms have been applied successfully to specific tasks, it is hard to generalize existing PBL methods to a wide range of physics-based problems. Such generalization would require an architecture that can adapt to variations in the correctness of the physics, or in the quality of training data. No such architecture exists. In this paper, we aim to generalize PBL, by making a first attempt to bring neural architecture search (NAS) to the realm of PBL. We introduce a new method known as physics-based neural architecture search (PhysicsNAS) that is a top-performer across a diverse range of quality in the physical model and the dataset.
Mistakes in Games
Venue: arXiv cs.GT 2019
Year: 2019

We define a new concept of "mistake" strategies and actions for strategic-form and extensive-form games, analyze the relationship to prior main game-theoretic solution concepts, study algorithms for computation, and explore practicality. This concept has potential applications to cybersecurity, for example detecting whether a human player is illegally using real-time assistance in games like poker.
Sampling Unknown Decision Functions to Build Classifier Copies
Venue: arXiv cs.LG 2019
Year: 2019

Copies have been proposed as a viable alternative to endow machine learning models with properties and features that adapt them to changing needs. A fundamental step of the copying process is generating an unlabelled set of points to explore the decision behavior of the targeted classifier throughout the input space. In this article we propose two sampling strategies to produce such sets. We validate them in six well-known problems and compare them with two standard methods. We evaluate our proposals in terms of both their accuracy performance and their computational cost.
Unsupervised Generative 3D Shape Learning from Natural Images
Venue: arXiv cs.CV 2019
Year: 2019

In this paper we present, to the best of our knowledge, the first method to learn a generative model of 3D shapes from natural images in a fully unsupervised way. For example, we do not use any ground truth 3D or 2D annotations, stereo video, and ego-motion during the training. Our approach follows the general strategy of Generative Adversarial Networks, where an image generator network learns to create image samples that are realistic enough to fool a discriminator network into believing that they are natural images. In contrast, in our approach the image generation is split into 2 stages. In the first stage a generator network outputs 3D objects. In the second, a differentiable renderer produces an image of the 3D objects from random viewpoints. The key observation is that a realistic 3D object should yield a realistic rendering from any plausible viewpoint. Thus, by randomizing the choice of the viewpoint our proposed training forces the generator network to learn an interpretable 3D representation disentangled from the viewpoint. In this work, a 3D representation consists of a triangle mesh and a texture map that is used to color the triangle surface by using the UV-mapping technique. We provide analysis of our learning approach, expose its ambiguities and show how to overcome them. Experimentally, we demonstrate that our method can learn realistic 3D shapes of faces by using only the natural images of the FFHQ dataset.
Identifying Supporting Facts for Multi-hop Question Answering with Document Graph Networks
Venue: arXiv cs.AI 2019
Year: 2019

Recent advances in reading comprehension have resulted in models that surpass human performance when the answer is contained in a single, continuous passage of text. However, complex Question Answering (QA) typically requires multi-hop reasoning - i.e. the integration of supporting facts from different sources, to infer the correct answer. This paper proposes Document Graph Network (DGN), a message passing architecture for the identification of supporting facts over a graph-structured representation of text. The evaluation on HotpotQA shows that DGN obtains competitive results when compared to a reading comprehension baseline operating on raw text, confirming the relevance of structured representations for supporting multi-hop reasoning.
Distance-Based Approaches to Repair Semantics in Ontology-based Data Access
Venue: arXiv cs.AI 2019
Year: 2019

In the presence of inconsistencies, repair techniques thrive to restore consistency by reasoning with several repairs. However, since the number of repairs can be large, standard inconsistent tolerant semantics usually yield few answers. In this paper, we use the notion of syntactic distance between repairs following the intuition that it can allow us to cluster some repairs "close" to each other. In this way, we propose a generic framework to answer queries in a more personalise fashion.
Teaching Hardware Reverse Engineering: Educational Guidelines and Practical Insights
Venue: arXiv cs.CR 2019
Year: 2019

Since underlying hardware components form the basis of trust in virtually any computing system, security failures in hardware pose a devastating threat to our daily lives. Hardware reverse engineering is commonly employed by security engineers in order to identify security vulnerabilities, to detect IP violations, or to conduct very-large-scale integration (VLSI) failure analysis. Even though industry and the scientific community demand experts with expertise in hardware reverse engineering, there is a lack of educational offerings, and existing training is almost entirely unstructured and on the job. To the best of our knowledge, we have developed the first course to systematically teach students hardware reverse engineering based on insights from the fields of educational research, cognitive science, and hardware security. The contribution of our work is threefold: (1) we propose underlying educational guidelines for practice-oriented courses which teach hardware reverse engineering; (2) we develop such a lab course with a special focus on gate-level netlist reverse engineering and provide the required tools to support it; (3) we conduct an educational evaluation of our pilot course. Based on our results, we provide valuable insights on the structure and content necessary to design and teach future courses on hardware reverse engineering.
Towards Cognitive Obfuscation: Impeding Hardware Reverse Engineering Based on Psychological Insights
Venue: arXiv cs.CR 2019
Year: 2019

In contrast to software reverse engineering, there are hardly any tools available that support hardware reversing. Therefore, the reversing process is conducted by human analysts combining several complex semi-automated steps. However, countermeasures against reversing are evaluated solely against mathematical models. Our research goal is the establishment of cognitive obfuscation based on the exploration of underlying psychological processes. We aim to identify problems which are hard to solve for human analysts and derive novel quantification metrics, thus enabling stronger obfuscation techniques.
VOnDA: A Framework for Ontology-Based Dialogue Management
Venue: arXiv cs.HC 2019
Year: 2019

We present VOnDA, a framework to implement the dialogue management functionality in dialogue systems. Although domain-independent, VOnDA is tailored towards dialogue systems with a focus on social communication, which implies the need of long-term memory and high user adaptivity. For these systems, which are used in health environments or elderly care, margin of error is very low and control over the dialogue process is of topmost importance. The same holds for commercial applications, where customer trust is at risk. VOnDA's specification and memory layer relies upon (extended) RDF/OWL, which provides a universal and uniform representation, and facilitates interoperability with external data sources, e.g., from physical sensors.
Autonomous Bimanual Functional Regrasping of Novel Object Class Instances
Venue: arXiv cs.RO 2019
Year: 2019

In human-made scenarios, robots need to be able to fully operate objects in their surroundings, i.e., objects are required to be functionally grasped rather than only picked. This imposes very strict constraints on the object pose such that a direct grasp can be performed. Inspired by the anthropomorphic nature of humanoid robots, we propose an approach that first grasps an object with one hand, obtaining full control over its pose, and performs the functional grasp with the second hand subsequently. Thus, we develop a fully autonomous pipeline for dual-arm functional regrasping of novel familiar objects, i.e., objects never seen before that belong to a known object category, e.g., spray bottles. This process involves semantic segmentation, object pose estimation, non-rigid mesh registration, grasp sampling, handover pose generation and in-hand pose refinement. The latter is used to compensate for the unpredictable object movement during the first grasp. The approach is applied to a human-like upper body. To the best knowledge of the authors, this is the first system that exhibits autonomous bimanual functional regrasping capabilities. We demonstrate that our system yields reliable success rates and can be applied on-line to real-world tasks using only one off-the-shelf RGB-D sensor.
A Survey of Methods to Leverage Monolingual Data in Low-resource Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Neural machine translation has become the state-of-the-art for language pairs with large parallel corpora. However, the quality of machine translation for low-resource languages leaves much to be desired. There are several approaches to mitigate this problem, such as transfer learning, semi-supervised and unsupervised learning techniques. In this paper, we review the existing methods, where the main idea is to exploit the power of monolingual data, which, compared to parallel, is usually easier to obtain and significantly greater in amount.
Type-aware Convolutional Neural Networks for Slot Filling
Venue: arXiv cs.CL 2019
Year: 2019

The slot filling task aims at extracting answers for queries about entities from text, such as "Who founded Apple". In this paper, we focus on the relation classification component of a slot filling system. We propose type-aware convolutional neural networks to benefit from the mutual dependencies between entity and relation classification. In particular, we explore different ways of integrating the named entity types of the relation arguments into a neural network for relation classification, including a joint training and a structured prediction approach. To the best of our knowledge, this is the first study on type-aware neural networks for slot filling. The type-aware models lead to the best results of our slot filling pipeline. Joint training performs comparable to structured prediction. To understand the impact of the different components of the slot filling pipeline, we perform a recall analysis, a manual error analysis and several ablation studies. Such analyses are of particular importance to other slot filling researchers since the official slot filling evaluations only assess pipeline outputs. The analyses show that especially coreference resolution and our convolutional neural networks have a large positive impact on the final performance of the slot filling pipeline. The presented models, the source code of our system as well as our coreference resource is publicy available.
Extraction of Efficient Programs in $IΣ_1$-arithmetic
Venue: arXiv cs.LO 2019
Year: 2019

Clausal Language (CL) is a declarative programming and verifying system used in our teaching of computer science. CL is an implementation of, what we call, $\mathit{PR}{+}IΣ_1$ paradigm (primitive recursive functions with $IΣ_1$-arithmetic). This paper introduces an extension of $IΣ_1$-proofs called extraction proofs where one can extract from the proofs of $Π_2$-specifications primitive recursive programs as efficient as the hand-coded ones. This is achieved by having the programming constructs correspond exactly to the proof rules with the computational content.
TagSLAM: Robust SLAM with Fiducial Markers
Venue: arXiv cs.RO 2019
Year: 2019

TagSLAM provides a convenient, flexible, and robust way of performing Simultaneous Localization and Mapping (SLAM) with AprilTag fiducial markers. By leveraging a few simple abstractions (bodies, tags, cameras), TagSLAM provides a front end to the GTSAM factor graph optimizer that makes it possible to rapidly design a range of experiments that are based on tags: full SLAM, extrinsic camera calibration with non-overlapping views, visual localization for ground truth, loop closure for odometry, pose estimation etc. We discuss in detail how TagSLAM initializes the factor graph in a robust way, and present loop closure as an application example. TagSLAM is a ROS based open source package and can be found at https://berndpfrommer.github.io/tagslam_web.
RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking
Venue: arXiv cs.CV 2019
Year: 2019

Accurate eye segmentation can improve eye-gaze estimation and support interactive computing based on visual attention; however, existing eye segmentation methods suffer from issues such as person-dependent accuracy, lack of robustness, and an inability to be run in real-time. Here, we present the RITnet model, which is a deep neural network that combines U-Net and DenseNet. RITnet is under 1 MB and achieves 95.3\% accuracy on the 2019 OpenEDS Semantic Segmentation challenge. Using a GeForce GTX 1080 Ti, RITnet tracks at $>$ 300Hz, enabling real-time gaze tracking applications. Pre-trained models and source code are available https://bitbucket.org/eye-ush/ritnet/.
Mμl: The Power of Dynamic Multi-Methods
Venue: arXiv cs.PL 2019
Year: 2019

Multi-methods are a straightforward extension of traditional (single) dynamic dispatch, which is the core of most object oriented languages. With multi-methods, a method call will select an appropriate implementation based on the values of multiple arguments, and not just the first/receiver. Language support for both single and multiple dispatch is typically designed to be used in conjunction with other object oriented features, in particular classes and inheritance. But are these extra features really necessary? Mμl is a dynamic language designed to be as simple as possible but still supporting flexible abstraction and polymorphism. Mμl provides only two forms of abstraction: (object) identities and (multi) methods. In Mμl method calls are dispatched based on the identity of arguments, as well as what other methods are defined on them. In order to keep Mμls design simple, when multiple method definitions are applicable, the most recently defined one is chosen, not the most specific (as is conventional with dynamic dispatch). In this paper we show how by defining methods at runtime, we obtain much of the power of classes and meta object protocols, in particular the ability to dynamically modify the state and behaviour of 'classes' of objects.
Emergence of Writing Systems Through Multi-Agent Cooperation
Venue: arXiv cs.MA 2019
Year: 2019

Learning to communicate is considered an essential task to develop a general AI. While recent literature in language evolution has studied emergent language through discrete or continuous message symbols, there has been little work in the emergence of writing systems in artificial agents. In this paper, we present a referential game setup with two agents, where the mode of communication is a written language system that emerges during the play. We show that the agents can learn to coordinate successfully using this mode of communication. Further, we study how the game rules affect the writing system taxonomy by proposing a consistency metric.
Cognitive Agent Based Simulation Model For Improving Disaster Response Procedures
Venue: arXiv cs.MA 2019
Year: 2019

In the event of a disaster, saving human lives is of utmost importance. For developing proper evacuation procedures and guidance systems, behavioural data on how people respond during panic and stress is crucial. In the absence of real human data on building evacuation, there is a need for a crowd simulator to model egress and decision-making under uncertainty. In this paper, we propose an agent-based simulation tool, which is grounded in human cognition and decision-making, for evaluating and improving the effectiveness of building evacuation procedures and guidance systems during a disaster. Specifically, we propose a predictive agent-wayfinding framework based on information theory that is applied at intersections with variable route choices where it fuses N dynamic information sources. The proposed framework can be used to visualize trajectories and prediction results (i.e., total evacuation time, number of people evacuated) for different combinations of reinforcing or contradicting information sources (i.e., signage, crowd flow, familiarity, and spatial layout). This tool can enable designers to recreate various disaster scenarios and generate simulation data for improving the evacuation procedures and existing guidance systems.
Abstractive Dialog Summarization with Semantic Scaffolds
Venue: arXiv cs.CL 2019
Year: 2019

The demand for abstractive dialog summary is growing in real-world applications. For example, customer service center or hospitals would like to summarize customer service interaction and doctor-patient interaction. However, few researchers explored abstractive summarization on dialogs due to the lack of suitable datasets. We propose an abstractive dialog summarization dataset based on MultiWOZ. If we directly apply previous state-of-the-art document summarization methods on dialogs, there are two significant drawbacks: the informative entities such as restaurant names are difficult to preserve, and the contents from different dialog domains are sometimes mismatched. To address these two drawbacks, we propose Scaffold Pointer Network (SPNet)to utilize the existing annotation on speaker role, semantic slot and dialog domain. SPNet incorporates these semantic scaffolds for dialog summarization. Since ROUGE cannot capture the two drawbacks mentioned, we also propose a new evaluation metric that considers critical informative entities in the text. On MultiWOZ, our proposed SPNet outperforms state-of-the-art abstractive summarization methods on all the automatic and human evaluation metrics.
A Unique Extension of Rich Words
Venue: arXiv cs.DM 2019
Year: 2019

A word $w$ is called rich if it contains $| w|+1$ palindromic factors, including the empty word. We say that a rich word $w$ can be extended in at least two ways if there are two distinct letters $x,y$ such that $wx,wy$ are rich. Let $R$ denote the set of all rich words. Given $w\in R$, let $K(w)$ denote the set of all words such that if $u\in K(w)$ then $wu\in R$ and $wu$ can be extended in at least two ways. Let $ω(w)=\min\{| u| \mid u\in K(w)\}$ and let $ϕ(n)=\max\{ω(w)\mid w\in R\mbox{ and }| w|=n\}$, where $n>0$. Vesti (2014) showed that $ϕ(n)\leq 2n$. In other words, it says that for each $w\in R$ there is a word $u$ with $| u|\leq 2| w|$ such that $wu\in R$ and $wu$ can be extended in at least two ways. We prove that $ϕ(n)\leq n$. In addition we prove that for each real constant $c>0$ and each integer $m>0$ there is $n>m$ such that $ϕ(n)\geq (\frac{2}{9}-c)n$. The results hold for each finite alphabet having at least two letters.
The merits of Universal Language Model Fine-tuning for Small Datasets -- a case with Dutch book reviews
Venue: arXiv cs.IR 2019
Year: 2019

We evaluated the effectiveness of using language models, that were pre-trained in one domain, as the basis for a classification model in another domain: Dutch book reviews. Pre-trained language models have opened up new possibilities for classification tasks with limited labelled data, because representation can be learned in an unsupervised fashion. In our experiments we have studied the effects of training set size (100-1600 items) on the prediction accuracy of a ULMFiT classifier, based on a language models that we pre-trained on the Dutch Wikipedia. We also compared ULMFiT to Support Vector Machines, which is traditionally considered suitable for small collections. We found that ULMFiT outperforms SVM for all training set sizes and that satisfactory results (~90%) can be achieved using training sets that can be manually annotated within a few hours. We deliver both our new benchmark collection of Dutch book reviews for sentiment classification as well as the pre-trained Dutch language model to the community.
Generalized DC loop current attack against the KLJN secure key exchange scheme
Venue: arXiv cs.ET 2019
Year: 2019

A new attack against the Kirchhoff Law Johnson Noise (KLJN) secure key distribution system is studied with unknown parasitic DC voltage sources at both Alices and Bobs ends. This paper is the generalization of our earlier investigation with a single end parasitic source. Under the assumption that Eve does not know the values of the parasitic sources, a new attack, utilizing the current generated by the parasitic dc voltage sources, is introduced. The attack is mathematically analyzed and demonstrated by computer simulations. Simple defense methods against the attack are shown. The earlier defense method based solely on the comparison of current/voltage data at Alice's and Bob's terminals is useless here since the wire currents and voltages are equal at both ends. However, the more expensive version of the earlier defense method, which is based on in situ system simulation and comparison with measurements, works efficiently.