OpenRaiser/Intern-Atlas huggingface dataset:
Higher-order Weighted Graph Convolutional Networks
Venue: arXiv cs.LG 2019
Year: 2019

Graph Convolution Network (GCN) has been recognized as one of the most effective graph models for semi-supervised learning, but it extracts merely the first-order or few-order neighborhood information through information propagation, which suffers performance drop-off for deeper structure. Existing approaches that deal with the higher-order neighbors tend to take advantage of adjacency matrix power. In this paper, we assume a seemly trivial condition that the higher-order neighborhood information may be similar to that of the first-order neighbors. Accordingly, we present an unsupervised approach to describe such similarities and learn the weight matrices of higher-order neighbors automatically through Lasso that minimizes the feature loss between the first-order and higher-order neighbors, based on which we formulate the new convolutional filter for GCN to learn the better node representations. Our model, called higher-order weighted GCN(HWGCN), has achieved the state-of-the-art results on a number of node classification tasks over Cora, Citeseer and Pubmed datasets.
Verifying OpenJDK's LinkedList using KeY
Venue: arXiv cs.LO 2019
Year: 2019

As a particular case study of the formal verification of state-of-the-art, real software, we discuss the specification and verification of a corrected version of the implementation of a linked list as provided by the Java Collection framework. Keywords: Java, standard library, deductive verification, KeY, Java Modeling Language, case study, bug
Information carefull worstcase DecreaseKey heaps with simple nonMeld variant
Venue: arXiv cs.DS 2019
Year: 2019

We analyze priority queues including DecreaseKey method in its interface. The paper is inspired by Strict Fibonacci Heaps [2], where G. S. Brodal, G. Lagogiannis, and R. E. Tarjan implemented the heap with DecreaseKey and Meld interface in assymptotically optimal worst case times (based on key comparisons). At the end of the paper there are mentioned possible variants of other structural properties an violations than they have used in the analysis. In the main variant a lot of information is wasted during violation reduction steps. Our goal is to concentrate on other variants and to invent natural strategy not losing that much in the information value. In other words we try to choose among them one which corresponds to superexpensive comparision principle as much as possible. The principle was described in [5] of myself, but after publication I have found these ideas in [4] of H. Kaplan, R. E. Tarjan, and U. Zwick.
RAD: On-line Anomaly Detection for Highly Unreliable Data
Venue: arXiv cs.LG 2019
Year: 2019

Classification algorithms have been widely adopted to detect anomalies for various systems, e.g., IoT, cloud and face recognition, under the common assumption that the data source is clean, i.e., features and labels are correctly set. However, data collected from the wild can be unreliable due to careless annotations or malicious data transformation for incorrect anomaly detection. In this paper, we present a two-layer on-line learning framework for robust anomaly detection (RAD) in the presence of unreliable anomaly labels, where the first layer is to filter out the suspicious data, and the second layer detects the anomaly patterns from the remaining data. To adapt to the on-line nature of anomaly detection, we extend RAD with additional features of repetitively cleaning, conflicting opinions of classifiers, and oracle knowledge. We on-line learn from the incoming data streams and continuously cleanse the data, so as to adapt to the increasing learning capacity from the larger accumulated data set. Moreover, we explore the concept of oracle learning that provides additional information of true labels for difficult data points. We specifically focus on three use cases, (i) detecting 10 classes of IoT attacks, (ii) predicting 4 classes of task failures of big data jobs, (iii) recognising 20 celebrities faces. Our evaluation results show that RAD can robustly improve the accuracy of anomaly detection, to reach up to 98% for IoT device attacks (i.e., +11%), up to 84% for cloud task failures (i.e., +20%) under 40% noise, and up to 74% for face recognition (i.e., +28%) under 30% noisy labels. The proposed RAD is general and can be applied to different anomaly detection algorithms.
A Biologically Plausible Benchmark for Contextual Bandit Algorithms in Precision Oncology Using in vitro Data
Venue: arXiv cs.LG 2019
Year: 2019

Precision oncology, the genetic sequencing of tumors to identify druggable targets, has emerged as the standard of care in the treatment of many cancers. Nonetheless, due to the pace of therapy development and variability in patient information, designing effective protocols for individual treatment assignment in a sample-efficient way remains a major challenge. One promising approach to this problem is to frame precision oncology treatment as a contextual bandit problem and to apply sequential decision-making algorithms designed to minimize regret in this setting. However, a clear prerequisite for considering this methodology in high-stakes clinical decisions is careful benchmarking to understand realistic costs and benefits. Here, we propose a benchmark dataset to evaluate contextual bandit algorithms based on real in vitro drug response of approximately 900 cancer cell lines. Specifically, we curated a dataset of complete treatment responses for a subset of 7 treatments from prior in vitro studies. This allows us to compute the regret of proposed decision policies using biologically plausible counterfactuals. We ran a suite of Bayesian bandit algorithms on our benchmark, and found that the methods accumulate less regret over a sequence of treatment assignment tasks than a rule-based baseline derived from current clinical practice. This effect was more pronounced when genomic information was included as context. We expect this work to be a starting point for evaluation of both the unique structural requirements and ethical implications for real-world testing of bandit based clinical decision support.
Structural Pruning in Deep Neural Networks: A Small-World Approach
Venue: arXiv cs.LG 2019
Year: 2019

Deep Neural Networks (DNNs) are usually over-parameterized, causing excessive memory and interconnection cost on the hardware platform. Existing pruning approaches remove secondary parameters at the end of training to reduce the model size; but without exploiting the intrinsic network property, they still require the full interconnection to prepare the network. Inspired by the observation that brain networks follow the Small-World model, we propose a novel structural pruning scheme, which includes (1) hierarchically trimming the network into a Small-World model before training, (2) training the network for a given dataset, and (3) optimizing the network for accuracy. The new scheme effectively reduces both the model size and the interconnection needed before training, achieving a locally clustered and globally sparse model. We demonstrate our approach on LeNet-5 for MNIST and VGG-16 for CIFAR-10, decreasing the number of parameters to 2.3% and 9.02% of the baseline model, respectively.
TENER: Adapting Transformer Encoder for Named Entity Recognition
Venue: arXiv cs.CL 2019
Year: 2019

The Bidirectional long short-term memory networks (BiLSTM) have been widely used as an encoder in models solving the named entity recognition (NER) task. Recently, the Transformer is broadly adopted in various Natural Language Processing (NLP) tasks owing to its parallelism and advantageous performance. Nevertheless, the performance of the Transformer in NER is not as good as it is in other NLP tasks. In this paper, we propose TENER, a NER architecture adopting adapted Transformer Encoder to model the character-level features and word-level features. By incorporating the direction and relative distance aware attention and the un-scaled attention, we prove the Transformer-like encoder is just as effective for NER as other NLP tasks.
Existential Types for Relaxed Noninterference
Venue: arXiv cs.PL 2019
Year: 2019

Information-flow security type systems ensure confidentiality by enforcing noninterference: a program cannot leak private data to public channels. However, in practice, programs need to selectively declassify information about private data. Several approaches have provided a notion of relaxed noninterference supporting selective and expressive declassification while retaining a formal security property. The labels-as-functions approach provides relaxed noninterference by means of declassification policies expressed as functions. The labels-as-types approach expresses declassification policies using type abstraction and faceted types, a pair of types representing the secret and public facets of values. The original proposal of labels-as-types is formulated in an object-oriented setting where type abstraction is realized by subtyping. The object-oriented approach however suffers from limitations due to its receiver-centric paradigm. In this work, we consider an alternative approach to labels-as-types, applicable in non-object-oriented languages, which allows us to express advanced declassification policies, such as extrinsic policies, based on a different form of type abstraction: existential types. An existential type exposes abstract types and operations on these; we leverage this abstraction mechanism to express secrets that can be declassified using the provided operations. We formalize the approach in a core functional calculus with existential types, define existential relaxed noninterference, and prove that well-typed programs satisfy this form of type-based relaxed noninterference.
Long-span language modeling for speech recognition
Venue: arXiv cs.CL 2019
Year: 2019

We explore neural language modeling for speech recognition where the context spans multiple sentences. Rather than encode history beyond the current sentence using a cache of words or document-level features, we focus our study on the ability of LSTM and Transformer language models to implicitly learn to carry over context across sentence boundaries. We introduce a new architecture that incorporates an attention mechanism into LSTM to combine the benefits of recurrent and attention architectures. We conduct language modeling and speech recognition experiments on the publicly available LibriSpeech corpus. We show that conventional training on a paragraph-level corpus results in significant reductions in perplexity compared to training on a sentence-level corpus. We also describe speech recognition experiments using long-span language models in second-pass re-ranking, and provide insights into the ability of such models to take advantage of context beyond the current sentence.
Achieving Differential Privacy in Vertically Partitioned Multiparty Learning
Venue: arXiv cs.LG 2019
Year: 2019

Preserving differential privacy has been well studied under centralized setting. However, it's very challenging to preserve differential privacy under multiparty setting, especially for the vertically partitioned case. In this work, we propose a new framework for differential privacy preserving multiparty learning in the vertically partitioned setting. Our core idea is based on the functional mechanism that achieves differential privacy of the released model by adding noise to the objective function. We show the server can simply dissect the objective function into single-party and cross-party sub-functions, and allocate computation and perturbation of their polynomial coefficients to local parties. Our method needs only one round of noise addition and secure aggregation. The released model in our framework achieves the same utility as applying the functional mechanism in the centralized setting. Evaluation on real-world and synthetic datasets for linear and logistic regressions shows the effectiveness of our proposed method.
Framing Effects on Privacy Concerns about a Home Telepresence Robot
Venue: arXiv cs.RO 2019
Year: 2019

Privacy-sensitive robotics is an emerging area of HRI research. Judgments about privacy would seem to be context-dependent, but none of the promising work on contextual "frames" has focused on privacy concerns. This work studies the impact of contextual "frames" on local users' privacy judgments in a home telepresence setting. Our methodology consists of using an online questionnaire to collect responses to animated videos of a telepresence robot after framing people with an introductory paragraph. The results of four studies indicate a large effect of manipulating the robot operator's identity between a stranger and a close confidante. It also appears that this framing effect persists throughout several videos. These findings serve to caution HRI researchers that a change in frame could cause their results to fail to replicate or generalize. We also recommend that robots be designed to encourage or discourage certain frames.
Equalization Loss for Large Vocabulary Instance Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Recent object detection and instance segmentation tasks mainly focus on datasets with a relatively small set of categories, e.g. Pascal VOC with 20 classes and COCO with 80 classes. The new large vocabulary dataset LVIS brings new challenges to conventional methods. In this work, we propose an equalization loss to solve the long tail of rare categories problem. Combined with exploiting the data from detection datasets to alleviate the effect of missing-annotation problems during the training, our method achieves 5.1\% overall AP gain and 11.4\% AP gain of rare categories on LVIS benchmark without any bells and whistles compared to Mask R-CNN baseline. Finally we achieve 28.9 mask AP on the test-set of the LVIS and rank 1st place in LVIS Challenge 2019.
Mirheo: High-Performance Mesoscale Simulations for Microfluidics
Venue: arXiv cs.CE 2019
Year: 2019

The transport and manipulation of particles and cells in microfluidic devices has become a core methodology in domains ranging from molecular biology to manufacturing and drug design. The rational design and operation of such devices can benefit from simulations that resolve flow-structure interactions at sub-micron resolution. We present a computational tool for large scale, efficient and high throughput mesoscale simulations of fluids and deformable objects at complex microscale geometries. The code employs Dissipative Particle Dynamics for the description of the flow coupled with visco-elastic membrane model for red blood cells and can also handle rigid bodies and complex geometries. The software (MiRheo) is deployed on hybrid GPU/CPU architectures exhibiting unprecedented time-to-solution performance and excellent weak and strong scaling for a number of benchmark problems. MiRheo exploits the capabilities of GPU clusters, leading to speedup of up to 10 in terms of time to solution as compared to state-of-the-art software packages and reaches 90 - 99 percent weak scaling efficiency on 512 nodes of the Piz Daint supercomputer. The software MiRheo, relies on a Python interface to facilitate the solution of complex problems and it is open source. We believe that MiRheo constitutes a potent computational tool that can greatly assist studies of microfluidics.
Smart Contract Interactions in Coq
Venue: arXiv cs.LO 2019
Year: 2019

We present a model/executable specification of smart contract execution in Coq. Our formalization allows for inter-contract communication and generalizes existing work by allowing modelling of both depth-first execution blockchains (like Ethereum) and breadth-first execution blockchains (like Tezos). We represent smart contracts programs in Coq's functional language Gallina, enabling easier reasoning about functional correctness of concrete contracts than other approaches. In particular we develop a Congress contract in this style. This contract -- a simplified version of the infamous DAO -- is interesting because of its very dynamic communication pattern with other contracts. We give a high-level partial specification of the Congress's behavior, related to reentrancy, and prove that the Congress satisfies it for all possible smart contract execution orders.
Network Creation Games with Local Information and Edge Swaps
Venue: arXiv cs.AI 2019
Year: 2019

In the swap game (SG) selfish players, each of which is associated to a vertex, form a graph by edge swaps, i.e., a player changes its strategy by simultaneously removing an adjacent edge and forming a new edge (Alon et al., 2013). The cost of a player considers the average distance to all other players or the maximum distance to other players. Any SG by $n$ players starting from a tree converges to an equilibrium with a constant Price of Anarchy (PoA) within $O(n^3)$ edge swaps (Lenzner, 2011). We focus on SGs where each player knows the subgraph induced by players within distance $k$. Therefore, each player cannot compute its cost nor a best response. We first consider pessimistic players who consider the worst-case global graph. We show that any SG starting from a tree (i) always converges to an equilibrium within $O(n^3)$ edge swaps irrespective of the value of $k$, (ii) the PoA is $Θ(n)$ for $k=1,2,3$, and (iii) the PoA is constant for $k \geq 4$. We then introduce weakly pessimistic players and optimistic players and show that these less pessimistic players achieve constant PoA for $k \leq 3$ at the cost of best response cycles.
Gathering on Rings for Myopic Asynchronous Robots with Lights
Venue: arXiv cs.DC 2019
Year: 2019

We investigate gathering algorithms for asynchronous autonomous mobile robots moving in uniform ring-shaped networks. Different from most work using the Look-Compute-Move (LCM) model, we assume that robots have limited visibility and lights. That is, robots can observe nodes only within a certain fixed distance, and emit a color from a set of constant number of colors. We consider gathering algorithms depending on two parameters related to the initial configuration: $M_{init}$, which denotes the number of nodes between two border nodes, and $O_{init}$, which denotes the number of nodes hosting robots between two border nodes. In both cases, a border node is a node hosting one or more robots that cannot see other robots on at least one side. Our main contribution is to prove that, if $M_{init}$ or $O_{init}$ is odd, gathering is always feasible with three or four colors. The proposed algorithms do not require additional assumptions, such as knowledge of the number of robots, multiplicity detection capabilities, or the assumption of towerless initial configurations. These results demonstrate the power of lights to achieve gathering of robots with limited visibility.
OntoScene, A Logic-based Scene Interpreter: Implementation and Application in the Rock Art Domain
Venue: arXiv cs.AI 2019
Year: 2019

We present OntoScene, a framework aimed at understanding the semantics of visual scenes starting from the semantics of their elements and the spatial relations holding between them. OntoScene exploits ontologies for representing knowledge and Prolog for specifying the interpretation rules that domain experts may adopt, and for implementing the SceneInterpreter engine. Ontologies allow the designer to formalize the domain in a reusable way, and make the system modular and interoperable with existing multiagent systems, while Prolog provides a solid basis to define complex rules of interpretation in a way that can be affordable even for people with no background in Computational Logics. The domain selected for experimenting OntoScene is that of prehistoric rock art, which provides us with a fascinating and challenging testbed. Under consideration in Theory and Practice of Logic Programming (TPLP)
Spectrally accurate Ewald summation for the Yukawa potential in two dimensions
Venue: arXiv cs.CE 2019
Year: 2019

An Ewald decomposition of the two-dimensional Yukawa potential and its derivative is presented for both the periodic and the free-space case. These modified Bessel functions of the second kind of zeroth and first degrees are used e.g. when solving the modified Helmholtz equation using a boundary integral method. The spectral Ewald method is used to compute arising sums at O(N log N) cost for N source and target points. To facilitate parameter selection, truncation-error estimates are developed for both the real-space sum and the Fourier-space sum, and are shown to estimate the errors well.
A Survey on Why-Type Question Answering Systems
Venue: arXiv cs.CL 2019
Year: 2019

Search engines such as Google, Yahoo and Baidu yield information in the form of a relevant set of web pages according to the need of the user. Question Answering Systems reduce the time taken to get an answer, to a query asked in natural language by providing the one most relevant answer. To the best of our knowledge, major research in Why-type questions began in early 2000's and our work on Why-type questions can help explore newer avenues for fact-finding and analysis. The paper presents a survey on Why-type Question Answering System, details the architecture, the processes involved in the system and suggests further areas of research.
EntropyDB: A Probabilistic Approach to Approximate Query Processing
Venue: arXiv cs.DB 2019
Year: 2019

We present EntropyDB, an interactive data exploration system that uses a probabilistic approach to generate a small, query-able summary of a dataset. Departing from traditional summarization techniques, we use the Principle of Maximum Entropy to generate a probabilistic representation of the data that can be used to give approximate query answers. We develop the theoretical framework and formulation of our probabilistic representation and show how to use it to answer queries. We then present solving techniques, give two critical optimizations to improve preprocessing time and query execution time, and explore methods to reduce query error. Lastly, we experimentally evaluate our work using a 5 GB dataset of flights within the United States and a 210 GB dataset from an astronomy particle simulation. While our current work only supports linear queries, we show that our technique can successfully answer queries faster than sampling while introducing, on average, no more error than sampling and can better distinguish between rare and nonexistent values. We also discuss extensions that can allow for data updates and linear queries over joins.
Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research
Venue: arXiv cs.CV 2019
Year: 2019

We present Kaolin, a PyTorch library aiming to accelerate 3D deep learning research. Kaolin provides efficient implementations of differentiable 3D modules for use in deep learning systems. With functionality to load and preprocess several popular 3D datasets, and native functions to manipulate meshes, pointclouds, signed distance functions, and voxel grids, Kaolin mitigates the need to write wasteful boilerplate code. Kaolin packages together several differentiable graphics modules including rendering, lighting, shading, and view warping. Kaolin also supports an array of loss functions and evaluation metrics for seamless evaluation and provides visualization functionality to render the 3D results. Importantly, we curate a comprehensive model zoo comprising many state-of-the-art 3D deep learning architectures, to serve as a starting point for future research endeavours. Kaolin is available as open-source software at https://github.com/NVIDIAGameWorks/kaolin/.
Popularity of patterns over $d$-equivalence classes of words and permutations
Venue: arXiv cs.DM 2019
Year: 2019

Two same length words are $d$-equivalent if they have same descent set and same underlying alphabet. In particular, two same length permutations are $d$-equivalent if they have same descent set. The popularity of a pattern in a set of words is the overall number of copies of the pattern within the words of the set. We show the far-from-trivial fact that two patterns are $d$-equivalent if and only if they are equipopular over any $d$-equivalence class, and this equipopularity does not follow obviously from a trivial equidistribution.