OpenRaiser/Intern-Atlas huggingface dataset:
Stochastic Weight Matrix-based Regularization Methods for Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

The aim of this paper is to introduce two widely applicable regularization methods based on the direct modification of weight matrices. The first method, Weight Reinitialization, utilizes a simplified Bayesian assumption with partially resetting a sparse subset of the parameters. The second one, Weight Shuffling, introduces an entropy- and weight distribution-invariant non-white noise to the parameters. The latter can also be interpreted as an ensemble approach. The proposed methods are evaluated on benchmark datasets, such as MNIST, CIFAR-10 or the JSB Chorales database, and also on time series modeling tasks. We report gains both regarding performance and entropy of the analyzed networks. We also made our code available as a GitHub repository (https://github.com/rpatrik96/lod-wmm-2019).
Synergistic Team Composition: A Computational Approach to Foster Diversity in Teams
Venue: arXiv cs.AI 2019
Year: 2019

Co-operative learning in heterogeneous teams refers to learning methods in which teams are organised both to accomplish academic tasks and for individuals to gain knowledge. Competencies, personality and the gender of team members are key factors that influence team performance. Here, we introduce a team composition problem, the so-called synergistic team composition problem (STCP), which incorporates such key factors when arranging teams. Thus, the goal of the STCP is to partition a set of individuals into a set of synergistic teams: teams that are diverse in personality and gender and whose members cover all required competencies to complete a task. Furthermore, the STCP requires that all teams are balanced in that they are expected to exhibit similar performances when completing the task. We propose two efficient algorithms to solve the STCP. Our first algorithm is based on a linear programming formulation and is appropriate to solve small instances of the problem. Our second algorithm is an anytime heuristic that is effective for large instances of the STCP. Finally, we thoroughly study the computational properties of both algorithms in an educational context when grouping students in a classroom into teams using actual-world data.
Complexity of Liveness in Parameterized Systems
Venue: arXiv cs.FL 2019
Year: 2019

We investigate the fine-grained complexity of liveness verification for leader contributor systems. These consist of a designated leader thread and an arbitrary number of identical contributor threads communicating via a shared memory. The liveness verification problem asks whether there is an infinite computation of the system in which the leader reaches a final state infinitely often. Like its reachability counterpart, the problem is known to be NP-complete. Our results show that, even from a fine-grained point of view, the complexities differ only by a polynomial factor. Liveness verification decomposes into reachability and cycle detection. We present a fixed point iteration solving the latter in polynomial time. For reachability, we reconsider the two standard parameterizations. When parameterized by the number of states of the leader L and the size of the data domain D, we show an (L + D)^O(L + D)-time algorithm. It improves on a previous algorithm, thereby settling an open problem. When parameterized by the number of states of the contributor C, we reuse an O*(2^C)-time algorithm. We show how to connect both algorithms with the cycle detection to obtain algorithms for liveness verification. The running times of the composed algorithms match those of reachability, proving that the fine-grained lower bounds for liveness verification are met.
Query Optimization Properties of Modified VBS
Venue: arXiv cs.AI 2019
Year: 2019

Valuation-Based~System can represent knowledge in different domains including probability theory, Dempster-Shafer theory and possibility theory. More recent studies show that the framework of VBS is also appropriate for representing and solving Bayesian decision problems and optimization problems. In this paper after introducing the valuation based system (VBS) framework, we present Markov-like properties of VBS and a method for resolving queries to VBS.
Artificial Intelligence BlockCloud (AIBC) Technical Whitepaper
Venue: arXiv cs.AI 2019
Year: 2019

The AIBC is an Artificial Intelligence and blockchain technology based large-scale decentralized ecosystem that allows system-wide low-cost sharing of computing and storage resources. The AIBC consists of four layers: a fundamental layer, a resource layer, an application layer, and an ecosystem layer. The AIBC implements a two-consensus scheme to enforce upper-layer economic policies and achieve fundamental layer performance and robustness: the DPoEV incentive consensus on the application and resource layers, and the DABFT distributed consensus on the fundamental layer. The DABFT uses deep learning techniques to predict and select the most suitable BFT algorithm in order to achieve the best balance of performance, robustness, and security. The DPoEV uses the knowledge map algorithm to accurately assess the economic value of digital assets.
Towards Explainable Artificial Intelligence
Venue: arXiv cs.AI 2019
Year: 2019

In recent years, machine learning (ML) has become a key enabling technology for the sciences and industry. Especially through improvements in methodology, the availability of large databases and increased computational power, today's ML algorithms are able to achieve excellent performance (at times even exceeding the human level) on an increasing number of complex tasks. Deep learning models are at the forefront of this development. However, due to their nested non-linear structure, these powerful models have been generally considered "black boxes", not providing any information about what exactly makes them arrive at their predictions. Since in many applications, e.g., in the medical domain, such lack of transparency may be not acceptable, the development of methods for visualizing, explaining and interpreting deep learning models has recently attracted increasing attention. This introductory paper presents recent developments and applications in this field and makes a plea for a wider use of explainable learning algorithms in practice.
Programmable Event Detection for In-Band Network Telemetry
Venue: arXiv cs.NI 2019
Year: 2019

In-Band Network Telemetry (INT) is a novel framework for collecting telemetry items and switch internal state information from the data plane at line rate. With the support of programmable data planes and programming language P4, switches parse telemetry instruction headers and determine which telemetry items to attach using custom metadata. At the network edge, telemetry information is removed and the original packets are forwarded while telemetry reports are sent to a distributed stream processor for further processing by a network monitoring platform. In order to avoid excessive load on the stream processor, telemetry items should not be sent for each individual packet but rather when certain events are triggered. In this paper, we develop a programmable INT event detection mechanism in P4 that allows customization of which events to report to the monitoring system, on a per-flow basis, from the control plane. At the stream processor, we implement a fast INT report collector using the kernel bypass technique AF_XDP, which parses telemetry reports and streams them to a distributed Kafka cluster, which can apply machine learning, visualization and further monitoring tasks. In our evaluation, we use real-world traces from different data center workloads and show that our approach is highly scalable and significantly reduces the network overhead and stream processor load due to effective event pre-filtering inside the switch data plane. While the INT report collector can process around 3 Mpps telemetry reports per core, using event pre-filtering increases the capacity by 10-15x.
Action Selection for MDPs: Anytime AO* vs. UCT
Venue: arXiv cs.AI 2019
Year: 2019

In the presence of non-admissible heuristics, A* and other best-first algorithms can be converted into anytime optimal algorithms over OR graphs, by simply continuing the search after the first solution is found. The same trick, however, does not work for best-first algorithms over AND/OR graphs, that must be able to expand leaf nodes of the explicit graph that are not necessarily part of the best partial solution. Anytime optimal variants of AO* must thus address an exploration-exploitation tradeoff: they cannot just "exploit", they must keep exploring as well. In this work, we develop one such variant of AO* and apply it to finite-horizon MDPs. This Anytime AO* algorithm eventually delivers an optimal policy while using non-admissible random heuristics that can be sampled, as when the heuristic is the cost of a base policy that can be sampled with rollouts. We then test Anytime AO* for action selection over large infinite-horizon MDPs that cannot be solved with existing off-line heuristic search and dynamic programming algorithms, and compare it with UCT.
Explaining and Interpreting LSTMs
Venue: arXiv cs.LG 2019
Year: 2019

While neural networks have acted as a strong unifying force in the design of modern AI systems, the neural network architectures themselves remain highly heterogeneous due to the variety of tasks to be solved. In this chapter, we explore how to adapt the Layer-wise Relevance Propagation (LRP) technique used for explaining the predictions of feed-forward networks to the LSTM architecture used for sequential data modeling and forecasting. The special accumulators and gated interactions present in the LSTM require both a new propagation scheme and an extension of the underlying theoretical framework to deliver faithful explanations.
WiderPerson: A Diverse Dataset for Dense Pedestrian Detection in the Wild
Venue: arXiv cs.CV 2019
Year: 2019

Pedestrian detection has achieved significant progress with the availability of existing benchmark datasets. However, there is a gap in the diversity and density between real world requirements and current pedestrian detection benchmarks: 1) most of existing datasets are taken from a vehicle driving through the regular traffic scenario, usually leading to insufficient diversity; 2) crowd scenarios with highly occluded pedestrians are still under represented, resulting in low density. To narrow this gap and facilitate future pedestrian detection research, we introduce a large and diverse dataset named WiderPerson for dense pedestrian detection in the wild. This dataset involves five types of annotations in a wide range of scenarios, no longer limited to the traffic scenario. There are a total of $13,382$ images with $399,786$ annotations, i.e., $29.87$ annotations per image, which means this dataset contains dense pedestrians with various kinds of occlusions. Hence, pedestrians in the proposed dataset are extremely challenging due to large variations in the scenario and occlusion, which is suitable to evaluate pedestrian detectors in the wild. We introduce an improved Faster R-CNN and the vanilla RetinaNet to serve as baselines for the new pedestrian detection benchmark. Several experiments are conducted on previous datasets including Caltech-USA and CityPersons to analyze the generalization capabilities of the proposed dataset and we achieve state-of-the-art performances on these previous datasets without bells and whistles. Finally, we analyze common failure cases and find the classification ability of pedestrian detector needs to be improved to reduce false alarm and miss detection rates. The proposed dataset is available at http://www.cbsr.ia.ac.cn/users/sfzhang/WiderPerson
MinWikiSplit: A Sentence Splitting Corpus with Minimal Propositions
Venue: arXiv cs.CL 2019
Year: 2019

We compiled a new sentence splitting corpus that is composed of 203K pairs of aligned complex source and simplified target sentences. Contrary to previously proposed text simplification corpora, which contain only a small number of split examples, we present a dataset where each input sentence is broken down into a set of minimal propositions, i.e. a sequence of sound, self-contained utterances with each of them presenting a minimal semantic unit that cannot be further decomposed into meaningful propositions. This corpus is useful for developing sentence splitting approaches that learn how to transform sentences with a complex linguistic structure into a fine-grained representation of short sentences that present a simple and more regular structure which is easier to process for downstream applications and thus facilitates and improves their performance.
Higher-Dimensional Potential Heuristics for Optimal Classical Planning
Venue: arXiv cs.AI 2019
Year: 2019

Potential heuristics for state-space search are defined as weighted sums over simple state features. Atomic features consider the value of a single state variable in a factored state representation, while binary features consider joint assignments to two state variables. Previous work showed that the set of all admissible and consistent potential heuristics using atomic features can be characterized by a compact set of linear constraints. We generalize this result to binary features and prove a hardness result for features of higher dimension. Furthermore, we prove a tractability result based on the treewidth of a new graphical structure we call the context-dependency graph. Finally, we study the relationship of potential heuristics to transition cost partitioning. Experimental results show that binary potential heuristics are significantly more informative than the previously considered atomic ones.
Adversarial Machine Learning Attack on Modulation Classification
Venue: arXiv cs.CR 2019
Year: 2019

Modulation classification is an important component of cognitive self-driving networks. Recently many ML-based modulation classification methods have been proposed. We have evaluated the robustness of 9 ML-based modulation classifiers against the powerful Carlini \& Wagner (C-W) attack and showed that the current ML-based modulation classifiers do not provide any deterrence against adversarial ML examples. To the best of our knowledge, we are the first to report the results of the application of the C-W attack for creating adversarial examples against various ML models for modulation classification.
K-Means Clustering on Noisy Intermediate Scale Quantum Computers
Venue: arXiv cs.ET 2019
Year: 2019

Real-time clustering of big performance data generated by the telecommunication networks requires domain-specific high performance compute infrastructure to detect anomalies. In this paper, we evaluate noisy intermediate-scale quantum (NISQ) computers characterized by low decoherence times, for K-means clustering and propose three strategies to generate shorter-depth quantum circuits needed to overcome the limitation of NISQ computers. The strategies are based on exploiting; i) quantum interference, ii) negative rotations and iii) destructive interference. By comparing our implementations on IBMQX2 machine for representative data sets, we show that NISQ computers can solve the K-means clustering problem with the same level of accuracy as that of classical computers.
A Decision-Based Dynamic Ensemble Selection Method for Concept Drift
Venue: arXiv cs.LG 2019
Year: 2019

We propose an online method for concept driftdetection based on dynamic classifier ensemble selection. Theproposed method generates a pool of ensembles by promotingdiversity among classifier members and chooses expert ensemblesaccording to global prequential accuracy values. Unlike currentdynamic ensemble selection approaches that use only local knowl-edge to select the most competent ensemble for each instance,our method focuses on selection taking into account the decisionspace. Consequently, it is well adapted to the context of driftdetection in data stream problems. The results of the experimentsshow that the proposed method attained the highest detection pre-cision and the lowest number of false alarms, besides competitiveclassification accuracy rates, in artificial datasets representingdifferent types of drifts. Moreover, it outperformed baselines indifferent real-problem datasets in terms of classification accuracy.
An Investigation into the Effectiveness of Enhancement in ASR Training and Test for CHiME-5 Dinner Party Transcription
Venue: arXiv cs.CL 2019
Year: 2019

Despite the strong modeling power of neural network acoustic models, speech enhancement has been shown to deliver additional word error rate improvements if multi-channel data is available. However, there has been a longstanding debate whether enhancement should also be carried out on the ASR training data. In an extensive experimental evaluation on the acoustically very challenging CHiME-5 dinner party data we show that: (i) cleaning up the training data can lead to substantial error rate reductions, and (ii) enhancement in training is advisable as long as enhancement in test is at least as strong as in training. This approach stands in contrast and delivers larger gains than the common strategy reported in the literature to augment the training database with additional artificially degraded speech. Together with an acoustic model topology consisting of initial CNN layers followed by factorized TDNN layers we achieve with 41.6% and 43.2% WER on the DEV and EVAL test sets, respectively, a new single-system state-of-the-art result on the CHiME-5 data. This is a 8% relative improvement compared to the best word error rate published so far for a speech recognizer without system combination.
Storage Class Memory: Principles, Problems, and Possibilities
Venue: arXiv cs.AR 2019
Year: 2019

Storage Class Memory (SCM) is a class of memory technology which has recently become viable for use. Their namearises from the fact that they exhibit non-volatility of data, similar to secondary storage while also having latencies comparable toprimary memory and byte-addressibility. In this area, Phase Change Memory (PCM), Spin-Transfer-Torque Random Access Memory(STT-RAM), and Resistive RAM (ReRAM) have emerged as the major contenders for commercial and industrial use. In this paper, wedescribe how these memory types function, while highlighting the problems of endurance and performance that these memory typesface. We also discuss the future possibilities of Multi-Level Cells (MLCs), as well as how SCM can be used to construct accelerators.
Keyphrase Generation for Scientific Articles using GANs
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we present a keyphrase generation approach using conditional Generative Adversarial Networks (GAN). In our GAN model, the generator outputs a sequence of keyphrases based on the title and abstract of a scientific article. The discriminator learns to distinguish between machine-generated and human-curated keyphrases. We evaluate this approach on standard benchmark datasets. Our model achieves state-of-the-art performance in generation of abstractive keyphrases and is also comparable to the best performing extractive techniques. We also demonstrate that our method generates more diverse keyphrases and make our implementation publicly available.
Circuit equivalence in 2-nilpotent algebras
Venue: arXiv cs.CC 2019
Year: 2019

The circuit equivalence problem of a finite algebra $\mathbf A$ is the computational problem of deciding whether two circuits over $\mathbf A$ define the same function or not. This problem not just generalises the equivalence problem for Boolean circuits, but is also of high interest in universal algebra, as it models the problems of checking identities in $\mathbf A$. In this paper we discuss the complexity for algebras from congruence modular varieties. A partial classification was already given by Idziak and Krzaczkowski, leaving essentially only a gap for nilpotent but not supernilpotent algebras. We start a systematic study of this open case, proving that the circuit equivalence problem is in P for $2$-nilpotent such algebras.
Exercising with an "Iron Man": Design for a Robot Exercise Coach for Persons with Dementia
Venue: arXiv cs.RO 2019
Year: 2019

Socially assistive robots are increasingly being designed to interact with humans in various therapeutical scenarios. We believe that one useful scenario is providing exercise coaching for Persons with Dementia (PwD), which involves unique challenges related to memory and communication. We present a design for a robot that can seek to help a PWD to conduct exercises by recognizing behaviors and providing feedback, in an online, multimodal, and engaging way. Additionally, in line with a mid-fidelity prototyping approach, we report on some feedback from an exploratory user study using a Baxter robot, which provided some confirmation of the usefulness of the general scenario; furthermore, the results suggested the degree of a robot's feedback could be tailored to provide impressions of attentiveness or fun. Limitations and possibilities for future improvement are outlined, touching on deep learning and haptic feedback, toward informing next designs.
Fine-Grained, Language-Based Access Control for Database-Backed Applications
Venue: arXiv cs.PL 2019
Year: 2019

Context: Database-backed applications often run queries with more authority than necessary. Since programs can access more data than they legitimately need, flaws in security checks at the application level can enable malicious or buggy code to view or modify data in violation of intended access control policies. Inquiry: Although database management systems provide tools to control access to data, these tools are not well-suited for modern applications which often have many users and consist of many different software components. First, databases are unaware of application users, and creating a new database user for each application user is impractical for applications with many users. Second, different components of the same application may require different authority, which would require creating different database users for different software components. Thus, it is difficult to use existing tools to properly limit the authority an application has when executing queries. For this reason, we consider a new, language-based approach to application-specific database security. Approach: Prior work has addressed the difficulty of running applications with least privilege using capability-based security and software contracts, which we adapt to the setting of database-backed applications. Knowledge: This paper's main contribution is the design and implementation of ShillDB, a language for writing secure database-backed applications. ShillDB enables reasoning about database access at the language level through capabilities, which limit which database tables a program can access, and contracts, which limit what operations a program can perform on those tables. ShillDB contracts are expressed as part of function interfaces, making it easy to specify different access control policies for different components. Contracts act as executable security documentation for ShillDB programs and are enforced by the language runtime. Further, ShillDB provides database access control guarantees independent of (and in addition to) the security mechanisms of the underlying database management system. Grounding: We have implemented a prototype of ShillDB and have used it to implement the backend for a lending library reservation system with contracts for each endpoint to evaluate the performance and usability of ShillDB. Further, we benchmark individual database operations in ShillDB to better understand the language's performance overhead. Importance: Our experience indicates that ShillDB is a practical language for enforcing database access control policies in realistic, multi-user applications and has reasonable performance overhead. ShillDB allows developers to reason about security at the component level, safely compose components, and reuse third-party components with their own application-specific database security policies.