OpenRaiser/Intern-Atlas huggingface dataset:
Encoding Syntactic Constituency Paths for Frame-Semantic Parsing with Graph Convolutional Networks
Venue: arXiv cs.CL 2020
Year: 2020

We study the problem of integrating syntactic information from constituency trees into a neural model in Frame-semantic parsing sub-tasks, namely Target Identification (TI), FrameIdentification (FI), and Semantic Role Labeling (SRL). We use a Graph Convolutional Network to learn specific representations of constituents, such that each constituent is profiled as the production grammar rule it corresponds to. We leverage these representations to build syntactic features for each word in a sentence, computed as the sum of all the constituents on the path between a word and a task-specific node in the tree, e.g. the target predicate for SRL. Our approach improves state-of-the-art results on the TI and SRL of ~1%and~3.5% points, respectively (+2.5% additional points are gained with BERT as input), when tested on FrameNet 1.5, while yielding comparable results on the CoNLL05 dataset to other syntax-aware systems.
AMLSI: A Novel Accurate Action Model Learning Algorithm
Venue: arXiv cs.AI 2020
Year: 2020

This paper presents new approach based on grammar induction called AMLSI Action Model Learning with State machine Interactions. The AMLSI approach does not require a training dataset of plan traces to work. AMLSI proceeds by trial and error: it queries the system to learn with randomly generated action sequences, and it observes the state transitions of the system, then AMLSI returns a PDDL domain corresponding to the system. A key issue for domain learning is the ability to plan with the learned domains. It often happens that a small learning error leads to a domain that is unusable for planning. Unlike other algorithms, we show that AMLSI is able to lift this lock by learning domains from partial and noisy observations with sufficient accuracy to allow planners to solve new problems.
Totally and Partially Ordered Hierarchical Planners in PDDL4J Library
Venue: arXiv cs.AI 2020
Year: 2020

In this paper, we outline the implementation of the TFD (Totally Ordered Fast Downward) and the PFD (Partially ordered Fast Downward) hierarchical planners that participated in the first HTN IPC competition in 2020. These two planners are based on forward-chaining task decomposition coupled with a compact grounding of actions, methods, tasks and HTN problems.
A Game Theoretic Model for Strategic Coopetition in Business Networks
Venue: arXiv cs.GT 2020
Year: 2020

Private blockchain is driving the creation of business networks, resulting in the creation of new value or new business models to the enterprises participating in the network. Such business networks form when enterprises come together to derive value through a network which is greater than the value that can be derived solely by any single company. This results in a setting that combines both competitive and cooperative behavior, and which we call strategic coopetition. Traditionally, cooperative and competitive behavior have been analyzed separately in game theory. In this article, we provide a formal model enabling to jointly analyze these different types of behaviors and the interdependencies between them. Using this model, we formally demonstrate and analyze the incentives for both cooperative and competitive behavior.
A Metric for Linear Symmetry-Based Disentanglement
Venue: arXiv cs.LG 2020
Year: 2020

The definition of Linear Symmetry-Based Disentanglement (LSBD) proposed by (Higgins et al., 2018) outlines the properties that should characterize a disentangled representation that captures the symmetries of data. However, it is not clear how to measure the degree to which a data representation fulfills these properties. We propose a metric for the evaluation of the level of LSBD that a data representation achieves. We provide a practical method to evaluate this metric and use it to evaluate the disentanglement of the data representations obtained for three datasets with underlying $SO(2)$ symmetries.
Modeling of Online Echo-Chamber Effect Based on the Concept of Spontaneous Symmetry Breaking
Venue: arXiv cs.SI 2020
Year: 2020

The online echo-chamber effect is a phenomenon in which beliefs that are far from common sense are strengthened within relatively small communities formed within online social networks. Since it is significantly degrading social activities in the real world, we should understand how the echo-chamber effect arises in an engineering framework to realize countermeasure technologies. This paper proposes a model of the online echo-chamber effect by introducing the concept of spontaneous symmetry breaking to the oscillation model framework used for describing online user dynamics.
Exposing the Robustness and Vulnerability of Hybrid 8T-6T SRAM Memory Architectures to Adversarial Attacks in Deep Neural Networks
Venue: arXiv cs.CR 2020
Year: 2020

Deep Learning is able to solve a plethora of once impossible problems. However, they are vulnerable to input adversarial attacks preventing them from being autonomously deployed in critical applications. Several algorithm-centered works have discussed methods to cause adversarial attacks and improve adversarial robustness of a Deep Neural Network (DNN). In this work, we elicit the advantages and vulnerabilities of hybrid 6T-8T memories to improve the adversarial robustness and cause adversarial attacks on DNNs. We show that bit-error noise in hybrid memories due to erroneous 6T-SRAM cells have deterministic behaviour based on the hybrid memory configurations (V_DD, 8T-6T ratio). This controlled noise (surgical noise) can be strategically introduced into specific DNN layers to improve the adversarial accuracy of DNNs. At the same time, surgical noise can be carefully injected into the DNN parameters stored in hybrid memory to cause adversarial attacks. To improve the adversarial robustness of DNNs using surgical noise, we propose a methodology to select appropriate DNN layers and their corresponding hybrid memory configurations to introduce the required surgical noise. Using this, we achieve 2-8% higher adversarial accuracy without re-training against white-box attacks like FGSM, than the baseline models (with no surgical noise introduced). To demonstrate adversarial attacks using surgical noise, we design a novel, white-box attack on DNN parameters stored in hybrid memory banks that causes the DNN inference accuracy to drop by more than 60% with over 90% confidence value. We support our claims with experiments, performed using benchmark datasets-CIFAR10 and CIFAR100 on VGG19 and ResNet18 networks.
On the Applicability of PEBS based Online Memory Access Tracking for Heterogeneous Memory Management at Scale
Venue: arXiv cs.OS 2020
Year: 2020

Operating systems have historically had to manage only a single type of memory device. The imminent availability of heterogeneous memory devices based on emerging memory technologies confronts the classic single memory model and opens a new spectrum of possibilities for memory management. Transparent data movement between different memory devices based on access patterns of applications is a desired feature to make optimal use of such devices and to hide the complexity of memory management to the end-user. However, capturing memory access patterns of an application at runtime comes at a cost, which is particularly challenging for large scale parallel applications that may be sensitive to system noise. In this work, we focus on the access pattern profiling phase prior to the actual memory relocation. We study the feasibility of using Intel's Processor Event-Based Sampling (PEBS) feature to record memory accesses by sampling at runtime and study the overhead at scale. We have implemented a custom PEBS driver in the IHK/McKernel lightweight multi-kernel operating system, one of whose advantages is minimal system interference due to the lightweight kernel's simple design compared to other OS kernels such as Linux. We present the PEBS overhead of a set of scientific applications and show the access patterns identified in noise-sensitive HPC applications. Our results show that clear access patterns can be captured with a 10% overhead in the worst-case and 1% in the best case when running on up to 128k CPU cores (2,048 Intel Xeon Phi Knights Landing nodes). We conclude that online memory access profiling using PEBS at large scale is promising for memory management in heterogeneous memory environments.
A Hierarchical Self-attentive Convolution Network for Review Modeling in Recommendation Systems
Venue: arXiv cs.IR 2020
Year: 2020

Using reviews to learn user and item representations is important for recommender system. Current review based methods can be divided into two categories: (1) the Convolution Neural Network (CNN) based models that extract n-gram features from user/item reviews; (2) the Recurrent Neural Network (RNN) based models that learn global contextual representations from reviews for users and items. Despite their success, both CNN and RNN based models in previous studies suffer from their own drawbacks. While CNN based models are weak in modeling long-dependency relation in text, RNN based models are slow in training and inference due to their incapability with parallel computing. To alleviate these problems, we propose a new text encoder module for review modeling in recommendation by combining convolution networks with self-attention networks to model local and global interactions in text together.As different words, sentences, reviews have different importance for modeling user and item representations, we construct review models hierarchically in sentence-level, review-level, and user/item level by encoding words for sentences, encoding sentences for reviews, and encoding reviews for user and item representations. Experiments on Amazon Product Benchmark show that our model can achieve significant better performance comparing to the state-of-the-art review based recommendation models.
Faster Projective Clustering Approximation of Big Data
Venue: arXiv cs.DS 2020
Year: 2020

In projective clustering we are given a set of n points in $R^d$ and wish to cluster them to a set $S$ of $k$ linear subspaces in $R^d$ according to some given distance function. An $\eps$-coreset for this problem is a weighted (scaled) subset of the input points such that for every such possible $S$ the sum of these distances is approximated up to a factor of $(1+\eps)$. We suggest to reduce the size of existing coresets by suggesting the first $O(\log(m))$ approximation for the case of $m$ lines clustering in $O(ndm)$ time, compared to the existing $\exp(m)$ solution. We then project the points on these lines and prove that for a sufficiently large $m$ we obtain a coreset for projective clustering. Our algorithm also generalize to handle outliers. Experimental results and open code are also provided.
TaylorGAN: Neighbor-Augmented Policy Update for Sample-Efficient Natural Language Generation
Venue: arXiv cs.CL 2020
Year: 2020

Score function-based natural language generation (NLG) approaches such as REINFORCE, in general, suffer from low sample efficiency and training instability problems. This is mainly due to the non-differentiable nature of the discrete space sampling and thus these methods have to treat the discriminator as a black box and ignore the gradient information. To improve the sample efficiency and reduce the variance of REINFORCE, we propose a novel approach, TaylorGAN, which augments the gradient estimation by off-policy update and the first-order Taylor expansion. This approach enables us to train NLG models from scratch with smaller batch size -- without maximum likelihood pre-training, and outperforms existing GAN-based methods on multiple metrics of quality and diversity. The source code and data are available at https://github.com/MiuLab/TaylorGAN
Minmax Regret 1-Sink Location Problems on Dynamic Flow Path Networks with Parametric Weights
Venue: arXiv cs.DS 2020
Year: 2020

This paper addresses the minmax regret 1-sink location problem on dynamic flow path networks with parametric weights. We are given a dynamic flow network consisting of an undirected path with positive edge lengths, positive edge capacities, and nonnegative vertex weights. A path can be considered as a road, an edge length as the distance along the road and a vertex weight as the number of people at the site. An edge capacity limits the number of people that can enter the edge per unit time. We consider the problem of locating a sink in the network, to which all the people evacuate from the vertices as quickly as possible. In our model, each weight is represented by a linear function in a common parameter $t$, and the decision maker who determines the location of a sink does not know the value of $t$. We formulate the sink location problem under such uncertainty as the minmax regret problem. Given $t$ and a sink location $x$, the cost of $x$ under $t$ is the sum of arrival times at $x$ for all the people determined by $t$. The regret for $x$ under $t$ is the gap between the cost of $x$ under $t$ and the optimal cost under $t$. The task of the problem is formulated as the one to find a sink location that minimizes the maximum regret over all $t$. For the problem, we propose an $O(n^4 2^{α(n)} α(n) \log n)$ time algorithm where $n$ is the number of vertices in the network and $α(\cdot)$ is the inverse Ackermann function. Also for the special case in which every edge has the same capacity, we show that the complexity can be reduced to $O(n^3 2^{α(n)} α(n) \log n)$.
Chinese Medical Question Answer Matching Based on Interactive Sentence Representation Learning
Venue: arXiv cs.CL 2020
Year: 2020

Chinese medical question-answer matching is more challenging than the open-domain question answer matching in English. Even though the deep learning method has performed well in improving the performance of question answer matching, these methods only focus on the semantic information inside sentences, while ignoring the semantic association between questions and answers, thus resulting in performance deficits. In this paper, we design a series of interactive sentence representation learning models to tackle this problem. To better adapt to Chinese medical question-answer matching and take the advantages of different neural network structures, we propose the Crossed BERT network to extract the deep semantic information inside the sentence and the semantic association between question and answer, and then combine with the multi-scale CNNs network or BiGRU network to take the advantage of different structure of neural networks to learn more semantic features into the sentence representation. The experiments on the cMedQA V2.0 and cMedQA V1.0 dataset show that our model significantly outperforms all the existing state-of-the-art models of Chinese medical question answer matching.
Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction
Venue: arXiv cs.CL 2020
Year: 2020

Relation Extraction (RE) is a vital step to complete Knowledge Graph (KG) by extracting entity relations from texts.However, it usually suffers from the long-tail issue. The training data mainly concentrates on a few types of relations, leading to the lackof sufficient annotations for the remaining types of relations. In this paper, we propose a general approach to learn relation prototypesfrom unlabeled texts, to facilitate the long-tail relation extraction by transferring knowledge from the relation types with sufficient trainingdata. We learn relation prototypes as an implicit factor between entities, which reflects the meanings of relations as well as theirproximities for transfer learning. Specifically, we construct a co-occurrence graph from texts, and capture both first-order andsecond-order entity proximities for embedding learning. Based on this, we further optimize the distance from entity pairs tocorresponding prototypes, which can be easily adapted to almost arbitrary RE frameworks. Thus, the learning of infrequent or evenunseen relation types will benefit from semantically proximate relations through pairs of entities and large-scale textual information.We have conducted extensive experiments on two publicly available datasets: New York Times and Google Distant Supervision.Compared with eight state-of-the-art baselines, our proposed model achieves significant improvements (4.1% F1 on average). Furtherresults on long-tail relations demonstrate the effectiveness of the learned relation prototypes. We further conduct an ablation study toinvestigate the impacts of varying components, and apply it to four basic relation extraction models to verify the generalization ability.Finally, we analyze several example cases to give intuitive impressions as qualitative analysis. Our codes will be released later.
Road Scene Graph: A Semantic Graph-Based Scene Representation Dataset for Intelligent Vehicles
Venue: arXiv cs.CV 2020
Year: 2020

Rich semantic information extraction plays a vital role on next-generation intelligent vehicles. Currently there is great amount of research focusing on fundamental applications such as 6D pose detection, road scene semantic segmentation, etc. And this provides us a great opportunity to think about how shall these data be organized and exploited. In this paper we propose road scene graph,a special scene-graph for intelligent vehicles. Different to classical data representation, this graph provides not only object proposals but also their pair-wise relationships. By organizing them in a topological graph, these data are explainable, fully-connected, and could be easily processed by GCNs (Graph Convolutional Networks). Here we apply scene graph on roads using our Road Scene Graph dataset, including the basic graph prediction model. This work also includes experimental evaluations using the proposed model.
Manipulating Medical Image Translation with Manifold Disentanglement
Venue: arXiv cs.CV 2020
Year: 2020

Medical image translation (e.g. CT to MR) is a challenging task as it requires I) faithful translation of domain-invariant features (e.g. shape information of anatomical structures) and II) realistic synthesis of target-domain features (e.g. tissue appearance in MR). In this work, we propose Manifold Disentanglement Generative Adversarial Network (MDGAN), a novel image translation framework that explicitly models these two types of features. It employs a fully convolutional generator to model domain-invariant features, and it uses style codes to separately model target-domain features as a manifold. This design aims to explicitly disentangle domain-invariant features and domain-specific features while gaining individual control of both. The image translation process is formulated as a stylisation task, where the input is "stylised" (translated) into diverse target-domain images based on style codes sampled from the learnt manifold. We test MDGAN for multi-modal medical image translation, where we create two domain-specific manifold clusters on the manifold to translate segmentation maps into pseudo-CT and pseudo-MR images, respectively. We show that by traversing a path across the MR manifold cluster, the target output can be manipulated while still retaining the shape information from the input.
Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup
Venue: arXiv cs.CL 2020
Year: 2020

Pre-trained language models, such as BERT, have achieved significant accuracy gain in many natural language processing tasks. Despite its effectiveness, the huge number of parameters makes training a BERT model computationally very challenging. In this paper, we propose an efficient multi-stage layerwise training (MSLT) approach to reduce the training time of BERT. We decompose the whole training process into several stages. The training is started from a small model with only a few encoder layers and we gradually increase the depth of the model by adding new encoder layers. At each stage, we only train the top (near the output layer) few encoder layers which are newly added. The parameters of the other layers which have been trained in the previous stages will not be updated in the current stage. In BERT training, the backward computation is much more time-consuming than the forward computation, especially in the distributed training setting in which the backward computation time further includes the communication time for gradient synchronization. In the proposed training strategy, only top few layers participate in backward computation, while most layers only participate in forward computation. Hence both the computation and communication efficiencies are greatly improved. Experimental results show that the proposed method can achieve more than 110% training speedup without significant performance degradation.
Human Computations in Citizen Crowds: A Knowledge Management Solution Framework
Venue: arXiv cs.HC 2020
Year: 2020

KG (Knowledge Generation) and understanding have traditionally been a Human-centric activity. KE (Knowledge Engineering) and KM (Knowledge Management) have tried to augment human knowledge on two separate planes: the first deals with machine interpretation of knowledge while the later explore interactions in human networks for KG and understanding. However, both remain computer-centric. Crowdsourced HC (Human Computations) have recently utilized human cognition and memory to generate diverse knowledge streams on specific tasks, which are mostly easy for humans to solve but remain challenging for machine algorithms. Literature shows little work on KM frameworks for citizen crowds, which gather input from the diverse category of Humans, organize that knowledge concerning tasks and knowledge categories and recreate new knowledge as a computer-centric activity. In this paper, we present an attempt to create a framework by implementing a simple solution, called ExamCheck, to focus on the generation of knowledge, feedback on that knowledge and recording the results of that knowledge in academic settings. Our solution, based on HC, shows that a structured KM framework can address a complex problem in a context that is important for participants themselves.
Numerical and experimental study of tonal noise sources at the outlet of an isolated centrifugal fan
Venue: arXiv cs.CE 2020
Year: 2020

In this study, tonal noise produced by an isolated centrifugal fan is investigated using unsteady Reynolds-averaged Navier-Stokes (URANS) equations. This type of fans is used in ventilation systems. As the fan propagates tonal noise in the system, it can severely affect the life quality of people that reside in the buildings. Our simulation shows that turbulence kinetic energy (TKE) is unevenly distributed around the rotation axis. Large TKE exists near the shroud at the pressure sides of the blades. It is caused by the recirculating flow. Moreover, the position of the largest TKE periodically varies among the blades. The period corresponds to approximately 4 times the fan rotation period, it was also found in acoustic measurements. The magnitude of the tonal noise at the blade passing frequencies agrees well with experimental data. By analyzing the wall-pressure fluctuations, it is found that the recirculating flow regions with large TKE are dominant sources of the tonal noise.