OpenRaiser/Intern-Atlas huggingface dataset:
A Structured Distributional Model of Sentence Meaning and Processing
Venue: arXiv cs.CL 2019
Year: 2019

Most compositional distributional semantic models represent sentence meaning with a single vector. In this paper, we propose a Structured Distributional Model (SDM) that combines word embeddings with formal semantics and is based on the assumption that sentences represent events and situations. The semantic representation of a sentence is a formal structure derived from Discourse Representation Theory and containing distributional vectors. This structure is dynamically and incrementally built by integrating knowledge about events and their typical participants, as they are activated by lexical items. Event knowledge is modeled as a graph extracted from parsed corpora and encoding roles and relationships between participants that are represented as distributional vectors. SDM is grounded on extensive psycholinguistic research showing that generalized knowledge about events stored in semantic memory plays a key role in sentence comprehension. We evaluate SDM on two recently introduced compositionality datasets, and our results show that combining a simple compositional model with event knowledge constantly improves performances, even with different types of word embeddings.
Tabula nearly rasa: Probing the Linguistic Knowledge of Character-Level Neural Language Models Trained on Unsegmented Text
Venue: arXiv cs.CL 2019
Year: 2019

Recurrent neural networks (RNNs) have reached striking performance in many natural language processing tasks. This has renewed interest in whether these generic sequence processing devices are inducing genuine linguistic knowledge. Nearly all current analytical studies, however, initialize the RNNs with a vocabulary of known words, and feed them tokenized input during training. We present a multi-lingual study of the linguistic knowledge encoded in RNNs trained as character-level language models, on input data with word boundaries removed. These networks face a tougher and more cognitively realistic task, having to discover any useful linguistic unit from scratch based on input statistics. The results show that our "near tabula rasa" RNNs are mostly able to solve morphological, syntactic and semantic tasks that intuitively presuppose word-level knowledge, and indeed they learned, to some extent, to track word boundaries. Our study opens the door to speculations about the necessity of an explicit, rigid word lexicon in language learning and usage.
Knowledge Network System (KNS) by Evolutionary Collective Intelligence (ECI): Model, Algorithm and Applications
Venue: arXiv cs.SI 2019
Year: 2019

Aiming at overcoming some inherent drawbacks and bottlenecks encountered by the conventional Knowledge, Recommendation, Search, and Social Systems, in this article we introduce the Knowledge Network System (KNS), a novel type of knowledge graph which is constructed by a new proposed algorithm, the Evolutionary Collective Intelligence (ECI) algorithm. The ECI, an agent-machine interactive algorithm, constructs the KNS by iteratively recommending interesting/matched samples/files to the agents, and meanwhile taking advantages of the collective intelligence of the agents. The ECI based KNS, to the best of our knowledge, is the first attempt in literature that integrates the functions of knowledge network construction, high-quality recommendation, new types of search and social in a same framework. Some real and potential applications of KNS and ECI are discussed, and a real system named VISVA is provided to demonstrate their efficacy. Some open problems for future works are also summarized in the end.
Sample-efficient Adversarial Imitation Learning from Observation
Venue: arXiv cs.LG 2019
Year: 2019

Imitation from observation is the framework of learning tasks by observing demonstrated state-only trajectories. Recently, adversarial approaches have achieved significant performance improvements over other methods for imitating complex behaviors. However, these adversarial imitation algorithms often require many demonstration examples and learning iterations to produce a policy that is successful at imitating a demonstrator's behavior. This high sample complexity often prohibits these algorithms from being deployed on physical robots. In this paper, we propose an algorithm that addresses the sample inefficiency problem by utilizing ideas from trajectory centric reinforcement learning algorithms. We test our algorithm and conduct experiments using an imitation task on a physical robot arm and its simulated version in Gazebo and will show the improvement in learning rate and efficiency.
Curriculum Learning Strategies for Hindi-English Codemixed Sentiment Analysis
Venue: arXiv cs.CL 2019
Year: 2019

Sentiment Analysis and other semantic tasks are commonly used for social media textual analysis to gauge public opinion and make sense from the noise on social media. The language used on social media not only commonly diverges from the formal language, but is compounded by codemixing between languages, especially in large multilingual societies like India. Traditional methods for learning semantic NLP tasks have long relied on end to end task specific training, requiring expensive data creation process, even more so for deep learning methods. This challenge is even more severe for resource scarce texts like codemixed language pairs, with lack of well learnt representations as model priors, and task specific datasets can be few and small in quantities to efficiently exploit recent deep learning approaches. To address above challenges, we introduce curriculum learning strategies for semantic tasks in code-mixed Hindi-English (Hi-En) texts, and investigate various training strategies for enhancing model performance. Our method outperforms the state of the art methods for Hi-En codemixed sentiment analysis by 3.31% accuracy, and also shows better model robustness in terms of convergence, and variance in test performance.
The Workflow Trace Archive: Open-Access Data from Public and Private Computing Infrastructures -- Technical Report
Venue: arXiv cs.DC 2019
Year: 2019

Realistic, relevant, and reproducible experiments often need input traces collected from real-world environments. We focus in this work on traces of workflows---common in datacenters, clouds, and HPC infrastructures. We show that the state-of-the-art in using workflow-traces raises important issues: (1) the use of realistic traces is infrequent, and (2) the use of realistic, {\it open-access} traces even more so. Alleviating these issues, we introduce the Workflow Trace Archive (WTA), an open-access archive of workflow traces from diverse computing infrastructures and tooling to parse, validate, and analyze traces. The WTA includes ${>}48$ million workflows captured from ${>}10$ computing infrastructures, representing a broad diversity of trace domains and characteristics. To emphasize the importance of trace diversity, we characterize the WTA contents and analyze in simulation the impact of trace diversity on experiment results. Our results indicate significant differences in characteristics, properties, and workflow structures between workload sources, domains, and fields.
Transfer Learning for Causal Sentence Detection
Venue: arXiv cs.CL 2019
Year: 2019

We consider the task of detecting sentences that express causality, as a step towards mining causal relations from texts. To bypass the scarcity of causal instances in relation extraction datasets, we exploit transfer learning, namely ELMO and BERT, using a bidirectional GRU with self-attention (BIGRUATT) as a baseline. We experiment with both generic public relation extraction datasets and a new biomedical causal sentence detection dataset, a subset of which we make publicly available. We find that transfer learning helps only in very small datasets. With larger datasets, BIGRUATT reaches a performance plateau, then larger datasets and transfer learning do not help.
Simple Algorithms for Dueling Bandits
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we present simple algorithms for Dueling Bandits. We prove that the algorithms have regret bounds for time horizon T of order O(T^rho ) with 1/2 <= rho <= 3/4, which importantly do not depend on any preference gap between actions, Delta. Dueling Bandits is an important extension of the Multi-Armed Bandit problem, in which the algorithm must select two actions at a time and only receives binary feedback for the duel outcome. This is analogous to comparisons in which the rater can only provide yes/no or better/worse type responses. We compare our simple algorithms to the current state-of-the-art for Dueling Bandits, ISS and DTS, discussing complexity and regret upper bounds, and conducting experiments on synthetic data that demonstrate their regret performance, which in some cases exceeds state-of-the-art.
3D Geometric salient patterns analysis on 3D meshes
Venue: arXiv cs.CV 2019
Year: 2019

Pattern analysis is a wide domain that has wide applicability in many fields. In fact, texture analysis is one of those fields, since the texture is defined as a set of repetitive or quasi-repetitive patterns. Despite its importance in analyzing 3D meshes, geometric texture analysis is less studied by geometry processing community. This paper presents a new efficient approach for geometric texture analysis on 3D triangular meshes. The proposed method is a scale-aware approach that takes as input a 3D mesh and a user-scale. It provides, as a result, a similarity-based clustering of texels in meaningful classes. Experimental results of the proposed algorithm are presented for both real-world and synthetic meshes within various textures. Furthermore, the efficiency of the proposed approach was experimentally demonstrated under mesh simplification and noise addition on the mesh surface. In this paper, we present a practical application for semantic annotation of 3D geometric salient texels.
ADA-Tucker: Compressing Deep Neural Networks via Adaptive Dimension Adjustment Tucker Decomposition
Venue: arXiv cs.CV 2019
Year: 2019

Despite the recent success of deep learning models in numerous applications, their widespread use on mobile devices is seriously impeded by storage and computational requirements. In this paper, we propose a novel network compression method called Adaptive Dimension Adjustment Tucker decomposition (ADA-Tucker). With learnable core tensors and transformation matrices, ADA-Tucker performs Tucker decomposition of arbitrary-order tensors. Furthermore, we propose that weight tensors in networks with proper order and balanced dimension are easier to be compressed. Therefore, the high flexibility in decomposition choice distinguishes ADA-Tucker from all previous low-rank models. To compress more, we further extend the model to Shared Core ADA-Tucker (SCADA-Tucker) by defining a shared core tensor for all layers. Our methods require no overhead of recording indices of non-zero elements. Without loss of accuracy, our methods reduce the storage of LeNet-5 and LeNet-300 by ratios of 691 times and 233 times, respectively, significantly outperforming state of the art. The effectiveness of our methods is also evaluated on other three benchmarks (CIFAR-10, SVHN, ILSVRC12) and modern newly deep networks (ResNet, Wide-ResNet).
Secure Architectures Implementing Trusted Coalitions for Blockchained Distributed Learning (TCLearn)
Venue: arXiv cs.CR 2019
Year: 2019

Distributed learning across a coalition of organizations allows the members of the coalition to train and share a model without sharing the data used to optimize this model. In this paper, we propose new secure architectures that guarantee preservation of data privacy, trustworthy sequence of iterative learning and equitable sharing of the learned model among each member of the coalition by using adequate encryption and blockchain mechanisms. We exemplify its deployment in the case of the distributed optimization of a deep learning convolutional neural network trained on medical images.
Impact of queue feedback on the stability and dynamics of a Rate Control Protocol (RCP) with two delays
Venue: arXiv cs.NI 2019
Year: 2019

Rate Control Protocol (RCP) uses feedback from routers to assign flows their fair rate. RCP estimates the fair rate using two forms of feedback: rate mismatch and queue size. An outstanding design question for RCP is whether the queue size feedback is useful or not. To address this, we analyze stability and the bifurcation properties of RCP in both the cases i.e., with and without queue size feedback. The model considers flows with two different round-trip times, operating over a single bottleneck link. By using an exogenous bifurcation parameter, we show that the system loses stability via a Hopf bifurcation and hence we can expect a limit cycle branching from the fixed point. We highlight that the presence of queue feedback can readily destabilize the system. Using Poincar{è} normal forms and the center manifold theorem, we show that the Hopf bifurcation is super-critical in the case of RCP without queue feedback. Whereas, in the presence of queue feedback, we show that the system can undergoes a sub-critical Hopf bifurcation for some parameter values. A sub-critical Hopf bifurcation can result in either large amplitude limit cycles or unstable limit cycles, and hence should be avoided in engineering applications. Thus, the presence of queue feedback would create adverse effects on the stability of the emerging limit cycles. In essence, the analytical results of RCP with two delays favor the design choice that uses feedback based only on rate mismatch. The theoretical analysis is validated with numerical computations and some packet level simulations as well.
On the Constrained Least-cost Tour Problem
Venue: arXiv cs.DS 2019
Year: 2019

We introduce the Constrained Least-cost Tour (CLT) problem: given an undirected graph with weight and cost functions on the edges, minimise the total cost of a tour rooted at a start vertex such that the total weight lies within a given range. CLT is related to the family of Travelling Salesman Problems with Profits, but differs by defining the weight function on edges instead of vertices, and by requiring the total weight to be within a range instead of being at least some quota. We prove CLT is $\mathcal{NP}$-hard, even in the simple case when the input graph is a path. We derive an informative lower bound by relaxing the integrality of edges and propose a heuristic motivated by this relaxation. For the case that requires the tour to be a simple cycle, we develop two heuristics which exploit Suurballe's algorithm to find low-cost, weight-feasible cycles. We demonstrate our algorithms by addressing a real-world problem that affects urban populations: finding routes that minimise air pollution exposure for walking, running and cycling in the city of London.
Declarative Learning-Based Programming as an Interface to AI Systems
Venue: arXiv cs.AI 2019
Year: 2019

Data-driven approaches are becoming more common as problem-solving techniques in many areas of research and industry. In most cases, machine learning models are the key component of these solutions, but a solution involves multiple such models, along with significant levels of reasoning with the models' output and input. Current technologies do not make such techniques easy to use for application experts who are not fluent in machine learning nor for machine learning experts who aim at testing ideas and models on real-world data in the context of the overall AI system. We review key efforts made by various AI communities to provide languages for high-level abstractions over learning and reasoning techniques needed for designing complex AI systems. We classify the existing frameworks based on the type of techniques and the data and knowledge representations they use, provide a comparative study of the way they address the challenges of programming real-world applications, and highlight some shortcomings and future directions.
Formal Modeling and SMT-Based Parameterized Verification of Data-Aware BPMN (Extended Version)
Venue: arXiv cs.LO 2019
Year: 2019

We propose DAB -- a data-aware extension of BPMN where the process operates over case and persistent data (partitioned into a read-only database called catalog and a read-write database called repository). The model trades off between expressiveness and the possibility of supporting parameterized verification of safety properties on top of it. Specifically, taking inspiration from the literature on verification of artifact systems, we study verification problems where safety properties are checked irrespectively of the content of the read-only catalog, and accepting the potential presence of unboundedly many tuples in the catalog and repository. We tackle such problems using an array-based backward reachability procedure fully implemented in MCMT -- a state-of-the-art array-based SMT model checker. Notably, we prove that the procedure is sound and complete for checking safety of DABs, and single out additional conditions that guarantee its termination and, in turn, show decidability of checking safety.
Sealed Computation: Abstract Requirements for Mechanisms to Support Trustworthy Cloud Computing
Venue: arXiv cs.CR 2019
Year: 2019

In cloud computing, data processing is delegated to a remote party for efficiency and flexibility reasons. A practical user requirement usually is that the confidentiality and integrity of data processing needs to be protected. In the common scenarios of cloud computing today, this can only be achieved by assuming that the remote party does not in any form act maliciously. In this paper, we propose an approach that avoids having to trust a single entity. Our approach is based on two concepts: (1) the technical abstraction of sealed computation, i.e., a technical mechanism to confine the processing of data within a tamper-proof hardware container, and (2) the additional role of an auditing party that itself cannot add functionality to the system but is able to check whether the system (including the mechanism for sealed computation) works as expected. We discuss the abstract technical and procedural requirements of these concepts and explain how they can be applied in practice.
Symbolic regression by uniform random global search
Venue: arXiv cs.NE 2019
Year: 2019

Symbolic regression (SR) is a data analysis problem where we search for the mathematical expression that best fits a numerical dataset. It is a global optimization problem. The most popular approach to SR is by genetic programming (SRGP). It is a common paradigm to compare an algorithm's performance to that of random search, but the data comparing SRGP to random search is lacking. We describe a novel algorithm for SR, namely SR by uniform random global search (SRURGS), also known as pure random search. We conduct experiments comparing SRURGS with SRGP using 100 randomly generated equations. Our results suggest that a SRGP is faster than SRURGS in producing equations with good R^2 for simple problems. However, our experiments suggest that SRURGS is more robust than SRGP, able to produce good output in more challenging problems. As SRURGS is arguably the simplest global search algorithm, we believe it should serve as a control algorithm against which other symbolic regression algorithms are compared. SRURGS has only one tuning parameter, and is conceptually very simple, making it a useful tool in solving SR problems. The method produces random equations, which is useful for the generation of symbolic regression benchmark problems. We have released well documented and open-source python code, currently under formal peer-review, so that interested researchers can deploy the tool in practice.
Event-based Star Tracking via Multiresolution Progressive Hough Transforms
Venue: arXiv cs.CV 2019
Year: 2019

Star trackers are state-of-the-art attitude estimation devices which function by recognising and tracking star patterns. Most commercial star trackers use conventional optical sensors. A recent alternative is to use event sensors, which could enable more energy efficient and faster star trackers. However, this demands new algorithms that can efficiently cope with high-speed asynchronous data, and are feasible on resource-constrained computing platforms. To this end, we propose an event-based processing approach for star tracking. Our technique operates on the event stream from a star field, by using multiresolution Hough Transforms to time-progressively integrate event data and produce accurate relative rotations. Optimisation via rotation averaging is then used to fuse the relative rotations and jointly refine the absolute orientations. Our technique is designed to be feasible for asynchronous operation on standard hardware. Moreover, compared to state-of-the-art event-based motion estimation schemes, our technique is much more efficient and accurate.
Tutorial on algebraic deletion correction codes
Venue: arXiv cs.DS 2019
Year: 2019

The deletion channel is known to be a notoriously diffcult channel to design error-correction codes for. In spite of this difficulty, there are some beautiful code constructions which give some intuition about the channel and about what good deletion codes look like. In this tutorial we will take a look at some of them. This document is a transcript of my talk at the coding theory reading group on some interesting works on deletion channel. It is not intended to be an exhaustive survey of works on deletion channel, but more as a tutorial to some of the important and cute ideas in this area. For a comprehensive survey, we refer the reader to the cited sources and surveys. We also provide an implementation of VT codes that correct single insertion/deletion errors for general alphabets at https://github.com/shubhamchandak94/VT_codes/.
Existence of a Convex Polyhedron with Respect to the Given Radii
Venue: arXiv cs.CG 2019
Year: 2019

Given a set of radii measured from a fixed point, the existence of a convex configuration with respect to the set of distinct radii in the two-dimensional case is proved when radii are distinct or repeated at most four points. However, we proved that there always exists a convex configuration in the three-dimensional case. In the application, we can imply the existence of the non-empty spherical Laguerre Voronoi diagram.
Imbalanced Learning-based Automatic SAR Images Change Detection by Morphologically Supervised PCA-Net
Venue: arXiv cs.CV 2019
Year: 2019

Change detection is a quite challenging task due to the imbalance between unchanged and changed class. In addition, the traditional difference map generated by log-ratio is subject to the speckle, which will reduce the accuracy. In this letter, an imbalanced learning-based change detection is proposed based on PCA network (PCA-Net), where a supervised PCA-Net is designed to obtain the robust features directly from given multitemporal SAR images instead of a difference map. Furthermore, to tackle with the imbalance between changed and unchanged classes, we propose a morphologically supervised learning method, where the knowledge in the pixels near the boundary between two classes are exploited to guide network training. Finally, our proposed PCA-Net can be trained by the datasets with available reference maps and applied to a new dataset, which is quite practical in change detection projects. Our proposed method is verified on five sets of multiple temporal SAR images. It is demonstrated from the experiment results that with the knowledge in training samples from the boundary, the learned features benefit for change detection and make the proposed method outperforms than supervised methods trained by randomly drawing samples.