OpenRaiser/Intern-Atlas huggingface dataset:
Requirements Engineering for Machine Learning: Perspectives from Data Scientists
Venue: arXiv cs.LG 2019
Year: 2019

Machine learning (ML) is used increasingly in real-world applications. In this paper, we describe our ongoing endeavor to define characteristics and challenges unique to Requirements Engineering (RE) for ML-based systems. As a first step, we interviewed four data scientists to understand how ML experts approach elicitation, specification, and assurance of requirements and expectations. The results show that changes in the development paradigm, i.e., from coding to training, also demands changes in RE. We conclude that development of ML systems demands requirements engineers to: (1) understand ML performance measures to state good functional requirements, (2) be aware of new quality requirements such as explainability, freedom from discrimination, or specific legal requirements, and (3) integrate ML specifics in the RE process. Our study provides a first contribution towards an RE methodology for ML systems.
Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamics
Venue: arXiv cs.AI 2019
Year: 2019

Continuous control and planning remains a major challenge in robotics and machine learning. Neuroscience offers the possibility of learning from animal brains that implement highly successful controllers, but it is unclear how to relate an animal's behavior to control principles. Animals may not always act optimally from the perspective of an external observer, but may still act rationally: we hypothesize that animals choose actions with highest expected future subjective value according to their own internal model of the world. Their actions thus result from solving a different optimal control problem from those on which they are evaluated in neuroscience experiments. With this assumption, we propose a novel framework of model-based inverse rational control that learns the agent's internal model that best explains their actions in a task described as a partially observable Markov decision process (POMDP). In this approach we first learn optimal policies generalized over the entire model space of dynamics and subjective rewards, using an extended Kalman filter to represent the belief space, a neural network in the actor-critic framework to optimize the policy, and a simplified basis for the parameter space. We then compute the model that maximizes the likelihood of the experimentally observable data comprising the agent's sensory observations and chosen actions. Our proposed method is able to recover the true model of simulated agents within theoretical error bounds given by limited data. We illustrate this method by applying it to a complex naturalistic task currently used in neuroscience experiments. This approach provides a foundation for interpreting the behavioral and neural dynamics of highly adapted controllers in animal brains.
Towards Self-Explainable Cyber-Physical Systems
Venue: arXiv cs.AI 2019
Year: 2019

With the increasing complexity of CPSs, their behavior and decisions become increasingly difficult to understand and comprehend for users and other stakeholders. Our vision is to build self-explainable systems that can, at run-time, answer questions about the system's past, current, and future behavior. As hitherto no design methodology or reference framework exists for building such systems, we propose the MAB-EX framework for building self-explainable systems that leverage requirements- and explainability models at run-time. The basic idea of MAB-EX is to first Monitor and Analyze a certain behavior of a system, then Build an explanation from explanation models and convey this EXplanation in a suitable way to a stakeholder. We also take into account that new explanations can be learned, by updating the explanation models, should new and yet un-explainable behavior be detected by the system.
Micro-architectural Analysis of OLAP: Limitations and Opportunities
Venue: arXiv cs.DB 2019
Year: 2019

Understanding micro-architectural behavior is profound in efficiently using hardware resources. Recent work has shown that, despite being aggressively optimized for modern hardware, in-memory online transaction processing (OLTP) systems severely underutilize their core micro-architecture resources [25]. Online analytical processing (OLAP) workloads, on the other hand, exhibit a completely different computing pattern. OLAP workloads are read-only, bandwidth-intensive and include various data access patterns including both sequential and random data accesses. In addition, with the rise of column-stores, they run on high performance engines that are tightly optimized for the efficient use of modern hardware. Hence, the micro-architectural behavior of modern OLAP systems remains unclear. This work presents the micro-architectural analysis of a breadth of OLAP systems. We examine CPU cycles and memory bandwidth utilization. The results show that, unlike the traditional, commercial OLTP systems, traditional, commercial OLAP systems do not suffer from instruction cache misses. Nevertheless, they suffer from their large instruction footprint resulting in slow response times. High performance OLAP engines execute tight instruction streams; however, they spend 25 to 82% of the CPU cycles on stalls regardless of the workload being sequential- or random-access-heavy. In addition, high performance OLAP engines underutilize the multi-core CPU or memory bandwidth resources due to their disproportional compute and memory demands. Hence, analytical processing engines should carefully assign their compute and memory resources for efficient multi-core micro-architectural utilization.
Improving Generalization in Coreference Resolution via Adversarial Training
Venue: arXiv cs.CL 2019
Year: 2019

In order for coreference resolution systems to be useful in practice, they must be able to generalize to new text. In this work, we demonstrate that the performance of the state-of-the-art system decreases when the names of PER and GPE named entities in the CoNLL dataset are changed to names that do not occur in the training set. We use the technique of adversarial gradient-based training to retrain the state-of-the-art system and demonstrate that the retrained system achieves higher performance on the CoNLL dataset (both with and without the change of named entities) and the GAP dataset.
Fine-grained Information Status Classification Using Discourse Context-Aware Self-Attention
Venue: arXiv cs.CL 2019
Year: 2019

Previous work on bridging anaphora recognition (Hou et al., 2013a) casts the problem as a subtask of learning fine-grained information status (IS). However, these systems heavily depend on many hand-crafted linguistic features. In this paper, we propose a discourse context-aware self-attention neural network model for fine-grained IS classification. On the ISNotes corpus (Markert et al., 2012), our model with the contextually-encoded word representations (BERT) (Devlin et al., 2018) achieves new state-of-the-art performances on fine-grained IS classification, obtaining a 4.1% absolute overall accuracy improvement compared to Hou et al. (2013a). More importantly, we also show an improvement of 3.9% F1 for bridging anaphora recognition without using any complex hand-crafted semantic features designed for capturing the bridging phenomenon.
How often should I access my online social networks?
Venue: arXiv cs.SI 2019
Year: 2019

Users of online social networks are faced with a conundrum of trying to be always informed without having enough time or attention budget to do so. The retention of users on online social networks has important implications, encompassing economic, psychological and infrastructure aspects. In this paper, we pose the following question: what is the optimal rate at which users should access a social network? To answer this question, we propose an analytical model to determine the value of an access (VoA) to the social network. In the simple setting considered in this paper, VoA is defined as the chance of a user accessing the network and obtaining new content. Clearly, VoA depends on the rate at which sources generate content and on the filtering imposed by the social network. Then, we pose an optimization problem wherein the utility of users grows with respect to VoA but is penalized by costs incurred to access the network. Using the proposed framework, we provide insights on the optimal access rate. Our results are parameterized using Facebook data, indicating the predictive power of the approach.
Local Score Dependent Model Explanation for Time Dependent Covariates
Venue: arXiv cs.LG 2019
Year: 2019

The use of deep neural networks to make high risk decisions creates a need for global and local explanations so that users and experts have confidence in the modeling algorithms. We introduce a novel technique to find global and local explanations for time series data used in binary classification machine learning systems. We identify the most salient of the original features used by a black box model to distinguish between classes. The explanation can be made on categorical, continuous, and time series data and can be generalized to any binary classification model. The analysis is conducted on time series data to train a long short-term memory deep neural network and uses the time dependent structure of the underlying features in the explanation. The proposed technique attributes weights to features to explain an observations risk of belonging to a class as a multiplicative factor of a base hazard rate. We use a variation of the Cox Proportional Hazards regression, a Generalized Additive Model, to explain the effect of variables upon the probability of an in-class response for a score output from the black box model. The covariates incorporate time dependence structure in the features so the explanation is inclusive of the underlying time series data structure.
Constrained Multi-Objective Optimization for Automated Machine Learning
Venue: arXiv cs.LG 2019
Year: 2019

Automated machine learning has gained a lot of attention recently. Building and selecting the right machine learning models is often a multi-objective optimization problem. General purpose machine learning software that simultaneously supports multiple objectives and constraints is scant, though the potential benefits are great. In this work, we present a framework called Autotune that effectively handles multiple objectives and constraints that arise in machine learning problems. Autotune is built on a suite of derivative-free optimization methods, and utilizes multi-level parallelism in a distributed computing environment for automatically training, scoring, and selecting good models. Incorporation of multiple objectives and constraints in the model exploration and selection process provides the flexibility needed to satisfy trade-offs necessary in practical machine learning applications. Experimental results from standard multi-objective optimization benchmark problems show that Autotune is very efficient in capturing Pareto fronts. These benchmark results also show how adding constraints can guide the search to more promising regions of the solution space, ultimately producing more desirable Pareto fronts. Results from two real-world case studies demonstrate the effectiveness of the constrained multi-objective optimization capability offered by Autotune.
Architecture and evolution of semantic networks in mathematics texts
Venue: arXiv cs.CL 2019
Year: 2019

Knowledge is a network of interconnected concepts. Yet, precisely how the topological structure of knowledge constrains its acquisition remains unknown, hampering the development of learning enhancement strategies. Here we study the topological structure of semantic networks reflecting mathematical concepts and their relations in college-level linear algebra texts. We hypothesize that these networks will exhibit structural order, reflecting the logical sequence of topics that ensures accessibility. We find that the networks exhibit strong core-periphery architecture, where a dense core of concepts presented early is complemented with a sparse periphery presented evenly throughout the exposition; the latter is composed of many small modules each reflecting more narrow domains. Using tools from applied topology, we find that the expositional evolution of the semantic networks produces and subsequently fills knowledge gaps, and that the density of these gaps tracks negatively with community ratings of each textbook. Broadly, our study lays the groundwork for future efforts developing optimal design principles for textbook exposition and teaching in a classroom setting.
Generalised Zero-Shot Learning with Domain Classification in a Joint Semantic and Visual Space
Venue: arXiv cs.CV 2019
Year: 2019

Generalised zero-shot learning (GZSL) is a classification problem where the learning stage relies on a set of seen visual classes and the inference stage aims to identify both the seen visual classes and a new set of unseen visual classes. Critically, both the learning and inference stages can leverage a semantic representation that is available for the seen and unseen classes. Most state-of-the-art GZSL approaches rely on a mapping between latent visual and semantic spaces without considering if a particular sample belongs to the set of seen or unseen classes. In this paper, we propose a novel GZSL method that learns a joint latent representation that combines both visual and semantic information. This mitigates the need for learning a mapping between the two spaces. Our method also introduces a domain classification that estimates whether a sample belongs to a seen or an unseen class. Our classifier then combines a class discriminator with this domain classifier with the goal of reducing the natural bias that GZSL approaches have toward the seen classes. Experiments show that our method achieves state-of-the-art results in terms of harmonic mean, the area under the seen and unseen curve and unseen classification accuracy on public GZSL benchmark data sets. Our code will be available upon acceptance of this paper.
Stop the Open Data Bus, We Want to Get Off
Venue: arXiv cs.CR 2019
Year: 2019

The subject of this report is the re-identification of individuals in the Myki public transport dataset released as part of the Melbourne Datathon 2018. We demonstrate the ease with which we were able to re-identify ourselves, our co-travellers, and complete strangers; our analysis raises concerns about the nature and granularity of the data released, in particular the ability to identify vulnerable or sensitive groups.
Robust Translational Force Control of Multi-Rotor UAV for Precise Acceleration Tracking
Venue: arXiv cs.RO 2019
Year: 2019

In this paper, we introduce a translational force control method with disturbance observer (DOB)-based force disturbance cancellation for precise three-dimensional acceleration control of a multi-rotor UAV. The acceleration control of the multi-rotor requires conversion of the desired acceleration signal to the desired roll, pitch, and total thrust. But because the attitude dynamics and the thrust dynamics are different, simple kinematic signal conversion without consideration of those difference can cause serious performance degradation in acceleration tracking. Unlike most existing translational force control techniques that are based on such simple inversion, our new method allows controlling the acceleration of the multi-rotor more precisely by considering the dynamics of the multi-rotor during the kinematic inversion. By combining the DOB with the translational force system that includes the improved conversion technique, we achieve robustness with respect to the external force disturbances that hinders the accurate acceleration control. mu-analysis is performed to ensure the robust stability of the overall closed-loop system, considering the combined effect of various possible model uncertainties. Both simulation and experiment are conducted to validate the proposed technique, which confirms the satisfactory performance to track the desired acceleration of the multi-rotor.
FlexNER: A Flexible LSTM-CNN Stack Framework for Named Entity Recognition
Venue: arXiv cs.CL 2019
Year: 2019

Named entity recognition (NER) is a foundational technology for information extraction. This paper presents a flexible NER framework compatible with different languages and domains. Inspired by the idea of distant supervision (DS), this paper enhances the representation by increasing the entity-context diversity without relying on external resources. We choose different layer stacks and sub-network combinations to construct the bilateral networks. This strategy can generally improve model performance on different datasets. We conduct experiments on five languages, such as English, German, Spanish, Dutch and Chinese, and biomedical fields, such as identifying the chemicals and gene/protein terms from scientific works. Experimental results demonstrate the good performance of this framework.
Sketched Representations and Orthogonal Planarity of Bounded Treewidth Graphs
Venue: arXiv cs.CG 2019
Year: 2019

Given a planar graph $G$ and an integer $b$, OrthogonalPlanarity is the problem of deciding whether $G$ admits an orthogonal drawing with at most $b$ bends in total. We show that OrthogonalPlanarity can be solved in polynomial time if $G$ has bounded treewidth. Our proof is based on an FPT algorithm whose parameters are the number of bends, the treewidth and the number of degree-2 vertices of $G$. This result is based on the concept of sketched orthogonal representation that synthetically describes a family of equivalent orthogonal representations. Our approach can be extended to related problems such as HV-Planarity and FlexDraw. In particular, both OrthogonalPlanarity and HV-Planarity can be decided in $O(n^3 \log n)$ time for series-parallel graphs, which improves over the previously known $O(n^4)$ bounds.
Complexity of universal access structures
Venue: arXiv cs.CR 2019
Year: 2019

An important parameter in a secret sharing scheme is the number of minimal qualified sets. Given this number, the universal access structure is the richest possible structure, namely the one in which there are one or more participants in every possible Boolean combination of the minimal qualified sets. Every access structure is a substructure of the universal structure for the same number of minimal qualified subsets, thus universal access structures have the highest complexity given the number of minimal qualified sets. We show that the complexity of the universal structure with $n$ minimal qualified sets is between $n/\log_2 n$ and $n/2.7182$ asymptotically.
Algorithms the min-max regret 0-1 Integer Linear Programming Problem with Interval Data
Venue: arXiv cs.DS 2019
Year: 2019

We address the Interval Data Min-Max Regret 0-1 Integer Linear Programming problem (MMR-ILP), a variant of the 0-1 Integer Linear Programming problem where the objective function coefficients are uncertain. We solve MMR-ILP using a Benders-like Decomposition Algorithm and two metaheuristics for min-max regret problems with interval data. Computational experiments developed on variations of MIPLIB instances show that the heuristics obtain good results in a reasonable computational time when compared to the Benders-like Decomposition algorithm.
Unsupervised Behavior Change Detection in Multidimensional Data Streams for Maritime Traffic Monitoring
Venue: arXiv cs.LG 2019
Year: 2019

The worldwide growth of maritime traffic and the development of the Automatic Identification System (AIS) has led to advances in monitoring systems for preventing vessel accidents and detecting illegal activities. In this work, we describe research gaps and challenges in machine learning for vessel behavior change and event detection, considering several constraints imposed by real-time data streams and the maritime monitoring domain. As a starting point, we investigate how unsupervised and semi-supervised change detection methods may be employed for identifying shifts in vessel behavior, aiming to detect and label unusual events.
WiFi-based Real-time Breathing and Heart Rate Monitoring during Sleep
Venue: arXiv cs.HC 2019
Year: 2019

Good quality sleep is essential for good health and sleep monitoring becomes a vital research topic. This paper provides a low cost, continuous and contactless WiFi-based vital signs (breathing and heart rate) monitoring method. In particular, we set up the antennas based on Fresnel diffraction model and signal propagation theory, which enhances the detection of weak breathing/heartbeat motion. We implement a prototype system using the off-shelf devices and a real-time processing system to monitor vital signs in real time. The experimental results indicate the accurate breathing rate and heart rate detection performance. To the best of our knowledge, this is the first work to use a pair of WiFi devices and omnidirectional antennas to achieve real-time individual breathing rate and heart rate monitoring in different sleeping postures.
Semi-supervised Learning with Adaptive Neighborhood Graph Propagation Network
Venue: arXiv cs.CV 2019
Year: 2019

Graph Convolutional Networks (GCNs) have been widely studied for compact data representation and semi-supervised learning tasks. However, existing GCNs usually use a fixed neighborhood graph which is not guaranteed to be optimal for semi-supervised learning tasks. In this paper, we first re-interpret graph convolution operation in GCNs as a composition of feature propagation and (non-linear) transformation. Based on this observation, we then propose a unified adaptive neighborhood feature propagation model and derive a novel Adaptive Neighborhood Graph Propagation Network (ANGPN) for data representation and semi-supervised learning. The aim of ANGPN is to conduct both graph construction and graph convolution simultaneously and cooperatively in a unified formulation and thus can learn an optimal neighborhood graph that best serves graph convolution for data representation and semi-supervised learning. One main benefit of ANGPN is that the learned (convolutional) representation can provide useful weakly supervised information for constructing a better neighborhood graph which meanwhile facilitates data representation and learning. Experimental results on four benchmark datasets demonstrate the effectiveness and benefit of the proposed ANGPN.
Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding
Venue: arXiv cs.LG 2019
Year: 2019

Recent state-of-the-art natural language understanding models, such as BERT and XLNet, score a pair of sentences (A and B) using multiple cross-attention operations - a process in which each word in sentence A attends to all words in sentence B and vice versa. As a result, computing the similarity between a query sentence and a set of candidate sentences, requires the propagation of all query-candidate sentence-pairs throughout a stack of cross-attention layers. This exhaustive process becomes computationally prohibitive when the number of candidate sentences is large. In contrast, sentence embedding techniques learn a sentence-to-vector mapping and compute the similarity between the sentence vectors via simple elementary operations. In this paper, we introduce Distilled Sentence Embedding (DSE) - a model that is based on knowledge distillation from cross-attentive models, focusing on sentence-pair tasks. The outline of DSE is as follows: Given a cross-attentive teacher model (e.g. a fine-tuned BERT), we train a sentence embedding based student model to reconstruct the sentence-pair scores obtained by the teacher model. We empirically demonstrate the effectiveness of DSE on five GLUE sentence-pair tasks. DSE significantly outperforms several ELMO variants and other sentence embedding methods, while accelerating computation of the query-candidate sentence-pairs similarities by several orders of magnitude, with an average relative degradation of 4.6% compared to BERT. Furthermore, we show that DSE produces sentence embeddings that reach state-of-the-art performance on universal sentence representation benchmarks. Our code is made publicly available at https://github.com/microsoft/Distilled-Sentence-Embedding.