OpenRaiser/Intern-Atlas huggingface dataset:
Representing Formal Languages: A Comparison Between Finite Automata and Recurrent Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

We investigate the internal representations that a recurrent neural network (RNN) uses while learning to recognize a regular formal language. Specifically, we train a RNN on positive and negative examples from a regular language, and ask if there is a simple decoding function that maps states of this RNN to states of the minimal deterministic finite automaton (MDFA) for the language. Our experiments show that such a decoding function indeed exists, and that it maps states of the RNN not to MDFA states, but to states of an {\em abstraction} obtained by clustering small sets of MDFA states into "superstates". A qualitative analysis reveals that the abstraction often has a simple interpretation. Overall, the results suggest a strong structural relationship between internal representations used by RNNs and finite automata, and explain the well-known ability of RNNs to recognize formal grammatical structure.
A Periodogram-Based Method to Identify Forced and Natural Oscillations Using PMUs
Venue: arXiv cs.SY 2019
Year: 2019

Sustained oscillations in power systems are dangerous. There are various mechanisms, for instance, limit cycle and forced oscillation, that may lead to sustained oscillations, which nevertheless are hard to differentiate. In this paper, a novel periodogram-based method to distinguish different oscillation mechanism is proposed, which can quantitatively extract essential signatures of different mechanisms from power spectral density. Numerical study shows that the proposed method can accurately distinguish different oscillation mechanisms even when the forced oscillation frequency is close to the natural frequency.
Query Term Weighting based on Query Performance Prediction
Venue: arXiv cs.IR 2019
Year: 2019

This work presents a general query term weighting approach based on query performance prediction (QPP). To this end, a given term is weighed according to its predicted effect on query performance. Such an effect is assumed to be manifested in the responses made by the underlying retrieval method for the original query and its (simple) variants in the form of a single-term expanded query. Focusing on search re-ranking as the underlying application, the effectiveness of the proposed term weighting approach is demonstrated using several state-of-the-art QPP methods evaluated over TREC corpora.
Social Credibility Incorporating Semantic Analysis and Machine Learning: A Survey of the State-of-the-Art and Future Research Directions
Venue: arXiv cs.SI 2019
Year: 2019

The wealth of Social Big Data (SBD) represents a unique opportunity for organisations to obtain the excessive use of such data abundance to increase their revenues. Hence, there is an imperative need to capture, load, store, process, analyse, transform, interpret, and visualise such manifold social datasets to develop meaningful insights that are specific to an application domain. This paper lays the theoretical background by introducing the state-of-the-art literature review of the research topic. This is associated with a critical evaluation of the current approaches, and fortified with certain recommendations indicated to bridge the research gap.
Hypernetwork functional image representation
Venue: arXiv cs.LG 2019
Year: 2019

Motivated by the human way of memorizing images we introduce their functional representation, where an image is represented by a neural network. For this purpose, we construct a hypernetwork which takes an image and returns weights to the target network, which maps point from the plane (representing positions of the pixel) into its corresponding color in the image. Since the obtained representation is continuous, one can easily inspect the image at various resolutions and perform on it arbitrary continuous operations. Moreover, by inspecting interpolations we show that such representation has some properties characteristic to generative models. To evaluate the proposed mechanism experimentally, we apply it to image super-resolution problem. Despite using a single model for various scaling factors, we obtained results comparable to existing super-resolution methods.
Learning to Generate Questions by Learning What not to Generate
Venue: arXiv cs.CL 2019
Year: 2019

Automatic question generation is an important technique that can improve the training of question answering, help chatbots to start or continue a conversation with humans, and provide assessment materials for educational purposes. Existing neural question generation models are not sufficient mainly due to their inability to properly model the process of how each word in the question is selected, i.e., whether repeating the given passage or being generated from a vocabulary. In this paper, we propose our Clue Guided Copy Network for Question Generation (CGC-QG), which is a sequence-to-sequence generative model with copying mechanism, yet employing a variety of novel components and techniques to boost the performance of question generation. In CGC-QG, we design a multi-task labeling strategy to identify whether a question word should be copied from the input passage or be generated instead, guiding the model to learn the accurate boundaries between copying and generation. Furthermore, our input passage encoder takes as input, among a diverse range of other features, the prediction made by a clue word predictor, which helps identify whether each word in the input passage is a potential clue to be copied into the target question. The clue word predictor is designed based on a novel application of Graph Convolutional Networks onto a syntactic dependency tree representation of each passage, thus being able to predict clue words only based on their context in the passage and their relative positions to the answer in the tree. We jointly train the clue prediction as well as question generation with multi-task learning and a number of practical strategies to reduce the complexity. Extensive evaluations show that our model significantly improves the performance of question generation and out-performs all previous state-of-the-art neural question generation models by a substantial margin.
Attack-Defense Quantification Based On Game-Theory
Venue: arXiv cs.CR 2019
Year: 2019

With the developing of the attack and defense technology, the cyber environment has been more and more sophisticated. We failed to give an accurate evaluation of network security situation, as we lack a more accurate quantitative evaluation of attack-defense behaviors. In response to this situation, we proposed an attack-defense stochastic game model (ADSGM), analyzed the different security property of distinct defense mechanism, and put forward a corresponding utility calculation coping with the distinct defense mechanism. Through a case study, we showed the impact of active defense and the risk of attack exposure, demonstrated the effectiveness of our methods on attack-defense behavior quantification. This paper filled with the gap in the quantitative assessment of defensive measures, to make the quantitative evaluation of attack-defense more comprehensive and accurate.
Algebraic Invariants for Linear Hybrid Automata
Venue: arXiv cs.LO 2019
Year: 2019

We exhibit an algorithm to compute the strongest algebraic (or polynomial) invariants that hold at each location of a given unguarded linear hybrid automaton (i.e., a hybrid automaton having only unguarded transitions, all of whose assignments are given by affine expressions, and all of whose continuous dynamics are given by linear differential equations). Our main tool is a control-theoretic result of independent interest: given such a linear hybrid automaton, we show how to discretise the continuous dynamics in such a way that the resulting automaton has precisely the same algebraic invariants.
RESTful or RESTless -- Current State of Today's Top Web APIs
Venue: arXiv cs.SE 2019
Year: 2019

Recent developments in the world of services on the Web show that both the number of available Web APIs as well as the applications built on top is constantly increasing. This trend is commonly attributed to the wide adoption of the REST architectural principles. Still, the development of Web APIs is rather autonomous and it is up to the providers to decide how to implement, expose and describe the Web APIs. The individual implementations are then commonly documented in textual form as part of a webpage, showing a wide variety in terms of content, structure and level of detail. As a result, client application developers are forced to manually process and interpret the documentation. Before we can achieve a higher level of automation and can make any significant improvement to current practices and technologies, we need to reach a deeper understanding of their similarities and differences. Therefore, in this paper we present a thorough analysis of the most popular Web APIs through the examination of their documentation. We provide conclusions about common description forms, output types, usage of API parameters, invocation support, level of reusability, API granularity and authentication details. The collected data builds a solid foundation for identifying deficiencies and can be used as a basis for devising common standards and guidelines for Web API development.
Matchings under Preferences: Strength of Stability and Trade-offs
Venue: arXiv cs.GT 2019
Year: 2019

We propose two solution concepts for matchings under preferences: robustness and near stability. The former strengthens while the latter relaxes the classic definition of stability by Gale and Shapley (1962). Informally speaking, robustness requires that a matching must be stable in the classic sense, even if the agents slightly change their preferences. Near stability, on the other hand, imposes that a matching must become stable (again, in the classic sense) provided the agents are willing to adjust their preferences a bit. Both of our concepts are quantitative; together they provide means for a fine-grained analysis of stability for matchings. Moreover, our concepts allow to explore the trade-offs between stability and other criteria of societal optimality, such as the egalitarian cost and the number of unmatched agents. We investigate the computational complexity of finding matchings that implement certain predefined trade-offs. We provide a polynomial-time algorithm that given agents' preferences returns a socially optimal robust matching, and we prove that finding a socially optimal and nearly stable matching is computationally hard.
Technical report of "Empirical Study on Human Evaluation of Complex Argumentation Frameworks"
Venue: arXiv cs.AI 2019
Year: 2019

In abstract argumentation, multiple argumentation semantics have been proposed that allow to select sets of jointly acceptable arguments from a given argumentation framework, i.e. based only on the attack relation between arguments. The existence of multiple argumentation semantics raises the question which of these semantics predicts best how humans evaluate arguments. Previous empirical cognitive studies that have tested how humans evaluate sets of arguments depending on the attack relation between them have been limited to a small set of very simple argumentation frameworks, so that some semantics studied in the literature could not be meaningfully distinguished by these studies. In this paper we report on an empirical cognitive study that overcomes these limitations by taking into consideration twelve argumentation frameworks of three to eight arguments each. These argumentation frameworks were mostly more complex than the argumentation frameworks considered in previous studies. All twelve argumentation framework were systematically instantiated with natural language arguments based on a certain fictional scenario, and participants were shown both the natural language arguments and a graphical depiction of the attack relation between them. Our data shows that grounded and CF2 semantics were the best predictors of human argument evaluation. A detailed analysis revealed that part of the participants chose a cognitively simpler strategy that is predicted very well by grounded semantics, while another part of the participants chose a cognitively more demanding strategy that is mostly predicted well by CF2 semantics.
TrIK-SVM : an alternative decomposition for kernel methods in Krein spaces
Venue: arXiv cs.LG 2019
Year: 2019

The proposed work aims at proposing a alternative kernel decomposition in the context of kernel machines with indefinite kernels. The original paper of KSVM (SVM in Kreǐn spaces) uses the eigen-decomposition, our proposition avoids this decompostion. We explain how it can help in designing an algorithm that won't require to compute the full kernel matrix. Finally we illustrate the good behavior of the proposed method compared to KSVM.
Multiresolution Graph Attention Networks for Relevance Matching
Venue: arXiv cs.CL 2019
Year: 2019

A large number of deep learning models have been proposed for the text matching problem, which is at the core of various typical natural language processing (NLP) tasks. However, existing deep models are mainly designed for the semantic matching between a pair of short texts, such as paraphrase identification and question answering, and do not perform well on the task of relevance matching between short-long text pairs. This is partially due to the fact that the essential characteristics of short-long text matching have not been well considered in these deep models. More specifically, these methods fail to handle extreme length discrepancy between text pieces and neither can they fully characterize the underlying structural information in long text documents. In this paper, we are especially interested in relevance matching between a piece of short text and a long document, which is critical to problems like query-document matching in information retrieval and web searching. To extract the structural information of documents, an undirected graph is constructed, with each vertex representing a keyword and the weight of an edge indicating the degree of interaction between keywords. Based on the keyword graph, we further propose a Multiresolution Graph Attention Network to learn multi-layered representations of vertices through a Graph Convolutional Network (GCN), and then match the short text snippet with the graphical representation of the document with the attention mechanisms applied over each layer of the GCN. Experimental results on two datasets demonstrate that our graph approach outperforms other state-of-the-art deep matching models.
Architecting Dependable Learning-enabled Autonomous Systems: A Survey
Venue: arXiv cs.SE 2019
Year: 2019

We provide a summary over architectural approaches that can be used to construct dependable learning-enabled autonomous systems, with a focus on automated driving. We consider three technology pillars for architecting dependable autonomy, namely diverse redundancy, information fusion, and runtime monitoring. For learning-enabled components, we additionally summarize recent architectural approaches to increase the dependability beyond standard convolutional neural networks. We conclude the study with a list of promising research directions addressing the challenges of existing approaches.
Linear Time Visualization and Search in Big Data using Pixellated Factor Space Mapping
Venue: arXiv cs.IR 2019
Year: 2019

It is demonstrated how linear computational time and storage efficient approaches can be adopted when analyzing very large data sets. More importantly, interpretation is aided and furthermore, basic processing is easily supported. Such basic processing can be the use of supplementary, i.e. contextual, elements, or particular associations. Furthermore pixellated grid cell contents can be utilized as a basic form of imposed clustering. For a given resolution level, here related to an associated m-adic ($m$ here is a non-prime integer) or p-adic ($p$ is prime) number system encoding, such pixellated mapping results in partitioning. The association of a range of m-adic and p-adic representations leads naturally to an imposed hierarchical clustering, with partition levels corresponding to the m-adic-based and p-adic-based representations and displays. In these clustering embedding and imposed cluster structures, some analytical visualization and search applications are described
Validation of smartphone based pavement roughness measures
Venue: arXiv cs.HC 2019
Year: 2019

Smartphones are equipped with sensors such as accelerometers, gyroscope, and GPS in one cost-effective device with an acceptable level of accuracy. There have been some research studies carried out in terms of using smartphones to measure the pavement roughness. However, a little attention has been paid to investigate the validity of the measured pavement roughness by smartphones via other subjective methods such as the user opinion. This paper aims at calculating the pavement roughness data with a smartphone using its embedded sensors and investigating its correlation with a user opinion about the ride quality. In addition, the applicability of using smartphones to assess the pavement surface distresses is examined. Furthermore, to validate the smartphone sensor outputs objectively, the Road Surface Profiler is applied. Finally, a good roughness model is developed which demonstrates an acceptable level of correlation between the pavement roughness measured by smartphones and the ride quality rated by users.
Learning Task Knowledge and its Scope of Applicability in Experience-Based Planning Domains
Venue: arXiv cs.AI 2019
Year: 2019

Experience-based planning domains (EBPDs) have been recently proposed to improve problem solving by learning from experience. EBPDs provide important concepts for long-term learning and planning in robotics. They rely on acquiring and using task knowledge, i.e., activity schemata, for generating concrete solutions to problem instances in a class of tasks. Using Three-Valued Logic Analysis (TVLA), we extend previous work to generate a set of conditions as the scope of applicability for an activity schema. The inferred scope is a bounded representation of a set of problems of potentially unbounded size, in the form of a 3-valued logical structure, which allows an EBPD system to automatically find an applicable activity schema for solving task problems. We demonstrate the utility of our approach in a set of classes of problems in a simulated domain and a class of real world tasks in a fully physically simulated PR2 robot in Gazebo.
Recent Advances in Analysis and Design of Cyber-physical Systems using Passivity Indices
Venue: arXiv cs.SY 2019
Year: 2019

Analysis and resilient design of Cyber-physical Systems have greatly benefited from energy based concepts of passivity and dissipativity. Recently, there has been much research devoted to the use of passivity indices in different components of Cyber-physical systems. Passivity indices are measures of passivity, indicating how passive a system is or how far is it from being passive and generalize passivity based methods to systems that might not be passive. In this paper, we will review recent advances in the use of passivity indices in Cyber-physical systems. We will overview how the indices have been defined and applied to different components of Cyber-physical systems and how they are used in the resilient design of compositional Cyber-physical systems.
Lipschitz Adaptivity with Multiple Learning Rates in Online Learning
Venue: arXiv cs.LG 2019
Year: 2019

We aim to design adaptive online learning algorithms that take advantage of any special structure that might be present in the learning task at hand, with as little manual tuning by the user as possible. A fundamental obstacle that comes up in the design of such adaptive algorithms is to calibrate a so-called step-size or learning rate hyperparameter depending on variance, gradient norms, etc. A recent technique promises to overcome this difficulty by maintaining multiple learning rates in parallel. This technique has been applied in the MetaGrad algorithm for online convex optimization and the Squint algorithm for prediction with expert advice. However, in both cases the user still has to provide in advance a Lipschitz hyperparameter that bounds the norm of the gradients. Although this hyperparameter is typically not available in advance, tuning it correctly is crucial: if it is set too small, the methods may fail completely; but if it is taken too large, performance deteriorates significantly. In the present work we remove this Lipschitz hyperparameter by designing new versions of MetaGrad and Squint that adapt to its optimal value automatically. We achieve this by dynamically updating the set of active learning rates. For MetaGrad, we further improve the computational efficiency of handling constraints on the domain of prediction, and we remove the need to specify the number of rounds in advance.
Learning Logistic Circuits
Venue: arXiv cs.LG 2019
Year: 2019

This paper proposes a new classification model called logistic circuits. On MNIST and Fashion datasets, our learning algorithm outperforms neural networks that have an order of magnitude more parameters. Yet, logistic circuits have a distinct origin in symbolic AI, forming a discriminative counterpart to probabilistic-logical circuits such as ACs, SPNs, and PSDDs. We show that parameter learning for logistic circuits is convex optimization, and that a simple local search algorithm can induce strong model structures from data.
Learning Everywhere: Pervasive Machine Learning for Effective High-Performance Computation
Venue: arXiv cs.DC 2019
Year: 2019

The convergence of HPC and data-intensive methodologies provide a promising approach to major performance improvements. This paper provides a general description of the interaction between traditional HPC and ML approaches and motivates the Learning Everywhere paradigm for HPC. We introduce the concept of effective performance that one can achieve by combining learning methodologies with simulation-based approaches, and distinguish between traditional performance as measured by benchmark scores. To support the promise of integrating HPC and learning methods, this paper examines specific examples and opportunities across a series of domains. It concludes with a series of open computer science and cyberinfrastructure questions and challenges that the Learning Everywhere paradigm presents.
Efficient Grasp Planning and Execution with Multi-Fingered Hands by Surface Fitting
Venue: arXiv cs.RO 2019
Year: 2019

This paper introduces a framework to plan grasps with multi-fingered hands. The framework includes a multi-dimensional iterative surface fitting (MDISF) for grasp planning and a grasp trajectory optimization (GTO) for grasp imagination. The MDISF algorithm searches for optimal contact regions and hand configurations by minimizing the collision and surface fitting error, and the GTO algorithm generates optimal finger trajectories to reach the highly ranked grasp configurations and avoid collision with the environment. The proposed grasp planning and imagination framework considers the collision avoidance and the kinematics of the hand-robot system, and is able to plan grasps and trajectories of different categories efficiently with gradient-based methods using the captured point cloud. The found grasps and trajectories are robust to sensing noises and underlying uncertainties. The effectiveness of the proposed framework is verified by both simulations and experiments.