OpenRaiser/Intern-Atlas huggingface dataset:
Injecting Prior Knowledge for Transfer Learning into Reinforcement Learning Algorithms using Logic Tensor Networks
Venue: arXiv cs.LG 2019
Year: 2019

Human ability at solving complex tasks is helped by priors on object and event semantics of their environment. This paper investigates the use of similar prior knowledge for transfer learning in Reinforcement Learning agents. In particular, the paper proposes to use a first-order-logic language grounded in deep neural networks to represent facts about objects and their semantics in the real world. Facts are provided as background knowledge a priori to learning a policy for how to act in the world. The priors are injected with the conventional input in a single agent architecture. As proof-of-concept, the paper tests the system in simple experiments that show the importance of symbolic abstraction and flexible fact derivation. The paper shows that the proposed system can learn to take advantage of both the symbolic layer and the image layer in a single decision selection module.
Query and Resource Optimizations: A Case for Breaking the Wall in Big Data Systems
Venue: arXiv cs.DB 2019
Year: 2019

Modern big data systems run on cloud environments where resources are shared amongst several users and applications. As a result, declarative user queries in these environments need to be optimized and executed over resources that constantly change and are provisioned on demand for each job. This requires us to rethink traditional query optimizers designed for systems that run on dedicated resources. In this paper, we show evidence that the choice of query plans depends heavily on the available resources, and the current practice of choosing query plans before picking the resources could lead to significant performance loss in two popular big data systems, namely Hive and SparkSQL. Therefore, we make a case for Resource and Query Optimization (or RAQO), i.e., choosing both the query plan and the resource configuration at the same time. We describe rule-based RAQO and present alternate decisions trees to make resource-aware query planning in Hive and Spark. We further present cost-based RAQO that integrates resource planning within a query planner, and show techniques to significantly reduce the resource planning overheads. We evaluate cost-based RAQO using state-of-the-art System R query planner as well as a recently proposed multi-objective query planner. Our evaluation on TPC-H and randomly generated schemas show that: (i) we can reduce the resource planning overhead by up to 16x, and (ii) RAQO can scale to schemas as large as 100 table joins as well as clusters as big as 100K containers with 100GB each.
The Anatomy of Requirements
Venue: arXiv cs.SE 2019
Year: 2019

Requirements engineering is crucial to software development but lacks a precise definition of its fundamental concepts. Even the basic definitions in the literature and in industry standards are often vague and verbose. To remedy this situation and provide a solid basis for discussions of requirements, this work provides precise definitions of the fundamental requirements concepts and two systematic classifications: a taxonomy of requirement elements (such as components, goals, constraints...) ; and a taxonomy of possible relations between these elements (such as "extends", "excepts", "belongs"...). The discussion evaluates the taxonomies on published requirements documents; readers can test the concepts in two online quizzes. The intended result of this work is to spur new advances in the study and practice of software requirements by clarifying the fundamental concepts.
Conditional Computation for Continual Learning
Venue: arXiv cs.LG 2019
Year: 2019

Catastrophic forgetting of connectionist neural networks is caused by the global sharing of parameters among all training examples. In this study, we analyze parameter sharing under the conditional computation framework where the parameters of a neural network are conditioned on each input example. At one extreme, if each input example uses a disjoint set of parameters, there is no sharing of parameters thus no catastrophic forgetting. At the other extreme, if the parameters are the same for every example, it reduces to the conventional neural network. We then introduce a clipped version of maxout networks which lies in the middle, i.e. parameters are shared partially among examples. Based on the parameter sharing analysis, we can locate a limited set of examples that are interfered when learning a new example. We propose to perform rehearsal on this set to prevent forgetting, which is termed as conditional rehearsal. Finally, we demonstrate the effectiveness of the proposed method in an online non-stationary setup, where updates are made after each new example and the distribution of the received example shifts over time.
QSPR analysis of some novel neighborhood degree based topological descriptors
Venue: arXiv cs.DM 2019
Year: 2019

Topological index is a numerical value associated with chemical constitution for correlation of chemical structure with various physical properties, chemical reactivity or biological activity. In this work, some new indices based on neighbourhood degree sum of nodes are proposed. To make the computation of the novel indices convenient, an algorithm is designed. QSPR analysis of these newly introduced indices are studied here which reveals their predicting power. Some mathematical properties of these indices are also discussed here.
On the Self-Similarity of Natural Stochastic Textures
Venue: arXiv cs.CV 2019
Year: 2019

Self-similarity is the essence of fractal images and, as such, characterizes natural stochastic textures. This paper is concerned with the property of self-similarity in the statistical sense in the case of fully-textured images that contain both stochastic texture and structural (mostly deterministic) information. We firstly decompose a textured image into two layers corresponding to its texture and structure, and show that the layer representing the stochastic texture is characterized by random phase of uniform distribution, unlike the phase of the structured information which is coherent. The uniform distribution of the the random phase is verified by using a suitable hypothesis testing framework. We proceed by proposing two approaches to assessment of self-similarity. The first is based on patch-wise calculation of the mutual information, while the second measures the mutual information that exists across scales. Quantifying the extent of self-similarity by means of mutual information is of paramount importance in the analysis of natural stochastic textures that are encountered in medical imaging, geology, agriculture and in computer vision algorithms that are designed for application on fully-textures images.
Gateway Placement and Selection Solutions in WMNs: A Survey
Venue: arXiv cs.NI 2019
Year: 2019

Due to the high demand of Internet access by users, and the tremendous success of wireless technologies, Wireless Mesh Networks (WMNs) have become a promising solution. IGW Placement and Selection (GPS) are significantly investigated problems to achieve QoS requirements, network performance, and reduce deployment cost in WMNs. Best effort is made to classify different works in the literature based on network characteristics. At first, one of the most principal capabilities of WMNs, which is taking advantage of using multi-radio routers in a multi-channel network, is studied. In this article, GPS protocols considering a definition of three types of WMN are investigated based on channel-radio association including Single Radio Single Channel (SRSC), Single Radio Multi- Channel (SRMC), and Multi-Radio Multi-Channel (MRMC) WMNs. Furthermore, a classification regarding static and dynamic channel allocation policies is derived. In addition, the reported works from the viewpoint of network solutions are classified. The first perspective is: centralized, distributed or hybrid architectures. Following this classification, the studies are categorized regarding optimization techniques, which are operation research-based solutions, heuristic algorithms, and meta-heuristic-based algorithms.
SEntNet: Source-aware Recurrent Entity Network for Dialogue Response Selection
Venue: arXiv cs.IR 2019
Year: 2019

Dialogue response selection is an important part of Task-oriented Dialogue Systems (TDSs); it aims to predict an appropriate response given a dialogue context. Obtaining key information from a complex, long dialogue context is challenging, especially when different sources of information are available, e.g., the user's utterances, the system's responses, and results retrieved from a knowledge base (KB). Previous work ignores the type of information source and merges sources for response selection. However, accounting for the source type may lead to remarkable differences in the quality of response selection. We propose the Source-aware Recurrent Entity Network (SEntNet), which is aware of different information sources for the response selection process. SEntNet achieves this by employing source-specific memories to exploit differences in the usage of words and syntactic structure from different information sources (user, system, and KB). Experimental results show that SEntNet obtains 91.0% accuracy on the Dialog bAbI dataset, outperforming prior work by 4.7%. On the DSTC2 dataset, SEntNet obtains an accuracy of 41.2%, beating source unaware recurrent entity networks by 2.4%.
Floors are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction
Venue: arXiv cs.CV 2019
Year: 2019

We propose 4 insights that help to significantly improve the performance of deep learning models that predict surface normals and semantic labels from a single RGB image. These insights are: (1) denoise the "ground truth" surface normals in the training set to ensure consistency with the semantic labels; (2) concurrently train on a mix of real and synthetic data, instead of pretraining on synthetic and finetuning on real; (3) jointly predict normals and semantics using a shared model, but only backpropagate errors on pixels that have valid training labels; (4) slim down the model and use grayscale instead of color inputs. Despite the simplicity of these steps, we demonstrate consistently improved results on several datasets, using a model that runs at 12 fps on a standard mobile phone.
A new approach to forecast service parts demand by integrating user preferences into multi-objective optimization
Venue: arXiv cs.AI 2019
Year: 2019

Service supply chain management is to prepare spare parts for failed products under warranty. Their goal is to reach agreed service level at the minimum cost. We convert this business problem into a preference based multi-objective optimization problem, where two quality criteria must be simultaneously optimized. One criterion is accuracy of demand forecast and the other is service level. Here we propose a general framework supporting solving preference-based multi-objective optimization problems (MOPs) by multi-gradient descent algorithm (MGDA), which is well suited for training deep neural network. The proposed framework treats agreed service level as a constrained criterion that must be met and generate a Pareto-optimal solution with highest forecasting accuracy. The neural networks used here are two Encoder-Decoder LSTM modes: one is used for pre-training phase to learn distributed representation of former generations' service parts consumption data, and the other is used for supervised learning phase to generate forecast quantities of current generations' service parts. Evaluated under the service parts consumption data in Lenovo Group Ltd, the proposed method clearly outperform baseline methods.
Asymptotic Risk of Bezier Simplex Fitting
Venue: arXiv cs.LG 2019
Year: 2019

The Bezier simplex fitting is a novel data modeling technique which exploits geometric structures of data to approximate the Pareto front of multi-objective optimization problems. There are two fitting methods based on different sampling strategies. The inductive skeleton fitting employs a stratified subsampling from each skeleton of a simplex, whereas the all-at-once fitting uses a non-stratified sampling which treats a simplex as a whole. In this paper, we analyze the asymptotic risks of those Bézier simplex fitting methods and derive the optimal subsample ratio for the inductive skeleton fitting. It is shown that the inductive skeleton fitting with the optimal ratio has a smaller risk when the degree of a Bezier simplex is less than three. Those results are verified numerically under small to moderate sample sizes. In addition, we provide two complementary applications of our theory: a generalized location problem and a multi-objective hyper-parameter tuning of the group lasso. The former can be represented by a Bezier simplex of degree two where the inductive skeleton fitting outperforms. The latter can be represented by a Bezier simplex of degree three where the all-at-once fitting gets an advantage.
A Fast and Stable Omnidirectional Walking Engine for the Nao Humanoid Robot
Venue: arXiv cs.RO 2019
Year: 2019

This paper proposes a framework designed to generate a closed-loop walking engine for a humanoid robot. In particular, the core of this framework is an abstract dynamics model which is composed of two masses that represent the lower and the upper body of a humanoid robot. Moreover, according to the proposed dynamics model, the low-level controller is formulated as a Linear-Quadratic-Gaussian (LQG) controller that is able to robustly track the desired trajectories. Besides, this framework is fully parametric which allows using an optimization algorithm to find the optimum parameters. To examine the performance of the proposed framework, a set of simulation using a simulated Nao robot in the RoboCup 3D simulation environment has been carried out. Simulation results show that the proposed framework is capable of providing fast and reliable omnidirectional walking. After optimizing the parameters using genetic algorithm (GA), the maximum forward walking velocity that we have achieved was $80.5cm/s$.
Scalable Distributed Subtrajectory Clustering
Venue: arXiv cs.DB 2019
Year: 2019

Trajectory clustering is an important operation of knowledge discovery from mobility data. Especially nowadays, the need for performing advanced analytic operations over massively produced data, such as mobility traces, in efficient and scalable ways is imperative. However, discovering clusters of complete trajectories can overlook significant patterns that exist only for a small portion of their lifespan. In this paper, we address the problem of Distributed Subtrajectory Clustering in an efficient and highly scalable way. The problem is challenging because the subtrajectories to be clustered are not known in advance, but they need to be discovered dynamically based on adjacent subtrajectories in space and time. Towards this objective, we split the original problem to three sub-problems, namely Subtrajectory Join, Trajectory Segmentation and Clustering and Outlier Detection, and deal with each one in a distributed fashion by utilizing the MapReduce programming model. The efficiency and the effectiveness of our solution is demonstrated experimentally over a synthetic and two large real datasets from the maritime and urban domains and through comparison with two state of the art subtrajectory clustering algorithms.
Scrubbing Sensitive PHI Data from Medical Records made Easy by SpaCy -- A Scalable Model Implementation Comparisons
Venue: arXiv cs.LG 2019
Year: 2019

De-identification of clinical records is an extremely important process which enables the use of the wealth of information present in them. There are a lot of techniques available for this but none of the method implementation has evaluated the scalability, which is an important benchmark. We evaluated numerous deep learning techniques such as BiLSTM-CNN, IDCNN, CRF, BiLSTM-CRF, SpaCy, etc. on both the performance and efficiency. We propose that the SpaCy model implementation for scrubbing sensitive PHI data from medical records is both well performing and extremely efficient compared to other published models.
Attention-based Modeling for Emotion Detection and Classification in Textual Conversations
Venue: arXiv cs.CL 2019
Year: 2019

This paper addresses the problem of modeling textual conversations and detecting emotions. Our proposed model makes use of 1) deep transfer learning rather than the classical shallow methods of word embedding; 2) self-attention mechanisms to focus on the most important parts of the texts and 3) turn-based conversational modeling for classifying the emotions. The approach does not rely on any hand-crafted features or lexicons. Our model was evaluated on the data provided by the SemEval-2019 shared task on contextual emotion detection in text. The model shows very competitive results.
Boosting Supervision with Self-Supervision for Few-shot Learning
Venue: arXiv cs.CV 2019
Year: 2019

We present a technique to improve the transferability of deep representations learned on small labeled datasets by introducing self-supervised tasks as auxiliary loss functions. While recent approaches for self-supervised learning have shown the benefits of training on large unlabeled datasets, we find improvements in generalization even on small datasets and when combined with strong supervision. Learning representations with self-supervised losses reduces the relative error rate of a state-of-the-art meta-learner by 5-25% on several few-shot learning benchmarks, as well as off-the-shelf deep networks on standard classification tasks when training from scratch. We find the benefits of self-supervision increase with the difficulty of the task. Our approach utilizes the images within the dataset to construct self-supervised losses and hence is an effective way of learning transferable representations without relying on any external training data.
Assessing the Quality of the Steps to Reproduce in Bug Reports
Venue: arXiv cs.SE 2019
Year: 2019

A major problem with user-written bug reports, indicated by developers and documented by researchers, is the (lack of high) quality of the reported steps to reproduce the bugs. Low-quality steps to reproduce lead to excessive manual effort spent on bug triage and resolution. This paper proposes Euler, an approach that automatically identifies and assesses the quality of the steps to reproduce in a bug report, providing feedback to the reporters, which they can use to improve the bug report. The feedback provided by Euler was assessed by external evaluators and the results indicate that Euler correctly identified 98% of the existing steps to reproduce and 58% of the missing ones, while 73% of its quality annotations are correct.
A Note on Sequences of Lattices
Venue: arXiv cs.CG 2019
Year: 2019

We investigate the relation between the convergence of a sequence of lattices and the set-theoretic convergence of their corresponding Voronoi cells sequence. We prove that if a sequence of full rank lattices converges to a full rank lattice, then the closures of the limit infimum and limit supremum of the Voronoi cells converges to the corresponding Voronoi cell. It remains an open question if the converse is also true.
Hardware Aware Neural Network Architectures using FbNet
Venue: arXiv cs.CV 2019
Year: 2019

We implement a differentiable Neural Architecture Search (NAS) method inspired by FBNet for discovering neural networks that are heavily optimized for a particular target device. The FBNet NAS method discovers a neural network from a given search space by optimizing over a loss function which accounts for accuracy and target device latency. We extend this loss function by adding an energy term. This will potentially enhance the ``hardware awareness" and help us find a neural network architecture that is optimal in terms of accuracy, latency and energy consumption, given a target device (Raspberry Pi in our case). We name our trained child architecture obtained at the end of search process as Hardware Aware Neural Network Architecture (HANNA). We prove the efficacy of our approach by benchmarking HANNA against two other state-of-the-art neural networks designed for mobile/embedded applications, namely MobileNetv2 and CondenseNet for CIFAR-10 dataset. Our results show that HANNA provides a speedup of about 2.5x and 1.7x, and reduces energy consumption by 3.8x and 2x compared to MobileNetv2 and CondenseNet respectively. HANNA is able to provide such significant speedup and energy efficiency benefits over the state-of-the-art baselines at the cost of a tolerable 4-5% drop in accuracy.
How to Avoid Making a Billion-Dollar Mistake: Type-Safe Data Plane Programming with SafeP4
Venue: arXiv cs.PL 2019
Year: 2019

The P4 programming language offers high-level, declarative abstractions that bring the flexibility of software to the domain of networking. Unfortunately, the main abstraction used to represent packet data in P4, namely header types, lacks basic safety guarantees. Over the last few years, experience with an increasing number of programs has shown the risks of the unsafe approach, which often leads to subtle software bugs. This paper proposes SafeP4, a domain-specific language for programmable data planes in which all packet data is guaranteed to have a well-defined meaning and satisfy essential safety guarantees. We equip SafeP4 with a formal semantics and a static type system that statically guarantees header validity---a common source of safety bugs according to our analysis of real-world P4 programs. Statically ensuring header validity is challenging because the set of valid headers can be modified at runtime, making it a dynamic program property. Our type system achieves static safety by using a form of path-sensitive reasoning that tracks dynamic information from conditional statements, routing tables, and the control plane. Our evaluation shows that SafeP4's type system can effectively eliminate common failures in many real-world programs.
The Little Phone That Could Ch-Ch-Chroot
Venue: arXiv cs.CR 2019
Year: 2019

Security testing has been a career path that many are beginning to take. In doing so, security testing can hit the realms of many different types of engagements, ranging from web, infrastructure and social engineering. With the risk of sabotage, corporate espionage it has been seen that many organisations are beginning to develop a tactical capability. In doing so, the term 'Red Team' has been coined to market such engagements. Red Teaming is the method of having almost free reign towards a target. By doing such an engagement a target will be able to fully understand the breadth of vulnerabilities facing their organisation. However, Red Teaming can be an expensive and resource intensive task. Through this paper, it is discussed that it is possible to make a covert disposable phone to help aide Red Teamer's with the reconnaissance phase without drawing attention to themselves within a day to day task.