OpenRaiser/Intern-Atlas huggingface dataset:
RobOT: Robustness-Oriented Testing for Deep Learning Systems
Venue: arXiv cs.SE 2021
Year: 2021

Recently, there has been a significant growth of interest in applying software engineering techniques for the quality assurance of deep learning (DL) systems. One popular direction is deep learning testing, where adversarial examples (a.k.a.~bugs) of DL systems are found either by fuzzing or guided search with the help of certain testing metrics. However, recent studies have revealed that the commonly used neuron coverage metrics by existing DL testing approaches are not correlated to model robustness. It is also not an effective measurement on the confidence of the model robustness after testing. In this work, we address this gap by proposing a novel testing framework called Robustness-Oriented Testing (RobOT). A key part of RobOT is a quantitative measurement on 1) the value of each test case in improving model robustness (often via retraining), and 2) the convergence quality of the model robustness improvement. RobOT utilizes the proposed metric to automatically generate test cases valuable for improving model robustness. The proposed metric is also a strong indicator on how well robustness improvement has converged through testing. Experiments on multiple benchmark datasets confirm the effectiveness and efficiency of RobOT in improving DL model robustness, with 67.02% increase on the adversarial robustness that is 50.65% higher than the state-of-the-art work DeepGini.
Using Machine Intelligence to Prioritise Code Review Requests
Venue: arXiv cs.SE 2021
Year: 2021

Modern Code Review (MCR) is the process of reviewing new code changes that need to be merged with an existing codebase. As a developer, one may receive many code review requests every day, i.e., the review requests need to be prioritised. Manually prioritising review requests is a challenging and time-consuming process. To address the above problem, we conducted an industrial case study at Ericsson aiming at developing a tool called Pineapple, which uses a Bayesian Network to prioritise code review requests. To validate our approach/tool, we deployed it in a live software development project at Ericsson, wherein more than 150 developers develop a telecommunication product. We focused on evaluating the predictive performance, feasibility, and usefulness of our approach. The results indicate that Pineapple has competent predictive performance (RMSE = 0.21 and MAE = 0.15). Furthermore, around 82.6% of Pineapple's users believe the tool can support code review request prioritisation by providing reliable results, and around 56.5% of the users believe it helps reducing code review lead time. As future work, we plan to evaluate Pineapple's predictive performance, usefulness, and feasibility through a longitudinal investigation.
QFL: Data-Driven Feedback Loop to Manage Quality in Agile Development
Venue: arXiv cs.SE 2021
Year: 2021

Background: Quality requirements (QRs) describe desired system qualities, playing an important role in the success of software projects. In the context of agile software development (ASD), where the main objective is the fast delivery of functionalities, QRs are often ill-defined and not well addressed during the development process. Software analytics tools help to control quality though the measurement of quality-related software aspects to support decision-makers in the process of QR management. Aim: The goal of this research is to explore the benefits of integrating a concrete software analytics tool, Q-Rapids Tool, to assess software quality and support QR management processes. Method: In the context of a technology transfer project, the Softeam company has integrated Q-Rapids Tool in their development process. We conducted a series of workshops involving Softeam members working in the Modelio product development. Results: We present the Quality Feedback Loop (QFL) process to be integrated in software development processes to control the complete QR life-cycle, from elicitation to validation. As a result of the implementation of QFL in Softeam, Modelio's team members highlight the benefits of integrating a data analytics tool with their project planning tool and the fact that project managers can control the whole process making the final decisions. Conclusions: Practitioners can benefit from the integration of software analytics tools as part of their software development toolchain to control software quality. The implementation of QFL promotes quality in the organization and the integration of software analytics and project planning tools also improves the communication between teams.
Empirical Analysis on Productivity Prediction and Locality for Use Case Points Method
Venue: arXiv cs.SE 2021
Year: 2021

Use Case Points (UCP) method has been around for over two decades. Although, there was a substantial criticism concerning the algebraic construction and factors assessment of UCP, it remains an efficient early size estimation method. Predicting software effort from UCP is still an ever-present challenge. The earlier version of UCP method suggested using productivity as a cost driver, where fixed or a few pre-defined productivity ratios have been widely agreed. While this approach was successful when no enough historical data is available, it is no longer acceptable because software projects are different in terms of development aspects. Therefore, it is better to understand the relationship between productivity and other UCP variables. This paper examines the impact of data locality approaches on productivity and effort prediction from multiple UCP variables. The environmental factors are used as partitioning factors to produce local homogeneous data either based on their influential levels or using clustering algorithms. Different machine learning methods, including solo and ensemble methods, are used to construct productivity and effort prediction models based on the local data. The results demonstrate that the prediction models that are created based on local data surpass models that use entire data. Also, the results show that conforming the hypothetical assumption between productivity and environmental factors is not necessarily a requirement for success of locality.
Language Independent Emotion Quantification using Non linear Modelling of Speech
Venue: arXiv cs.SD 2021
Year: 2021

At present emotion extraction from speech is a very important issue due to its diverse applications. Hence, it becomes absolutely necessary to obtain models that take into consideration the speaking styles of a person, vocal tract information, timbral qualities and other congenital information regarding his voice. Our speech production system is a nonlinear system like most other real world systems. Hence the need arises for modelling our speech information using nonlinear techniques. In this work we have modelled our articulation system using nonlinear multifractal analysis. The multifractal spectral width and scaling exponents reveals essentially the complexity associated with the speech signals taken. The multifractal spectrums are well distinguishable the in low fluctuation region in case of different emotions. The source characteristics have been quantified with the help of different non-linear models like Multi-Fractal Detrended Fluctuation Analysis, Wavelet Transform Modulus Maxima. The Results obtained from this study gives a very good result in emotion clustering.
Terminating grid exploration with myopic luminous robots
Venue: arXiv cs.DC 2021
Year: 2021

We investigate the terminating grid exploration for autonomous myopic luminous robots. Myopic robots mean that they can observe nodes only within a certain fixed distance, and luminous robots mean that they have light devices that can emit colors. First, we prove that, in the semi-synchronous and asynchronous models, three myopic robots are necessary to achieve the terminating grid exploration if the visible distance is one. Next, we give fourteen algorithms for the terminating grid exploration in various assumptions of synchrony (fully-synchronous, semi-synchronous, and asynchronous models), visible distance, the number of colors, and a chirality. Six of them are optimal in terms of the number of robots.
An Ensemble Deep Convolutional Neural Network Model for Electricity Theft Detection in Smart Grids
Venue: arXiv cs.LG 2021
Year: 2021

Smart grids extremely rely on Information and Communications Technology (ICT) and smart meters to control and manage numerous parameters of the network. However, using these infrastructures make smart grids more vulnerable to cyber threats especially electricity theft. Electricity Theft Detection (EDT) algorithms are typically used for such purpose since this Non-Technical Loss (NTL) may lead to significant challenges in the power system. In this paper, an Ensemble Deep Convolutional Neural Network (EDCNN) algorithm for ETD in smart grids has been proposed. As the first layer of the model, a random under bagging technique is applied to deal with the imbalance data, and then Deep Convolutional Neural Networks (DCNN) are utilized on each subset. Finally, a voting system is embedded, in the last part. The evaluation results based on the Area Under Curve (AUC), precision, recall, f1-score, and accuracy verify the efficiency of the proposed method compared to the existing method in the literature.
A GeoSPARQL Compliance Benchmark
Venue: arXiv cs.DB 2021
Year: 2021

We propose a series of tests that check for the compliance of RDF triplestores with the GeoSPARQL standard. The purpose of the benchmark is to test how many of the requirements outlined in the standard a tested system supports and to push triplestores forward in achieving a full GeoSPARQL compliance. This topic is of concern because the support of GeoSPARQL varies greatly between different triplestore implementations, and such support is of great importance for the domain of geospatial RDF data. Additionally, we present a comprehensive comparison of triplestores, providing an insight into their current GeoSPARQL support.
A Logic for Conditional Local Strategic Reasoning
Venue: arXiv cs.GT 2021
Year: 2021

We consider systems of rational agents who act and interact in pursuit of their individual and collective objectives. We study and formalise the reasoning of an agent, or of an external observer, about the expected choices of action of the other agents based on their objectives, in order to assess the reasoner's ability, or expectation, to achieve their own objective. To formalize such reasoning we extend Pauly's Coalition Logic with three new modal operators of conditional strategic reasoning, thus introducing the Logic for Local Conditional Strategic Reasoning ConStR. We provide formal semantics for the new conditional strategic operators in concurrent game models, introduce the matching notion of bisimulation for each of them, prove bisimulation invariance and Hennessy-Milner property for each of them, and discuss and compare briefly their expressiveness. Finally, we also propose systems of axioms for each of the basic operators of ConStR and for the full logic.
Testing Framework for Black-box AI Models
Venue: arXiv cs.LG 2021
Year: 2021

With widespread adoption of AI models for important decision making, ensuring reliability of such models remains an important challenge. In this paper, we present an end-to-end generic framework for testing AI Models which performs automated test generation for different modalities such as text, tabular, and time-series data and across various properties such as accuracy, fairness, and robustness. Our tool has been used for testing industrial AI models and was very effective to uncover issues present in those models. Demo video link: https://youtu.be/984UCU17YZI
Adversarial Segmentation Loss for Sketch Colorization
Venue: arXiv cs.CV 2021
Year: 2021

We introduce a new method for generating color images from sketches or edge maps. Current methods either require some form of additional user-guidance or are limited to the "paired" translation approach. We argue that segmentation information could provide valuable guidance for sketch colorization. To this end, we propose to leverage semantic image segmentation, as provided by a general purpose panoptic segmentation network, to create an additional adversarial loss function. Our loss function can be integrated to any baseline GAN model. Our method is not limited to datasets that contain segmentation labels, and it can be trained for "unpaired" translation tasks. We show the effectiveness of our method on four different datasets spanning scene level indoor, outdoor, and children book illustration images using qualitative, quantitative and user study analysis. Our model improves its baseline up to 35 points on the FID metric. Our code and pretrained models can be found at https://github.com/giddyyupp/AdvSegLoss.
To Reuse or Not To Reuse? A Framework and System for Evaluating Summarized Knowledge
Venue: arXiv cs.HC 2021
Year: 2021

As the amount of information online continues to grow, a correspondingly important opportunity is for individuals to reuse knowledge which has been summarized by others rather than starting from scratch. However, appropriate reuse requires judging the relevance, trustworthiness, and thoroughness of others' knowledge in relation to an individual's goals and context. In this work, we explore augmenting judgements of the appropriateness of reusing knowledge in the domain of programming, specifically of reusing artifacts that result from other developers' searching and decision making. Through an analysis of prior research on sensemaking and trust, along with new interviews with developers, we synthesized a framework for reuse judgements. The interviews also validated that developers express a desire for help with judging whether to reuse an existing decision. From this framework, we developed a set of techniques for capturing the initial decision maker's behavior and visualizing signals calculated based on the behavior, to facilitate subsequent consumers' reuse decisions, instantiated in a prototype system called Strata. Results of a user study suggest that the system significantly improves the accuracy, depth, and speed of reusing decisions. These results have implications for systems involving user-generated content in which other users need to evaluate the relevance and trustworthiness of that content.
Securing RPL using Network Coding: The Chained Secure Mode (CSM)
Venue: arXiv cs.NI 2021
Year: 2021

As the de facto routing protocol for many Internet of Things (IoT) networks nowadays, and to assure the confidentiality and integrity of its control messages, the Routing Protocol for Low Power and Lossy Networks (RPL) incorporates three modes of security: the Unsecured Mode (UM), Preinstalled Secure Mode (PSM), and the Authenticated Secure Mode (ASM). While the PSM and ASM are intended to protect against external routing attacks and some replay attacks (through an optional replay protection mechanism), recent research showed that RPL in PSM is still vulnerable to many routing attacks, both internal and external. In this paper, we propose a novel secure mode for RPL, the Chained Secure Mode (CSM), based on the concept of intraflow Network Coding (NC). The CSM is designed to enhance RPL resilience and mitigation capability against replay attacks while allowing the integration with external security measures such as Intrusion Detection Systems (IDSs). The security and performance of the proposed CSM were evaluated and compared against RPL in UM and PSM (with and without the optional replay protection) under several routing attacks: the Neighbor attack (NA), Wormhole (WH), and CloneID attack (CA), using average packet delivery rate (PDR), End-to-End (E2E) latency, and power consumption as metrics. It showed that CSM has better performance and more enhanced security than both the UM and PSM with the replay protection, while mitigating both the NA and WH attacks and significantly reducing the effect of the CA in the investigated scenarios.
Unsupervised Extractive Summarization using Pointwise Mutual Information
Venue: arXiv cs.CL 2021
Year: 2021

Unsupervised approaches to extractive summarization usually rely on a notion of sentence importance defined by the semantic similarity between a sentence and the document. We propose new metrics of relevance and redundancy using pointwise mutual information (PMI) between sentences, which can be easily computed by a pre-trained language model. Intuitively, a relevant sentence allows readers to infer the document content (high PMI with the document), and a redundant sentence can be inferred from the summary (high PMI with the summary). We then develop a greedy sentence selection algorithm to maximize relevance and minimize redundancy of extracted sentences. We show that our method outperforms similarity-based methods on datasets in a range of domains including news, medical journal articles, and personal anecdotes.
kPAM 2.0: Feedback Control for Category-Level Robotic Manipulation
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we explore generalizable, perception-to-action robotic manipulation for precise, contact-rich tasks. In particular, we contribute a framework for closed-loop robotic manipulation that automatically handles a category of objects, despite potentially unseen object instances and significant intra-category variations in shape, size and appearance. Previous approaches typically build a feedback loop on top of a real-time 6-DOF pose estimator. However, representing an object with a parameterized transformation from a fixed geometric template does not capture large intra-category shape variation. Hence we adopt the keypoint-based object representation proposed in kPAM for category-level pick-and-place, and extend it to closed-loop manipulation policies with contact-rich tasks. We first augment keypoints with local orientation information. Using the oriented keypoints, we propose a novel object-centric action representation in terms of regulating the linear/angular velocity or force/torque of these oriented keypoints. This formulation is surprisingly versatile -- we demonstrate that it can accomplish contact-rich manipulation tasks that require precision and dexterity for a category of objects with different shapes, sizes and appearances, such as peg-hole insertion for pegs and holes with significant shape variation and tight clearance. With the proposed object and action representation, our framework is also agnostic to the robot grasp pose and initial object configuration, making it flexible for integration and deployment.
Large Scale Distributed Collaborative Unlabeled Motion Planning with Graph Policy Gradients
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we present a learning method to solve the unlabelled motion problem with motion constraints and space constraints in 2D space for a large number of robots. To solve the problem of arbitrary dynamics and constraints we propose formulating the problem as a multi-agent problem. We are able to demonstrate the scalability of our methods for a large number of robots by employing a graph neural network (GNN) to parameterize policies for the robots. The GNN reduces the dimensionality of the problem by learning filters that aggregate information among robots locally, similar to how a convolutional neural network is able to learn local features in an image. Additionally, by employing a GNN we are also able to overcome the computational overhead of training policies for a large number of robots by first training graph filters for a small number of robots followed by zero-shot policy transfer to a larger number of robots. We demonstrate the effectiveness of our framework through various simulations.
Does Culture Matter? Impact of Individualism and Uncertainty Avoidance on App Reviews
Venue: arXiv cs.SE 2021
Year: 2021

Mobile applications are often used by an international audience and therefore receive a high daily amount of user reviews from various countries. Previous work found evidence that app store reviews contain helpful information for software evolution processes. However, the cultural diversity of the reviews and its consequences on specific user feedback characteristics has only been researched to a limited extent so far. In this paper, we examine the influence of two cultural dimensions, Individualism and Uncertainty Avoidance on user feedback in Apple app store reviews written in different languages. For this purpose, we collected 647,141 reviews from eight countries and written in five languages over a period of six months. We then used manual content analysis and automated processing to examine a sample of 3,120 reviews. The results show that there is a statistically significant influence of Individualism and Uncertainty Avoidance on user feedback characteristics. The results of this study will help researchers and practitioners to reduce algorithm bias caused by less diversified training and test data and to raise awareness of the importance of analyzing diversified user feedback.
Verifying High-Level Latency-Insensitive Designs with Formal Model Checking
Venue: arXiv cs.LO 2021
Year: 2021

Latency-insensitive design mitigates increasing interconnect delay and enables productive component reuse in complex digital systems. This design style has been adopted in high-level design flows because untimed functional blocks connected through latency-insensitive interfaces provide a natural communication abstraction. However, latency-insensitive design with high-level languages also introduces a unique set of verification challenges that jeopardize functional correctness. In particular, bugs due to invalid consumption of inputs and deadlocks can be difficult to detect and debug with dynamic simulation methods. To tackle these two classes of bugs, we propose formal model checking methods to guarantee that a high-level latency-insensitive design is unaffected by invalid input data and is free of deadlock. We develop a well-structured verification wrapper for each property to automatically construct the corresponding formal model for checking. Our experiments demonstrate that the formal checks are effective in realistic bug scenarios from high-level designs.
ReRankMatch: Semi-Supervised Learning with Semantics-Oriented Similarity Representation
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes integrating semantics-oriented similarity representation into RankingMatch, a recently proposed semi-supervised learning method. Our method, dubbed ReRankMatch, aims to deal with the case in which labeled and unlabeled data share non-overlapping categories. ReRankMatch encourages the model to produce the similar image representations for the samples likely belonging to the same class. We evaluate our method on various datasets such as CIFAR-10, CIFAR-100, SVHN, STL-10, and Tiny ImageNet. We obtain promising results (4.21% error rate on CIFAR-10 with 4000 labels, 22.32% error rate on CIFAR-100 with 10000 labels, and 2.19% error rate on SVHN with 1000 labels) when the amount of labeled data is sufficient to learn semantics-oriented similarity representation. The code is made publicly available at https://github.com/tqtrunghnvn/ReRankMatch.