OpenRaiser/Intern-Atlas huggingface dataset:
Hypervector Design for Efficient Hyperdimensional Computing on Edge Devices
Venue: arXiv cs.LG 2021
Year: 2021

Hyperdimensional computing (HDC) has emerged as a new light-weight learning algorithm with smaller computation and energy requirements compared to conventional techniques. In HDC, data points are represented by high-dimensional vectors (hypervectors), which are mapped to high-dimensional space (hyperspace). Typically, a large hypervector dimension ($\geq1000$) is required to achieve accuracies comparable to conventional alternatives. However, unnecessarily large hypervectors increase hardware and energy costs, which can undermine their benefits. This paper presents a technique to minimize the hypervector dimension while maintaining the accuracy and improving the robustness of the classifier. To this end, we formulate the hypervector design as a multi-objective optimization problem for the first time in the literature. The proposed approach decreases the hypervector dimension by more than $32\times$ while maintaining or increasing the accuracy achieved by conventional HDC. Experiments on a commercial hardware platform show that the proposed approach achieves more than one order of magnitude reduction in model size, inference time, and energy consumption. We also demonstrate the trade-off between accuracy and robustness to noise and provide Pareto front solutions as a design parameter in our hypervector design.
Multi-objective discovery of PDE systems using evolutionary approach
Venue: arXiv cs.NE 2021
Year: 2021

Usually, the systems of partial differential equations (PDEs) are discovered from observational data in the single vector equation form. However, this approach restricts the application to the real cases, where, for example, the form of the external forcing is of interest. In the paper, a multi-objective co-evolution algorithm is described. The single equations within the system and the system itself are evolved simultaneously to obtain the system. This approach allows discovering the systems with the form-independent equations. In contrast to the single vector equation, a component-wise system is more suitable for expert interpretation and, therefore, for applications. The example of the two-dimensional Navier-Stokes equation is considered.
Practical Encrypted Computing for IoT Clients
Venue: arXiv cs.CR 2021
Year: 2021

Privacy and energy are primary concerns for sensor devices that offload compute to a potentially untrusted edge server or cloud. Homomorphic Encryption (HE) enables offload processing of encrypted data. HE offload processing retains data privacy, but is limited by the need for frequent communication between the client device and the offload server. Existing client-aided encrypted computing systems are optimized for performance on the offload server, failing to sufficiently address client costs, and precluding HE offload for low-resource (e.g., IoT) devices. We introduce Client-aided HE for Opaque Compute Offloading (CHOCO), a client-optimized system for encrypted offload processing. CHOCO introduces rotational redundancy, an algorithmic optimization to minimize computing and communication costs. We design Client-Aided HE for Opaque Compute Offloading Through Accelerated Cryptographic Operations (CHOCO-TACO), a comprehensive architectural accelerator for client-side cryptographic operations that eliminates most of their time and energy costs. Our evaluation shows that CHOCO makes client-aided HE offloading feasible for resource-constrained clients. Compared to existing encrypted computing solutions, CHOCO reduces communication cost by up to 2948x. With hardware support, client-side encryption/decryption is faster by 1094x and uses 648x less energy. In our end-to-end implementation of a large-scale DNN (VGG16), CHOCO uses 37% less energy than local (unencrypted) computation.
Wasserstein Robust Classification with Fairness Constraints
Venue: arXiv cs.LG 2021
Year: 2021

We propose a distributionally robust classification model with a fairness constraint that encourages the classifier to be fair in view of the equality of opportunity criterion. We use a type-$\infty$ Wasserstein ambiguity set centered at the empirical distribution to model distributional uncertainty and derive a conservative reformulation for the worst-case equal opportunity unfairness measure. We establish that the model is equivalent to a mixed binary optimization problem, which can be solved by standard off-the-shelf solvers. To improve scalability, we further propose a convex, hinge-loss-based model for large problem instances whose reformulation does not incur any binary variables. Moreover, we also consider the distributionally robust learning problem with a generic ground transportation cost to hedge against the uncertainties in the label and sensitive attribute. Finally, we numerically demonstrate that our proposed approaches improve fairness with negligible loss of predictive accuracy.
Evaluation of Morphological Embeddings for English and Russian Languages
Venue: arXiv cs.CL 2021
Year: 2021

This paper evaluates morphology-based embeddings for English and Russian languages. Despite the interest and introduction of several morphology-based word embedding models in the past and acclaimed performance improvements on word similarity and language modeling tasks, in our experiments, we did not observe any stable preference over two of our baseline models - SkipGram and FastText. The performance exhibited by morphological embeddings is the average of the two baselines mentioned above.
Linnaeus: A highly reusable and adaptable ML based log classification pipeline
Venue: arXiv cs.LG 2021
Year: 2021

Logs are a common way to record detailed run-time information in software. As modern software systems evolve in scale and complexity, logs have become indispensable to understanding the internal states of the system. At the same time however, manually inspecting logs has become impractical. In recent times, there has been more emphasis on statistical and machine learning (ML) based methods for analyzing logs. While the results have shown promise, most of the literature focuses on algorithms and state-of-the-art (SOTA), while largely ignoring the practical aspects. In this paper we demonstrate our end-to-end log classification pipeline, Linnaeus. Besides showing the more traditional ML flow, we also demonstrate our solutions for adaptability and re-use, integration towards large scale software development processes, and how we cope with lack of labelled data. We hope Linnaeus can serve as a blueprint for, and inspire the integration of, various ML based solutions in other large scale industrial settings.
An Efficient Hypergraph Approach to Robust Point Cloud Resampling
Venue: arXiv cs.CV 2021
Year: 2021

Efficient processing and feature extraction of largescale point clouds are important in related computer vision and cyber-physical systems. This work investigates point cloud resampling based on hypergraph signal processing (HGSP) to better explore the underlying relationship among different cloud points and to extract contour-enhanced features. Specifically, we design hypergraph spectral filters to capture multi-lateral interactions among the signal nodes of point clouds and to better preserve their surface outlines. Without the need and the computation to first construct the underlying hypergraph, our low complexity approach directly estimates hypergraph spectrum of point clouds by leveraging hypergraph stationary processes from the observed 3D coordinates. Evaluating the proposed resampling methods with several metrics, our test results validate the high efficacy of hypergraph characterization of point clouds and demonstrate the robustness of hypergraph-based resampling under noisy observations.
Using Relative Lines of Code to Guide Automated Test Generation for Python
Venue: arXiv cs.SE 2021
Year: 2021

Raw lines of code (LOC) is a metric that does not, at first glance, seem extremely useful for automated test generation. It is both highly language-dependent and not extremely meaningful, semantically, within a language: one coder can produce the same effect with many fewer lines than another. However, relative LOC, between components of the same project, turns out to be a highly useful metric for automated testing. In this paper, we make use of a heuristic based on LOC counts for tested functions to dramatically improve the effectiveness of automated test generation. This approach is particularly valuable in languages where collecting code coverage data to guide testing has a very high overhead.We apply the heuristic to property-based Python testing using the TSTL (Template Scripting Testing Language) tool. In our experiments, the simple LOC heuristic can improve branch and statement coverage by large margins (often more than 20%, up to 40% or more), and improve fault detection by an even larger margin (usually more than 75%, and up to 400% or more). The LOC heuristic is also easy to combine with other approaches, and is comparable to, and possibly more effective than, two well-established approaches for guiding random testing.
On Satisfying the Android OS Community: User Feedback Still Central to Developers' Portfolios
Venue: arXiv cs.SE 2021
Year: 2021

End-users play an integral role in identifying requirements, validating software features' usefulness, locating defects, and in software product evolution in general. Their role in these activities is especially prominent in online application distribution platforms (OADPs), where software is developed for many potential users, and for which the traditional processes of requirements gathering and negotiation with a single group of end-users do not apply. With such vast access to end-users, however, comes the challenge of how to prioritize competing requirements in order to satisfy previously unknown user groups, especially with early releases of a product. One highly successful product that has managed to overcome this challenge is the Android Operating System (OS). While the requirements of early versions of the Android OS likely benefited from market research, new features in subsequent releases appear to have benefitted extensively from user reviews. Thus, lessons learned about how Android developers have managed to satisfy the user community over time could usefully inform other software products. We have used data mining and natural language processing (NLP) techniques to investigate the issues that were logged by the Android community, and how Google's remedial efforts correlated with users' requests. We found very strong alignment between end-users' top feature requests and Android developers' responses, particularly for the more recent Android releases. Our findings suggest that effort spent responding to end-users' loudest calls may be integral to software systems' survival, and a product's overall success.
JITLine: A Simpler, Better, Faster, Finer-grained Just-In-Time Defect Prediction
Venue: arXiv cs.SE 2021
Year: 2021

A Just-In-Time (JIT) defect prediction model is a classifier to predict if a commit is defect-introducing. Recently, CC2Vec -- a deep learning approach for Just-In-Time defect prediction -- has been proposed. However, CC2Vec requires the whole dataset (i.e., training + testing) for model training, assuming that all unlabelled testing datasets would be available beforehand, which does not follow the key principles of just-in-time defect predictions. Our replication study shows that, after excluding the testing dataset for model training, the F-measure of CC2Vec is decreased by 38.5% for OpenStack and 45.7% for Qt, highlighting the negative impact of excluding the testing dataset for Just-In-Time defect prediction. In addition, CC2Vec cannot perform fine-grained predictions at the line level (i.e., which lines are most risky for a given commit). In this paper, we propose JITLine -- a Just-In-Time defect prediction approach for predicting defect-introducing commits and identifying lines that are associated with that defect-introducing commit (i.e., defective lines). Through a case study of 37,524 commits from OpenStack and Qt, we find that our JITLine approach is at least 26%-38% more accurate (F-measure), 17%-51% more cost-effective (PCI@20%LOC), 70-100 times faster than the state-of-the-art approaches (i.e., CC2Vec and DeepJIT) and the fine-grained predictions at the line level by our approach are 133%-150% more accurate (Top-10 Accuracy) than the baseline NLP approach. Therefore, our JITLine approach may help practitioners to better prioritize defect-introducing commits and better identify defective lines.
Predicting Community Smells' Occurrence on Individual Developers by Sentiments
Venue: arXiv cs.SE 2021
Year: 2021

Community smells appear in sub-optimal software development community structures, causing unforeseen additional project costs, e.g., lower productivity and more technical debt. Previous studies analyzed and predicted community smells in the granularity of community sub-groups using socio-technical factors. However, refactoring such smells requires the effort of developers individually. To eliminate them, supportive measures for every developer should be constructed according to their motifs and working states. Recent work revealed developers' personalities could influence community smells' variation, and their sentiments could impact productivity. Thus, sentiments could be evaluated to predict community smells' occurrence on them. To this aim, this paper builds a developer-oriented and sentiment-aware community smell prediction model considering 3 smells such as Organizational Silo, Lone Wolf, and Bottleneck. Furthermore, it also predicts if a developer quitted the community after being affected by any smell. The proposed model achieves cross- and within-project prediction F-Measure ranging from 76% to 93%. Research also reveals 6 sentimental features having stronger predictive power compared with activeness metrics. Imperative and indicative expressions, politeness, and several emotions are the most powerful predictors. Finally, we test statistically the mean and distribution of sentimental features. Based on our findings, we suggest developers should communicate in a straightforward and polite way.
Performance Exploration of Virtualization Systems
Venue: arXiv cs.DC 2021
Year: 2021

Virtualization has gained astonishing popularity in recent decades. It is applied in several application domains, including mainframes, personal computers, data centers, and embedded systems. While the benefits of virtualization are no longer to be demonstrated, it often comes at the price of performance degradation compared to native execution. In this work, we conduct a comparative study on the performance outcome of VMWare, KVM, and Docker against compute-intensive, IO-intensive, and system benchmarks. The experiments reveal that containers are the way-to-go for the fast execution of applications. It also shows that VMWare and KVM perform similarly on most of the benchmarks.
On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models
Venue: arXiv cs.LG 2021
Year: 2021

With an increase in low-cost machine learning APIs, advanced machine learning models may be trained on private datasets and monetized by providing them as a service. However, privacy researchers have demonstrated that these models may leak information about records in the training dataset via membership inference attacks. In this paper, we take a closer look at another inference attack reported in literature, called attribute inference, whereby an attacker tries to infer missing attributes of a partially known record used in the training dataset by accessing the machine learning model as an API. We show that even if a classification model succumbs to membership inference attacks, it is unlikely to be susceptible to attribute inference attacks. We demonstrate that this is because membership inference attacks fail to distinguish a member from a nearby non-member. We call the ability of an attacker to distinguish the two (similar) vectors as strong membership inference. We show that membership inference attacks cannot infer membership in this strong setting, and hence inferring attributes is infeasible. However, under a relaxed notion of attribute inference, called approximate attribute inference, we show that it is possible to infer attributes close to the true attributes. We verify our results on three publicly available datasets, five membership, and three attribute inference attacks reported in literature.
Inductive Relation Prediction by BERT
Venue: arXiv cs.CL 2021
Year: 2021

Relation prediction in knowledge graphs is dominated by embedding based methods which mainly focus on the transductive setting. Unfortunately, they are not able to handle inductive learning where unseen entities and relations are present and cannot take advantage of prior knowledge. Furthermore, their inference process is not easily explainable. In this work, we propose an all-in-one solution, called BERTRL (BERT-based Relational Learning), which leverages pre-trained language model and fine-tunes it by taking relation instances and their possible reasoning paths as training samples. BERTRL outperforms the SOTAs in 15 out of 18 cases in both inductive and transductive settings. Meanwhile, it demonstrates strong generalization capability in few-shot learning and is explainable.
Pairwise Symmetry Reasoning for Multi-Agent Path Finding Search
Venue: arXiv cs.AI 2021
Year: 2021

Multi-Agent Path Finding (MAPF) is a challenging combinatorial problem that asks us to plan collision-free paths for a team of cooperative agents. In this work, we show that one of the reasons why MAPF is so hard to solve is due to a phenomenon called pairwise symmetry, which occurs when two agents have many different paths to their target locations, all of which appear promising, but every combination of them results in a collision. We identify several classes of pairwise symmetries and show that each one arises commonly in practice and can produce an exponential explosion in the space of possible collision resolutions, leading to unacceptable runtimes for current state-of-the-art (bounded-sub)optimal MAPF algorithms. We propose a variety of reasoning techniques that detect the symmetries efficiently as they arise and resolve them by using specialized constraints to eliminate all permutations of pairwise colliding paths in a single branching step. We implement these ideas in the context of the leading optimal MAPF algorithm CBS and show that the addition of the symmetry reasoning techniques can have a dramatic positive effect on its performance - we report a reduction in the number of node expansions by up to four orders of magnitude and an increase in scalability by up to thirty times. These gains allow us to solve to optimality a variety of challenging MAPF instances previously considered out of reach for CBS.
Goal-Driven Autonomous Exploration Through Deep Reinforcement Learning
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we present an autonomous navigation system for goal-driven exploration of unknown environments through deep reinforcement learning (DRL). Points of interest (POI) for possible navigation directions are obtained from the environment and an optimal waypoint is selected, based on the available data. Following the waypoints, the robot is guided towards the global goal and the local optimum problem of reactive navigation is mitigated. Then, a motion policy for local navigation is learned through a DRL framework in a simulation. We develop a navigation system where this learned policy is integrated into a motion planning stack as the local navigation layer to move the robot between waypoints towards a global goal. The fully autonomous navigation is performed without any prior knowledge while a map is recorded as the robot moves through the environment. Experiments show that the proposed method has an advantage over similar exploration methods, without reliance on a map or prior information in complex static as well as dynamic environments.
UIEC^2-Net: CNN-based Underwater Image Enhancement Using Two Color Space
Venue: arXiv cs.CV 2021
Year: 2021

Underwater image enhancement has attracted much attention due to the rise of marine resource development in recent years. Benefit from the powerful representation capabilities of Convolution Neural Networks(CNNs), multiple underwater image enhancement algorithms based on CNNs have been proposed in the last few years. However, almost all of these algorithms employ RGB color space setting, which is insensitive to image properties such as luminance and saturation. To address this problem, we proposed Underwater Image Enhancement Convolution Neural Network using 2 Color Space (UICE^2-Net) that efficiently and effectively integrate both RGB Color Space and HSV Color Space in one single CNN. To our best knowledge, this method is the first to use HSV color space for underwater image enhancement based on deep learning. UIEC^2-Net is an end-to-end trainable network, consisting of three blocks as follow: a RGB pixel-level block implements fundamental operations such as denoising and removing color cast, a HSV global-adjust block for globally adjusting underwater image luminance, color and saturation by adopting a novel neural curve layer, and an attention map block for combining the advantages of RGB and HSV block output images by distributing weight to each pixel. Experimental results on synthetic and real-world underwater images show the good performance of our proposed method in both subjective comparisons and objective metrics. The code are available at https://github.com/BIGWangYuDong/UWEnhancement.
Block sparse signal recovery via minimizing the block $q$-ratio sparsity
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we propose a method for block sparse signal recovery that minimizes the block $q$-ratio sparsity $\left(\lVert z\rVert_{2,1}/\lVert z\rVert_{2,q}\right)^{\frac{q}{q-1}}$ with $q\in[0,\infty]$. For the case of $1<q\leq\infty$, we present the theoretical analyses and the computing algorithms for both cases of the $\ell_2$-bounded and $\ell_{2,\infty}$-bounded noises. The corresponding unconstrained model is also investigated. Its superior performance in block sparse signal reconstruction is demonstrated by numerical experiments.
Neural Architecture Search based on Cartesian Genetic Programming Coding Method
Venue: arXiv cs.NE 2021
Year: 2021

Neural architecture search (NAS) is a hot topic in the field of automated machine learning and outperforms humans in designing neural architectures on quite a few machine learning tasks. Motivated by the natural representation form of neural networks by the Cartesian genetic programming (CGP), we propose an evolutionary approach of NAS based on CGP, called CGPNAS, to solve sentence classification task. To evolve the architectures under the framework of CGP, the operations such as convolution are identified as the types of function nodes of CGP, and the evolutionary operations are designed based on Evolutionary Strategy. The experimental results show that the searched architectures are comparable with the performance of human-designed architectures. We verify the ability of domain transfer of our evolved architectures. The transfer experimental results show that the accuracy deterioration is lower than 2-5%. Finally, the ablation study identifies the Attention function as the single key function node and the linear transformations along could keep the accuracy similar with the full evolved architectures, which is worthy of investigation in the future.