Reviewerly/exHarmony huggingface dataset:
Self-supervised Distillation for Computer Vision Onboard Planetary Robots

In situ exploration of planets beyond Mars will largely depend on autonomous robotic agents for the foreseeable future. These autonomous planetary explorers need to perceive and understand their surroundings in order to make decisions that maximize science return and minimize risk. Deep learning has demonstrated strong performance on a variety of computer vision and image processing tasks, and has become the main approach for powering terrestrial autonomous systems from robotic vacuum cleaners to self-driving cars. However, deep learning systems require significant volumes of annotated data to optimize the models' parameters, which is a luxury not afforded by in situ missions to new locations in our Solar Sys-tem. Moreover, space-qualified hardware used on robotic space missions relies on legacy technologies due to power constraints and extensive flight qualification requirements (e.g., radiation tolerance), resulting in computational limitations that prevent the use of deep learning models for real-time robotic perception tasks (e.g., obstacle detection, terrain segmentation). In this paper, we address these two challenges by leveraging self-supervised distillation to train small, efficient deep learning models that can match or outperform state-of-the-art results obtained by significantly larger models on Mars image classification and terrain segmentation tasks. Using a set of 100,000 unlabeled images taken by Curiosity and large self-supervised vision models, we distill a variety of small model architectures and evaluate their performance on the published test sets for the MSL classification benchmark and the AI4Mars segmentation benchmark. Experimental results show that on the MSL v2.1 classification task, the best-performing student ResNet-18 model is able to achieve a model compression ratio of 5.2 when distilled from a pretrained ResNet-152 teacher model. In addition, we show that using in-domain images for distillation and increasing the dataset size for distillation has a positive effect on downstream vision tasks. Overall, results indicate that self-supervised distillation enables small models to achieve state-of-the-art performance on the benchmark datasets, supporting the feasibility of performing real-time inference using these small distilled models on next-generation flight hardware such as the High Performance Spaceflight Computer (HPSC).
Artificial Potential Field-Based Path Planning for Cluttered Environments

In this paper, we study path planning algorithms of resource constrained mobile agents in unknown cluttered environments, which include but are not limited to various terrestrial missions e.g., search and rescue missions by drones in jungles, and space missions e.g., navigation of rovers on the Moon. In particular, we focus our attention on artificial potential field (APF) based methods, in which the target is attractive while the obstacles are repulsive to the mobile agent. In this paper, we propose two major updates to the classical APF algorithm which significantly improve the performance of path planning using APF. First, we propose to improve an existing classical method that replaces the gradient descent optimization of the potential field cost function on a continuous domain with a combinatorial optimization on a set of predefined points (called bacteria points) around the agent's current location. Our proposition includes an adaptive hyperparameter that changes the value of the potential function associated to each bacteria point based on the current environmental measurements. Our proposed solution improves the navigation performance in terms of convergence to the target at the expense of minimal increase in computational complexity. Second, we propose an improved potential field cost function of the bacteria points by introducing a new branching cost function which further improves the navigation performance. The algorithms were tested on a set of Monte Carlo simulation trials where the environment changes for each trial. Our simulation results show 25% lower navigation time and around 300% higher success rate compared to the conventional potential field method, and we present future directions for research.
Cross-Language Call Graph Construction Supporting Different Host Languages

Modern software systems are increasingly multi-lingual, which consist of components developed in different programming languages to reuse existing libraries and com-bine language features. Foreign function interface (FFI) is a mechanism that enables interoperation between a host language and a guest language. CFFI interoperating with external C is part of the language standard for almost all languages. For example, Python/C API is the CFFI between Python and C/C++. Python host with C guest can achieve both productivity and performance, and is widely used by many mainstream software systems in different application domains. The popularity of these software systems makes high demands on program analysis of multilingual codebases. A fundamental challenge is to construct call graphs that capture the connectivity between host and guest languages. In this work, we present a novel approach to call graph construction for calls from different host languages to C/C++ foreign functions. The semantics of the foreign function declaration interfaces are modeled to establish cross-language call relationships, taking into account the semantic abstraction to support different host languages. Graph transformation and function node fusions are defined to build the complete call graphs. We demonstrate experimentally that static call graphs for Python and JavaScript calling C/C++ can be constructed effectively and automatically. The call graph construction can further efficiently work as an IDE language server and integrate with other tools.
Multi-Modal API Recommendation

Too many options can be a problem, which is the case for Application Programming Interfaces (APIs). As there are many such APIs, with many more being introduced periodically, it raises the problem of choosing which API to be recommended. Furthermore, numerous APIs are commonly used together with other complementary third-party APIs. It can be challenging for developers to understand how to use each API and to remember all the complementary APIs for the API they want to use. Therefore, an accurate API recommendation approach can improve developers' efficiency in implementing certain functionality. Several approaches have been developed to automatically recommend APIs based on either a natural language query or source code context However, none of these API recommendation approaches have utilized these two sources of information at the same time (i.e., leveraging natural language query and source code context together). In this work, we propose an approach named MulARec, which leverages the information from natural language query (annotation) and source code context The results confirm that our approach outperforms state-of-the-art API recommendation approaches which only leverage a single type of information as the input Our work also demonstrates that multi-modal information can boost the performance of API recommendation approaches by 20%-50% better in terms of BLEU-score than the baselines.
Weak Labelling for File-level Source Code Classification

Software repository hosting services contain large amounts of open-source software, with GitHub hosting over 200 million repositories, from new to established ones. However, these repositories are not easy to find, calling for various attempts to classify their application domains automatically. However, most proposed approaches use artifacts, like README files, as a proxy for the project, losing the information in the source code and the interaction between files. Furthermore, they all focus on the project-level, ignoring the decomposition of software projects into components and modules.This work presents a weak labelling approach based on keyword extraction to annotate source files in a software project.Our findings suggest that using keywords to perform file-level annotations is an effective approach that can capture enough information from the source file so that new labels can be predicted.The long-term goal of our research is to classify source code files and use these annotations to identify semantic components in software projects. In addition, these annotations can be used for semantic reverse engineering, software reuse, and more. We plan to train machine learning models that use our proposed weak supervision to better annotate source files inside software projects.
Characterize Software Release Notes of GitHub Projects: Structure, Writing Style, and Content

Release notes (RNs) summarize important changes between two successive software releases to facilitate software upgrades and serve as means of communication between software and its users. However, existing research has shown that many users cannot extract the information they want from RNs effectively and efficiently due to poor structure and insufficient content. Many efforts have been devoted to categorizing documented information in RNs, however, how exactly RNs are organized, in what way RNs are written, and what is written in RNs with respect to project domains and release types remain under investigation. To bridge this knowledge gap, we manually analyzed 612 RNs from 233 top popular GitHub projects to characterize their Structure, Writing Style, and Content. We find 64.54% of RNs organize changes into hierarchical structures following three strategies, i.e., by Change Type, Affected Module, or Change Priority. And 11.60% of RNs adopt multiple strategies to present the changes. Among the three strategies to organize changes, by Change Type is mostly adopted. RNs of major releases and System Software are more likely to organize changes by Affected Module. We also find three types of Writing Styles: Expository, Descriptive, and Persuasive with increasing explanation information and manual effort, taking 30.07%, 34.80%, and 35.13% of RNs, respectively. 83.10% of RNs in System Software projects adopt the Descriptive and Persuasive writing style. For Content, we find System Software and Libraries & Frameworks projects are more likely to record Breaking Changes, while Software Tools projects emphasize Enhancements. Besides, Fixed Bugs and Security Changes are less common in RNs of major releases. Our findings not only serve practitioners with a roadmap for customizing high-quality RNs but also shed light on future research on automating RN.
D-ACT: Towards Diff-Aware Code Transformation for Code Review Under a Time-Wise Evaluation

Code review is a software quality assurance practice, yet remains time-consuming (e.g., due to slow feedback from reviewers). Recent Neural Machine Translation (NMT)-based code transformation approaches were proposed to automatically generate an approved version of changed methods for a given submitted patch. The existing approaches could change code tokens in any area in a changed method. However, not all code tokens need to be changed. Intuitively, the changed code tokens in the method should be paid more attention to than the others as they are more prone to be defective. In this paper, we present an NMT-based Diff-Aware Code Transformation approach (DACT) by leveraging token-level change information to enable the NMT models to better focus on the changed tokens in a changed method. We evaluate our D-ACT and the baseline approaches based on a time-wise evaluation (that is ignored by the existing work) with 5,758 changed methods. Under the time-wise evaluation scenario, our results show that (1) D-ACT can correctly transform 107 - 245 changed methods, which is at least 62% higher than the existing approaches; (2) the performance of the existing approaches drops by 57% to 94% when the time-wise evaluation is ignored; and (3) D-ACT is improved by 17% - 82% with an average of 29% when considering the token-level change information. Our results suggest that (1) NMT-based code transformation approaches for code review should be evaluated under the time-wise evaluation; and (2) the token-level change information can substantially improve the performance of NMTbased code transformation approaches for code review.
Self-Admitted Library Migrations in Java, JavaScript, and Python Packaging Ecosystems: A Comparative Study

Reusing open-source software libraries has become the norm in modern software development, but libraries can fail due to various reasons, e.g., security vulnerabilities, lacking features, and end of maintenance. In some cases, developers need to replace a library with another competent library with similar functionalities, i.e., library migration. Previous studies have leveraged library migrations as a unique lens of observation to reveal insights into library selection and dependency management in general. However, they are heavily biased toward Java while the generalizability of their findings remains unknown.In this paper, we present a comparative study on self-admitted library migrations (SALMs) from three packaging ecosystems: Java/Maven, JavaScript/npm, and Python/PyPI. For this study, we design a set of semi-automatic methods that accurately locate SALMs, their domains, and their rationales from git repositories. We reveal that SALMs are prevalent and highly unidirectional in all three ecosystems, and the underlying rationales can be well covered by a previous theoretical framework. Also, SALMs in these ecosystems present domain similarity (testing frameworks, web frameworks, HTTP clients, and serialization). However, we observe differences in the longitudinal trends, the distributions of rationales, the ecosystem-specific domains, and the levels of unidirectionality, all of which indicate that Python/PyPI sees increasingly intense competition between libraries and deserves more research on library recommendation and migration.
An Experience Report on Regression-Free Repair of Deep Neural Network Model

Systems based on Deep Neural Networks (DNNs) are increasingly being used in industry. In the process of system operation, DNNs need to be updated in order to improve their performance. When updating DNNs, systems used in companies that require high reliability must have as few regressions as possible. Since the update of DNNs has a data-driven nature, it is difficult to suppress regressions as expected by developers. This paper identifies the requirements for DNN updating in industry and presents a case study using techniques to meet those requirements. In the case study, we worked on satisfying the requirement to update models trained on car images collected in Fujitsu assuming security applications without regression for a specific class. We were able to suppress regression by customizing the objective function based on NeuRecover, a DNN repair technique. Moreover, we discuss some of the challenges identified in the case study.
Interpretable Software Defect Prediction Incorporating Multiple Rules

Software defect prediction models are of great importance in software testing, however, they also face the problem of model uninterpretability. Association rules have good accuracy and interpretability, being widely used in interpretable rule mining scenarios, but there are some common problems with current research: 1) Data unbalance seriously affects the accuracy of mined rules; 2) Most studies treat features as equally important and ignore feature contribution degree; 3) Classification by default rules easily reduces the accuracy of defect classification. Therefore, in the class unbalance scenario, we propose a weighted association rule based on the contribution degree of features, which solves the problem that defective rules are difficult to mine and considers the contribution degree of features. The process of rule generation, ranking, pruning and prediction is optimized according to the weighted support of the rules, and an ensemble model incorporating multiple rules is built. Experimental results on the PROMISE dataset show that the model proposed in this paper obtains an average F1 and MCC improvement of 6.4 % and 9.8 %, respectively, compared with current state-of-the-art classifiers; in terms of interpretability, rule-based interpretation in this paper can provide developers with better guidance on defect repair and risk avoidance compared with model-agnostic methods. From the experimental results, it can be concluded that the contribution degree of features helps to improve the quality of the rule set, and the construction of diversified rules can improve the accuracy of rule prediction.
Exploring and Repairing Gender Fairness Violations in Word Embedding-based Sentiment Analysis Model through Adversarial Patches

With the advancement of sentiment analysis (SA) models and their incorporation into our daily lives, fairness testing on these models is crucial, since unfair decisions can cause discrimination to a large population. Nevertheless, some challenges in fairness testing include the unknown oracle, the difficulty in generating suitable test inputs, and the lack of a reliable way of fixing the issues. To fill in these gaps, BiasRV, a tool based on metamorphic testing (MT), was introduced and succeeded in uncovering fairness issues in a transformer-based model. However, the extent of unfairness in other SA models has not been thoroughly investigated. Our work conducts a more comprehensive empirical study to reveal the extent of fairness violations, specifically gender fairness, exhibited by other popular word embedding-based SA models. We define fairness violation as the behavior in which an SA model predicts variants created from a text, which merely differ in gender classes, to have different sentiments. Our inspection utilizing BiasRV uncovers at least 30 fairness violations (at BiasRV’s default threshold) in all three SA models. Realizing the importance of addressing such significant violations, we introduce adversarial patches (AP) as a way of patch generation in an automated program repair (APR) system to fix them. We adopt adversarial fine-tuning in AP by retraining SA models using adversarial examples, which are bias-uncovering test cases dynamically generated by a tool named BiasFinder at runtime. Evaluation of the SA models shows that our proposed AP reduces fairness violations by at least 25%.
An Empirical Investigation on the Performance of Domain Adaptation for T5 Code Completion

Code completion has the benefit of improving coding speed and reducing the chance of inducing bugs. In recent years, DL-based code completion techniques have been proposed. In particular, pre-trained models have shown outstanding performance because they can complete code by considering the context before and after it is completed. While the model can generate the set of candidate codes, some of those might need to be modified by developers because projects can have different coding rules.In this study, to complete code that fits a specific project appropriately, we train the CodeT5 model with additional data from the target project. This fine-tuning approach is called do-main adaptation, and is often used in neural machine translation. Our preliminary experiment observes that our domain-adapted model improves 5.3% of the perfect prediction rate and, 3.4% of the edit distance rate, compared to the fine-tuned model with the out-of-domain dataset. Furthermore, we discover that the improvement is greater with a larger repository size. The model that is trained with a small dataset, however, hardly improves or performs worse.
Cloud Integration in the Educational Sector Using Amazon Web Services

Web Services (AWS) is a prominent cloud service provider in the information technology industry. Businesses in the contemporary period may benefit from its propensity to develop and expand, as well as from its scalability and flexibility. However, it has not yet achieved widespread acceptance in the academic community. In this article, we will cover the fundamentals of AWS, such as interfacing with EC2 instances, and then compare the costs of deploying in the cloud vs on-premises. This will be useful in understanding why AWS is not more widely used in the academic community. The results of this comparison show that when the time frame of the demand is short and the number of individuals served is low, implementation on AWS is more cost-effective than on premises. When the quantity and duration of users in the education sector grow, the consequences change.
One Case of THOUGHT: Industry-University Converged Education Practice on Open Source

Nowadays, more and more data are produced in our life. The rapid development of big data and intelligent computing industry has put forward new demands for talents, while colleges and universities with limited industry experiences are striving to promote the talent cultivation. Therefore, the "Innovative Community of Industry-Education Convergence", named THOUGHT (innovaTive Hub for the cO-development of indUstry and hiGHer educaTion) [1], was founded in 2019. THOUGHT aims to improve education cooperation between teaching staff and industry professionals in the related field. The industry-university converged courses are different from technical trainings in enterprises or the speeches of the leaders. In order to cultivate talents, both sides of industry and education cooperate closely in nearly every aspects in course building, e.g., targets, contents, methods, and assessments. And it is necessary to make the courses systematic for learners in terms of knowledge transmission, ability training and value shaping, so as to make them more competent for challenging work in the future of big data and AI. The industry-university converged courses focus on breaking the barrier of the traditional campus. Compared with the emphasis on theories and ideal experiments of traditional campus, it focuses more on practices and applications. Therefore, it needs to rely on a bridge which is convenient for both sides to participate in. Compared with traditional tools of business platforms, open source platforms are better choices. And open source makes it easier for more learners to stand on the giants’ shoulders. This paper shares the experience of a case of THOUGHT: the industry-university converged course "Foundamentals and Application of Distributed Data System" jointly built by Tsinghua University and Greenplum open source community. The target, team, organization, structure and contents, practice project, and future work of this course are all introduced in this paper, which may be a vivid case for professors and professionals in this field.
Image Stitching in Dynamic Scene for Computer Vision Project-Base Learning

In this paper, we present a novel computer vision course project. The project aims to stitch images captured in the dynamic scene, which is challenging since traditional methods rely on the assumption of static scenes. However, it provides the opportunity to solve the frontier computer vision problem by using the theories and skills taught in the lectures. The proposed project has three milestones that cover most of the content in the course syllabus and require the students to use interdisciplinary CV techniques, including traditional and deep-learning-based ones. We provide a detailed description of the proposed course project, including the goal, milestones, schedule, and evaluation strategy. We believe it can serve as a practical CV education resource for other higher educators by applying to their CV-related courses.