OpenRaiser/Intern-Atlas huggingface dataset:
Is this Change the Answer to that Problem? Correlating Descriptions of Bug and Code Changes for Evaluating Patch Correctness
Venue: arXiv cs.SE 2022
Year: 2022

In this work, we propose a novel perspective to the problem of patch correctness assessment: a correct patch implements changes that "answer" to a problem posed by buggy behaviour. Concretely, we turn the patch correctness assessment into a Question Answering problem. To tackle this problem, our intuition is that natural language processing can provide the necessary representations and models for assessing the semantic correlation between a bug (question) and a patch (answer). Specifically, we consider as inputs the bug reports as well as the natural language description of the generated patches. Our approach, Quatrain, first considers state of the art commit message generation models to produce the relevant inputs associated to each generated patch. Then we leverage a neural network architecture to learn the semantic correlation between bug reports and commit messages. Experiments on a large dataset of 9135 patches generated for three bug datasets (Defects4j, Bugs.jar and Bears) show that Quatrain can achieve an AUC of 0.886 on predicting patch correctness, and recalling 93% correct patches while filtering out 62% incorrect patches. Our experimental results further demonstrate the influence of inputs quality on prediction performance. We further perform experiments to highlight that the model indeed learns the relationship between bug reports and code change descriptions for the prediction. Finally, we compare against prior work and discuss the benefits of our approach.
[Reproducibility Report] Path Planning using Neural A* Search
Venue: arXiv cs.AI 2022
Year: 2022

The following paper is a reproducibility report for "Path Planning using Neural A* Search" published in ICML2 2021 as part of the ML Reproducibility Challenge 2021. The original paper proposes the Neural A* planner, and claims it achieves an optimal balance between the reduction of node expansions and path accuracy. We verify this claim by reimplementing the model in a different framework and reproduce the data published in the original paper. We have also provided a code-flow diagram to aid comprehension of the code structure. As extensions to the original paper, we explore the effects of (1) generalizing the model by training it on a shuffled dataset, (2) introducing dropout, (3) implementing empirically chosen hyperparameters as trainable parameters in the model, (4) altering the network model to Generative Adversarial Networks (GANs) to introduce stochasticity, (5) modifying the encoder from Unet to Unet++, (6) incorporating cost maps obtained from the Neural A* module in other variations of A* search.
Understanding Masked Image Modeling via Learning Occlusion Invariant Feature
Venue: arXiv cs.CV 2022
Year: 2022

Recently, Masked Image Modeling (MIM) achieves great success in self-supervised visual recognition. However, as a reconstruction-based framework, it is still an open question to understand how MIM works, since MIM appears very different from previous well-studied siamese approaches such as contrastive learning. In this paper, we propose a new viewpoint: MIM implicitly learns occlusion-invariant features, which is analogous to other siamese methods while the latter learns other invariance. By relaxing MIM formulation into an equivalent siamese form, MIM methods can be interpreted in a unified framework with conventional methods, among which only a) data transformations, i.e. what invariance to learn, and b) similarity measurements are different. Furthermore, taking MAE (He et al.) as a representative example of MIM, we empirically find the success of MIM models relates a little to the choice of similarity functions, but the learned occlusion invariant feature introduced by masked image -- it turns out to be a favored initialization for vision transformers, even though the learned feature could be less semantic. We hope our findings could inspire researchers to develop more powerful self-supervised methods in computer vision community.
Institutional Collaboration Recommendation: An expertise-based framework using NLP and Network Analysis
Venue: arXiv cs.DL 2022
Year: 2022

The shift from 'trust-based funding' to 'performance-based funding' is one of the factors that has forced institutions to strive for continuous improvement of performance. Several studies have established the importance of collaboration in enhancing the performance of paired institutions. However, identification of suitable institutions for collaboration is sometimes difficult and therefore institutional collaboration recommendation systems can be vital. Currently, there are no well-developed institutional collaboration recommendation systems. In order to bridge this gap, we design a framework that recognizes thematic strengths and core competencies of institutions, which can in turn be used for collaboration recommendations. The framework, based on NLP and network analysis techniques, is capable of determining the strengths of an institution in different thematic areas within a field and thereby determining the core competency and potential core competency areas of that institution. A major advantage of the system is that it can help to determine and improve the research portfolio of an institution within a field through suitable collaboration, which may lead to the overall improvement of the performance of the institution in that field. The framework is demonstrated by analyzing the performance of 195 Indian institutions in the field of 'Computer Science'. Upon validation using standard metrics for novelty, coverage and diversity of recommendation systems, the framework is found to be of sufficient coverage and capable of tossing novel and diverse recommendations. The article thus presents an institutional collaboration recommendation system which can be used by institutions to identify potential collaborators.
GREASE: Generate Factual and Counterfactual Explanations for GNN-based Recommendations
Venue: arXiv cs.IR 2022
Year: 2022

Recently, graph neural networks (GNNs) have been widely used to develop successful recommender systems. Although powerful, it is very difficult for a GNN-based recommender system to attach tangible explanations of why a specific item ends up in the list of suggestions for a given user. Indeed, explaining GNN-based recommendations is unique, and existing GNN explanation methods are inappropriate for two reasons. First, traditional GNN explanation methods are designed for node, edge, or graph classification tasks rather than ranking, as in recommender systems. Second, standard machine learning explanations are usually intended to support skilled decision-makers. Instead, recommendations are designed for any end-user, and thus their explanations should be provided in user-understandable ways. In this work, we propose GREASE, a novel method for explaining the suggestions provided by any black-box GNN-based recommender system. Specifically, GREASE first trains a surrogate model on a target user-item pair and its $l$-hop neighborhood. Then, it generates both factual and counterfactual explanations by finding optimal adjacency matrix perturbations to capture the sufficient and necessary conditions for an item to be recommended, respectively. Experimental results conducted on real-world datasets demonstrate that GREASE can generate concise and effective explanations for popular GNN-based recommender models.
Snowpack Estimation in Key Mountainous Water Basins from Openly-Available, Multimodal Data Sources
Venue: arXiv cs.CV 2022
Year: 2022

Accurately estimating the snowpack in key mountainous basins is critical for water resource managers to make decisions that impact local and global economies, wildlife, and public policy. Currently, this estimation requires multiple LiDAR-equipped plane flights or in situ measurements, both of which are expensive, sparse, and biased towards accessible regions. In this paper, we demonstrate that fusing spatial and temporal information from multiple, openly-available satellite and weather data sources enables estimation of snowpack in key mountainous regions. Our multisource model outperforms single-source estimation by 5.0 inches RMSE, as well as outperforms sparse in situ measurements by 1.2 inches RMSE.
Challenges and Opportunities for Simultaneous Multi-functional Networks in the UHF Bands
Venue: arXiv cs.NI 2022
Year: 2022

Multi-functional wireless networks are rapidly evolving and aspire to become a promising attribute of the upcoming 6G networks. Enabling multiple simultaneous networking functions with a single radio fosters the development of more integrated and simpler equipment, overcoming design and technology barriers inherited from radio systems of the past. We are seeing numerous trends exploiting these features in newly designed radios, such as those operating on the mmWave band. In this article, however, we carefully analyze the challenges and opportunities for multi-functional wireless networks in UHF bands, advocating the reuse of existing infrastructures and technologies, and exploring the possibilities of expanding their functionality without requiring architectural changes. We believe that both modern and legacy technologies can be turned into multi-functional systems if the right scientific and technological challenges are properly addressed. This transformation can foster the development of new applications and extend the useful life of these systems, contributing to a more sustainable digitization by delaying equipment obsolescence.
Distinctive Image Captioning via CLIP Guided Group Optimization
Venue: arXiv cs.CV 2022
Year: 2022

Image captioning models are usually trained according to human annotated ground-truth captions, which could generate accurate but generic captions. In this paper, we focus on generating distinctive captions that can distinguish the target image from other similar images. To evaluate the distinctiveness of captions, we introduce a series of metrics that use large-scale vision-language pre-training model CLIP to quantify the distinctiveness. To further improve the distinctiveness of captioning models, we propose a simple and effective training strategy that trains the model by comparing target image with similar image group and optimizing the group embedding gap. Extensive experiments are conducted on various baseline models to demonstrate the wide applicability of our strategy and the consistency of metric results with human evaluation. By comparing the performance of our best model with existing state-of-the-art models, we claim that our model achieves new state-of-the-art towards distinctiveness objective.
3D Vision with Transformers: A Survey
Venue: arXiv cs.CV 2022
Year: 2022

The success of the transformer architecture in natural language processing has recently triggered attention in the computer vision field. The transformer has been used as a replacement for the widely used convolution operators, due to its ability to learn long-range dependencies. This replacement was proven to be successful in numerous tasks, in which several state-of-the-art methods rely on transformers for better learning. In computer vision, the 3D field has also witnessed an increase in employing the transformer for 3D convolution neural networks and multi-layer perceptron networks. Although a number of surveys have focused on transformers in vision in general, 3D vision requires special attention due to the difference in data representation and processing when compared to 2D vision. In this work, we present a systematic and thorough review of more than 100 transformers methods for different 3D vision tasks, including classification, segmentation, detection, completion, pose estimation, and others. We discuss transformer design in 3D vision, which allows it to process data with various 3D representations. For each application, we highlight key properties and contributions of proposed transformer-based methods. To assess the competitiveness of these methods, we compare their performance to common non-transformer methods on 12 3D benchmarks. We conclude the survey by discussing different open directions and challenges for transformers in 3D vision. In addition to the presented papers, we aim to frequently update the latest relevant papers along with their corresponding implementations at: https://github.com/lahoud/3d-vision-transformers.
EFI: A Toolbox for Feature Importance Fusion and Interpretation in Python
Venue: arXiv cs.LG 2022
Year: 2022

This paper presents an open-source Python toolbox called Ensemble Feature Importance (EFI) to provide machine learning (ML) researchers, domain experts, and decision makers with robust and accurate feature importance quantification and more reliable mechanistic interpretation of feature importance for prediction problems using fuzzy sets. The toolkit was developed to address uncertainties in feature importance quantification and lack of trustworthy feature importance interpretation due to the diverse availability of machine learning algorithms, feature importance calculation methods, and dataset dependencies. EFI merges results from multiple machine learning models with different feature importance calculation approaches using data bootstrapping and decision fusion techniques, such as mean, majority voting and fuzzy logic. The main attributes of the EFI toolbox are: (i) automatic optimisation of ML algorithms, (ii) automatic computation of a set of feature importance coefficients from optimised ML algorithms and feature importance calculation techniques, (iii) automatic aggregation of importance coefficients using multiple decision fusion techniques, and (iv) fuzzy membership functions that show the importance of each feature to the prediction task. The key modules and functions of the toolbox are described, and a simple example of their application is presented using the popular Iris dataset.
QSAM-Net: Rain streak removal by quaternion neural network with self-attention module
Venue: arXiv cs.CV 2022
Year: 2022

Images captured in real-world applications in remote sensing, image or video retrieval, and outdoor surveillance suffer degraded quality introduced by poor weather conditions. Conditions such as rain and mist, introduce artifacts that make visual analysis challenging and limit the performance of high-level computer vision methods. For time-critical applications where a rapid response is necessary, it becomes crucial to develop algorithms that automatically remove rain, without diminishing the quality of the image contents. This article aims to develop a novel quaternion multi-stage multiscale neural network with a self-attention module called QSAM-Net to remove rain streaks. The novelty of this algorithm is that it requires significantly fewer parameters by a factor of 3.98, over prior methods, while improving visual quality. This is demonstrated by the extensive evaluation and benchmarking on synthetic and real-world rainy images. This feature of QSAM-Net makes the network suitable for implementation on edge devices and applications requiring near real-time performance. The experiments demonstrate that by improving the visual quality of images. In addition, object detection accuracy and training speed are also improved.
Almost optimal searching of maximal subrepetitions in a word
Venue: arXiv cs.DS 2022
Year: 2022

For $0<δ<1$ a $δ$-subrepetition in a word is a factor which exponent is less than~2 but is not less than $1+δ$ (the exponent of the factor is the ratio of the factor length to its minimal period). The $δ$-subrepetition is maximal if it cannot be extended to the left or to the right by at least one letter with preserving its minimal period. In the paper we propose an algorithm for searching all maximal $δ$-subrepetitions in a word of length~$n$ in $O(\frac{n}δ\log\frac{1}δ)$ time (the lower bound for this time is $Ω(\frac{n}δ)$).
Share with Me: A Study on a Social Robot Collecting Mental Health Data
Venue: arXiv cs.RO 2022
Year: 2022

Social robots have been used to assist with mental well-being in various ways such as to help children with autism improve on their social skills and executive functioning such as joint attention and bodily awareness. They are also used to help older adults by reducing feelings of isolation and loneliness, as well as supporting mental well-being of teens and children. However, existing work in this sphere has only shown support for mental health through social robots by responding interactively to human activity to help them learn relevant skills. We hypothesize that humans can also get help from social robots in mental well-being by releasing or sharing their mental health data with the social robots. In this paper, we present a human-robot interaction (HRI) study to evaluate this hypothesis. During the five-day study, a total of fifty-five (n=55) participants shared their in-the-moment mood and stress levels with a social robot. We saw a majority of positive results indicating it is worth conducting future work in this direction, and the potential of social robots to largely support mental well-being.
Improving Vaccine Stance Detection by Combining Online and Offline Data
Venue: arXiv cs.SI 2022
Year: 2022

Differing opinions about COVID-19 have led to various online discourses regarding vaccines. Due to the detrimental effects and the scale of the COVID-19 pandemic, detecting vaccine stance has become especially important and is attracting increasing attention. Communication during the pandemic is typically done via online and offline sources, which provide two complementary avenues for detecting vaccine stance. Therefore, this paper aims to (1) study the importance of integrating online and offline data to vaccine stance detection; and (2) identify the critical online and offline attributes that influence an individual's vaccine stance. We model vaccine hesitancy as a surrogate for identifying the importance of online and offline factors. With the aid of explainable AI and combinatorial analysis, we conclude that both online and offline factors help predict vaccine stance.
Aesthetic Attributes Assessment of Images with AMANv2 and DPC-CaptionsV2
Venue: arXiv cs.CV 2022
Year: 2022

Image aesthetic quality assessment is popular during the last decade. Besides numerical assessment, nature language assessment (aesthetic captioning) has been proposed to describe the generally aesthetic impression of an image. In this paper, we propose aesthetic attribute assessment, which is the aesthetic attributes captioning, i.e., to assess the aesthetic attributes such as composition, lighting usage and color arrangement. It is a non-trivial task to label the comments of aesthetic attributes, which limit the scale of the corresponding datasets. We construct a novel dataset, named DPC-CaptionsV2, by a semi-automatic way. The knowledge is transferred from a small-scale dataset with full annotations to large-scale professional comments from a photography website. Images of DPC-CaptionsV2 contain comments up to 4 aesthetic attributes: composition, lighting, color, and subject. Then, we propose a new version of Aesthetic Multi-Attributes Networks (AMANv2) based on the BUTD model and the VLPSA model. AMANv2 fuses features of a mixture of small-scale PCCD dataset with full annotations and large-scale DPCCaptionsV2 dataset with full annotations. The experimental results of DPCCaptionsV2 show that our method can predict the comments on 4 aesthetic attributes, which are closer to aesthetic topics than those produced by the previous AMAN model. Through the evaluation criteria of image captioning, the specially designed AMANv2 model is better to the CNN-LSTM model and the AMAN model.
Disentangled Representation Learning Using ($β$-)VAE and GAN
Venue: arXiv cs.CV 2022
Year: 2022

Given a dataset of images containing different objects with different features such as shape, size, rotation, and x-y position; and a Variational Autoencoder (VAE); creating a disentangled encoding of these features in the hidden space vector of the VAE was the task of interest in this paper. The dSprite dataset provided the desired features for the required experiments in this research. After training the VAE combined with a Generative Adversarial Network (GAN), each dimension of the hidden vector was disrupted to explore the disentanglement in each dimension. Note that the GAN was used to improve the quality of output image reconstruction.
Multi-target Tracking of Zebrafish based on Particle Filter
Venue: arXiv cs.CV 2022
Year: 2022

Zebrafish is an excellent model organism, which has been widely used in the fields of biological experiments, drug screening, and swarm intelligence. In recent years, there are a large number of techniques for tracking of zebrafish involved in the study of behaviors, which makes it attack much attention of scientists from many fields. Multi-target tracking of zebrafish is still facing many challenges. The high mobility and uncertainty make it difficult to predict its motion; the similar appearances and texture features make it difficult to establish an appearance model; it is even hard to link the trajectories because of the frequent occlusion. In this paper, we use particle filter to approximate the uncertainty of the motion. Firstly, by analyzing the motion characteristics of zebrafish, we establish an efficient hybrid motion model to predict its positions; then we establish an appearance model based on the predicted positions to predict the postures of every targets, meanwhile weigh the particles by comparing the difference of predicted pose and observation pose ; finally, we get the optimal position of single zebrafish through the weighted position, and use the joint particle filter to process trajectory linking of multiple zebrafish.
High Recall Data-to-text Generation with Progressive Edit
Venue: arXiv cs.CL 2022
Year: 2022

Data-to-text (D2T) generation is the task of generating texts from structured inputs. We observed that when the same target sentence was repeated twice, Transformer (T5) based model generates an output made up of asymmetric sentences from structured inputs. In other words, these sentences were different in length and quality. We call this phenomenon "Asymmetric Generation" and we exploit this in D2T generation. Once asymmetric sentences are generated, we add the first part of the output with a no-repeated-target. As this goes through progressive edit (ProEdit), the recall increases. Hence, this method better covers structured inputs than before editing. ProEdit is a simple but effective way to improve performance in D2T generation and it achieves the new stateof-the-art result on the ToTTo dataset
Classification of electromagnetic interference induced image noise in an analog video link
Venue: arXiv cs.CV 2022
Year: 2022

With the ever-increasing electrification of the vehicle showing no sign of retreating, electronic systems deployed in automotive applications are subject to more stringent Electromagnetic Immunity compliance constraints than ever before, to ensure the proximity of nearby electronic systems will not affect their operation. The EMI compliance testing of an analog camera link requires video quality to be monitored and assessed to validate such compliance, which up to now, has been a manual task. Due to the nature of human interpretation, this is open to inconsistency. Here, we propose a solution using deep learning models that analyse, and grade video content derived from an EMI compliance test. These models are trained using a dataset built entirely from real test image data to ensure the accuracy of the resultant model(s) is maximised. Starting with the standard AlexNet, we propose four models to classify the EMI noise level