OpenRaiser/Intern-Atlas huggingface dataset:
What Makes a Good Commit Message?
Venue: arXiv cs.SE 2022
Year: 2022

A key issue in collaborative software development is communication among developers. One modality of communication is a commit message, in which developers describe the changes they make in a repository. As such, commit messages serve as an "audit trail" by which developers can understand how the source code of a project has changed-and why. Hence, the quality of commit messages affects the effectiveness of communication among developers. Commit messages are often of poor quality as developers lack time and motivation to craft a good message. Several automatic approaches have been proposed to generate commit messages. However, these are based on uncurated datasets including considerable proportions of poorly phrased commit messages. In this multi-method study, we first define what constitutes a "good" commit message, and then establish what proportion of commit messages lack information using a sample of almost 1,600 messages from five highly active open source projects. We find that an average of circa 44% of messages could be improved, suggesting the use of uncurated datasets may be a major threat when commit message generators are trained with such data. We also observe that prior work has not considered semantics of commit messages, and there is surprisingly little guidance available for writing good commit messages. To that end, we develop a taxonomy based on recurring patterns in commit messages' expressions. Finally, we investigate whether "good" commit messages can be automatically identified; such automation could prompt developers to write better commit messages.
Comparison and Combination of Sentence Embeddings Derived from Different Supervision Signals
Venue: arXiv cs.CL 2022
Year: 2022

There have been many successful applications of sentence embedding methods. However, it has not been well understood what properties are captured in the resulting sentence embeddings depending on the supervision signals. In this paper, we focus on two types of sentence embedding methods with similar architectures and tasks: one fine-tunes pre-trained language models on the natural language inference task, and the other fine-tunes pre-trained language models on word prediction task from its definition sentence, and investigate their properties. Specifically, we compare their performances on semantic textual similarity (STS) tasks using STS datasets partitioned from two perspectives: 1) sentence source and 2) superficial similarity of the sentence pairs, and compare their performances on the downstream and probing tasks. Furthermore, we attempt to combine the two methods and demonstrate that combining the two methods yields substantially better performance than the respective methods on unsupervised STS tasks and downstream tasks.
Automatic defect segmentation by unsupervised anomaly learning
Venue: arXiv cs.CV 2022
Year: 2022

This paper addresses the problem of defect segmentation in semiconductor manufacturing. The input of our segmentation is a scanning-electron-microscopy (SEM) image of the candidate defect region. We train a U-net shape network to segment defects using a dataset of clean background images. The samples of the training phase are produced automatically such that no manual labeling is required. To enrich the dataset of clean background samples, we apply defect implant augmentation. To that end, we apply a copy-and-paste of a random image patch in the clean specimen. To improve the robustness of the unlabeled data scenario, we train the features of the network with unsupervised learning methods and loss functions. Our experiments show that we succeed to segment real defects with high quality, even though our dataset contains no defect examples. Our approach performs accurately also on the problem of supervised and labeled defect segmentation.
Partial Encryption after Encoding for Security and Reliability in Data Systems
Venue: arXiv cs.IT 2022
Year: 2022

We consider the problem of secure and reliable communication over a noisy multipath network. Previous work considering a noiseless version of our problem proposed a hybrid universal network coding cryptosystem (HUNCC). By combining an information-theoretically secure encoder together with partial encryption, HUNCC is able to obtain security guarantees, even in the presence of an all-observing eavesdropper. In this paper, we propose a version of HUNCC for noisy channels (N-HUNCC). This modification requires four main novelties. First, we present a network coding construction which is jointly, individually secure and error-correcting. Second, we introduce a new security definition which is a computational analogue of individual security, which we call individual indistinguishability under chosen ciphertext attack (individual IND-CCA1), and show that NHUNCC satisfies it. Third, we present a noise based decoder for N-HUNCC, which permits the decoding of the encoded-thenencrypted data. Finally, we discuss how to select parameters for N-HUNCC and its error-correcting capabilities.
Encoding and Decoding of Several Optimal Rank Metric Codes
Venue: arXiv cs.IT 2022
Year: 2022

This paper presents encoding and decoding algorithms for several families of optimal rank metric codes whose codes are in restricted forms of symmetric, alternating and Hermitian matrices. First, we show the evaluation encoding is the right choice for these codes and then we provide easily reversible encoding methods for each family. Later unique decoding algorithms for the codes are described. The decoding algorithms are interpolation-based and can uniquely correct errors for each code with rank up to $\lfloor(d-1)/2\rfloor$ in polynomial-time, where $d$ is the minimum distance of the code.
Artificial Intelligence based tool wear and defect prediction for special purpose milling machinery using low-cost acceleration sensor retrofits
Venue: arXiv cs.LG 2022
Year: 2022

Milling machines form an integral part of many industrial processing chains. As a consequence, several machine learning based approaches for tool wear detection have been proposed in recent years, yet these methods mostly deal with standard milling machines, while machinery designed for more specialized tasks has gained only limited attention so far. This paper demonstrates the application of an acceleration sensor to allow for convenient condition monitoring of such a special purpose machine, i.e. round seam milling machine. We examine a variety of conditions including blade wear and blade breakage as well as improper machine mounting or insufficient transmission belt tension. In addition, we presents different approaches to supervised failure recognition with limited amounts of training data. Hence, aside theoretical insights, our analysis is of high, practical importance, since retrofitting older machines with acceleration sensors and an on-edge classification setup comes at low cost and effort, yet provides valuable insights into the state of the machine and tools in particular and the production process in general.
Addressing modern and practical challenges in machine learning: A survey of online federated and transfer learning
Venue: arXiv cs.LG 2022
Year: 2022

Online federated learning (OFL) and online transfer learning (OTL) are two collaborative paradigms for overcoming modern machine learning challenges such as data silos, streaming data, and data security. This survey explored OFL and OTL throughout their major evolutionary routes to enhance understanding of online federated and transfer learning. Besides, practical aspects of popular datasets and cutting-edge applications for online federated and transfer learning are highlighted in this work. Furthermore, this survey provides insight into potential future research areas and aims to serve as a resource for professionals developing online federated and transfer learning frameworks.
Effective Variable Depth Local Search for the Budgeted Maximum Coverage Problem
Venue: arXiv cs.DS 2022
Year: 2022

We address the Budgeted Maximum Coverage Problem (BMCP), which is a natural and more practical extension of the standard 0-1 knapsack problem and the set cover problem. Given m elements with nonnegative weights, n subsets of elements with nonnegative costs, and a total budget, BMCP aims to select some subsets such that the total cost of selected subsets does not exceed the budget, and the total weight of associated elements is maximized. In this paper, we propose a variable depth local search algorithm (VDLS) for the BMCP. VDLS first generates an initial solution by a greedy algorithm, then iteratively improves the solution through a partial depth-first search method, that can improve the solution by simultaneously changing the states (selected or not) of multiple subsets. Such method allows VDLS to explore the solution space widely and deeply, and to yield high-quality solutions. We further propose a neighbour structure to boost the algorithm performance, that is, both subsets have a neighbour relation if they have at least one common associated element. By applying the neighbour structure, VDLS can adjust the selected subsets while losing as few covered elements as possible. Since the existing BMCP benchmarks only have simple structures and small scales, we design 60 new instances with relatively large scales and complex structures to enrich the diversity of the BMCP instances. Experimental results on 30 public instances and 60 new instances we designed demonstrate that VDLS significantly outperforms the existing heuristic and the general CPLEX exact solver, for the BMCP.
Transformers in Self-Supervised Monocular Depth Estimation with Unknown Camera Intrinsics
Venue: arXiv cs.CV 2022
Year: 2022

The advent of autonomous driving and advanced driver assistance systems necessitates continuous developments in computer vision for 3D scene understanding. Self-supervised monocular depth estimation, a method for pixel-wise distance estimation of objects from a single camera without the use of ground truth labels, is an important task in 3D scene understanding. However, existing methods for this task are limited to convolutional neural network (CNN) architectures. In contrast with CNNs that use localized linear operations and lose feature resolution across the layers, vision transformers process at constant resolution with a global receptive field at every stage. While recent works have compared transformers against their CNN counterparts for tasks such as image classification, no study exists that investigates the impact of using transformers for self-supervised monocular depth estimation. Here, we first demonstrate how to adapt vision transformers for self-supervised monocular depth estimation. Thereafter, we compare the transformer and CNN-based architectures for their performance on KITTI depth prediction benchmarks, as well as their robustness to natural corruptions and adversarial attacks, including when the camera intrinsics are unknown. Our study demonstrates how transformer-based architecture, though lower in run-time efficiency, achieves comparable performance while being more robust and generalizable.
Conversational Agents: Theory and Applications
Venue: arXiv cs.CL 2022
Year: 2022

In this chapter, we provide a review of conversational agents (CAs), discussing chatbots, intended for casual conversation with a user, as well as task-oriented agents that generally engage in discussions intended to reach one or several specific goals, often (but not always) within a specific domain. We also consider the concept of embodied conversational agents, briefly reviewing aspects such as character animation and speech processing. The many different approaches for representing dialogue in CAs are discussed in some detail, along with methods for evaluating such agents, emphasizing the important topics of accountability and interpretability. A brief historical overview is given, followed by an extensive overview of various applications, especially in the fields of health and education. We end the chapter by discussing benefits and potential risks regarding the societal impact of current and future CA technology.
A Tensor Based Regression Approach for Human Motion Prediction
Venue: arXiv cs.RO 2022
Year: 2022

Collaborative robotic systems will be a key enabling technology for current and future industrial applications. The main aspect of such applications is to guarantee safety for humans. To detect hazardous situations, current commercially available robotic systems rely on direct physical contact to the co-working person. To further advance this technology, there are multiple efforts to develop predictive capabilities for such systems. Using motion tracking sensors and pose estimation systems combined with adequate predictive models, potential episodes of hazardous collisions between humans and robots can be predicted. Based on the provided predictive information, the robotic system can avoid physical contact by adjusting speed or position. A potential approach for such systems is to perform human motion prediction with machine learning methods like Artificial Neural Networks. In our approach, the motion patterns of past seconds are used to predict future ones by applying a linear Tensor-on-Tensor regression model, selected according to a similarity measure between motion sequences obtained by Dynamic TimeWarping. For test and validation of our proposed approach, industrial pseudo assembly tasks were recorded with a motion capture system, providing unique traceable Cartesian coordinates $(x, y, z)$ for each human joint. The prediction of repetitive human motions associated with assembly tasks, whose data vary significantly in length and have highly correlated variables, has been achieved in real time.
TransFollower: Long-Sequence Car-Following Trajectory Prediction through Transformer
Venue: arXiv cs.AI 2022
Year: 2022

Car-following refers to a control process in which the following vehicle (FV) tries to keep a safe distance between itself and the lead vehicle (LV) by adjusting its acceleration in response to the actions of the vehicle ahead. The corresponding car-following models, which describe how one vehicle follows another vehicle in the traffic flow, form the cornerstone for microscopic traffic simulation and intelligent vehicle development. One major motivation of car-following models is to replicate human drivers' longitudinal driving trajectories. To model the long-term dependency of future actions on historical driving situations, we developed a long-sequence car-following trajectory prediction model based on the attention-based Transformer model. The model follows a general format of encoder-decoder architecture. The encoder takes historical speed and spacing data as inputs and forms a mixed representation of historical driving context using multi-head self-attention. The decoder takes the future LV speed profile as input and outputs the predicted future FV speed profile in a generative way (instead of an auto-regressive way, avoiding compounding errors). Through cross-attention between encoder and decoder, the decoder learns to build a connection between historical driving and future LV speed, based on which a prediction of future FV speed can be obtained. We train and test our model with 112,597 real-world car-following events extracted from the Shanghai Naturalistic Driving Study (SH-NDS). Results show that the model outperforms the traditional intelligent driver model (IDM), a fully connected neural network model, and a long short-term memory (LSTM) based model in terms of long-sequence trajectory prediction accuracy. We also visualized the self-attention and cross-attention heatmaps to explain how the model derives its predictions.
ALDER: Unlocking blockchain performance by multiplexing consensus protocols
Venue: arXiv cs.DC 2022
Year: 2022

Most of today's online services (e.g., social networks, search engines, market places) are centralized, which is recognized as unsatisfactory by a majority of users for various reasons (e.g., centralized governance, censorship, loss of control over personal data). Blockchain technologies promise a new Web revolution (Web 3.0) through the decentralization of online services. However, one of the key limitations for this revolution to happen at a planetary scale is the poor performance of today's blockchains. We propose in this paper ALDER, a solution for unlocking the performance of off-the-shelf leader-based blockchains by multiplexing their consensus protocol. Our solution leverages the existence of multiple potential leaders to alleviate the bottleneck that exists at different levels of consensus protocols. To illustrate the benefits it brings to Blockchain performance, we apply ALDER to three representative blockchains, namely Algorand (Proof-of-Stake), RapidChain (Sharding-based) and Bitcoin (Proof-of-Work). Our evaluation, involving up to 10,000 nodes deployed on 100 physical machines, shows that using ALDER can provide up to a 300% improvement in both throughput and latency reduction.
One-Year In: COVID-19 Research at the International Level in CORD-19 Data
Venue: arXiv cs.DL 2022
Year: 2022

The appearance of a novel coronavirus in late 2019 radically changed the community of researchers working on coronaviruses since the 2002 SARS epidemic. In 2020, coronavirus-related publications grew by 20 times over the previous two years, with 130,000 more researchers publishing on related topics. The United States, the United Kingdom and China led dozens of nations working on coronavirus prior to the pandemic, but leadership consolidated among these three nations in 2020, which collectively accounted for 50% of all papers, garnering well more than 60% of citations. China took an early lead on COVID-19 research, but dropped rapidly in production and international participation through the year. Europe showed an opposite pattern, beginning slowly in publications but growing in contributions during the year. The share of internationally collaborative publications dropped from pre-pandemic rates; single-authored publications grew. For all nations, including China, the number of publications about COVID track closely with the outbreak of COVID-19 cases. Lower-income nations participate very little in COVID-19 research in 2020. Topic maps of internationally collaborative work show the rise of patient care and public health clusters, two topics that were largely absent from coronavirus research in the two years prior to 2020. Findings are consistent with global science as a self-organizing system operating on a reputation-based dynamic.
Almost Optimal Proper Learning and Testing Polynomials
Venue: arXiv cs.LG 2022
Year: 2022

We give the first almost optimal polynomial-time proper learning algorithm of Boolean sparse multivariate polynomial under the uniform distribution. For $s$-sparse polynomial over $n$ variables and $ε=1/s^β$, $β>1$, our algorithm makes $$q_U=\left(\frac{s}ε\right)^{\frac{\log β}β+O(\frac{1}β)}+ \tilde O\left(s\right)\left(\log\frac{1}ε\right)\log n$$ queries. Notice that our query complexity is sublinear in $1/ε$ and almost linear in $s$. All previous algorithms have query complexity at least quadratic in $s$ and linear in $1/ε$. We then prove the almost tight lower bound $$q_L=\left(\frac{s}ε\right)^{\frac{\log β}β+Ω(\frac{1}β)}+ Ω\left(s\right)\left(\log\frac{1}ε\right)\log n,$$ Applying the reduction in~\cite{Bshouty19b} with the above algorithm, we give the first almost optimal polynomial-time tester for $s$-sparse polynomial. Our tester, for $β>3.404$, makes $$\tilde O\left(\frac{s}ε\right)$$ queries.
AI-based artistic representation of emotions from EEG signals: a discussion on fairness, inclusion, and aesthetics
Venue: arXiv cs.AI 2022
Year: 2022

While Artificial Intelligence (AI) technologies are being progressively developed, artists and researchers are investigating their role in artistic practices. In this work, we present an AI-based Brain-Computer Interface (BCI) in which humans and machines interact to express feelings artistically. This system and its production of images give opportunities to reflect on the complexities and range of human emotions and their expressions. In this discussion, we seek to understand the dynamics of this interaction to reach better co-existence in fairness, inclusion, and aesthetics.
Probabilistic Consensus on Feature Distribution for Multi-robot Systems with Markovian Exploration Dynamics
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, we present a consensus-based decentralized multi-robot approach to reconstruct a discrete distribution of features, modeled as an occupancy grid map, that represent information contained in a bounded planar 2D environment, such as visual cues used for navigation or semantic labels associated with object detection. The robots explore the environment according to a random walk modeled by a discrete-time discrete-state (DTDS) Markov chain and estimate the feature distribution from their own measurements and the estimates communicated by neighboring robots, using a distributed Chernoff fusion protocol. We prove that under this decentralized fusion protocol, each robot's feature distribution converges to the ground truth distribution in an almost sure sense. We verify this result in numerical simulations that show that the Hellinger distance between the estimated and ground truth feature distributions converges to zero over time for each robot. We also validate our strategy through Software-In-The-Loop (SITL) simulations of quadrotors that search a bounded square grid for a set of visual features distributed on a discretized circle.
Robust Dialogue State Tracking with Weak Supervision and Sparse Data
Venue: arXiv cs.CL 2022
Year: 2022

Generalising dialogue state tracking (DST) to new data is especially challenging due to the strong reliance on abundant and fine-grained supervision during training. Sample sparsity, distributional shift and the occurrence of new concepts and topics frequently lead to severe performance degradation during inference. In this paper we propose a training strategy to build extractive DST models without the need for fine-grained manual span labels. Two novel input-level dropout methods mitigate the negative impact of sample sparsity. We propose a new model architecture with a unified encoder that supports value as well as slot independence by leveraging the attention mechanism. We combine the strengths of triple copy strategy DST and value matching to benefit from complementary predictions without violating the principle of ontology independence. Our experiments demonstrate that an extractive DST model can be trained without manual span labels. Our architecture and training strategies improve robustness towards sample sparsity, new concepts and topics, leading to state-of-the-art performance on a range of benchmarks. We further highlight our model's ability to effectively learn from non-dialogue data.