OpenRaiser/Intern-Atlas huggingface dataset:
Deep Metric Learning for Unsupervised Remote Sensing Change Detection
Venue: arXiv cs.CV 2023
Year: 2023

Remote Sensing Change Detection (RS-CD) aims to detect relevant changes from Multi-Temporal Remote Sensing Images (MT-RSIs), which aids in various RS applications such as land cover, land use, human development analysis, and disaster response. The performance of existing RS-CD methods is attributed to training on large annotated datasets. Furthermore, most of these models are less transferable in the sense that the trained model often performs very poorly when there is a domain gap between training and test datasets. This paper proposes an unsupervised CD method based on deep metric learning that can deal with both of these issues. Given an MT-RSI, the proposed method generates corresponding change probability map by iteratively optimizing an unsupervised CD loss without training it on a large dataset. Our unsupervised CD method consists of two interconnected deep networks, namely Deep-Change Probability Generator (D-CPG) and Deep-Feature Extractor (D-FE). The D-CPG is designed to predict change and no change probability maps for a given MT-RSI, while D-FE is used to extract deep features of MT-RSI that will be further used in the proposed unsupervised CD loss. We use transfer learning capability to initialize the parameters of D-FE. We iteratively optimize the parameters of D-CPG and D-FE for a given MT-RSI by minimizing the proposed unsupervised ``similarity-dissimilarity loss''. This loss is motivated by the principle of metric learning where we simultaneously maximize the distance between change pair-wise pixels while minimizing the distance between no-change pair-wise pixels in bi-temporal image domain and their deep feature domain. The experiments conducted on three CD datasets show that our unsupervised CD method achieves significant improvements over the state-of-the-art supervised and unsupervised CD methods. Code available at https://github.com/wgcban/Metric-CD
Classification of Primitive Manufacturing Tasks from Filtered Event Data
Venue: arXiv cs.CV 2023
Year: 2023

Collaborative robots are increasingly present in industry to support human activities. However, to make the human-robot collaborative process more effective, there are several challenges to be addressed. Collaborative robotic systems need to be aware of the human activities to (1) anticipate collaborative/assistive actions, (2) learn by demonstration, and (3) activate safety procedures in shared workspace. This study proposes an action classification system to recognize primitive assembly tasks from human motion events data captured by a Dynamic and Active-pixel Vision Sensor (DAVIS). Several filters are compared and combined to remove event data noise. Task patterns are classified from a continuous stream of event data using advanced deep learning and recurrent networks to classify spatial and temporal features. Experiments were conducted on a novel dataset, the dataset of manufacturing tasks (DMT22), featuring 5 classes of representative manufacturing primitives (PickUp, Place, Screw, Hold, Idle) from 5 participants. Results show that the proposed filters remove about 65\% of all events (noise) per recording, conducting to a classification accuracy up to 99,37\% for subjects that trained the system and 97.08\% for new subjects. Data from a left-handed subject were successfully classified using only right-handed training data. These results are object independent.
Static Analysis for Android GDPR Compliance Assurance
Venue: arXiv cs.SE 2023
Year: 2023

Many Android applications collect data from users. When they do, they must protect this collected data according to the current legal frameworks. Such data protection has become even more important since the European Union rolled out the General Data Protection Regulation (GDPR). App developers have limited tool support to reason about data protection throughout their app development process. Although many Android applications state a privacy policy, privacy policy compliance checks are currently manual, expensive, and prone to error. One of the major challenges in privacy audits is the significant gap between legal privacy statements (in English text) and technical measures that Android apps use to protect their user's privacy. In this thesis, we will explore to what extent we can use static analysis to answer important questions regarding data protection. Our main goal is to design a tool based approach that aids app developers and auditors in ensuring data protection in Android applications, based on automated static program analysis.
SUD$^2$: Supervision by Denoising Diffusion Models for Image Reconstruction
Venue: arXiv cs.CV 2023
Year: 2023

Many imaging inverse problems$\unicode{x2014}$such as image-dependent in-painting and dehazing$\unicode{x2014}$are challenging because their forward models are unknown or depend on unknown latent parameters. While one can solve such problems by training a neural network with vast quantities of paired training data, such paired training data is often unavailable. In this paper, we propose a generalized framework for training image reconstruction networks when paired training data is scarce. In particular, we demonstrate the ability of image denoising algorithms and, by extension, denoising diffusion models to supervise network training in the absence of paired training data.
On the Use of Geometric Deep Learning for the Iterative Classification and Down-Selection of Analog Electric Circuits
Venue: arXiv cs.CE 2023
Year: 2023

Many complex engineering systems can be represented in a topological form, such as graphs. This paper utilizes a machine learning technique called Geometric Deep Learning (GDL) to aid designers with challenging, graph-centric design problems. The strategy presented here is to take the graph data and apply GDL to seek the best realizable performing solution effectively and efficiently with lower computational costs. This case study used here is the synthesis of analog electrical circuits that attempt to match a specific frequency response within a particular frequency range. Previous studies utilized an enumeration technique to generate 43,249 unique undirected graphs presenting valid potential circuits. Unfortunately, determining the sizing and performance of many circuits can be too expensive. To reduce computational costs with a quantified trade-off in accuracy, the fraction of the circuit graphs and their performance are used as input data to a classification-focused GDL model. Then, the GDL model can be used to predict the remainder cheaply, thus, aiding decision-makers in the search for the best graph solutions. The results discussed in this paper show that additional graph-based features are useful, favorable total set classification accuracy of 80\% in using only 10\% of the graphs, and iteratively-built GDL models can further subdivide the graphs into targeted groups with medians significantly closer to the best and containing 88.2 of the top 100 best-performing graphs on average using 25\% of the graphs.
ChameleonIDE: Untangling Type Errors Through Interactive Visualization and Exploration
Venue: arXiv cs.HC 2023
Year: 2023

Dynamically typed programming languages are popular in education and the software industry. While presenting a low barrier to entry, they suffer from run-time type errors and longer-term problems in code quality and maintainability. Statically typed languages, while showing strength in these aspects, lack in learnability and ease of use. In particular, fixing type errors poses challenges to both novice users and experts. Further, compiler-type error messages are presented in a static way that is biased toward the first occurrence of the error in the program code. To help users resolve such type errors, we introduce ChameleonIDE, a type debugging tool that presents type errors to the user in an unbiased way, allowing them to explore the full context of where the errors could occur. Programmers can interactively verify the steps of reasoning against their intention. Through three studies involving real programmers, we showed that ChameleonIDE is more effective in fixing type errors than traditional text-based error messages. This difference is more significant in harder tasks. Further, programmers actively using ChameleonIDE's interactive features are shown to be more efficient in fixing type errors than passively reading the type error output.
What Do Users Ask in Open-Source AI Repositories? An Empirical Study of GitHub Issues
Venue: arXiv cs.SE 2023
Year: 2023

Artificial Intelligence systems, which benefit from the availability of large-scale datasets and increasing computational power, have become effective solutions to various critical tasks, such as natural language understanding, speech recognition, and image processing. The advancement of these AI systems is inseparable from open-source software (OSS). This paper presents an empirical study that investigates the issues in the repositories of open-source AI repositories to assist developers in understanding problems during the process of employing AI systems. We collect 576 repositories from the PapersWithCode platform. Among these repositories, we find 24,953 issues by utilizing GitHub REST APIs. Our empirical study includes three phases. First, we manually analyze these issues to categorize the problems that developers are likely to encounter in open-source AI repositories. Specifically, we provide a taxonomy of 13 categories related to AI systems. The two most common issues are runtime errors (23.18%) and unclear instructions (19.53%). Second, we see that 67.5% of issues are closed. We also find that half of these issues resolve within four days. Moreover, issue management features, e.g., label and assign, are not widely adopted in open-source AI repositories. In particular, only 7.81% and 5.9% of repositories label issues and assign these issues to assignees, respectively. Finally, we empirically show that employing GitHub issue management features and writing issues with detailed descriptions facilitate the resolution of issues. Based on our findings, we make recommendations for developers to help better manage the issues of open-source AI repositories and improve their quality.
MMFace4D: A Large-Scale Multi-Modal 4D Face Dataset for Audio-Driven 3D Face Animation
Venue: arXiv cs.CV 2023
Year: 2023

Audio-Driven Face Animation is an eagerly anticipated technique for applications such as VR/AR, games, and movie making. With the rapid development of 3D engines, there is an increasing demand for driving 3D faces with audio. However, currently available 3D face animation datasets are either scale-limited or quality-unsatisfied, which hampers further developments of audio-driven 3D face animation. To address this challenge, we propose MMFace4D, a large-scale multi-modal 4D (3D sequence) face dataset consisting of 431 identities, 35,904 sequences, and 3.9 million frames. MMFace4D exhibits two compelling characteristics: 1) a remarkably diverse set of subjects and corpus, encompassing actors spanning ages 15 to 68, and recorded sentences with durations ranging from 0.7 to 11.4 seconds. 2) It features synchronized audio and 3D mesh sequences with high-resolution face details. To capture the subtle nuances of 3D facial expressions, we leverage three synchronized RGBD cameras during the recording process. Upon MMFace4D, we construct a non-autoregressive framework for audio-driven 3D face animation. Our framework considers the regional and composite natures of facial animations, and surpasses contemporary state-of-the-art approaches both qualitatively and quantitatively. The code, model, and dataset will be publicly available.
TypeScript's Evolution: An Analysis of Feature Adoption Over Time
Venue: arXiv cs.SE 2023
Year: 2023

TypeScript is a quickly evolving superset of JavaScript with active development of new features. Our paper seeks to understand how quickly these features are adopted by the developer community. Existing work in JavaScript shows the adoption of dynamic language features can be a major hindrance to static analysis. As TypeScript evolves the addition of features makes the underlying standard more and more difficult to keep up with. In our work we present an analysis of 454 open source TypeScript repositories and study the adoption of 13 language features over the past three years. We show that while new versions of the TypeScript compiler are aggressively adopted by the community, the same cannot be said for language features. While some experience strong growth others are rarely adopted by projects. Our work serves as a starting point for future study of the adoption of features in TypeScript. We also release our analysis and data gathering software as open source in the hope it helps the programming languages community.
Decoding algorithm for HL-codes and performance of the DHH-cryptosystem -- a candidate for post-quantum cryptography
Venue: arXiv cs.CR 2023
Year: 2023

We give a decoding algorithm for a class of error-correcting codes, which can be used in the DHH-cryptosystem, which is a candidate for post-quantum cryptography, since it is of McEliece type. Furthermore, we implement the encryption and decryption algorithms for this cryptosystem and investigate its performance.
Moving Target Defense for Service-oriented Mission-critical Networks
Venue: arXiv cs.CR 2023
Year: 2023

Modern mission-critical systems (MCS) are increasingly softwarized and interconnected. As a result, their complexity increased, and so their vulnerability against cyber-attacks. The current adoption of virtualization and service-oriented architectures (SOA) in MCSs provides additional flexibility that can be leveraged to withstand and mitigate attacks, e.g., by moving critical services or data flows. This enables the deployment of strategies for moving target defense (MTD), which allows stripping attackers of their asymmetric advantage from the long reconnaissance of MCSs. However, it is challenging to design MTD strategies, given the diverse threat landscape, resource limitations, and potential degradation in service availability. In this paper, we combine two optimization models to explore feasible service configurations for SOA-based systems and to derive subsequent MTD actions with their time schedule based on an attacker-defender game. Our results indicate that even for challenging and diverse attack scenarios, our models can defend the system by up to 90% of the system operation time with a limited MTD defender budget.
mCPT at SemEval-2023 Task 3: Multilingual Label-Aware Contrastive Pre-Training of Transformers for Few- and Zero-shot Framing Detection
Venue: arXiv cs.CL 2023
Year: 2023

This paper presents the winning system for the zero-shot Spanish framing detection task, which also achieves competitive places in eight additional languages. The challenge of the framing detection task lies in identifying a set of 14 frames when only a few or zero samples are available, i.e., a multilingual multi-label few- or zero-shot setting. Our developed solution employs a pre-training procedure based on multilingual Transformers using a label-aware contrastive loss function. In addition to describing the system, we perform an embedding space analysis and ablation study to demonstrate how our pre-training procedure supports framing detection to advance computational framing analysis.
Contrastive Self-supervised Learning in Recommender Systems: A Survey
Venue: arXiv cs.IR 2023
Year: 2023

Deep learning-based recommender systems have achieved remarkable success in recent years. However, these methods usually heavily rely on labeled data (i.e., user-item interactions), suffering from problems such as data sparsity and cold-start. Self-supervised learning, an emerging paradigm that extracts information from unlabeled data, provides insights into addressing these problems. Specifically, contrastive self-supervised learning, due to its flexibility and promising performance, has attracted considerable interest and recently become a dominant branch in self-supervised learning-based recommendation methods. In this survey, we provide an up-to-date and comprehensive review of current contrastive self-supervised learning-based recommendation methods. Firstly, we propose a unified framework for these methods. We then introduce a taxonomy based on the key components of the framework, including view generation strategy, contrastive task, and contrastive objective. For each component, we provide detailed descriptions and discussions to guide the choice of the appropriate method. Finally, we outline open issues and promising directions for future research.
Using causal inference and Bayesian statistics to explain the capability of a test suite in exposing software faults
Venue: arXiv cs.SE 2023
Year: 2023

Test effectiveness refers to the capability of a test suite in exposing faults in software. It is crucial to be aware of factors that influence this capability. We aim at inferring the causal relationship between the two factors (i.e., Cover/Exec) and the capability of a test suite to expose and discover faults in software. Cover refers to the number of distinct test cases covering the statement and Exec equals the number of times a test suite executes a statement. We analyzed 459166 software faults from {12} Java programs. Bayesian statistics along with the back-door criterion was exploited for the purpose of causal inference. Furthermore, we examined the common pitfall measuring association, the mixture of causal and noncausal relationships, instead of causal association. The results show that Cover is of more causal association as against \textit{Exec}, and the causal association and noncausal one for those variables are statistically different. Software developers could exploit the results to design and write more effective test cases, which lead to discovering more bugs hidden in software.
Extensions of Egghe g-index: Improvements of Hirsch h-index
Venue: arXiv cs.DL 2023
Year: 2023

A few new indices to characterize the scientific output of scientists are defined in the paper. These indices are compared with -index and its alternative indices using some proven assertions. The gd-indices are introduced as extensions of the g-index to define H-index as an improvement of the h-index. Numerous computational results which are conducted indicate the good behaviour of defined indices to evaluate the scientific impact of scientists. There exist very good approximations between some pairs of all considered indices in the sense that related ratios are often very close to 1.
Coverage Analysis of Hybrid RF/THz Networks With Best Relay Selection
Venue: arXiv cs.IT 2023
Year: 2023

Utilizing terahertz (THz) transmission to enhance coverage has proven various benefits compared to traditional radio frequency (RF) counterparts. This letter proposes a dual-hop decode-and-forward (DF) routing protocol in a hybrid RF and THz relay network named hybrid relay selection (HRS). The coverage probability of the HRS protocol is derived. The HRS protocol prioritizes THz relays for higher data rates or short source-destination distances; and RF relays for lower data rates or large source-destination distances. The proposed HRS protocol offers nearly the same performance as the optimal selection protocol, which requires complete instantaneous channel state information (CSI) of all the nodes.
Connectivity with uncertainty regions given as line segments
Venue: arXiv cs.CG 2023
Year: 2023

For a set $Q$ of points in the plane and a real number $δ\ge 0$, let $\mathbb{G}_δ(Q)$ be the graph defined on $Q$ by connecting each pair of points at distance at most $δ$. We consider the connectivity of $\mathbb{G}_δ(Q)$ in the best scenario when the location of a few of the points is uncertain, but we know for each uncertain point a line segment that contains it. More precisely, we consider the following optimization problem: given a set $P$ of $n-k$ points in the plane and a set $S$ of $k$ line segments in the plane, find the minimum $δ\ge 0$ with the property that we can select one point $p_s\in s$ for each segment $s\in S$ and the corresponding graph $\mathbb{G}_δ( P\cup \{ p_s\mid s\in S\})$ is connected. It is known that the problem is NP-hard. We provide an algorithm to exactly compute an optimal solution in $O(f(k) n \log n)$ time, for a computable function $f(\cdot)$. This implies that the problem is FPT when parameterized by $k$. The best previous algorithm uses $O((k!)^k k^{k+1}\cdot n^{2k})$ time and computes the solution up to fixed precision.
Multi-modal Expression Recognition with Ensemble Method
Venue: arXiv cs.CV 2023
Year: 2023

This paper presents our submission to the Expression Classification Challenge of the fifth Affective Behavior Analysis in-the-wild (ABAW) Competition. In our method, multimodal feature combinations extracted by several different pre-trained models are applied to capture more effective emotional information. For these combinations of visual and audio modal features, we utilize two temporal encoders to explore the temporal contextual information in the data. In addition, we employ several ensemble strategies for different experimental settings to obtain the most accurate expression recognition results. Our system achieves the average F1 Score of 0.45774 on the validation set.
On the Reconstructability and Rediscoverability of Typed Jackson Nets (Extended Version)
Venue: arXiv cs.FL 2023
Year: 2023

A process discovery algorithm aims to construct a model from data generated by historical system executions such that the model describes the system well. Consequently, one desired property of a process discovery algorithm is rediscoverability, which ensures that the algorithm can construct a model that is behaviorally equivalent to the original system. A system often simultaneously executes multiple processes that interact through object manipulations. This paper presents a framework for developing process discovery algorithms for constructing models that describe interacting processes based on typed Jackson Nets that use identifiers to refer to the objects they manipulate. Typed Jackson Nets enjoy the reconstructability property which states that the composition of the processes and the interactions of a decomposed typed Jackson Net yields a model that is bisimilar to the original system. We exploit this property to demonstrate that if a process discovery algorithm ensures rediscoverability, the system of interacting processes is rediscoverable.
ShaRPy: Shape Reconstruction and Hand Pose Estimation from RGB-D with Uncertainty
Venue: arXiv cs.CV 2023
Year: 2023

Despite their potential, markerless hand tracking technologies are not yet applied in practice to the diagnosis or monitoring of the activity in inflammatory musculoskeletal diseases. One reason is that the focus of most methods lies in the reconstruction of coarse, plausible poses, whereas in the clinical context, accurate, interpretable, and reliable results are required. Therefore, we propose ShaRPy, the first RGB-D Shape Reconstruction and hand Pose tracking system, which provides uncertainty estimates of the computed pose, e.g., when a finger is hidden or its estimate is inconsistent with the observations in the input, to guide clinical decision-making. Besides pose, ShaRPy approximates a personalized hand shape, promoting a more realistic and intuitive understanding of its digital twin. Our method requires only a light-weight setup with a single consumer-level RGB-D camera yet it is able to distinguish similar poses with only small joint angle deviations in a metrically accurate space. This is achieved by combining a data-driven dense correspondence predictor with traditional energy minimization. To bridge the gap between interactive visualization and biomedical simulation we leverage a parametric hand model in which we incorporate biomedical constraints and optimize for both, its pose and hand shape. We evaluate ShaRPy on a keypoint detection benchmark and show qualitative results of hand function assessments for activity monitoring of musculoskeletal diseases.