OpenRaiser/Intern-Atlas huggingface dataset:
Intention Recognition for Multiple Agents
Venue: arXiv cs.AI 2022
Year: 2022

Intention recognition is an important step to facilitate collaboration among multiple agents. Existing work mainly focuses on intention recognition in a single-agent setting and uses a descriptive model, e.g. Bayesian networks, in the recognition process. In this article, we develop a new approach of identifying intentions for multiple agents through a clustering algorithm. We first define an intention model for multiple agents of interest. We use a prescriptive approach to model agents' behaviours where their intentions are hidden in the implementation of their plans. We introduce landmarks into the behavioural model therefore enhancing informative features to identify common intentions for multiple agents. Subsequently, we further refine the model by focusing only action sequences in their plan and provide a light model for identifying and comparing their intentions. The new models provide a simple approach of grouping agents' common intentions upon partial plans observed in agents' interactions. Then, we transform the intention recognition into an un-supervised learning problem and adapt a clustering algorithm to group intentions of multiple agents through comparing their behavioural models. We conduct the clustering process by measuring similarity of probability distributions over potential landmarks in intention models so as to discover agents' common intentions. Finally, we examine the new intention recognition approaches in two problem domains. We demonstrate importance of recognising common intentions of multiple agents in achieving their goals and provide experimental results to show performance of the new approaches.
Designing Enhanced Multi-dimensional Constellations for Code-Domain NOMA
Venue: arXiv cs.IT 2022
Year: 2022

This paper presents an enhanced design of multi-dimensional (MD) constellations which play a pivotal role in many communication systems such as code-domain non-orthogonal multiple access (CD-NOMA). MD constellations are attractive as their structural properties, if properly designed, lead to signal space diversity and hence improved error rate performance. Unlike the existing works which mostly focus on MD constellations with large minimum Euclidean distance (MED), we look for new MD constellations with additional feature that the minimum product distance (MPD) is also large. To this end, a non-convex optimization problem is formulated and then solved by the convex-concave procedure (CCCP). Compared with the state-of-the-art literature, our proposed MD constellations lead to significant error performance enhancement over Rayleigh fading channels whilst maintaining almost the same performance over the Gaussian channels. To demonstrate their application, we also show that these MD constellations give rise to good codebooks in sparse code multiple access systems. All the obtained MD constellations can be found in https://github.com/Aureliano1/Multi-dimensional-constellation.
Windmills of the minds: an algorithm for Fermat's Two Squares Theorem
Venue: arXiv cs.LO 2022
Year: 2022

The two squares theorem of Fermat is a gem in number theory, with a spectacular one-sentence "proof from the Book". Here is a formalisation of this proof, with an interpretation using windmill patterns. The theory behind involves involutions on a finite set, especially the parity of the number of fixed points in the involutions. Starting as an existence proof that is non-constructive, there is an ingenious way to turn it into a constructive one. This gives an algorithm to compute the two squares by iterating the two involutions alternatively from a known fixed point.
PSI: A Pedestrian Behavior Dataset for Socially Intelligent Autonomous Car
Venue: arXiv cs.CV 2022
Year: 2022

Prediction of pedestrian behavior is critical for fully autonomous vehicles to drive in busy city streets safely and efficiently. The future autonomous cars need to fit into mixed conditions with not only technical but also social capabilities. As more algorithms and datasets have been developed to predict pedestrian behaviors, these efforts lack the benchmark labels and the capability to estimate the temporal-dynamic intent changes of the pedestrians, provide explanations of the interaction scenes, and support algorithms with social intelligence. This paper proposes and shares another benchmark dataset called the IUPUI-CSRC Pedestrian Situated Intent (PSI) data with two innovative labels besides comprehensive computer vision labels. The first novel label is the dynamic intent changes for the pedestrians to cross in front of the ego-vehicle, achieved from 24 drivers with diverse backgrounds. The second one is the text-based explanations of the driver reasoning process when estimating pedestrian intents and predicting their behaviors during the interaction period. These innovative labels can enable several computer vision tasks, including pedestrian intent/behavior prediction, vehicle-pedestrian interaction segmentation, and video-to-language mapping for explainable algorithms. The released dataset can fundamentally improve the development of pedestrian behavior prediction models and develop socially intelligent autonomous cars to interact with pedestrians efficiently. The dataset has been evaluated with different tasks and is released to the public to access.
Multi-View Active Learning for Short Text Classification in User-Generated Data
Venue: arXiv cs.CL 2022
Year: 2022

Mining user-generated data often suffers from the lack of enough labeled data, short document lengths, and the informal user language. In this paper, we propose a novel active learning model to overcome these obstacles in the tasks tailored for query phrases--e.g., detecting positive reports of natural disasters. Our model has three novelties: 1) It is the first approach to employ multi-view active learning in this domain. 2) It uses the Parzen-Rosenblatt window method to integrate the representativeness measure into multi-view active learning. 3) It employs a query-by-committee strategy, based on the agreement between predictors, to address the usually noisy language of the documents in this domain. We evaluate our model in four publicly available Twitter datasets with distinctly different applications. We also compare our model with a wide range of baselines including those with multiple classifiers. The experiments testify that our model is highly consistent and outperforms existing models.
Seeing Objects in dark with Continual Contrastive Learning
Venue: arXiv cs.CV 2022
Year: 2022

Object Detection, a fundamental computer vision problem, has paramount importance in smart camera systems. However, a truly reliable camera system could be achieved if and only if the underlying object detection component is robust enough across varying imaging conditions (or domains), for instance, different times of the day, adverse weather conditions, etc. In an effort to achieving a reliable camera system, in this paper, we make an attempt to train such a robust detector. Unfortunately, to build a well-performing detector across varying imaging conditions, one would require labeled training images (often in large numbers) from a plethora of corner cases. As manually obtaining such a large labeled dataset may be infeasible, we suggest using synthetic images, to mimic different training image domains. We propose a novel, contrastive learning method to align the latent representations of a pair of real and synthetic images, to make the detector robust to the different domains. However, we found that merely contrasting the embeddings may lead to catastrophic forgetting of the information essential for object detection. Hence, we employ a continual learning based penalty, to alleviate the issue of forgetting, while contrasting the representations. We showcase that our proposed method outperforms a wide range of alternatives to address the extremely challenging, yet under-studied scenario of object detection at night-time.
From Coarse to Fine-grained Concept based Discrimination for Phrase Detection
Venue: arXiv cs.CV 2022
Year: 2022

Phrase detection requires methods to identify if a phrase is relevant to an image and localize it, if applicable. A key challenge for training more discriminative detection models is sampling negatives. Sampling techniques from prior work focus primarily on hard, often noisy, negatives disregarding the broader distribution of negative samples. Our proposed CFCD-Net addresses this through two novels methods. First, we generate groups of semantically similar words we call concepts (\eg, \{dog, cat, horse\} and \ \{car, truck, SUV\}), and then train our CFCD-Net to discriminate between a region of interest and its unrelated concepts. Second, for phrases containing fine-grained mutually-exclusive words (\eg, colors), we force the model to select only one applicable phrase for each region using our novel fine-grained module (FGM). We evaluate our approach on Flickr30K Entities and RefCOCO+, where we improve mAP over the state-of-the-art by 1.5-2 points. When considering only the phrases affected by our FGM module, we improve by 3-4 points on both datasets.
Synthetic ECG Signal Generation Using Generative Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Electrocardiogram (ECG) datasets tend to be highly imbalanced due to the scarcity of abnormal cases. Additionally, the use of real patients' ECGs is highly regulated due to privacy issues. Therefore, there is always a need for more ECG data, especially for the training of automatic diagnosis machine learning models, which perform better when trained on a balanced dataset. We studied the synthetic ECG generation capability of 5 different models from the generative adversarial network (GAN) family and compared their performances, the focus being only on Normal cardiac cycles. Dynamic Time Warping (DTW), Fréchet, and Euclidean distance functions were employed to quantitatively measure performance. Five different methods for evaluating generated beats were proposed and applied. We also proposed 3 new concepts (threshold, accepted beat and productivity rate) and employed them along with the aforementioned methods as a systematic way for comparison between models. The results show that all the tested models can, to an extent, successfully mass-generate acceptable heartbeats with high similarity in morphological features, and potentially all of them can be used to augment imbalanced datasets. However, visual inspections of generated beats favors BiLSTM-DC GAN and WGAN, as they produce statistically more acceptable beats. Also, with regards to productivity rate, the Classic GAN is superior with a 72% productivity rate. We also designed a simple experiment with the state-of-the-art classifier (ECGResNet34) to show empirically that the augmentation of the imbalanced dataset by synthetic ECG signals could improve the performance of classification significantly.
BDFA: A Blind Data Adversarial Bit-flip Attack on Deep Neural Networks
Venue: arXiv cs.CR 2022
Year: 2022

Adversarial bit-flip attack (BFA) on Neural Network weights can result in catastrophic accuracy degradation by flipping a very small number of bits. A major drawback of prior bit flip attack techniques is their reliance on test data. This is frequently not possible for applications that contain sensitive or proprietary data. In this paper, we propose Blind Data Adversarial Bit-flip Attack (BDFA), a novel technique to enable BFA without any access to the training or testing data. This is achieved by optimizing for a synthetic dataset, which is engineered to match the statistics of batch normalization across different layers of the network and the targeted label. Experimental results show that BDFA could decrease the accuracy of ResNet50 significantly from 75.96\% to 13.94\% with only 4 bits flips.
Generative Adversarial Networks for Labeled Acceleration Data Augmentation for Structural Damage Detection
Venue: arXiv cs.LG 2022
Year: 2022

There has been a major advance in the field of Data Science in the last few decades, and these have been utilized for different engineering disciplines and applications. Artificial Intelligence (AI), Machine Learning (ML) and Deep Learning (DL) algorithms have been utilized for civil Structural Health Monitoring (SHM) especially for damage detection applications using sensor data. Although ML and DL methods show superior learning skills for complex data structures, they require plenty of data for training. However, in SHM, data collection from civil structures can be expensive and time taking; particularly getting useful data (damage associated data) can be challenging. The objective of this study is to address the data scarcity problem for damage detection applications. This paper employs 1-D Wasserstein Deep Convolutional Generative Adversarial Networks using Gradient Penalty (1-D WDCGAN-GP) for synthetic labelled acceleration data generation. Then, the generated data is augmented with varying ratios for the training dataset of a 1-D Deep Convolutional Neural Network (1-D DCNN) for damage detection application. The damage detection results show that the 1-D WDCGAN-GP can be successfully utilized to tackle data scarcity in vibration-based damage detection applications of civil structures. Keywords: Structural Health Monitoring (SHM), Structural Damage Detection, 1-D Deep Convolutional Neural Networks (1-D DCNN), 1-D Generative Adversarial Networks (1-D GAN), Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP)
Computational complexity of problems for deterministic presentations of sofic shifts
Venue: arXiv cs.CC 2022
Year: 2022

Sofic shifts are symbolic dynamical systems defined by the set of bi-infinite sequences on an edge-labeled directed graph, called a presentation. We study the computational complexity of an array of natural decision problems about presentations of sofic shifts, such as whether a given graph presents a shift of finite type, or an irreducible shift; whether one graph presents a subshift of another; and whether a given presentation is minimal, or has a synchronizing word. Leveraging connections to automata theory, we first observe that these problems are all decidable in polynomial time when the given presentation is irreducible (strongly connected), via algorithms both known and novel to this work. For the general (reducible) case, however, we show they are all PSPACE-complete. All but one of these problems (subshift) remain polynomial-time solvable when restricting to synchronizing deterministic presentations. We also study the size of synchronizing words and synchronizing deterministic presentations.
Understanding Distributed Tutorship in Online Language Tutoring
Venue: arXiv cs.HC 2022
Year: 2022

With the rise of the gig economy, online language tutoring platforms are becoming increasingly popular. They provide temporary and flexible jobs for native speakers as tutors and allow language learners to have one-on-one speaking practices on demand. However, the lack of stable relationships hinders tutors and learners from building long-term trust. "Distributed tutorship" -- temporally discontinuous learning experience with different tutors -- has been underexplored yet has many implications for modern learning platforms. In this paper, we analyzed tutorship sequences of 15,959 learners and found that around 40% of learners change to new tutors every session; 44% learners change to new tutors while reverting to previous tutors sometimes; only 16% learners change to new tutors and then fix on one tutor. We also found suggestive evidence that higher distributedness -- higher diversity and lower continuity in tutorship -- is correlated to slower improvements in speaking performance scores with a similar number of sessions. We further surveyed 519 and interviewed 40 learners and found that more learners preferred fixed tutorship while some do not have it due to various reasons. Finally, we conducted semi-structured interviews with three tutors and one product manager to discuss the implications for improving the continuity in learning under distributed tutorship.
Synthesis of Pure and Impure Petri nets With Restricted Place-environments: Complexity Issues
Venue: arXiv cs.FL 2022
Year: 2022

Petri net synthesis consists in deciding for a given transition system $A$ whether there exists a Petri net $N$ whose reachability graph is isomorphic to $A$. Several works examined the synthesis of Petri net subclasses that restrict, for every place $p$ of the net, the cardinality of its preset or of its postset or both in advance by small natural numbers $\varrho$ and $κ$, respectively, such as for example (weighted) marked graphs, (weighted) T-systems and choice-free nets. In this paper, we study the synthesis aiming at Petri nets which have such restricted place environments, from the viewpoint of classical and parameterized complexity: We first show that, for any fixed natural numbers $\varrho$ and $κ$, deciding whether for a given transition system $A$ there is a Petri net $N$ such that (1) its reachability graph is isomorphic to $A$ and (2) for every place $p$ of $N$ the preset of $p$ has at most $\varrho$ and the postset of $p$ has at most $κ$ elements is doable in polynomial time. Secondly, we introduce a modified version of the problem, namely Environment Restricted Synthesis (ERS, for short), where $\varrho$ and $κ$ are part of the input, and show that ERS is NP-complete, regardless whether the sought net is impure or pure. In case of the impure nets, our methods also imply that ERS parameterized by $\varrho+κ$ is $W[2]$-hard.
IntelliTC: Automating Type Changes in IntelliJ IDEA
Venue: arXiv cs.SE 2022
Year: 2022

Developers often change types of program elements. Such refactoring often involves updating not only the type of the element itself, but also the API of all type-dependent references in the code, thus it is tedious and time-consuming. Despite type changes being more frequent than renamings, just a few current IDE tools provide partially-automated support only for a small set of hard-coded types. Researchers have recently proposed a data-driven approach to inferring API rewrite rules for type change patterns in Java using code commits history. In this paper, we build upon these recent advances and introduce IntelliTC - a tool to perform Java type change refactoring. We implemented it as a plugin for IntelliJ IDEA, a popular Java IDE developed by JetBrains. We present 3 different ways of providing support for such a refactoring from the standpoint of the user experience: Classic mode, Suggested Refactoring, and Inspection mode. To evaluate these modalities of using IntelliTC, we surveyed 22 experienced software developers. They positively rated the usefulness of the tool. The source code and distribution of the plugin are available on GitHub: https://github.com/JetBrains-Research/data-driven-type-migration. A demonstration video is on YouTube: https://youtu.be/pdcfvADA1PY.
Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments
Venue: arXiv cs.CV 2022
Year: 2022

Indirect Time-of-Flight (I-ToF) imaging is a widespread way of depth estimation for mobile devices due to its small size and affordable price. Previous works have mainly focused on quality improvement for I-ToF imaging especially curing the effect of Multi Path Interference (MPI). These investigations are typically done in specifically constrained scenarios at close distance, indoors and under little ambient light. Surprisingly little work has investigated I-ToF quality improvement in real-life scenarios where strong ambient light and far distances pose difficulties due to an extreme amount of induced shot noise and signal sparsity, caused by the attenuation with limited sensor power and light scattering. In this work, we propose a new learning based end-to-end depth prediction network which takes noisy raw I-ToF signals as well as an RGB image and fuses their latent representation based on a multi step approach involving both implicit and explicit alignment to predict a high quality long range depth map aligned to the RGB viewpoint. We test our approach on challenging real-world scenes and show more than 40% RMSE improvement on the final depth map compared to the baseline approach.
Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learning
Venue: arXiv cs.LG 2022
Year: 2022

A common vision from science fiction is that robots will one day inhabit our physical spaces, sense the world as we do, assist our physical labours, and communicate with us through natural language. Here we study how to design artificial agents that can interact naturally with humans using the simplification of a virtual environment. We show that imitation learning of human-human interactions in a simulated world, in conjunction with self-supervised learning, is sufficient to produce a multimodal interactive agent, which we call MIA, that successfully interacts with non-adversarial humans 75% of the time. We further identify architectural and algorithmic techniques that improve performance, such as hierarchical action selection. Altogether, our results demonstrate that imitation of multi-modal, real-time human behaviour may provide a straightforward and surprisingly effective means of imbuing agents with a rich behavioural prior from which agents might then be fine-tuned for specific purposes, thus laying a foundation for training capable agents for interactive robots or digital assistants. A video of MIA's behaviour may be found at https://youtu.be/ZFgRhviF7mY
GIFdroid: Automated Replay of Visual Bug Reports for Android Apps
Venue: arXiv cs.SE 2022
Year: 2022

Bug reports are vital for software maintenance that allow users to inform developers of the problems encountered while using software. However, it is difficult for non-technical users to write clear descriptions about the bug occurrence. Therefore, more and more users begin to record the screen for reporting bugs as it is easy to be created and contains detailed procedures triggering the bug. But it is still tedious and time-consuming for developers to reproduce the bug due to the length and unclear actions within the recording. To overcome these issues, we propose GIFdroid, a light-weight approach to automatically replay the execution trace from visual bug reports. GIFdroid adopts image processing techniques to extract the keyframes from the recording, map them to states in GUI Transitions Graph, and generate the execution trace of those states to trigger the bug. Our automated experiments and user study demonstrate its accuracy, efficiency, and usefulness of the approach.
GCA-Net : Utilizing Gated Context Attention for Improving Image Forgery Localization and Detection
Venue: arXiv cs.CV 2022
Year: 2022

Forensic analysis of manipulated pixels requires the identification of various hidden and subtle features from images. Conventional image recognition models generally fail at this task because they are biased and more attentive toward the dominant local and spatial features. In this paper, we propose a novel Gated Context Attention Network (GCA-Net) that utilizes non-local attention in conjunction with a gating mechanism in order to capture the finer image discrepancies and better identify forged regions. The proposed framework uses high dimensional embeddings to filter and aggregate the relevant context from coarse feature maps at various stages of the decoding process. This improves the network's understanding of global differences and reduces false-positive localizations. Our evaluation on standard image forensic benchmarks shows that GCA-Net can both compete against and improve over state-of-the-art networks by an average of 4.7% AUC. Additional ablation studies also demonstrate the method's robustness against attributions and resilience to false-positive predictions.