OpenRaiser/Intern-Atlas huggingface dataset:
Deeply Unsupervised Patch Re-Identification for Pre-training Object Detectors
Venue: arXiv cs.CV 2022
Year: 2022

Unsupervised pre-training aims at learning transferable features that are beneficial for downstream tasks. However, most state-of-the-art unsupervised methods concentrate on learning global representations for image-level classification tasks instead of discriminative local region representations, which limits their transferability to region-level downstream tasks, such as object detection. To improve the transferability of pre-trained features to object detection, we present Deeply Unsupervised Patch Re-ID (DUPR), a simple yet effective method for unsupervised visual representation learning. The patch Re-ID task treats individual patch as a pseudo-identity and contrastively learns its correspondence in two views, enabling us to obtain discriminative local features for object detection. Then the proposed patch Re-ID is performed in a deeply unsupervised manner, appealing to object detection, which usually requires multilevel feature maps. Extensive experiments demonstrate that DUPR outperforms state-of-the-art unsupervised pre-trainings and even the ImageNet supervised pre-training on various downstream tasks related to object detection.
Injecting Knowledge in Data-driven Vehicle Trajectory Predictors
Venue: arXiv cs.AI 2022
Year: 2022

Vehicle trajectory prediction tasks have been commonly tackled from two distinct perspectives: either with knowledge-driven methods or more recently with data-driven ones. On the one hand, we can explicitly implement domain-knowledge or physical priors such as anticipating that vehicles will follow the middle of the roads. While this perspective leads to feasible outputs, it has limited performance due to the difficulty to hand-craft complex interactions in urban environments. On the other hand, recent works use data-driven approaches which can learn complex interactions from the data leading to superior performance. However, generalization, \textit{i.e.}, having accurate predictions on unseen data, is an issue leading to unrealistic outputs. In this paper, we propose to learn a "Realistic Residual Block" (RRB), which effectively connects these two perspectives. Our RRB takes any off-the-shelf knowledge-driven model and finds the required residuals to add to the knowledge-aware trajectory. Our proposed method outputs realistic predictions by confining the residual range and taking into account its uncertainty. We also constrain our output with Model Predictive Control (MPC) to satisfy kinematic constraints. Using a publicly available dataset, we show that our method outperforms previous works in terms of accuracy and generalization to new scenes. We will release our code and data split here: https://github.com/vita-epfl/RRB.
A Survey of Embodied AI: From Simulators to Research Tasks
Venue: arXiv cs.AI 2022
Year: 2022

There has been an emerging paradigm shift from the era of "internet AI" to "embodied AI", where AI algorithms and agents no longer learn from datasets of images, videos or text curated primarily from the internet. Instead, they learn through interactions with their environments from an egocentric perception similar to humans. Consequently, there has been substantial growth in the demand for embodied AI simulators to support various embodied AI research tasks. This growing interest in embodied AI is beneficial to the greater pursuit of Artificial General Intelligence (AGI), but there has not been a contemporary and comprehensive survey of this field. This paper aims to provide an encyclopedic survey for the field of embodied AI, from its simulators to its research. By evaluating nine current embodied AI simulators with our proposed seven features, this paper aims to understand the simulators in their provision for use in embodied AI research and their limitations. Lastly, this paper surveys the three main research tasks in embodied AI -- visual exploration, visual navigation and embodied question answering (QA), covering the state-of-the-art approaches, evaluation metrics and datasets. Finally, with the new insights revealed through surveying the field, the paper will provide suggestions for simulator-for-task selections and recommendations for the future directions of the field.
Believe The HiPe: Hierarchical Perturbation for Fast, Robust, and Model-Agnostic Saliency Mapping
Venue: arXiv cs.CV 2022
Year: 2022

Understanding the predictions made by Artificial Intelligence (AI) systems is becoming more and more important as deep learning models are used for increasingly complex and high-stakes tasks. Saliency mapping -- a popular visual attribution method -- is one important tool for this, but existing formulations are limited by either computational cost or architectural constraints. We therefore propose Hierarchical Perturbation, a very fast and completely model-agnostic method for interpreting model predictions with robust saliency maps. Using standard benchmarks and datasets, we show that our saliency maps are of competitive or superior quality to those generated by existing model-agnostic methods -- and are over 20 times faster to compute.
A New Metric on Symmetric Group and Applications to Block Permutation Codes
Venue: arXiv cs.IT 2022
Year: 2022

Permutation codes have received a great attention due to various applications. For different applications, one needs permutation codes under different metrics. The generalized Cayley metric was introduced by Chee and Vu [4] and this metric includes several other metrics as special cases. However, the generalized Cayley metric is not easily computable in general. Therefore the block permutation metric was introduced by Yang et al. [22] as the generalized Cayley metric and the block permutation metric have the same magnitude. However, the block permutation metric lacks the symmetry property which restricts more advanced algebraic tools to be involved. In this paper, by introducing a novel metric closely related to the block permutation metric, we build a bridge between some advanced algebraic methods and codes in the block permutation metric. More specifically, based on some techniques from algebraic function fields originated in [19], we give an algebraic-geometric construction of codes in the novel metric with reasonably good parameters. By observing a trivial relation between the novel metric and block permutation metric, we then produce non-systematic codes in block permutation metric that improve all known results given in [21, 22]. More importantly, based on our non-systematic codes, we provide an explicit and systematic construction of codes in block permutation metric which improves the systematic result shown in [22]. In the end, we demonstrate that our codes in the novel metric itself have reasonably good parameters by showing that our construction beats the corresponding Gilbert-Varshamov bound.
Error-and-Erasure Decoding of Product and Staircase Codes
Venue: arXiv cs.IT 2022
Year: 2022

High-rate product codes (PCs) and staircase codes (SCs) are ubiquitous codes in high-speed optical communication achieving near-capacity performance on the binary symmetric channel. Their success is mostly due to very efficient iterative decoding algorithms that require very little complexity. In this paper, we extend the density evolution (DE) analysis for PCs and SCs to a channel with ternary output and ternary message passing, where the third symbol marks an erasure. We investigate the performance of a standard error-and-erasure decoder and of its simplification using DE. The proposed analysis can be used to find component code configurations and quantizer levels for the channel output. We also show how the use of even-weight BCH subcodes as component codes can improve the decoding performance at high rates. The DE results are verified by Monte-Carlo simulations, which show that additional coding gains of up to 0.6 dB are possible by ternary decoding, at only a small additional increase in complexity compared to traditional binary message passing.
Session-based Social and Dependency-aware Software Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

With the increase of complexity of modern software, social collaborative coding and reuse of open source software packages become more and more popular, which thus greatly enhances the development efficiency and software quality. However, the explosive growth of open source software packages exposes developers to the challenge of information overload. While this can be addressed by conventional recommender systems, they usually do not consider particular constraints of social coding such as social influence among developers and dependency relations among software packages. In this paper, we aim to model the dynamic interests of developers with both social influence and dependency constraints, and propose the Session-based Social and Dependency-aware software Recommendation (SSDRec) model. This model integrates recurrent neural network (RNN) and graph attention network (GAT) into a unified framework. An RNN is employed to model the short-term dynamic interests of developers in each session and two GATs are utilized to capture social influence from friends and dependency constraints from dependent software packages, respectively. Extensive experiments are conducted on real-world datasets and the results demonstrate that our model significantly outperforms the competitive baselines.
HyperCI: A Higher Order Collective Influence Measure for Hypernetwork Dismantling
Venue: arXiv cs.SI 2022
Year: 2022

The connectivity of networked systems is often dependent on a small portion of critical nodes. Network dismantling studies the strategy to identify a subset of nodes the removal of which will maximally destroy the connectivity of a network and fragment it into disconnected components. However, conventional network dismantling approaches focus on simple network which models only pairwise interaction between nodes while groupwise interactions among arbitrary number of nodes are ubiquitous in networked systems like integrated circuits. Groupwise interactions modeled by hypernetwork introduce higher order connectivity patterns, which limits the application of conventional network dismantling methods on hypernetwork. In this brief, we propose HyperCI, a higher order collective influence measure for hypernetwork dismantling. It considers the node co-occurrence characteristics and higher order influence ability both introduced by hyperedges in hypernetwork. We evaluate the effectiveness of our proposed HyperCI on six real world hypernetworks including integrated circuits and citation networks and the results indicate our proposed HyperCI outperforms baseline network dismantling methods for both simple network and hypernetwork.
Hardness of Token Swapping on Trees
Venue: arXiv cs.DS 2022
Year: 2022

Given a graph where every vertex has exactly one labeled token, how can we most quickly execute a given permutation on the tokens? In (sequential) token swapping, the goal is to use the shortest possible sequence of swaps, each of which exchanges the tokens at the two endpoints of an edge of the graph. In parallel token swapping, the goal is to use the fewest rounds, each of which consists of one or more swaps on the edges of a matching. We prove that both of these problems remain NP-hard when the graph is restricted to be a tree. These token swapping problems have been studied by disparate groups of researchers in discrete mathematics, theoretical computer science, robot motion planning, game theory, and engineering. Previous work establishes NP-completeness on general graphs (for both problems); polynomial-time algorithms for simple graph classes such as cliques, stars, paths, and cycles; and constant-factor approximation algorithms in some cases. The two natural cases of sequential and parallel token swapping in trees were first studied over thirty years ago (as "sorting with a transposition tree") and over twenty-five years ago (as "routing permutations via matchings"), yet their complexities were previously unknown. We also show limitations on approximation of sequential token swapping on trees: we identify a broad class of algorithms that encompass all three known polynomial-time algorithms that achieve the best known approximation factor (which is $2$) and show that no such algorithm can achieve an approximation factor less than $2$.
Knowledge Graph Question Answering using Graph-Pattern Isomorphism
Venue: arXiv cs.AI 2022
Year: 2022

Knowledge Graph Question Answering (KGQA) systems are based on machine learning algorithms, requiring thousands of question-answer pairs as training examples or natural language processing pipelines that need module fine-tuning. In this paper, we present a novel QA approach, dubbed TeBaQA. Our approach learns to answer questions based on graph isomorphisms from basic graph patterns of SPARQL queries. Learning basic graph patterns is efficient due to the small number of possible patterns. This novel paradigm reduces the amount of training data necessary to achieve state-of-the-art performance. TeBaQA also speeds up the domain adaption process by transforming the QA system development task into a much smaller and easier data compilation task. In our evaluation, TeBaQA achieves state-of-the-art performance on QALD-8 and delivers comparable results on QALD-9 and LC-QuAD v1. Additionally, we performed a fine-grained evaluation on complex queries that deal with aggregation and superlative questions as well as an ablation study, highlighting future research challenges.
CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation
Venue: arXiv cs.CL 2022
Year: 2022

Pipelined NLP systems have largely been superseded by end-to-end neural modeling, yet nearly all commonly-used models still require an explicit tokenization step. While recent tokenization approaches based on data-derived subword lexicons are less brittle than manually engineered tokenizers, these techniques are not equally suited to all languages, and the use of any fixed vocabulary may limit a model's ability to adapt. In this paper, we present CANINE, a neural encoder that operates directly on character sequences, without explicit tokenization or vocabulary, and a pre-training strategy that operates either directly on characters or optionally uses subwords as a soft inductive bias. To use its finer-grained input effectively and efficiently, CANINE combines downsampling, which reduces the input sequence length, with a deep transformer stack, which encodes context. CANINE outperforms a comparable mBERT model by 2.8 F1 on TyDi QA, a challenging multilingual benchmark, despite having 28% fewer model parameters.
Is BERT a Cross-Disciplinary Knowledge Learner? A Surprising Finding of Pre-trained Models' Transferability
Venue: arXiv cs.CL 2022
Year: 2022

This paper investigates whether the power of the models pre-trained on text data, such as BERT, can be transferred to general token sequence classification applications. To verify pre-trained models' transferability, we test the pre-trained models on text classification tasks with meanings of tokens mismatches, and real-world non-text token sequence classification data, including amino acid, DNA, and music. We find that even on non-text data, the models pre-trained on text converge faster, perform better than the randomly initialized models, and only slightly worse than the models using task-specific knowledge. We also find that the representations of the text and non-text pre-trained models share non-trivial similarities.
Hippocampal formation-inspired probabilistic generative model
Venue: arXiv cs.AI 2022
Year: 2022

In building artificial intelligence (AI) agents, referring to how brains function in real environments can accelerate development by reducing the design space. In this study, we propose a probabilistic generative model (PGM) for navigation in uncertain environments by integrating the neuroscientific knowledge of hippocampal formation (HF) and the engineering knowledge in robotics and AI, namely, simultaneous localization and mapping (SLAM). We follow the approach of brain reference architecture (BRA) (Yamakawa, 2021) to compose the PGM and outline how to verify the model. To this end, we survey and discuss the relationship between the HF findings and SLAM models. The proposed hippocampal formation-inspired probabilistic generative model (HF-PGM) is designed to be highly consistent with the anatomical structure and functions of the HF. By referencing the brain, we elaborate on the importance of integration of egocentric/allocentric information from the entorhinal cortex to the hippocampus and the use of discrete-event queues.
Unsupervised Image Transformation Learning via Generative Adversarial Networks
Venue: arXiv cs.CV 2022
Year: 2022

In this work, we study the image transformation problem, which targets at learning the underlying transformations (e.g., the transition of seasons) from a collection of unlabeled images. However, there could be countless of transformations in the real world, making such a task incredibly challenging, especially under the unsupervised setting. To tackle this obstacle, we propose a novel learning framework built on generative adversarial networks (GANs), where the discriminator and the generator share a transformation space. After the model gets fully optimized, any two points within the shared space are expected to define a valid transformation. In this way, at the inference stage, we manage to adequately extract the variation factor between a customizable image pair by projecting both images onto the transformation space. The resulting transformation vector can further guide the image synthesis, facilitating image editing with continuous semantic change (e.g., altering summer to winter with fall as the intermediate step). Noticeably, the learned transformation space supports not only transferring image styles (e.g., changing day to night), but also manipulating image contents (e.g., adding clouds in the sky). In addition, we make in-depth analysis on the properties of the transformation space to help understand how various transformations are organized. Project page is at https://genforce.github.io/trgan/.
A Whole Brain Probabilistic Generative Model: Toward Realizing Cognitive Architectures for Developmental Robots
Venue: arXiv cs.AI 2022
Year: 2022

Building a humanlike integrative artificial cognitive system, that is, an artificial general intelligence (AGI), is the holy grail of the artificial intelligence (AI) field. Furthermore, a computational model that enables an artificial system to achieve cognitive development will be an excellent reference for brain and cognitive science. This paper describes an approach to develop a cognitive architecture by integrating elemental cognitive modules to enable the training of the modules as a whole. This approach is based on two ideas: (1) brain-inspired AI, learning human brain architecture to build human-level intelligence, and (2) a probabilistic generative model(PGM)-based cognitive system to develop a cognitive system for developmental robots by integrating PGMs. The development framework is called a whole brain PGM (WB-PGM), which differs fundamentally from existing cognitive architectures in that it can learn continuously through a system based on sensory-motor information. In this study, we describe the rationale of WB-PGM, the current status of PGM-based elemental cognitive modules, their relationship with the human brain, the approach to the integration of the cognitive modules, and future challenges. Our findings can serve as a reference for brain studies. As PGMs describe explicit informational relationships between variables, this description provides interpretable guidance from computational sciences to brain science. By providing such information, researchers in neuroscience can provide feedback to researchers in AI and robotics on what the current models lack with reference to the brain. Further, it can facilitate collaboration among researchers in neuro-cognitive sciences as well as AI and robotics.
Margin Preserving Self-paced Contrastive Learning Towards Domain Adaptation for Medical Image Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

To bridge the gap between the source and target domains in unsupervised domain adaptation (UDA), the most common strategy puts focus on matching the marginal distributions in the feature space through adversarial learning. However, such category-agnostic global alignment lacks of exploiting the class-level joint distributions, causing the aligned distribution less discriminative. To address this issue, we propose in this paper a novel margin preserving self-paced contrastive Learning (MPSCL) model for cross-modal medical image segmentation. Unlike the conventional construction of contrastive pairs in contrastive learning, the domain-adaptive category prototypes are utilized to constitute the positive and negative sample pairs. With the guidance of progressively refined semantic prototypes, a novel margin preserving contrastive loss is proposed to boost the discriminability of embedded representation space. To enhance the supervision for contrastive learning, more informative pseudo-labels are generated in target domain in a self-paced way, thus benefiting the category-aware distribution alignment for UDA. Furthermore, the domain-invariant representations are learned through joint contrastive learning between the two domains. Extensive experiments on cross-modal cardiac segmentation tasks demonstrate that MPSCL significantly improves semantic segmentation performance, and outperforms a wide variety of state-of-the-art methods by a large margin.
Deep Learning-based Patient Re-identification Is able to Exploit the Biometric Nature of Medical Chest X-ray Data
Venue: arXiv cs.CV 2022
Year: 2022

With the rise and ever-increasing potential of deep learning techniques in recent years, publicly available medical datasets became a key factor to enable reproducible development of diagnostic algorithms in the medical domain. Medical data contains sensitive patient-related information and is therefore usually anonymized by removing patient identifiers, e.g., patient names before publication. To the best of our knowledge, we are the first to show that a well-trained deep learning system is able to recover the patient identity from chest X-ray data. We demonstrate this using the publicly available large-scale ChestX-ray14 dataset, a collection of 112,120 frontal-view chest X-ray images from 30,805 unique patients. Our verification system is able to identify whether two frontal chest X-ray images are from the same person with an AUC of 0.9940 and a classification accuracy of 95.55%. We further highlight that the proposed system is able to reveal the same person even ten and more years after the initial scan. When pursuing a retrieval approach, we observe an mAP@R of 0.9748 and a precision@1 of 0.9963. Furthermore, we achieve an AUC of up to 0.9870 and a precision@1 of up to 0.9444 when evaluating our trained networks on external datasets such as CheXpert and the COVID-19 Image Data Collection. Based on this high identification rate, a potential attacker may leak patient-related information and additionally cross-reference images to obtain more information. Thus, there is a great risk of sensitive content falling into unauthorized hands or being disseminated against the will of the concerned patients. Especially during the COVID-19 pandemic, numerous chest X-ray datasets have been published to advance research. Therefore, such data may be vulnerable to potential attacks by deep learning-based re-identification algorithms.