OpenRaiser/Intern-Atlas huggingface dataset:
DETR for Crowd Pedestrian Detection
Venue: arXiv cs.CV 2021
Year: 2021

Pedestrian detection in crowd scenes poses a challenging problem due to the heuristic defined mapping from anchors to pedestrians and the conflict between NMS and highly overlapped pedestrians. The recently proposed end-to-end detectors(ED), DETR and deformable DETR, replace hand designed components such as NMS and anchors using the transformer architecture, which gets rid of duplicate predictions by computing all pairwise interactions between queries. Inspired by these works, we explore their performance on crowd pedestrian detection. Surprisingly, compared to Faster-RCNN with FPN, the results are opposite to those obtained on COCO. Furthermore, the bipartite match of ED harms the training efficiency due to the large ground truth number in crowd scenes. In this work, we identify the underlying motives driving ED's poor performance and propose a new decoder to address them. Moreover, we design a mechanism to leverage the less occluded visible parts of pedestrian specifically for ED, and achieve further improvements. A faster bipartite match algorithm is also introduced to make ED training on crowd dataset more practical. The proposed detector PED(Pedestrian End-to-end Detector) outperforms both previous EDs and the baseline Faster-RCNN on CityPersons and CrowdHuman. It also achieves comparable performance with state-of-the-art pedestrian detection methods. Code will be released soon.
Image-based numerical characterization and experimental validation of tensile behavior of octet-truss lattice structures
Venue: arXiv cs.CE 2021
Year: 2021

The production of lightweight metal lattice structures has received much attention due to the recent developments in additive manufacturing (AM). The design flexibility comes, however, with the complexity of the underlying physics. In fact, metal additive manufacturing introduces process-induced geometrical defects that mainly result in deviations of the effective geometry from the nominal one. This change in the final printed shape is the primary cause of the gap between the as-designed and as-manufactured mechanical behavior of AM products. Thus, the possibility to incorporate the precise manufactured geometries into the computational analysis is crucial for the quality and performance assessment of the final parts. Computed tomography (CT) is an accurate method for the acquisition of the manufactured shape. However, it is often not feasible to integrate the CT-based geometrical information into the traditional computational analysis due to the complexity of the meshing procedure for such high-resolution geometrical models and the prohibitive numerical costs. In this work, an embedded numerical framework is applied to efficiently simulate and compare the mechanical behavior of as-designed to as-manufactured octet-truss lattice structures. The parts are produced using laser powder bed fusion (LPBF). Employing an immersed boundary method, namely the Finite Cell Method (FCM), we perform direct numerical simulations (DNS) and first-order numerical homogenization analysis of a tensile test for a 3D printed octet-truss structure. Numerical results based on CT scan (as-manufactured geometry) show an excellent agreement with experimental measurements, whereas both DNS and first-order numerical homogenization performed directly on the 3D virtual model (as-designed geometry) of the structure show a significant deviation from experimental data.
Aggregative Self-Supervised Feature Learning from a Limited Sample
Venue: arXiv cs.CV 2021
Year: 2021

Self-supervised learning (SSL) is an efficient approach that addresses the issue of limited training data and annotation shortage. The key part in SSL is its proxy task that defines the supervisory signals and drives the learning toward effective feature representations. However, most SSL approaches usually focus on a single proxy task, which greatly limits the expressive power of the learned features and therefore deteriorates the network generalization capacity. In this regard, we hereby propose two strategies of aggregation in terms of complementarity of various forms to boost the robustness of self-supervised learned features. We firstly propose a principled framework of multi-task aggregative self-supervised learning from a limited sample to form a unified representation, with an intent of exploiting feature complementarity among different tasks. Then, in self-aggregative SSL, we propose to self-complement an existing proxy task with an auxiliary loss function based on a linear centered kernel alignment metric, which explicitly promotes the exploring of where are uncovered by the features learned from a proxy task at hand to further boost the modeling capability. Our extensive experiments on 2D natural image and 3D medical image classification tasks under limited data and annotation scenarios confirm that the proposed aggregation strategies successfully boost the classification accuracy.
Specializing Inter-Agent Communication in Heterogeneous Multi-Agent Reinforcement Learning using Agent Class Information
Venue: arXiv cs.AI 2021
Year: 2021

Inspired by recent advances in agent communication with graph neural networks, this work proposes the representation of multi-agent communication capabilities as a directed labeled heterogeneous agent graph, in which node labels denote agent classes and edge labels, the communication type between two classes of agents. We also introduce a neural network architecture that specializes communication in fully cooperative heterogeneous multi-agent tasks by learning individual transformations to the exchanged messages between each pair of agent classes. By also employing encoding and action selection modules with parameter sharing for environments with heterogeneous agents, we demonstrate comparable or superior performance in environments where a larger number of agent classes operates.
Quantizing data for distributed learning
Venue: arXiv cs.LG 2021
Year: 2021

We consider machine learning applications that train a model by leveraging data distributed over a trusted network, where communication constraints can create a performance bottleneck. A number of recent approaches propose to overcome this bottleneck through compression of gradient updates. However, as models become larger, so does the size of the gradient updates. In this paper, we propose an alternate approach to learn from distributed data that quantizes data instead of gradients, and can support learning over applications where the size of gradient updates is prohibitive. Our approach leverages the dependency of the computed gradient on data samples, which lie in a much smaller space in order to perform the quantization in the smaller dimension data space. At the cost of an extra gradient computation, the gradient estimate can be refined by conveying the difference between the gradient at the quantized data point and the original gradient using a small number of bits. Lastly, in order to save communication, our approach adds a layer that decides whether to transmit a quantized data sample or not based on its importance for learning. We analyze the convergence of the proposed approach for smooth convex and non-convex objective functions and show that we can achieve order optimal convergence rates with communication that mostly depends on the data rather than the model (gradient) dimension. We use our proposed algorithm to train ResNet models on the CIFAR-10 and ImageNet datasets, and show that we can achieve an order of magnitude savings over gradient compression methods. These communication savings come at the cost of increasing computation at the learning agent, and thus our approach is beneficial in scenarios where communication load is the main problem.
AsyncTaichi: On-the-fly Inter-kernel Optimizations for Imperative and Spatially Sparse Programming
Venue: arXiv cs.PL 2021
Year: 2021

Leveraging spatial sparsity has become a popular approach to accelerate 3D computer graphics applications. Spatially sparse data structures and efficient sparse kernels (such as parallel stencil operations on active voxels), are key to achieve high performance. Existing work focuses on improving performance within a single sparse computational kernel. We show that a system that looks beyond a single kernel, plus additional domain-specific sparse data structure analysis, opens up exciting new space for optimizing sparse computations. Specifically, we propose a domain-specific data-flow graph model of imperative and sparse computation programs, which describes kernel relationships and enables easy analysis and optimization. Combined with an asynchronous execution engine that exposes a wide window of kernels, the inter-kernel optimizer can then perform effective sparse computation optimizations, such as eliminating unnecessary voxel list generations and removing voxel activation checks. These domain-specific optimizations further make way for classical general-purpose optimizations that are originally challenging to directly apply to computations with sparse data structures. Without any computational code modification, our new system leads to $4.02\times$ fewer kernel launches and $1.87\times$ speed up on our GPU benchmarks, including computations on Eulerian grids, Lagrangian particles, meshes, and automatic differentiation.
Towards open and expandable cognitive AI architectures for large-scale multi-agent human-robot collaborative learning
Venue: arXiv cs.RO 2021
Year: 2021

Learning from Demonstration (LfD) constitutes one of the most robust methodologies for constructing efficient cognitive robotic systems. Despite the large body of research works already reported, current key technological challenges include those of multi-agent learning and long-term autonomy. Towards this direction, a novel cognitive architecture for multi-agent LfD robotic learning is introduced, targeting to enable the reliable deployment of open, scalable and expandable robotic systems in large-scale and complex environments. In particular, the designed architecture capitalizes on the recent advances in the Artificial Intelligence (AI) field, by establishing a Federated Learning (FL)-based framework for incarnating a multi-human multi-robot collaborative learning environment. The fundamental conceptualization relies on employing multiple AI-empowered cognitive processes (implementing various robotic tasks) that operate at the edge nodes of a network of robotic platforms, while global AI models (underpinning the aforementioned robotic tasks) are collectively created and shared among the network, by elegantly combining information from a large number of human-robot interaction instances. Regarding pivotal novelties, the designed cognitive architecture a) introduces a new FL-based formalism that extends the conventional LfD learning paradigm to support large-scale multi-agent operational settings, b) elaborates previous FL-based self-learning robotic schemes so as to incorporate the human in the learning loop and c) consolidates the fundamental principles of FL with additional sophisticated AI-enabled learning methodologies for modelling the multi-level inter-dependencies among the robotic tasks. The applicability of the proposed framework is explained using an example of a real-world industrial case study for agile production-based Critical Raw Materials (CRM) recovery.
Indecision Modeling
Venue: arXiv cs.AI 2021
Year: 2021

AI systems are often used to make or contribute to important decisions in a growing range of applications, including criminal justice, hiring, and medicine. Since these decisions impact human lives, it is important that the AI systems act in ways which align with human values. Techniques for preference modeling and social choice help researchers learn and aggregate peoples' preferences, which are used to guide AI behavior; thus, it is imperative that these learned preferences are accurate. These techniques often assume that people are willing to express strict preferences over alternatives; which is not true in practice. People are often indecisive, and especially so when their decision has moral implications. The philosophy and psychology literature shows that indecision is a measurable and nuanced behavior -- and that there are several different reasons people are indecisive. This complicates the task of both learning and aggregating preferences, since most of the relevant literature makes restrictive assumptions on the meaning of indecision. We begin to close this gap by formalizing several mathematical \emph{indecision} models based on theories from philosophy, psychology, and economics; these models can be used to describe (indecisive) agent decisions, both when they are allowed to express indecision and when they are not. We test these models using data collected from an online survey where participants choose how to (hypothetically) allocate organs to patients waiting for a transplant.
GTA: Global Temporal Attention for Video Action Understanding
Venue: arXiv cs.CV 2021
Year: 2021

Self-attention learns pairwise interactions to model long-range dependencies, yielding great improvements for video action recognition. In this paper, we seek a deeper understanding of self-attention for temporal modeling in videos. We first demonstrate that the entangled modeling of spatio-temporal information by flattening all pixels is sub-optimal, failing to capture temporal relationships among frames explicitly. To this end, we introduce Global Temporal Attention (GTA), which performs global temporal attention on top of spatial attention in a decoupled manner. We apply GTA on both pixels and semantically similar regions to capture temporal relationships at different levels of spatial granularity. Unlike conventional self-attention that computes an instance-specific attention matrix, GTA directly learns a global attention matrix that is intended to encode temporal structures that generalize across different samples. We further augment GTA with a cross-channel multi-head fashion to exploit channel interactions for better temporal modeling. Extensive experiments on 2D and 3D networks demonstrate that our approach consistently enhances temporal modeling and provides state-of-the-art performance on three video action recognition datasets.
Deep Learning to Segment Pelvic Bones: Large-scale CT Datasets and Baseline Models
Venue: arXiv cs.CV 2021
Year: 2021

Purpose: Pelvic bone segmentation in CT has always been an essential step in clinical diagnosis and surgery planning of pelvic bone diseases. Existing methods for pelvic bone segmentation are either hand-crafted or semi-automatic and achieve limited accuracy when dealing with image appearance variations due to the multi-site domain shift, the presence of contrasted vessels, coprolith and chyme, bone fractures, low dose, metal artifacts, etc. Due to the lack of a large-scale pelvic CT dataset with annotations, deep learning methods are not fully explored. Methods: In this paper, we aim to bridge the data gap by curating a large pelvic CT dataset pooled from multiple sources and different manufacturers, including 1, 184 CT volumes and over 320, 000 slices with different resolutions and a variety of the above-mentioned appearance variations. Then we propose for the first time, to the best of our knowledge, to learn a deep multi-class network for segmenting lumbar spine, sacrum, left hip, and right hip, from multiple-domain images simultaneously to obtain more effective and robust feature representations. Finally, we introduce a post-processing tool based on the signed distance function (SDF) to eliminate false predictions while retaining correctly predicted bone fragments. Results: Extensive experiments on our dataset demonstrate the effectiveness of our automatic method, achieving an average Dice of 0.987 for a metal-free volume. SDF post-processor yields a decrease of 10.5% in hausdorff distance by maintaining important bone fragments in post-processing phase. Conclusion: We believe this large-scale dataset will promote the development of the whole community and plan to open source the images, annotations, codes, and trained baseline models at https://github.com/ICT-MIRACLE-lab/CTPelvic1K.
A novel smoothed particle hydrodynamics formulation for thermo-capillary phase change problems with focus on metal additive manufacturing melt pool modeling
Venue: arXiv cs.CE 2021
Year: 2021

Laser-based metal processing including welding and three dimensional printing, involves localized melting of solid or granular raw material, surface tension-driven melt flow and significant evaporation of melt due to the applied very high energy densities. The present work proposes a weakly compressible smoothed particle hydrodynamics formulation for thermo-capillary phase change problems involving solid, liquid and gaseous phases with special focus on selective laser melting, an emerging metal additive manufacturing technique. Evaporation-induced recoil pressure, temperature-dependent surface tension and wetting forces are considered as mechanical interface fluxes, while a Gaussian laser beam heat source and evaporation-induced heat losses are considered as thermal interface fluxes. A novel interface stabilization scheme is proposed, which is shown to allow for a stable and smooth liquid-gas interface by effectively damping spurious interface flows as typically occurring in continuum surface force approaches. Moreover, discretization strategies for the tangential projection of the temperature gradient, as required for the discrete Marangoni forces, are critically reviewed. The proposed formulation is deemed especially suitable for modeling of the melt pool dynamics in metal additive manufacturing because the full range of relevant interface forces is considered and the explicit resolution of the atmospheric gas phase enables a consistent description of pore formation by gas inclusion. The accuracy and robustness of the individual model and method building blocks is verified by means of several selected examples in the context of the selective laser melting process.
SS-SFDA : Self-Supervised Source-Free Domain Adaptation for Road Segmentation in Hazardous Environments
Venue: arXiv cs.CV 2021
Year: 2021

We present a novel approach for unsupervised road segmentation in adverse weather conditions such as rain or fog. This includes a new algorithm for source-free domain adaptation (SFDA) using self-supervised learning. Moreover, our approach uses several techniques to address various challenges in SFDA and improve performance, including online generation of pseudo-labels and self-attention as well as use of curriculum learning, entropy minimization and model distillation. We have evaluated the performance on $6$ datasets corresponding to real and synthetic adverse weather conditions. Our method outperforms all prior works on unsupervised road segmentation and SFDA by at least 10.26%, and improves the training time by 18-180x. Moreover, our self-supervised algorithm exhibits similar accuracy performance in terms of mIOU score as compared to prior supervised methods.
ColorShapeLinks: A board game AI competition for educators and students
Venue: arXiv cs.CY 2021
Year: 2021

ColorShapeLinks is an AI board game competition framework specially designed for students and educators in videogame development, with openness and accessibility in mind. The competition is based on an arbitrarily-sized version of the Simplexity board game, the motto of which, "simple to learn, complex to master", is curiously also applicable to AI agents. ColorShapeLinks offers graphical and text-based frontends and a completely open and documented development framework built using industry standard tools and following software engineering best practices. ColorShapeLinks is not only a competition, but both a game and a framework which educators and students can extend and use to host their own competitions. It has been successfully used for running internal competitions in AI classes, as well as for hosting an international AI competition at the IEEE Conference on Games.
Learning Accurate Long-term Dynamics for Model-based Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Accurately predicting the dynamics of robotic systems is crucial for model-based control and reinforcement learning. The most common way to estimate dynamics is by fitting a one-step ahead prediction model and using it to recursively propagate the predicted state distribution over long horizons. Unfortunately, this approach is known to compound even small prediction errors, making long-term predictions inaccurate. In this paper, we propose a new parametrization to supervised learning on state-action data to stably predict at longer horizons -- that we call a trajectory-based model. This trajectory-based model takes an initial state, a future time index, and control parameters as inputs, and directly predicts the state at the future time index. Experimental results in simulated and real-world robotic tasks show that trajectory-based models yield significantly more accurate long term predictions, improved sample efficiency, and the ability to predict task reward. With these improved prediction properties, we conclude with a demonstration of methods for using the trajectory-based model for control.
Latent-CF: A Simple Baseline for Reverse Counterfactual Explanations
Venue: arXiv cs.LG 2021
Year: 2021

In the environment of fair lending laws and the General Data Protection Regulation (GDPR), the ability to explain a model's prediction is of paramount importance. High quality explanations are the first step in assessing fairness. Counterfactuals are valuable tools for explainability. They provide actionable, comprehensible explanations for the individual who is subject to decisions made from the prediction. It is important to find a baseline for producing them. We propose a simple method for generating counterfactuals by using gradient descent to search in the latent space of an autoencoder and benchmark our method against approaches that search for counterfactuals in feature space. Additionally, we implement metrics to concretely evaluate the quality of the counterfactuals. We show that latent space counterfactual generation strikes a balance between the speed of basic feature gradient descent methods and the sparseness and authenticity of counterfactuals generated by more complex feature space oriented techniques.
Do You Do Yoga? Understanding Twitter Users' Types and Motivations using Social and Textual Information
Venue: arXiv cs.CL 2021
Year: 2021

Leveraging social media data to understand people's lifestyle choices is an exciting domain to explore but requires a multiview formulation of the data. In this paper, we propose a joint embedding model based on the fusion of neural networks with attention mechanism by incorporating social and textual information of users to understand their activities and motivations. We use well-being related tweets from Twitter, focusing on 'Yoga'. We demonstrate our model on two downstream tasks: (i) finding user type such as either practitioner or promotional (promoting yoga studio/gym), other; (ii) finding user motivation i.e. health benefit, spirituality, love to tweet/retweet about yoga but do not practice yoga.