OpenRaiser/Intern-Atlas huggingface dataset:
Progressive Deblurring of Diffusion Models for Coarse-to-Fine Image Synthesis
Venue: arXiv cs.CV 2022
Year: 2022

Recently, diffusion models have shown remarkable results in image synthesis by gradually removing noise and amplifying signals. Although the simple generative process surprisingly works well, is this the best way to generate image data? For instance, despite the fact that human perception is more sensitive to the low frequencies of an image, diffusion models themselves do not consider any relative importance of each frequency component. Therefore, to incorporate the inductive bias for image data, we propose a novel generative process that synthesizes images in a coarse-to-fine manner. First, we generalize the standard diffusion models by enabling diffusion in a rotated coordinate system with different velocities for each component of the vector. We further propose a blur diffusion as a special case, where each frequency component of an image is diffused at different speeds. Specifically, the proposed blur diffusion consists of a forward process that blurs an image and adds noise gradually, after which a corresponding reverse process deblurs an image and removes noise progressively. Experiments show that the proposed model outperforms the previous method in FID on LSUN bedroom and church datasets. Code is available at https://github.com/sangyun884/blur-diffusion.
Domain Generalization for Activity Recognition via Adaptive Feature Fusion
Venue: arXiv cs.CV 2022
Year: 2022

Human activity recognition requires the efforts to build a generalizable model using the training datasets with the hope to achieve good performance in test datasets. However, in real applications, the training and testing datasets may have totally different distributions due to various reasons such as different body shapes, acting styles, and habits, damaging the model's generalization performance. While such a distribution gap can be reduced by existing domain adaptation approaches, they typically assume that the test data can be accessed in the training stage, which is not realistic. In this paper, we consider a more practical and challenging scenario: domain-generalized activity recognition (DGAR) where the test dataset \emph{cannot} be accessed during training. To this end, we propose \emph{Adaptive Feature Fusion for Activity Recognition~(AFFAR)}, a domain generalization approach that learns to fuse the domain-invariant and domain-specific representations to improve the model's generalization performance. AFFAR takes the best of both worlds where domain-invariant representations enhance the transferability across domains and domain-specific representations leverage the model discrimination power from each domain. Extensive experiments on three public HAR datasets show its effectiveness. Furthermore, we apply AFFAR to a real application, i.e., the diagnosis of Children's Attention Deficit Hyperactivity Disorder~(ADHD), which also demonstrates the superiority of our approach.
Forest and Water Bodies Segmentation Through Satellite Images Using U-Net
Venue: arXiv cs.CV 2022
Year: 2022

Global environment monitoring is a task that requires additional attention in the contemporary rapid climate change environment. This includes monitoring the rate of deforestation and areas affected by flooding. Satellite imaging has greatly helped monitor the earth, and deep learning techniques have helped to automate this monitoring process. This paper proposes a solution for observing the area covered by the forest and water. To achieve this task UNet model has been proposed, which is an image segmentation model. The model achieved a validation accuracy of 82.55% and 82.92% for the segmentation of areas covered by forest and water, respectively.
Face editing with GAN -- A Review
Venue: arXiv cs.CV 2022
Year: 2022

In recent years, Generative Adversarial Networks (GANs) have become a hot topic among researchers and engineers that work with deep learning. It has been a ground-breaking technique which can generate new pieces of content of data in a consistent way. The topic of GANs has exploded in popularity due to its applicability in fields like image generation and synthesis, and music production and composition. GANs have two competing neural networks: a generator and a discriminator. The generator is used to produce new samples or pieces of content, while the discriminator is used to recognize whether the piece of content is real or generated. What makes it different from other generative models is its ability to learn unlabeled samples. In this review paper, we will discuss the evolution of GANs, several improvements proposed by the authors and a brief comparison between the different models. Index Terms generative adversarial networks, unsupervised learning, deep learning.
A System-driven Automatic Ground Truth Generation Method for DL Inner-City Driving Corridor Detectors
Venue: arXiv cs.CV 2022
Year: 2022

Data-driven perception approaches are well-established in automated driving systems. In many fields even super-human performance is reached. Unlike prediction and planning approaches, mainly supervised learning algorithms are used for the perception domain. Therefore, a major remaining challenge is the efficient generation of ground truth data. As perception modules are positioned close to the sensor, they typically run on raw sensor data of high bandwidth. Due to that, the generation of ground truth labels typically causes a significant manual effort, which leads to high costs for the labelling itself and the necessary quality control. In this contribution, we propose an automatic labeling approach for semantic segmentation of the drivable ego corridor that reduces the manual effort by a factor of 150 and more. The proposed holistic approach could be used in an automated data loop, allowing a continuous improvement of the depending perception modules.
Receptive Field-based Segmentation for Distributed CNN Inference Acceleration in Collaborative Edge Computing
Venue: arXiv cs.DC 2022
Year: 2022

This paper studies inference acceleration using distributed convolutional neural networks (CNNs) in collaborative edge computing network. To avoid inference accuracy loss in inference task partitioning, we propose receptive field-based segmentation (RFS). To reduce the computation time and communication overhead, we propose a novel collaborative edge computing using fused-layer parallelization to partition a CNN model into multiple blocks of convolutional layers. In this scheme, the collaborative edge servers (ESs) only need to exchange small fraction of the sub-outputs after computing each fused block. In addition, to find the optimal solution of partitioning a CNN model into multiple blocks, we use dynamic programming, named as dynamic programming for fused-layer parallelization (DPFP). The experimental results show that DPFP can accelerate inference of VGG-16 up to 73% compared with the pre-trained model, which outperforms the existing work MoDNN in all tested scenarios. Moreover, we evaluate the service reliability of DPFP under time-variant channel, which shows that DPFP is an effective solution to ensure high service reliability with strict service deadline.
Distributed Deep Learning Inference Acceleration using Seamless Collaboration in Edge Computing
Venue: arXiv cs.DC 2022
Year: 2022

This paper studies inference acceleration using distributed convolutional neural networks (CNNs) in collaborative edge computing. To ensure inference accuracy in inference task partitioning, we consider the receptive-field when performing segment-based partitioning. To maximize the parallelization between the communication and computing processes, thereby minimizing the total inference time of an inference task, we design a novel task collaboration scheme in which the overlapping zone of the sub-tasks on secondary edge servers (ESs) is executed on the host ES, named as HALP. We further extend HALP to the scenario of multiple tasks. Experimental results show that HALP can accelerate CNN inference in VGG-16 by 1.7-2.0x for a single task and 1.7-1.8x for 4 tasks per batch on GTX 1080TI and JETSON AGX Xavier, which outperforms the state-of-the-art work MoDNN. Moreover, we evaluate the service reliability under time-variant channel, which shows that HALP is an effective solution to ensure high service reliability with strict service deadline.
Ergodic Rate Analysis of STAR-RIS Aided NOMA Systems
Venue: arXiv cs.IT 2022
Year: 2022

This letter analyzes the ergodic rates of a simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) aided non-orthogonal multiple access (NOMA) system, where the direct links from the base station to cell-edge users are non-line-of-sight due to obstacles, and STAR-RIS is used to provide line-of-sight links to these cell-edge users. By fitting the distribution of the composite channel power gain to a gamma distribution, we derive the closed-form expressions of ergodic rates and high signal-to-noise ratio (SNR) slopes for cell-edge users. Numerical results reveal that 1) the ergodic rates increase with the number of STAR-RIS elements, and the high SNR slopes are fixed as constants; 2) STAR-RIS aided NOMA systems achieve higher ergodic rates than conventional RIS aided NOMA systems.
Video Swin Transformers for Egocentric Video Understanding @ Ego4D Challenges 2022
Venue: arXiv cs.CV 2022
Year: 2022

We implemented Video Swin Transformer as a base architecture for the tasks of Point-of-No-Return temporal localization and Object State Change Classification. Our method achieved competitive performance on both challenges.
Dynamic Graph Reasoning for Multi-person 3D Pose Estimation
Venue: arXiv cs.CV 2022
Year: 2022

Multi-person 3D pose estimation is a challenging task because of occlusion and depth ambiguity, especially in the cases of crowd scenes. To solve these problems, most existing methods explore modeling body context cues by enhancing feature representation with graph neural networks or adding structural constraints. However, these methods are not robust for their single-root formulation that decoding 3D poses from a root node with a pre-defined graph. In this paper, we propose GR-M3D, which models the \textbf{M}ulti-person \textbf{3D} pose estimation with dynamic \textbf{G}raph \textbf{R}easoning. The decoding graph in GR-M3D is predicted instead of pre-defined. In particular, It firstly generates several data maps and enhances them with a scale and depth aware refinement module (SDAR). Then multiple root keypoints and dense decoding paths for each person are estimated from these data maps. Based on them, dynamic decoding graphs are built by assigning path weights to the decoding paths, while the path weights are inferred from those enhanced data maps. And this process is named dynamic graph reasoning (DGR). Finally, the 3D poses are decoded according to dynamic decoding graphs for each detected person. GR-M3D can adjust the structure of the decoding graph implicitly by adopting soft path weights according to input data, which makes the decoding graphs be adaptive to different input persons to the best extent and more capable of handling occlusion and depth ambiguity than previous methods. We empirically show that the proposed bottom-up approach even outperforms top-down methods and achieves state-of-the-art results on three 3D pose datasets.
Two-Aspect Information Fusion Model For ABAW4 Multi-task Challenge
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we propose the solution to the Multi-Task Learning (MTL) Challenge of the 4th Affective Behavior Analysis in-the-wild (ABAW) competition. The task of ABAW is to predict frame-level emotion descriptors from videos: discrete emotional state; valence and arousal; and action units. Although researchers have proposed several approaches and achieved promising results in ABAW, current works in this task rarely consider interactions between different emotion descriptors. To this end, we propose a novel end to end architecture to achieve full integration of different types of information. Experimental results demonstrate the effectiveness of our proposed solution.
Detection and Initial Assessment of Lunar Landing Sites Using Neural Networks
Venue: arXiv cs.RO 2022
Year: 2022

Robotic and human lunar landings are a focus of future NASA missions. Precision landing capabilities are vital to guarantee the success of the mission, and the safety of the lander and crew. During the approach to the surface there are multiple challenges associated with Hazard Relative Navigation to ensure safe landings. This paper will focus on a passive autonomous hazard detection and avoidance sub-system to generate an initial assessment of possible landing regions for the guidance system. The system uses a single camera and the MobileNetV2 neural network architecture to detect and discern between safe landing sites and hazards such as rocks, shadows, and craters. Then a monocular structure from motion will recreate the surface to provide slope and roughness analysis.
Rate-Splitting Multiple Access in Multi-cell Dense Networks: A Stochastic Geometry Approach
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, the potential benefits of applying the Rate-Splitting Multiple Access (RSMA) in multi-cell dense networks are explored. Using tools of stochastic geometry, the sum-rate of RSMA-enhanced multi-cell dense networks is evaluated mathematically based on a Moment Generating Function (MGF) based framework to prove that RSMA is a general and powerful strategy for multi-antenna downlink systems. Further elaboration of the systematic performance metrics is undertaken by developing analytical expressions for area spectral efficiency and sum-rate in the RSMA-enhanced multi-cell dense networks. Based on the tractable expressions, we then offer an optimization framework for energy efficiency in terms of the number of antennas. Additionally, simulation results are shown to verify the accuracy of our analytical results and provide some insightful insights into system design. Analytically, it has been shown that: 1) the sum-rate of RSMA-enhanced multi-cell dense networks is significantly influenced by the power splitting ratio, and there is a unique value that maximizes the sum-rate; 2) the RSMA-enhanced multi-cell dense networks transmission scheme has superior sum-rate performance compared with Non-Orthogonal Multiple Access (NOMA) and Space-Division Multiple Access (SDMA) in a wide range of power splitting ratio; 3) By increasing the number of antennas and BS density in an RSMA-enhanced multi-cell dense network, the area spectral efficiency can be substantially enhanced; 4) As for energy efficiency, there exists an optimal antenna number for maximizing this performance metric.
Facing Changes: Continual Entity Alignment for Growing Knowledge Graphs
Venue: arXiv cs.CL 2022
Year: 2022

Entity alignment is a basic and vital technique in knowledge graph (KG) integration. Over the years, research on entity alignment has resided on the assumption that KGs are static, which neglects the nature of growth of real-world KGs. As KGs grow, previous alignment results face the need to be revisited while new entity alignment waits to be discovered. In this paper, we propose and dive into a realistic yet unexplored setting, referred to as continual entity alignment. To avoid retraining an entire model on the whole KGs whenever new entities and triples come, we present a continual alignment method for this task. It reconstructs an entity's representation based on entity adjacency, enabling it to generate embeddings for new entities quickly and inductively using their existing neighbors. It selects and replays partial pre-aligned entity pairs to train only parts of KGs while extracting trustworthy alignment for knowledge augmentation. As growing KGs inevitably contain non-matchable entities, different from previous works, the proposed method employs bidirectional nearest neighbor matching to find new entity alignment and update old alignment. Furthermore, we also construct new datasets by simulating the growth of multilingual DBpedia. Extensive experiments demonstrate that our continual alignment method is more effective than baselines based on retraining or inductive learning.
Investigating the Validity of Botometer-based Social Bot Studies
Venue: arXiv cs.SI 2022
Year: 2022

The idea that social media platforms like Twitter are inhabited by vast numbers of social bots has become widely accepted in recent years. Social bots are assumed to be automated social media accounts operated by malicious actors with the goal of manipulating public opinion. They are credited with the ability to produce content autonomously and to interact with human users. Social bot activity has been reported in many different political contexts, including the U.S. presidential elections, discussions about migration, climate change, and COVID-19. However, the relevant publications either use crude and questionable heuristics to discriminate between supposed social bots and humans or -- in the vast majority of the cases -- fully rely on the output of automatic bot detection tools, most commonly Botometer. In this paper, we point out a fundamental theoretical flaw in the widely-used study design for estimating the prevalence of social bots. Furthermore, we empirically investigate the validity of peer-reviewed Botometer-based studies by closely and systematically inspecting hundreds of accounts that had been counted as social bots. We were unable to find a single social bot. Instead, we found mostly accounts undoubtedly operated by human users, the vast majority of them using Twitter in an inconspicuous and unremarkable fashion without the slightest traces of automation. We conclude that studies claiming to investigate the prevalence, properties, or influence of social bots based on Botometer have, in reality, just investigated false positives and artifacts of this approach.
HPS-Det: Dynamic Sample Assignment with Hyper-Parameter Search for Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Sample assignment plays a prominent part in modern object detection approaches. However, most existing methods rely on manual design to assign positive / negative samples, which do not explicitly establish the relationships between sample assignment and object detection performance. In this work, we propose a novel dynamic sample assignment scheme based on hyper-parameter search. We first define the number of positive samples assigned to each ground truth as the hyper-parameters and employ a surrogate optimization algorithm to derive the optimal choices. Then, we design a dynamic sample assignment procedure to dynamically select the optimal number of positives at each training iteration. Experiments demonstrate that the resulting HPS-Det brings improved performance over different object detection baselines. Moreover, We analyze the hyper-parameter reusability when transferring between different datasets and between different backbones for object detection, which exhibits the superiority and versatility of our method.
A general-purpose method for applying Explainable AI for Anomaly Detection
Venue: arXiv cs.LG 2022
Year: 2022

The need for explainable AI (XAI) is well established but relatively little has been published outside of the supervised learning paradigm. This paper focuses on a principled approach to applying explainability and interpretability to the task of unsupervised anomaly detection. We argue that explainability is principally an algorithmic task and interpretability is principally a cognitive task, and draw on insights from the cognitive sciences to propose a general-purpose method for practical diagnosis using explained anomalies. We define Attribution Error, and demonstrate, using real-world labeled datasets, that our method based on Integrated Gradients (IG) yields significantly lower attribution errors than alternative methods.
Robots Enact Malignant Stereotypes
Venue: arXiv cs.RO 2022
Year: 2022

Stereotypes, bias, and discrimination have been extensively documented in Machine Learning (ML) methods such as Computer Vision (CV) [18, 80], Natural Language Processing (NLP) [6], or both, in the case of large image and caption models such as OpenAI CLIP [14]. In this paper, we evaluate how ML bias manifests in robots that physically and autonomously act within the world. We audit one of several recently published CLIP-powered robotic manipulation methods, presenting it with objects that have pictures of human faces on the surface which vary across race and gender, alongside task descriptions that contain terms associated with common stereotypes. Our experiments definitively show robots acting out toxic stereotypes with respect to gender, race, and scientifically-discredited physiognomy, at scale. Furthermore, the audited methods are less likely to recognize Women and People of Color. Our interdisciplinary sociotechnical analysis synthesizes across fields and applications such as Science Technology and Society (STS), Critical Studies, History, Safety, Robotics, and AI. We find that robots powered by large datasets and Dissolution Models (sometimes called "foundation models", e.g. CLIP) that contain humans risk physically amplifying malignant stereotypes in general; and that merely correcting disparities will be insufficient for the complexity and scale of the problem. Instead, we recommend that robot learning methods that physically manifest stereotypes or other harmful outcomes be paused, reworked, or even wound down when appropriate, until outcomes can be proven safe, effective, and just. Finally, we discuss comprehensive policy changes and the potential of new interdisciplinary research on topics like Identity Safety Assessment Frameworks and Design Justice to better understand and address these harms.
Defining an action of SO(d)-rotations on images generated by projections of d-dimensional objects: Applications to pose inference with Geometric VAEs
Venue: arXiv cs.CV 2022
Year: 2022

Recent advances in variational autoencoders (VAEs) have enabled learning latent manifolds as compact Lie groups, such as $SO(d)$. Since this approach assumes that data lies on a subspace that is homeomorphic to the Lie group itself, we here investigate how this assumption holds in the context of images that are generated by projecting a $d$-dimensional volume with unknown pose in $SO(d)$. Upon examining different theoretical candidates for the group and image space, we show that the attempt to define a group action on the data space generally fails, as it requires more specific geometric constraints on the volume. Using geometric VAEs, our experiments confirm that this constraint is key to proper pose inference, and we discuss the potential of these results for applications and future work.
Experience with Abrupt Transition to Remote Teaching of Embedded Systems
Venue: arXiv cs.CY 2022
Year: 2022

Due to the pandemic of COVID-19, many university courses had to abruptly transform to enable remote teaching. Adjusting courses on embedded systems and micro-controllers was extra challenging since interaction with real hardware is their integral part. We start by comparing our experience with four basic alternatives of teaching embedded systems: 1) interacting with hardware at school, 2) having remote access to hardware, 3) lending hardware to students for at-home work and 4) virtualizing hardware. Afterward, we evaluate in detail our experience of the fast transition from traditional, offline at-school hardware programming course to using remote access to real hardware present in the lab. The somewhat unusual remote hardware access approach turned out to be a fully viable alternative for teaching embedded systems, enabling a relatively low-effort transition. Our setup is based on existing solutions and stable open technologies without the need for custom-developed applications that require high maintenance. We evaluate the experience of both the students and teachers and condense takeaways for future courses. The specific environment setup is available online as an inspiration for others.