OpenRaiser/Intern-Atlas huggingface dataset:
ACE: Towards Application-Centric Edge-Cloud Collaborative Intelligence
Venue: arXiv cs.DC 2022
Year: 2022

Intelligent applications based on machine learning are impacting many parts of our lives. They are required to operate under rigorous practical constraints in terms of service latency, network bandwidth overheads, and also privacy. Yet current implementations running in the Cloud are unable to satisfy all these constraints. The Edge-Cloud Collaborative Intelligence (ECCI) paradigm has become a popular approach to address such issues, and rapidly increasing applications are developed and deployed. However, these prototypical implementations are developer-dependent and scenario-specific without generality, which cannot be efficiently applied in large-scale or to general ECC scenarios in practice, due to the lack of supports for infrastructure management, edge-cloud collaborative service, complex intelligence workload, and efficient performance optimization. In this article, we systematically design and construct the first unified platform, ACE, that handles ever-increasing edge and cloud resources, user-transparent services, and proliferating intelligence workloads with increasing scale and complexity, to facilitate cost-efficient and high-performing ECCI application development and deployment. For verification, we explicitly present the construction process of an ACE-based intelligent video query application, and demonstrate how to achieve customizable performance optimization efficiently. Based on our initial experience, we discuss both the limitations and vision of ACE to shed light on promising issues to elaborate in the approaching ECCI ecosystem.
Multitask Emotion Recognition Model with Knowledge Distillation and Task Discriminator
Venue: arXiv cs.CV 2022
Year: 2022

Due to the collection of big data and the development of deep learning, research to predict human emotions in the wild is being actively conducted. We designed a multi-task model using ABAW dataset to predict valence-arousal, expression, and action unit through audio data and face images at in real world. We trained model from the incomplete label by applying the knowledge distillation technique. The teacher model was trained as a supervised learning method, and the student model was trained by using the output of the teacher model as a soft label. As a result we achieved 2.40 in Multi Task Learning task validation dataset.
A Preliminary Research on Space Situational Awareness Based on Event Cameras
Venue: arXiv cs.CV 2022
Year: 2022

Event camera is a new type of sensor that is different from traditional cameras. Each pixel is triggered asynchronously by an event. The trigger event is the change of the brightness irradiated on the pixel. If the increment or decrement is higher than a certain threshold, the event is output. Compared with traditional cameras, event cameras have the advantages of high temporal resolution, low latency, high dynamic range, low bandwidth and low power consumption. We carried out a series of observation experiments in a simulated space lighting environment. The experimental results show that the event camera can give full play to the above advantages in space situational awareness. This article first introduces the basic principles of the event camera, then analyzes its advantages and disadvantages, then introduces the observation experiment and analyzes the experimental results, and finally, a workflow of space situational awareness based on event cameras is given.
On Optimizing the Power Allocation and the Decoding Order in Uplink Cooperative NOMA
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we investigate for the first time the dynamic power allocation and decoding order at the base station (BS) of two-user uplink (UL) cooperative non-orthogonal multiple access (C-NOMA)-based cellular networks. In doing so, we formulate a joint optimization problem aiming at maximizing the minimum user achievable rate, which is a non-convex optimization problem and hard to be directly solved. To tackle this issue, an iterative algorithm based on successive convex approximation (SCA) is proposed. The numerical results reveal that the proposed scheme provides superior performance in comparison with the traditional UL NOMA. In addition, we demonstrated that in UL C-NOMA, decoding the far NOMA user first at the BS provides the best performance.
On the Kullback-Leibler divergence between pairwise isotropic Gaussian-Markov random fields
Venue: arXiv cs.IT 2022
Year: 2022

The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields are mathematical structures that models the interaction between these variables by means of an inverse temperature parameter, responsible for controlling the spatial dependence structure along the field. In this paper, we derive closed-form expressions for the Kullback-Leibler divergence between two pairwise isotropic Gaussian-Markov random fields in both univariate and multivariate cases. The proposed equation allows the development of novel similarity measures in image processing and machine learning applications, such as image denoising and unsupervised metric learning.
An Evolutionary Note on Smart City Development in China
Venue: arXiv cs.CY 2022
Year: 2022

In response to challenges posed by urbanization, David Bollier from the University of Southern California raised a new idea for city planning: a comprehensive network and applications of information technologies. IBM later echoed the idea and initiated its Smart Planet vision in 2008. After that, the smart city concept was quickly adopted by major cities throughout the world, and it has gradually evolved into a strategic choice by ambitious cities. This paper looks into the smart city trend by reviewing how the concept of smart city was proposed and what the essence of a smart city is. More specifically, the driving forces of the smart city development in China are investigated, and the key differences of smart cities between China and other countries are summarized. Finally, four big challenges to build future smart cities are discussed.
Quantum Motion Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Motion segmentation is a challenging problem that seeks to identify independent motions in two or several input images. This paper introduces the first algorithm for motion segmentation that relies on adiabatic quantum optimization of the objective function. The proposed method achieves on-par performance with the state of the art on problem instances which can be mapped to modern quantum annealers.
A Broad Comparative Evaluation of x86-64 Binary Rewriters
Venue: arXiv cs.SE 2022
Year: 2022

Binary rewriting is a rapidly-maturing technique for modifying software for instrumentation, customization, optimization, and hardening without access to source code. Unfortunately, the practical applications of binary rewriting tools are often unclear to users because their limitations are glossed over in the literature. This, among other challenges, has prohibited the widespread adoption of these tools. To address this shortcoming, we collect ten popular binary rewriters and assess their generality across a broad range of input binary classes and the functional reliability of the resulting rewritten binaries. Additionally, we evaluate the performance of the rewriters themselves as well as the rewritten binaries they produce. The goal of this broad evaluation is to establish a shared context for future research and development of binary rewriting tools by providing a state of the practice for their capabilities. To support potential binary rewriter users, we also identify input binary features that are predictive of tool success and show that a simple decision tree model can accurately predict whether a particular tool can rewrite a target binary. The binary rewriters, our corpus of 3344 sample binaries, and the evaluation infrastructure itself are all freely available as open-source software.
Open-set Recognition via Augmentation-based Similarity Learning
Venue: arXiv cs.CV 2022
Year: 2022

The primary assumption of conventional supervised learning or classification is that the test samples are drawn from the same distribution as the training samples, which is called closed set learning or classification. In many practical scenarios, this is not the case because there are unknowns or unseen class samples in the test data, which is called the open set scenario, and the unknowns need to be detected. This problem is referred to as the open set recognition problem and is important in safety-critical applications. We propose to detect unknowns (or unseen class samples) through learning pairwise similarities. The proposed method works in two steps. It first learns a closed set classifier using the seen classes that have appeared in training and then learns how to compare seen classes with pseudo-unseen (automatically generated unseen class samples). The pseudo-unseen generation is carried out by performing distribution shifting augmentations on the seen or training samples. We call our method OPG (Open set recognition based on Pseudo unseen data Generation). The experimental evaluation shows that the learned similarity-based features can successfully distinguish seen from unseen in benchmark datasets for open set recognition.
A Representation Separation Perspective to Correspondences-free Unsupervised 3D Point Cloud Registration
Venue: arXiv cs.CV 2022
Year: 2022

3D point cloud registration in remote sensing field has been greatly advanced by deep learning based methods, where the rigid transformation is either directly regressed from the two point clouds (correspondences-free approaches) or computed from the learned correspondences (correspondences-based approaches). Existing correspondences-free methods generally learn the holistic representation of the entire point cloud, which is fragile for partial and noisy point clouds. In this paper, we propose a correspondences-free unsupervised point cloud registration (UPCR) method from the representation separation perspective. First, we model the input point cloud as a combination of pose-invariant representation and pose-related representation. Second, the pose-related representation is used to learn the relative pose wrt a "latent canonical shape" for the source and target point clouds respectively. Third, the rigid transformation is obtained from the above two learned relative poses. Our method not only filters out the disturbance in pose-invariant representation but also is robust to partial-to-partial point clouds or noise. Experiments on benchmark datasets demonstrate that our unsupervised method achieves comparable if not better performance than state-of-the-art supervised registration methods.
Global Tracking Transformers
Venue: arXiv cs.CV 2022
Year: 2022

We present a novel transformer-based architecture for global multi-object tracking. Our network takes a short sequence of frames as input and produces global trajectories for all objects. The core component is a global tracking transformer that operates on objects from all frames in the sequence. The transformer encodes object features from all frames, and uses trajectory queries to group them into trajectories. The trajectory queries are object features from a single frame and naturally produce unique trajectories. Our global tracking transformer does not require intermediate pairwise grouping or combinatorial association, and can be jointly trained with an object detector. It achieves competitive performance on the popular MOT17 benchmark, with 75.3 MOTA and 59.1 HOTA. More importantly, our framework seamlessly integrates into state-of-the-art large-vocabulary detectors to track any objects. Experiments on the challenging TAO dataset show that our framework consistently improves upon baselines that are based on pairwise association, outperforming published works by a significant 7.7 tracking mAP. Code is available at https://github.com/xingyizhou/GTR.
Interpretability of Neural Network With Physiological Mechanisms
Venue: arXiv cs.NE 2022
Year: 2022

Deep learning continues to play as a powerful state-of-art technique that has achieved extraordinary accuracy levels in various domains of regression and classification tasks, including images, video, signal, and natural language data. The original goal of proposing the neural network model is to improve the understanding of complex human brains using a mathematical expression approach. However, recent deep learning techniques continue to lose the interpretations of its functional process by being treated mostly as a black-box approximator. To address this issue, such an AI model needs to be biological and physiological realistic to incorporate a better understanding of human-machine evolutionary intelligence. In this study, we compare neural networks and biological circuits to discover the similarities and differences from various perspective views. We further discuss the insights into how neural networks learn from data by investigating human biological behaviors and understandable justifications.
Immersive Visual Analysis of Cello Bow Movements
Venue: arXiv cs.HC 2022
Year: 2022

We propose a 3D immersive visualization environment for analyzing the right hand movements of a cello player. To achieve this, we track the position and orientation of the cello bow and record audio. As movements mostly occur in a shallow volume and the motion is therefore mostly two-dimensional, we use the third dimension to encode time. Our concept further explores various mappings from motion and audio data to spatial and other visual attributes. We work in close cooperation with a cellist and plan to evaluate our prototype through a user study with a group of cellists in the near future.
CrossFormer: Cross Spatio-Temporal Transformer for 3D Human Pose Estimation
Venue: arXiv cs.CV 2022
Year: 2022

3D human pose estimation can be handled by encoding the geometric dependencies between the body parts and enforcing the kinematic constraints. Recently, Transformer has been adopted to encode the long-range dependencies between the joints in the spatial and temporal domains. While they had shown excellence in long-range dependencies, studies have noted the need for improving the locality of vision Transformers. In this direction, we propose a novel pose estimation Transformer featuring rich representations of body joints critical for capturing subtle changes across frames (i.e., inter-feature representation). Specifically, through two novel interaction modules; Cross-Joint Interaction and Cross-Frame Interaction, the model explicitly encodes the local and global dependencies between the body joints. The proposed architecture achieved state-of-the-art performance on two popular 3D human pose estimation datasets, Human3.6 and MPI-INF-3DHP. In particular, our proposed CrossFormer method boosts performance by 0.9% and 0.3%, compared to the closest counterpart, PoseFormer, using the detected 2D poses and ground-truth settings respectively.
HetSched: Quality-of-Mission Aware Scheduling for Autonomous Vehicle SoCs
Venue: arXiv cs.AR 2022
Year: 2022

Systems-on-Chips (SoCs) that power autonomous vehicles (AVs) must meet stringent performance and safety requirements prior to deployment. With increasing complexity in AV applications, the system needs to meet these real-time demands of multiple safety-critical applications simultaneously. A typical AV-SoC is a heterogeneous multiprocessor consisting of accelerators supported by general-purpose cores. Such heterogeneity, while needed for power-performance efficiency, complicates the art of task scheduling. In this paper, we demonstrate that hardware heterogeneity impacts the scheduler's effectiveness and that optimizing for only the real-time aspect of applications is not sufficient in AVs. Therefore, a more holistic approach is required -- one that considers global Quality-of-Mission (QoM) metrics, as defined in the paper. We then propose HetSched, a multi-step scheduler that leverages dynamic runtime information about the underlying heterogeneous hardware platform, along with the applications' real-time constraints and the task traffic in the system to optimize overall mission performance. HetSched proposes two scheduling policies: MSstat and MSdyn and scheduling optimizations like task pruning, hybrid heterogeneous ranking and rank update. HetSched improves overall mission performance on average by 4.6x, 2.6x and 2.6x when compared against CPATH, ADS and 2lvl-EDF (state-of-the-art real-time schedulers built for heterogeneous systems), respectively, and achieves an average of 53.3% higher hardware utilization, while meeting 100% critical deadlines for real-world applications of autonomous vehicles. Furthermore, when used as part of an SoC design space exploration loop, in comparison to prior schedulers, HetSched reduces the number of processing elements required by an SoC to safely complete AV's missions by 35% on average while achieving 2.7x lower energy-mission time product.
Frame-level Prediction of Facial Expressions, Valence, Arousal and Action Units for Mobile Devices
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we consider the problem of real-time video-based facial emotion analytics, namely, facial expression recognition, prediction of valence and arousal and detection of action unit points. We propose the novel frame-level emotion recognition algorithm by extracting facial features with the single EfficientNet model pre-trained on AffectNet. As a result, our approach may be implemented even for video analytics on mobile devices. Experimental results for the large scale Aff-Wild2 database from the third Affective Behavior Analysis in-the-wild (ABAW) Competition demonstrate that our simple model is significantly better when compared to the VggFace baseline. In particular, our method is characterized by 0.15-0.2 higher performance measures for validation sets in uni-task Expression Classification, Valence-Arousal Estimation and Expression Classification. Due to simplicity, our approach may be considered as a new baseline for all four sub-challenges.
C to Checked C by 3C
Venue: arXiv cs.PL 2022
Year: 2022

Owing to the continued use of C (and C++), spatial safety violations (e.g., buffer overflows) still constitute one of today's most dangerous and prevalent security vulnerabilities. To combat these violations, Checked C extends C with bounds-enforced checked pointer types. Checked C is essentially a gradually typed spatially safe C - checked pointers are backwards-binary compatible with legacy pointers, and the language allows them to be added piecemeal, rather than necessarily all at once, so that safety retrofitting can be incremental. This paper presents a semi-automated process for porting a legacy C program to Checked C. The process centers on 3C, a static analysis-based annotation tool. 3C employs two novel static analysis algorithms - typ3c and boun3c - to annotate legacy pointers as checked pointers, and to infer array bounds annotations for pointers that need them. 3C performs a root cause analysis to direct a human developer to code that should be refactored; once done, 3C can be re-run to infer further annotations (and updated root causes). Experiments on 11 programs totaling 319KLoC show 3C to be effective at inferring checked pointer types, and experience with previously and newly ported code finds 3C works well when combined with human-driven refactoring.
A Comparative Survey of Deep Active Learning
Venue: arXiv cs.LG 2022
Year: 2022

While deep learning (DL) is data-hungry and usually relies on extensive labeled data to deliver good performance, Active Learning (AL) reduces labeling costs by selecting a small proportion of samples from unlabeled data for labeling and training. Therefore, Deep Active Learning (DAL) has risen as a feasible solution for maximizing model performance under a limited labeling cost/budget in recent years. Although abundant methods of DAL have been developed and various literature reviews conducted, the performance evaluation of DAL methods under fair comparison settings is not yet available. Our work intends to fill this gap. In this work, We construct a DAL toolkit, DeepAL+, by re-implementing 19 highly-cited DAL methods. We survey and categorize DAL-related works and construct comparative experiments across frequently used datasets and DAL algorithms. Additionally, we explore some factors (e.g., batch size, number of epochs in the training process) that influence the efficacy of DAL, which provides better references for researchers to design their DAL experiments or carry out DAL-related applications.
From MIM-Based GAN to Anomaly Detection:Event Probability Influence on Generative Adversarial Networks
Venue: arXiv cs.LG 2022
Year: 2022

In order to introduce deep learning technologies into anomaly detection, Generative Adversarial Networks (GANs) are considered as important roles in the algorithm design and realistic applications. In terms of GANs, event probability reflected in the objective function, has an impact on the event generation which plays a crucial part in GAN-based anomaly detection. The information metric, e.g. Kullback-Leibler divergence in the original GAN, makes the objective function have different sensitivity on different event probability, which provides an opportunity to refine GAN-based anomaly detection by influencing data generation. In this paper, we introduce the exponential information metric into the GAN, referred to as MIM-based GAN, whose superior characteristics on data generation are discussed in theory. Furthermore, we propose an anomaly detection method with MIM-based GAN, as well as explain its principle for the unsupervised learning case from the viewpoint of probability event generation. Since this method is promising to detect anomalies in Internet of Things (IoT), such as environmental, medical and biochemical outliers, we make use of several datasets from the online ODDS repository to evaluate its performance and compare it with other methods.
Smart Wireless Environments Enabled by RISs: Deployment Scenarios and Two Key Challenges
Venue: arXiv cs.IT 2022
Year: 2022

Reconfigurable Intelligent Surfaces (RISs) constitute the enabler for programmable propagation of electromagnetic signals, and are lately being considered as a candidate physical-layer technology for the demanding connectivity, reliability, localization, and sustainability requirements of next generation wireless communications networks. In this paper, we present various deployment scenarios for RIS-enabled smart wireless environments that have been recently designed by the ongoing EU H2020 RISE-6G project. The scenarios are taxonomized according to performance objectives, in particular, connectivity and reliability, localization and sensing, as well as sustainability and secrecy. We identify various deployment strategies and sketch the core architectural requirements in terms of RIS control and signaling, depending on the RIS hardware architectures and their respective capabilities. Furthermore, we introduce and discuss, via preliminary simulation results and reflectarray measurements, two key novel challenges with RIS-enabled smart wireless environments, namely, the area of influence and the bandwidth of influence of RISs, which corroborate the need for careful deployment and planning of this new technology.