Reviewerly/exHarmony huggingface dataset:
Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression

Even though 3D face reconstruction has achieved impressive progress, most orthogonal projection-based face reconstruction methods can not achieve accurate and consistent reconstruction results when the face is very close to the camera due to the distortion under the perspective projection. In this paper, we propose to simultaneously reconstruct 3D face mesh in the world space and predict 2D face landmarks on the image plane to address the problem of perspective 3D face reconstruction. Based on the predicted 3D vertices and 2D landmarks, the 6DoF (6 Degrees of Freedom) face pose can be easily estimated by the PnP solver to represent perspective projection. Our approach achieves 1st place on the leader-board of the ECCV 2022 WCPA challenge and our model is visually robust under different identities, expressions and poses. The training code and models are released to facilitate future research. https://github.com/deepinsight/insightface/tree/master/reconstruction/jmlr .
Multi-UAV Path Planning in Complex Obstacle Environments

The performance of UAV path planning algorithm is mainly reflected in the convergence time, flight distance and flight time. However, in practical applications, the environment is quite complex, and many interference factors must be considered. In many cases, when the flight path of UAV reaches the global optimum, the convergence time of the algorithm will be affected, that is, the search ability and convergence ability of the algorithm are conflicting. Aiming at these problems, this paper establishes a complex flight environment, uses the Nash equilibrium method to transform the convergence and searchability of the genetic algorithm into two players in the game, and uses the Nash equilibrium method to find a balance between the convergence and searchability of the algorithm, so as to realize path planning of multiple UAVs in complex obstacle environments.
LWA-HAND: Lightweight Attention Hand for Interacting Hand Reconstruction

Hand reconstruction has achieved great success in real-time applications such as visual reality and augmented reality while interacting with two-hand reconstruction through efficient transformers is left unexplored. In this paper, we propose a method called lightweight attention hand (LWA-HAND) to reconstruct hands in low flops from a single RGB image. To solve the occlusion and interaction challenges in efficient attention architectures, we introduce three mobile attention modules. The first module is a lightweight feature attention module that extracts both local occlusion representation and global image patch representation in a coarse-to-fine manner. The second module is a cross image and graph bridge module which fuses image context and hand vertex. The third module is a lightweight cross-attention mechanism that uses element-wise operation for cross attention of two hands in linear complexity. The resulting model achieves comparable performance on the InterHand2.6M benchmark in comparison with the state-of-the-art models. Simultaneously, it reduces the flops to 0.47GFlops while the state-of-the-art models have heavy computations between 10GFlops and 20GFlops.
CRRNet: Channel Relation Reasoning Network for Salient Object Detection

Channel map matters for salient object detection. Effectively exploring the relationship between channels can enhance the relevant channel maps and infer a better saliency map. The existing channel map enhancement methods focus on learning the weights to re-weight the channel maps or using the similarity between the channels to combine the channel maps and enhance them. We propose a Channel Relation Reasoning Module (CRRM) based on graph convolution, inferring the semantic relations between channels and enhancing the channel maps according to the mutual relations. First, we carefully designed a multi-level complementary feature extraction module, which is used to extract multi-level edge and salient object internal features simultaneously. Then, the rich multi-level edge and salient features are fed to CRRM which projects the features into the hidden space and uses the graph convolution to establish the semantic relationship between the channels to enhance the feature expression. Finally, two stacked convolution layers can produce high-quality saliency maps. Extensive experiments on multiple datasets show the effectiveness of our model.
Multi-scale Multi-task Distillation for Incremental 3D Medical Image Segmentation

Automatic medical image segmentation is the core component for many clinical applications. Substantial number of deep learning based methods have been proposed in past years, but deploying such methods in practice faces certain difficulties, such as the acquisition of massive annotated data for training and the high latency of model iteration. In contrast to the conventional cycle of "data collection, offline training, model update", developing a system that continuously generates robust predictions will be critical. Recently, incremental learning was widely investigated for classification and semantic segmentation on 2D natural images. Existing work showed the effectiveness of data rehearsal and knowledge distillation in counteracting catastrophic forgetting. Inspired by these advances, we propose a multi-scale multi-task distillation framework for incremental learning with 3D medical images. Different from the task-incremental scenario in literature, our proposed strategy focuses on improving robustness against implicit data distribution shift. We introduce knowledge distillation as multi-task regularization to resolve prediction confusions. At each step, the network is instructed to learn towards both the new ground truth and the uncertainty weighted predictions from the previous model. Simultaneously, image features at multiple scales in the segmentation network could participate in a contrastive learning scheme, aiming at more discriminant representations that inherit the past knowledge effectively. Experiments showed that our method improved overall continual learning robustness under the extremely challenging scenario of "seeing each image once in a batch of one" without any pre-training. In addition, the proposed method could work on top of any network architectures and existing incremental learning strategies. We also showed further improvements by combining our method with data rehearsal using a small buffer.
Real-Time Security Monitoring System for Intelligent Multi-region Transfer Platform

In order to realize the effective monitoring and early warning of foreign objects around the transfer platform without dead angle, this paper designs a real-time security monitoring system for intelligent multi-region transfer platform based on the YOLOv5 target detection algorithm, including a security monitoring server and a client. Firstly, the dangerous area is calibrated on the captured image, and then based on the multi-thread technology, YOLOv5 is used to detect the foreign object in multiple areas around the transfer platform, and finally the ray method is used to judge whether the foreign object invades the detection area. The practical application results of an industrial site show that the designed system can detect foreign objects (pedestrians and trolleys) in real time within the dangerous area around the transfer platform, and can also set the dangerous area range online according to the user's requirements. In addition, system can complete the detection within 0.1 s and give an alarm prompt immediately when there is an abnormality. The system has a low false alarm rate of 1% to 2%, which reduces the occurrence of foreign objects invading dangerous areas and improves the safety of the production process.
Mine-Microseismic-Signal Recognition Based on Meta-learning

The mine management robot must have the ability to identify microseismic signals from collected data. Classical deep learning models are more susceptible to overfitting when dealing with small samples. To address this issue, this paper proposes two meta-learning-based methods to identify microseismic signals in mines. First, a set of wavelet basis functions is used to transform the microseismic and blast signals to generate 2D feature maps of wavelet coefficients. Then, two models based on convolutional neural networks were constructed using MAML and ANIL. Finally, comparative experiments of the two methods are conducted on a real data set. The experimental results show that the proposed methods in this paper have good performance in terms of efficiency and accuracy.
Construction and Practice of “Three ‘Wholes’ and Six ‘Integrations’” Intelligent Robot Education System

In recent years, the number of universities successfully applying for robot engineering has increased year by year, however, the educational resources of most universities can't fulfill the learning needs of such a highly interdisciplinary subject. In order to solve several outstanding problems commonly existed in the practical education of intelligent robots, this paper puts forward the concept of “three ‘wholes’ and six ‘integrations’” to build a complete system and institutional mechanism system of practical education for innovation and entrepreneurship, to cultivate students’ two abilities of “practice” and “innovation and entrepreneurship”. The methods may solve some problems in the practical intelligent robot’s education of entrepreneurship and innovation, aiming to help to form a virtuous circle system like an ecosystem. This will help college teachers to build a complete education system, which is conducive to cultivating high-quality talents who are better at innovation and entrepreneurship. This paper has four innovations: orientation, systematization, modularization and achievement.
CL-GAN: Contrastive Learning-Based Generative Adversarial Network for Modality Transfer with Limited Paired Data

Separate acquisition of multiple modalities in medical imaging is time-consuming, costly and increases unnecessary irradiation to patients. This paper proposes a novel deep learning method, contrastive learning-based Generative Adversarial Network (CL-GAN) for modality transfer with limited paired data. We employ CL-GAN to generate synthetic PET (synPET) images from MRI data, and it has a three-phase training pipeline: 1) intra-modality training for separate source (MRI) and target (PET) domain encoders, 2) cross-modality encoder training with MRI-PET pairs and 3) GAN training. As obtaining paired MRI-PET training data in sufficient quantities is often very costly and cumbersome in clinical practice, we integrate contrastive learning (CL) in all three training phases to fully leverage paired and unpaired data, leading to more accurate and realistic synPET images. Experimental results on benchmark datasets demonstrate the superior performance of CL-GAN, both qualitatively and quantitatively, when compared with current state-of-the-art methods.
Research on Network Traffic Anomaly Detection for Class Imbalance

It is actually a classification that the anomalistic detection on traffic of network belongs to. The imbalance of data about network traffic, leading to the disability of anomaly model on learning the characteristics of the minority class samples, makes the detection accuracy in the minority samples poor. Aiming at the imbalance of network flow data, a network traffic anomaly detection method combining Deep Convolutional Generative Adversarial Networks (DCGAN) and Vision Transformer is proposed to make the detection accuracy of network traffic data more perfect. The network traffic dataset is sampled, and the sampled imbalanced dataset is class balanced by DCGAN. The class balanced network traffic data is fed into Vision Transformer for prediction. To simplify the model structure, the vision Transformer model is proposed, and only N encoders are included in the vision Transformer, and the decoder structure is removed. Using residual network in encoder to solve the problem of model degradation. The dataset uses CIC-IDS-2017 network intrusion detection data. After experiments, it was enough to hold that the model which was proposed can positively perform class balancing of data and increase the prediction precision of data about network traffic.
SpiderTailed: A Tool for Detecting Presentation Failures Using Screenshots and DOM Extraction

In smartphone and Web applications, a major problem arises when the rendering result of a GUI component does not meet a defined specification. In this study, we refer to such problems as Presentation Failure, and although many methods have been proposed to detect Presentation Failure, most of them are achieved by directly comparing images of the application output with the specified specifications. Therefore, a large difference between the two may induce misdetections of Presentation Failure. This study proposes a method to extract abstract visual properties from the output of before and after version web pages and compare visual properties extracted correspondingly. Specifically, screenshots are taken from each image for each element, and visual properties are extracted using computer vision techniques. Furthermore, we implement the proposed method as a tool (SpiderTailed) and describe the comparison experiment results with a manual observation test to confirm the effectiveness of the method.
Patient-Level Microsatellite Stability Assessment from Whole Slide Images by Combining Momentum Contrast Learning and Group Patch Embeddings

Assessing microsatellite stability status of a patient's colorectal cancer is crucial in personalizing treatment regime. Recently, convolutional-neural-networks (CNN) combined with transfer-learning approaches were proposed to circumvent traditional laboratory testing for determining microsatellite status from hematoxylin and eosin stained biopsy whole slide images (WSI). However, the high resolution of WSI practically prevent direct classification of the entire WSI. Current approaches bypass the WSI high resolution by first classifying small patches extracted from the WSI, and then aggregating patch-level classification logits to deduce the patient-level status. Such approaches limit the capacity to capture important information which resides at the high resolution WSI data. We introduce an effective approach to leverage WSI high resolution information by momentum contrastive learning of patch embeddings along with training a patient-level classifier on groups of those embeddings. Our approach achieves up to 7.4% better accuracy compared to the straightforward patch-level classification and patient level aggregation approach with a higher stability (AUC, $$0.91 \pm 0.01$$ vs. $$0.85 \pm 0.04$$ , p-value < 0.01). Our code can be found at https://github.com/TechnionComputationalMRILab/colorectal_cancer_ai .
Today’s Recomposition of the Money Supply

Today, the future belongs to digital money, for it is technologically superior to book money. Central Bank Digital Currency (CBDC) and private digital currencies—be they stablecoins or unbacked cryptos—are vying to succeed bank money as the dominant type of money. CBDC is likely to emerge from this as the winner, because CBDC is superior to private cryptos by virtue of being legal tender and safe-stock base money, and also because of the no longer negligible constitutional need to take back monetary control and be able to implement monetary policy effectively. Bank money, presently still dominant, competes against both CBDC and private digital monies to assert its position. Bank money may well persist for a longer period of time, albeit gradually declining in importance. Central-bank reserves will over time be going down together with bank money, or be replaced with CBDC early on. Solid cash is not a systemically defining factor any more.
Complementary Phase Encoding for Pair-Wise Neural Deblurring of Accelerated Brain MRI

MRI has become an invaluable tool for diagnostic brain imaging, providing unrivalled qualitative and quantitative information to the radiologist. However, due to long scanning times and capital costs, access to MRI lags behind CT. Typical brain protocols lasting over 30 min set a clear limitation to patient experience, scanner throughput, operation profitability, and lead to long waiting times for an appointment. As image quality, in terms of spatial resolution and noise, is strongly dependent on acquisition duration, significant scanning acceleration must successfully address challenging image degradation. In this work, we consider the scan acceleration scenario of a strongly anisotropic acquisition matrix. We propose a neural approach that jointly deblurs scan pairs acquired with mutually orthogonal phase encoding directions. This leverages the complementarity of the respective phase encoded information as blur directions are also mutually orthogonal between the scans in the pair. The proposed architecture, trained end-to-end, is applied to T1w scan pairs consisting of one scan with contrast media injection (CMI), and one without. Qualitative and quantitative validation is provided against state-of-the-art deblurring methods, for an acceleration factor of 4 beyond compressed sensing acceleration. The proposed method outperforms the compared methods, suggesting its possible clinical applicability for this challenging task.
Intelligent Security Video Surveillance System Based on GCN

As the competitive focus of video surveillance technology, intelligent system that recognizes abnormal human behavior for home security is increasingly in demand. We propose a framework aiming to take a holistic approach to analyze human behavior. When abnormal behavior is detected, users can receive alert through SMS. Moreover, database is involved to record abnormal behavior for view and search later on. This framework is developed by integrating state-of-art advancements and technologies in human skeleton recognition, human behavior recognition and data augmentation. Our system has been tested and proven to work well. It’s promising that this framework can conduce to the improvement of living standard in home security monitoring.
Self-attention Capsule Network for Tissue Classification in Case of Challenging Medical Image Statistics

We propose the first Self-Attention Capsule Network that was designed to deal with unique core challenges of medical imaging, specifically for tissue classification. These challenges are - significant data heterogeneity with statistics variability across imaging domains, insufficient spatial context and local fine-grained details, and limited training data. Moreover, our proposed method solves limitations of the baseline Capsule Networks (CapsNet) such as handling complicated challenging data and limited computational resources. To cope with these challenges, our method is composed of a self-attention module that simplifies the complexity of the input data such that the CapsNet routing mechanism can be efficiently used, while extracting much richer contextual information, compared with CNNs. To demonstrate the strengths of our method, it was extensively evaluated on three diverse medical datasets and three natural benchmarks. The proposed method outperformed other methods we compared with in classification accuracy but also in robustness, within and across different datasets and domains.
Racial Bias in the Beautyverse: Evaluation of Augmented-Reality Beauty Filters

This short paper proposes a preliminary and yet insightful investigation of racial biases in beauty filters techniques currently used on social media. The obtained results are a call to action for researchers in Computer Vision: such biases risk being replicated and exaggerated in the Metaverse and, as a consequence, they deserve more attention from the community.
Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI &amp; AIM 2022 Challenge: Report

Video super-resolution is one of the most popular tasks on mobile devices, being widely used for an automatic improvement of low-bitrate and low-resolution video streams. While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices. In this Mobile AI challenge, we address this problem and propose the participants to design an end-to-end real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. The participants were provided with the REDS training dataset containing video sequences for a 4X video upscaling task. The runtime and power efficiency of all models was evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of accelerating floating-point and quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt/30 FPS] power consumption. A detailed description of all models developed in the challenge is provided in this paper.
Autonomous Object Tracking System of UAV Based on Correlation Filtering

Aiming at the problem of UAV autonomous object tracking under the condition of limited computing resources, this paper combines the high-speed Kernel Correlation Filter (KCF) tracking algorithm with UAV motion control to construct a vision-based UAV autonomous object tracking system. Firstly, the RGB data is captured through the UAV on-board camera and transmitted to the control terminal; After calibrating the initial frame of the tracked object, the control terminal uses KCF algorithm to predict the position of the tracked object at the subsequent time; Then, the relative position of the UAV to the object is estimated based on the visual tracking results; Finally, the relative position of the UAV and the object is used to calculate the movement control amount of the UAV. The experimental results show that the vision-based UAV autonomous object tracking system constructed in this paper can achieve fast and robust UAV autonomous object tracking and relative position maintenance with CPU support and low memory usage.
Multi-agent Simulation for Risk Prediction in Student Projects with Real Clients

We have conducted a software development project course with real clients: non-profit organizations of local communities, local governments, and special support schools. Although such a project course can bring project work experiences with real software development to students, many risks are inherent in projects. In this paper, we focus on schedule delay and workload imbalance risks. We propose an analysis approach to predict such risks based on a multi-agent simulation model, including task dependencies expressed using program evaluation and review techniques (PERT). The simulation result shows that task allocation methods may affect the possibility of schedule delay and workload imbalance risks.