Reviewerly/exHarmony huggingface dataset:
img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation

We propose real-time, six degrees of freedom (6DoF), 3D face pose estimation without face detection or landmark localization. We observe that estimating the 6DoF rigid transformation of a face is a simpler problem than facial landmark detection, often used for 3D face alignment. In addition, 6DoF offers more information than face bounding box labels. We leverage these observations to make multiple contributions: (a) We describe an easily trained, efficient, Faster R-CNN–based model which regresses 6DoF pose for all faces in the photo, without preliminary face detection. (b) We explain how pose is converted and kept consistent between the input photo and arbitrary crops created while training and evaluating our model. (c) Finally, we show how face poses can replace detection bounding box training labels. Tests on AFLW2000-3D and BIWI show that our method runs at real-time and outperforms state of the art (SotA) face pose estimators. Remarkably, our method also surpasses SotA models of comparable complexity on the WIDER FACE detection benchmark, despite not been optimized on bounding box labels.
HVPR: Hybrid Voxel-Point Representation for Single-stage 3D Object Detection

We address the problem of 3D object detection, that is, estimating 3D object bounding boxes from point clouds. 3D object detection methods exploit either voxel-based or point-based features to represent 3D objects in a scene. Voxel-based features are efficient to extract, while they fail to preserve fine-grained 3D structures of objects. Point-based features, on the other hand, represent the 3D structures more accurately, but extracting these features is computationally expensive. We introduce in this paper a novel single-stage 3D detection method having the merit of both voxel-based and point-based features. To this end, we propose a new convolutional neural network (CNN) architecture, dubbed HVPR, that integrates both features into a single 3D representation effectively and efficiently. Specifically, we augment the point-based features with a memory module to reduce the computational cost. We then aggregate the features in the memory, semantically similar to each voxel-based one, to obtain a hybrid 3D representation in a form of a pseudo image, allowing to localize 3D objects in a single stage efficiently. We also propose an Attentive Multi-scale Feature Module (AMFM) that extracts scale-aware features considering the sparse and irregular patterns of point clouds. Experimental results on the KITTI dataset demonstrate the effectiveness and efficiency of our approach, achieving a better compromise in terms of speed and accuracy.
An Overview of Twenty-Five Years of Augmented Reality in Education

Augmented reality (AR) enables an interactive experience with the real world where real-world objects are enhanced with computer-generated perceptual information. Twenty-five years have passed since the first AR application designed exclusively to be used in educational settings. Since then, this technology has been successfully implemented to enrich educational contexts providing learning gains, motivation, enjoyment, autonomy, among other benefits. This study provides an overview of AR technology in education from its origins to the present. Consequently, based on the analysis of its evolution, the study defines three generations of AR in education. Moreover, the study identifies some major challenges from previous AR applications and, finally, it poses some insights to address these challenges to enhance the benefits of AR for education.
Residual Attention: A Simple but Effective Method for Multi-Label Recognition

Multi-label image recognition is a challenging computer vision task of practical use. Progresses in this area, how-ever, are often characterized by complicated methods, heavy computations, and lack of intuitive explanations. To effectively capture different spatial regions occupied by objects from different categories, we propose an embarrassingly simple module, named class-specific residual attention (CSRA). CSRA generates class-specific features for every category by proposing a simple spatial attention score, and then combines it with the class-agnostic average pooling feature. CSRA achieves state-of-the-art results on multi-label recognition, and at the same time is much simpler than them. Furthermore, with only 4 lines of code, CSRA also leads to consistent improvement across many diverse pretrained models and datasets without any extra training. CSRA is both easy to implement and light in computations, which also enjoys intuitive explanations and visualizations.
Business Innovation based on artificial intelligence and Blockchain technology

The growing business evolution and the latest Artificial Intelligence (AI) make the different business practices to be enhanced by the ability to create new means of collaboration. Such growing technology helps to deliver brand services and even some new kinds of corporate interactions with customers and staff. AI digitization simultaneously emphasized businesses to focus on the existing strategies and regularly and early pursue new market opportunities. While digital technology research in the framework of business innovation is gaining greater interest and the privacy of data can be maintained by Blockchain technology. Therefore in this paper, Business Innovation based on artificial intelligence and Blockchain technology (BI-AIBT) has been proposed to enhance the business practices and maintain the secured interaction among the various clients. The collection of qualitative empirical data is made up of few primary respondents from two distinct business sectors. BI-AIBT has been evaluated by undertaking and exploring the difference and similarities between digitalization's impact on value development, proposal, and business capture. Besides, organizational capacities and staff skills interaction issues can be improved by BT. The experimental result suggests that digital transformation is usually regarded as essential and improves business innovation strategies. The numerical result proposed BI-AIBT improves the demand prediction ratio (97.1%), product quality ratio (98.3%), Business development ratio (98.9%), customer behavior analysis ratio (96.3%), and customer satisfaction ratio (97.2%).
Bifurcated Backbone Strategy for RGB-D Salient Object Detection

Multi-level feature fusion is a fundamental topic in computer vision. It has been exploited to detect, segment and classify objects at various scales. When multi-level features meet multi-modal cues, the optimal feature aggregation and multi-modal learning strategy become a hot potato. In this paper, we leverage the inherent multi-modal and multi-level nature of RGB-D salient object detection to devise a novel Bifurcated Backbone Strategy Network (BBS-Net). Our architecture, is simple, efficient, and backbone-independent. In particular, first, we propose to regroup the multi-level features into teacher and student features using a bifurcated backbone strategy (BBS). Second, we introduce a depth-enhanced module (DEM) to excavate informative depth cues from the channel and spatial views. Then, RGB and depth modalities are fused in a complementary way. Extensive experiments show that BBS-Net significantly outperforms 18 state-of-the-art (SOTA) models on eight challenging datasets under five evaluation measures, demonstrating the superiority of our approach (~4% improvement in S-measure vs . the top-ranked model: DMRA). In addition, we provide a comprehensive analysis on the generalization ability of different RGB-D datasets and provide a powerful training set for future research. The complete algorithm, benchmark results, and post-processing toolbox are publicly available at https://github.com/zyjwuyan/BBS-Net.
Geometry Uncertainty Projection Network for Monocular 3D Object Detection

Geometry Projection is a powerful depth estimation method in monocular 3D object detection. It estimates depth dependent on heights, which introduces mathematical priors into the deep model. But projection process also introduces the error amplification problem, in which the error of the estimated height will be amplified and reflected greatly at the output depth. This property leads to uncontrollable depth inferences and also damages the training efficiency. In this paper, we propose a Geometry Uncertainty Projection Network (GUP Net) to tackle the error amplification problem at both inference and training stages. Specifically, a GUP module is proposed to obtains the geometry-guided uncertainty of the inferred depth, which not only provides high reliable confidence for each depth but also benefits depth learning. Furthermore, at the training stage, we propose a Hierarchical Task Learning strategy to reduce the instability caused by error amplification. This learning algorithm monitors the learning situation of each task by a proposed indicator and adaptively assigns the proper loss weights for different tasks according to their pre-tasks situation. Based on that, each task starts learning only when its pre-tasks are learned well, which can significantly improve the stability and efficiency of the training process. Extensive experiments demonstrate the effectiveness of the proposed method. The overall model can infer more reliable object depth than existing methods and outperforms the state-of-the-art image-based monocular 3D detectors by 3.74% and 4.7% AP <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">40</inf> of the car and pedestrian categories on the KITTI benchmark. The code and model will be released at https://github.com/SuperMHP/GUPNet.
Hierarchical graph representations in digital pathology

Cancer diagnosis, prognosis, and therapy response predictions from tissue specimens highly depend on the phenotype and topological distribution of constituting histological entities. Thus, adequate tissue representations for encoding histological entities is imperative for computer aided cancer patient care. To this end, several approaches have leveraged cell-graphs, capturing the cell-microenvironment, to depict the tissue. These allow for utilizing graph theory and machine learning to map the tissue representation to tissue functionality, and quantify their relationship. Though cellular information is crucial, it is incomplete alone to comprehensively characterize complex tissue structure. We herein treat the tissue as a hierarchical composition of multiple types of histological entities from fine to coarse level, capturing multivariate tissue information at multiple levels. We propose a novel multi-level hierarchical entity-graph representation of tissue specimens to model the hierarchical compositions that encode histological entities as well as their intra- and inter-entity level interactions. Subsequently, a hierarchical graph neural network is proposed to operate on the hierarchical entity-graph and map the tissue structure to tissue functionality. Specifically, for input histology images, we utilize well-defined cells and tissue regions to build HierArchical Cell-to-Tissue (HACT) graph representations, and devise HACT-Net, a message passing graph neural network, to classify the HACT representations. As part of this work, we introduce the BReAst Carcinoma Subtyping (BRACS) dataset, a large cohort of Haematoxylin & Eosin stained breast tumor regions-of-interest, to evaluate and benchmark our proposed methodology against pathologists and state-of-the-art computer-aided diagnostic approaches. Through comparative assessment and ablation studies, our proposed method is demonstrated to yield superior classification results compared to alternative methods as well as individual pathologists. The code, data, and models can be accessed at https://github.com/histocartography/hact-net.
A Haze Prediction Model in Chengdu Based on LSTM

Air pollution with fluidity can influence a large area for a long time and can be harmful to the ecological environment and human health. Haze, one form of air pollution, has been a critical problem since the industrial revolution. Though the actual cause of haze could be various and complicated, in this paper, we have found out that many gases’ distributions and wind power or temperature are related to PM2.5/10’s concentration. Thus, based on the correlation between PM2.5/PM10 and other gaseous pollutants and the timing continuity of PM2.5/PM10, we propose a multilayer long short-term memory haze prediction model. This model utilizes the concentration of O3, CO, NO2, SO2, and PM2.5/PM10 in the last 24 h as inputs to predict PM2.5/PM10 concentrations in the future. Besides pre-processing the data, the primary approach to boost the prediction performance is adding layers above a single-layer long short-term memory model. Moreover, it is proved that by doing so, we could let the network make predictions more accurately and efficiently. Furthermore, by comparison, in general, we have obtained a more accurate prediction.
Friendly or competent? The effects of perception of robot appearance and service context on usage intention

This study examines the influence of the perception of robot appearance and service context on customers'/tourists' intention to use robots. Four studies confirm a congruity effect between the perception of service robot appearance and service context. Results suggest that customers/tourists are more willing to use a service robot perceived as warm in hedonic service contexts, whereas they are more willing to use a service robot perceived as competent in utilitarian service contexts. These effects are driven by trust. This study offers a novel perspective on customers'/tourists' reactions to service robots. It also provides insights for hospitality and tourism practitioners making decisions about robot adoption.
Multilayer Reservoir Computing Based on Ferroelectric α‐In<sub>2</sub>Se<sub>3</sub> for Hierarchical Information Processing

Dynamic physical systems such as reservoir computing (RC) architectures show a great prospect in temporal information processing, whereas hierarchical information processing capability is still lacking due to the absence of advanced multilayer reservoir elements. Here, a stackable reservoir system is constructed based on ferroelectric α-In2 Se3 devices with voltage input and output, which is realized by dynamic voltage division between a ferroelectric field-effect transistor and a planar device and therefore allows the reservoirs to cascade, enabling multilayer RC. Fast Fourier transformation analysis shows high-harmonic generation in the first layer as a result of inherent nonlinearity of the reservoir, and progressive low-pass filtering effect is realized where higher-frequency components are progressively filtered in deeper-layer RCs. Time-series prediction and waveform classification tasks are also demonstrated, serving as evidence for the memory capacity and computing capability of the deep reservoir architecture. This work can provide a promising pathway in exploiting emerging 2D materials and dynamics for advanced neuromorphic computing architectures.
Channel-wise Knowledge Distillation for Dense Prediction

Knowledge distillation (KD) has been proven a simple and effective tool for training compact dense prediction models. Lightweight student networks are trained by extra supervision transferred from large teacher networks. Most previous KD variants for dense prediction tasks align the activation maps from the student and teacher network in the spatial domain, typically by normalizing the activation values on each spatial location and minimizing point-wise and/or pair-wise discrepancy. Different from the previous methods, here we propose to normalize the activation map of each channel to obtain a soft probability map. By simply minimizing the Kullback–Leibler (KL) divergence between the channel-wise probability map of the two networks, the distillation process pays more attention to the most salient regions of each channel, which are valuable for dense prediction tasks.We conduct experiments on a few dense prediction tasks, including semantic segmentation and object detection. Experiments demonstrate that our proposed method outperforms state-of-the-art distillation methods considerably, and can require less computational cost during training. In particular, we improve the RetinaNet detector (ResNet50 backbone) by 3.4% in mAP on the COCO dataset, and PSPNet (ResNet18 backbone) by 5.81% in mIoU on the Cityscapes dataset. Code is available at: https://git.io/Distiller
Anticipative Video Transformer

We propose Anticipative Video Transformer (AVT), an end-to-end attention-based video modeling architecture that attends to the previously observed video in order to anticipate future actions. We train the model jointly to predict the next action in a video sequence, while also learning frame feature encoders that are predictive of successive future frames’ features. Compared to existing temporal aggregation strategies, AVT has the advantage of both maintaining the sequential progression of observed actions while still capturing long-range dependencies—both critical for the anticipation task. Through extensive experiments, we show that AVT obtains the best reported performance on four popular action anticipation benchmarks: EpicKitchens-55, EpicKitchens-100, EGTEA Gaze+, and 50-Salads; and it wins first place in the EpicKitchens-100 CVPR’21 challenge.
Exploration in deep reinforcement learning: A survey

This paper reviews exploration techniques in deep reinforcement learning. Exploration techniques are of primary importance when solving sparse reward problems. In sparse reward problems, the reward is rare, which means that the agent will not find the reward often by acting randomly. In such a scenario, it is challenging for reinforcement learning to learn rewards and actions association. Thus more sophisticated exploration methods need to be devised. This review provides a comprehensive overview of existing exploration approaches, which are categorized based on the key contributions as follows reward novel states, reward diverse behaviours, goal-based methods, probabilistic methods, imitation-based methods, safe exploration and random-based methods. Then, the unsolved challenges are discussed to provide valuable future research directions. Finally, the approaches of different categories are compared in terms of complexity, computational effort and overall performance.
Light Field Image Compression Based on Bi-Level View Compensation With Rate-Distortion Optimization

Compared with conventional color images, light field images (LFIs) contain richer scene information, which allows a wide range of interesting applications. However, such additional information is obtained at the cost of generating substantially more data, which poses challenges to both data storage and transmission. In this paper, we propose a new hybrid framework for effective compression of LFIs. The proposed framework takes the particular characteristics of LFIs into account so that the inter- and intra-view correlations of LFIs can be more efficiently exploited to produce better compression performance. Specifically, the proposed scheme partitions sub-aperture images (SAIs) of an LFI into two groups, namely, key SAIs and non-key SAIs. Bi-level view compensation is proposed to exploit the inter-view correlation: first, based on the group of selected key SAIs, learning-based angular super-resolution is performed to compensate non-key SAIs in pixel-wise, during which heterogeneous inter-view correlation between the non-key SAIs is efficiently removed; second, the two groups of SAIs are respectively reorganized as pseudo-sequences, and block-wise motion compensation is carried out with a standard video encoder, during which the homogeneous inter-view correlation is subsequently exploited. The video encoder also helps to remove the intra-view correlation of the SAIs and finally generates the encoded bitstream. Moreover, the bits allocated to each group are optimally determined via model-based rate distortion optimization. Extensive experimental evaluations and comparisons demonstrate the advantage of the proposed framework over existing methods in terms of rate-distortion performance.
Wearable technologies for hand joints monitoring for rehabilitation: A survey

Hand deformities often become a major obstacle in conducting everyday tasks for many people around the globe. Rehabilitation procedures are widely used for strengthening the hand muscles, which in turn leads to the restoration of functionality of the affected hand. This paper conducts a survey of various wearable technologies that can be used to accurately quantify the rehabilitation progress in terms of fingers' hand joint angles. Based on the data acquisition methods, these technologies can be mainly divided into six categories: 1) Flex sensor based; 2) Accelerometer based; 3) Vision based; 4) Hall-effect based; 5) Stretch sensor based; and 6) Magnetic sensor based. The main focus of some of the discussed technologies has been on various other domains, like gaming gloves, tele-manipulation etc., and thus their usage for rehabilitation of hand joints could be quite interesting. This paper analyzes the strengths and weaknesses of these wearable technologies along with some examples of their implementations. Based on our survey results, we propose a wearable glove for accurately measuring hand joint angles with enhanced features for better diagnosis and rehabilitation.
Mashup Recommendation by Regularizing Matrix Factorization with API Co-Invocations

Mashups are a dominant approach for building data-centric applications, especially mobile applications, in recent years. Since mashups are predominantly based on public data sources and existing APIs, it requires no sophisticated programming knowledge of people to develop mashup applications. The recent prevalence of open APIs and open data sources in the Big Data era has provided new opportunities for mashup development, but at the same time increase the difficulty of selecting the right services for a given mashup task. The API recommendation for mashup differs from traditional service recommendation tasks in lacking the specific QoS information and formal semantic specification of the APIs, which limits the adoption of many existing methods. Although there are a significant number of service recommendation approaches, most of them focus on improving the recommendation accuracy and work pays attention to the diversity of the recommendation results. Another challenge comes from the existence of both explicit and implicit correlations among the different APIs, which are generally neglected by existing recommendation methods. In this paper, we address the above deficiencies of existing approaches by exploring API recommendation for mashups in the reusable composition context, with the goal of helping developers identify the most appropriate APIs for their composition tasks. In particular, we propose a probabilistic matrix factorization approach with implicit correlation regularization to solve the recommendation problem and enhance the recommendation diversity. We conjecture that the co-invocation of APIs in real-world mashups is driven by both the explicit textual similarity and implicit correlations of APIs such as the similarity or the complementary relationship of APIs. We develop a latent variable model to uncover the latent correlations between APIs by analyzing their co-invocation patterns. We further explore the relationships of topics/categories to the proposed approach. We demonstrate the effectiveness of our approach by conducting extensive experiments on a real dataset crawled from ProgrammableWeb.