Reviewerly/exHarmony huggingface dataset:
DunhuangGAN: A Generative Adversarial Network for Dunhuang Mural Art Style Transfer

Style transfer has been successfully applied to various visual art creation tasks. However, due to the inherent style of Dunhuang mural art, existing methods can not produce high-quality mural art stylized images. We propose a novel model of DunhuangGAN for Dunhuang mural art style transfer. DunhuangGAN is based on the improved contrastive learning framework and optimized under the proposed multiple loss. Firstly, we propose a content-biased contrastive loss to alleviate the negative impacts caused by the inter-domain style differences. Secondly, we propose the line loss and color loss to simulate the line drawing modeling and heavy color of Dunhuang mural art. In addition, we introduce semantic loss to improve the visual effect of certain content element areas in generated images that are rare in the Dunhuang murals. Extensive experiments based on the collected dataset show that our method outperforms existing methods in the Dunhuang mural art style transfer task.
Dynamic Transformer for Image Captioning

Transformer-based architectures have shown great success in image captioning. However, the calculation process of the model is predefined, which is heavily dependent on expert experience and lacks flexibility. In this paper, we propose a Dynamic Transformer to change the encoding procedure from sequential to adaptive, i.e., data-dependent computing paths. Specifically, we design three different types of feature extraction blocks, each containing a calculation unit to perform the corresponding operation and a routing gate to select the direction to pass the signal based on the input data. Three blocks are deployed in parallel at each layer to build a multi-layer routing space in a fully connected manner. In addition, we use a path regularization constraint model to learn similar paths for semantically similar images. We conduct extensive experiments on the COCO image captioning dataset and achieve new state-of-the-art performance, i.e., 133.2% CIDEr on Karpathy test split.
Mining Valuable Source Domain Instances for Privacy-Preserving Domain Adaptive Object Detection

Conventional Domain Adaptive Object Detection (DAOD) usually learns a well-performed detector by aligning the source/target representations in hidden space, which is un-practical in real situations because of data privacy. Recently, some methods based on pseudo-label technique have been proposed for privacy-preserving DAOD. However, they bring new difficulties in evaluating the quality of these labels. In this paper, we propose a Mining Valuable source domain instances (MiVi) algorithm for privacy-preserving DAOD where data from different domains cannot be aggregated. Specifically, we explore a novel Instances Refinement strategy to mine valuable instance-level samples from the source data, which are close to the feature distribution of the target domain. To achieve this, we are the first to introduce self-supervised learning to guide the detector to learn the instance-level features in different domains. Then, by reserving labeled source domain instance-level samples with domain-invariant features and discarding the ones with source domain-specific features, we transform unsupervised DAOD into supervised object detection. Extensive experiments in four representative DAOD scenarios show that our method outperforms existing methods.
Enhancing Localization Accuracy of Indoor Occupancy Tracking Using Optical Camera Communication and Human Pose Estimation

Precise location of personnel indoors is one of the crucial prerequisite of data analytics of workforce productivity as well as workplace safety and health. In this paper, we propose an enhanced indoor occupancy tracking system using optical camera communication (OCC) on top of conventional surveillance cam-eras. The proposed system is able to track workers who carry unique infrared LED beacons indoor. OCC with infrared beacons is adopted for identity check, while the localization accuracy is enhanced using deep-learning-based human pose estimation to infer the footfall location. Particularly, a comprehensive footfall estimation algorithm empowered by deep-learning-based human pose estimation is presented, which provides precise footfall estimation considering real-time actions of person and occlusion patterns. In order to validate the accuracy improvement, The experiment is conducted in an open lab with area of about 30 m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> using single surveillance camera sensors. According to the experimental results, the localization accuracy is improved by 9.36% on average and 44.19% at far end, in comparison with conventional methods.
Augmented Reality for Assisting Students in the Learning Riau Malay Culture Local Subject during COVID-19 Pandemic

Corona Virus Disease 2019 or known as COVID-19 disseminated Indonesia in March 2020. The presence of this virus leads to fear and worry, all work activities are carried out from home. This is also perceived by the Public Elementary School number 189 Pekanbaru (SDN 189), particularly in the subject Budaya Melayu Riau (BMR) or Riau Malay Culture. BMR is a subject on heritage that characterizes Riau Malay culture; it has material evidence objects which are the results of human cultural history, nature and the environment are notable to be preserved. The problem faced by SDN 189 is that the level of students' comprehension towards BMR subject is diminished during COVID-19. The solution offered in solving this problem is to create fun learning technology through online learning using Android-based Augmented Reality (AR). AR is a type of interactive technology combines actual and virtual objects to produce 3D objects that are real in life. Therefore, the ultimate goal of this research is to intensify the knowledge and teacher’s skills in teaching and enhance students' understanding of BMR subjects.
Searching Latent Sub-Goals in Hierarchical Reinforcement Learning as Riemannian Manifold Optimization

Hierarchical Reinforcement Learning (HRL) is promising to tackle the long-term sparse reward problem. However, goal conditioned HRL, which decomposes the goal into a series of sub-goals, suffers from sub-goal search inefficiency problems when the observation space is too large. This problem is more severe in a visual observation space, since its high latent dimensions, where the complete dynamics information is preserved, exponentially increase the difficulty of sub-goal search. In view of this, we propose to treat the latent space as a manifold, i.e., a Riemannian manifold. Assisted by the Riemannian manifold optimization, sub-goals can be efficiently searched in the higher-dimensional latent space, with the help of preserving the dynamics information efficiently. Experiments on a series of MuJoCo tasks with visual observation show that the proposed Riemannian manifold optimization, compared with the baseline that directly searches for sub-goals in bounded latent space, improves the success rate by 1.5 times on average. In much higher dimensions where the baseline no longer converges, the success rate of the proposed method is maintained.
Automated Pap Smear Cervical Cancer Detection Based on Multiscale Convolutional Neural Network

Cervical cancer is one deadly common gynecological malignant tumor, early accurate screening detection can save lives. However, the large-scale manual screening is limited by the few experienced cytologists, which is difficult to change under the existing conditions. In this paper, we propose an automatic detection framework based on deep learning to improve the detection accuracy and efficiency of cervical cancer cells. Different from the traditional convolutional neural network with a top-down structure, we design an isomorphic dual-branch residual-like structure as a multi-scale network. The multi-scale network can increase the range of the receptive field with each network layer at a fine-grained level to accurately learn the representation information of pathological images. In addition, we also perform ablation studies on different widths, residual block types, and patch extraction methods in the backbone network. The experimental results show that our framework has achieved an AP of 89.7% in cervical cancer cell detection, which is superior to other existing detection frameworks.
Group Correspondence: A Statistical Perspective for Incomplete Multi-View Clustering Augmentation

Cross-view consistency is the fundamental property of multiview clustering. However, in incomplete multi-view scenarios, existing methods can only pursue consistency through the paired data while ignoring the information in unpaired data. In this paper, we show a new insight from the data pattern and provide a novel perspective to incorporate unpaired data for consistency maximization by mining group correspondence. We first formulate cross-view consistency in a statistical perspective to by-pass the strict demand of instance correspondence, and then propose a technique to construct corresponding groups across views to enhance the objective of consistency maximization. Our proposal can be used as a universal plug-in to augment existing approaches. We test the efficacy and generality of our proposal by adapting it to two base methods as augmentations and comparing the augmented models against the original ones and other baselines. Experiment results demonstrate the effectiveness of our proposal and validate the value of our insight.
Towards Robust Video Text Detection with Spatio-Temporal Attention Modeling and Text Cues Fusion

Information carried by video text is of great value to various video applications. However, detecting text in videos of- ten faces great challenges due to the widely varied appearance of text and the complicated, dynamic video context. In this paper, we propose a robust video text detection network that adaptively combines relevant text cues in multiple frames with spatio-temporal attention and fusion mechanisms, which effectively enhance the accuracy and robustness of video text detection compared to single-frame detection. The network first localizes text region proposals and propagates them across frames with an R-CNN based framework. Then, a Transformer-based cross-frame feature fusion model is employed to attentively select and combine relevant text features, yielding an enhanced representation of text region integrating complementary text cues for robust text candidate prediction. The network achieves competitive text detection performance on standard video text benchmarks, demonstrating the effectiveness of the proposed method.
Shadow Removal Through Learning-Based Region Matching and Mapping Function Optimization

In this paper, we develop a novel shadow removal technique where the inputs are a single natural image to be restored and its corresponding shadow mask. We first decompose the image by super-pixels and cluster them into several sim-ilar regions. Then we train a random forest model to pre-dict matched pairs between shadow and non-shadow regions. By applying a distribution-based mapping function on the matched pairs, we can relight pixels in those shadow regions. An optimization framework based on half-quadratic splitting (HQS) method is also introduced to further improve the qual-ity of the mapping process. We also design a post-processing stage with a boundary inpainting function to generate bet-ter visual results. Our experiments show that the proposed method can remove shadows effectively and produce high-quality shadow-free images.
Development Based on Augmented Reality, 3D Hologram, Infographic, and Video Animation (ARKHANA) to Improve 21<sup>st</sup> Century Skills

This research was aimed to identify learning media development process of digital-based history in order to find 21st century skill on the students and find alternative solution which could fulfill student and teacher needs within the implementation of online history learning activities. To achieve the goals, the learning method should be contained of these following steps: (1) problem formulation, (2) problem and field data search, (3) literature review, (4) analysis, and (5) result presentation. Through those steps in the learning method, the researcher found that ARKHANA (Augmented Reality, 3D Hologram, Info graphic, and Video Animation) product was stated as a feasible alternative solution and could be used as the means to realize research aims. To put in an evidence, based on several analysis activities such as need analysis, competence analysis, technology analysis, and SWOT analysis, ARKHANA product was said to be an appropriate solution, since it was highly innovative, easy to use, and able to accommodate student learning styles including to visual style, and ease the teacher in teaching and student habituation to utilize 21st century skill.
Evaluating the Effect of Cinematography on the Viewing Experience in Immersive Environment

Cinematic Virtual Reality (CVR) is an increasingly popular digital art production technology that could enhance the sense of presence when a viewer explores immersive environments. There are three important viewing-experience-related aspects, attention, sustainability, and guidance, which can be affected by the cinematography principles. Attention indicates whether the viewer is focusing on the storytelling-related region or not. Sustainability refers to viewers' ability to continuously watch the CVR content, and guidance affects the understanding of the narrative. In this paper, we conducted within-subject repeated-measures experiments on 22 participants in an HMD-based immersive environment, to explore the correlation between viewing experience and comprehensive factors. According to experimental results, we suggest an attention-comfort-understanding analysis paradigm for directing the CVR shot, which could help creators effectively attract viewers' attention, minimize the cybersickness, and deepen their understanding of narratives.
Semantic-Driven Saliency-Context Separation for Video Captioning

Video captioning aims at generating a natural language de-scription for a given video clip including not only salient sce-narios but also contextual scenarios. The former reveal the highlight of a video and are usually the focus of most existing captioning methods. The latter, however, are not well ex-plored and even ignored easily, though they may provide cer-tain detailed and latent information that can help with a better understanding of the video. To effectively exploit the infor-mation contained in both, a novel video captioning network is proposed. It has two key modules: Cross-Modality Selection (CMS) and Saliency-Context Adaptive Decoder (SCAD). Specifically, CMS mainly focuses on utilizing the semantic information to distinguish saliency and context. Meanwhile, SCAD adaptively identifies both the saliency and context to generate more detailed and precise captions. Experiments on two benchmark datasets, i.e., MSVD and MSR-VTT, demon-strate the effectiveness of our model through the comparison with state-of-the-art methods.
SEPL-Net: A Semantics-Enhanced Pseudo Labeling Network for Semi-Supervised Image Analysis

As the mainstream solution for semi-supervised learning (SSL), pseudo-labeling-based approaches have achieved re-markable success. However, an obvious drawback of existing methods is that the valuable semantic relationships among categories are often ignored, thus leading to suboptimal encoded embeddings. To address this, we present a novel Semantics-Enhanced Pseudo Labeling Network, called SEPL-Net, for image analysis in a semi-supervised manner. SEPL-Net explores the prior knowledge of visual similarity between different classes to improve the quality of pseudo label decision making. Particularly, we encode semantic labels combined with the one-hot label to jointly train our network by exploiting their disagreement. To alleviate the difficulty of labeling unlabeled images due to the introduction of semantic labels, we further design different classifiers with differentiated strong augmentation modes to enable cooperative pseudo labeling. Extensive experimental results show that our SEPL-Net outperforms existing SSL methods with the averaged 1.84% accuracy improvement on image classification task. Code is available at https://github.com/sweetvicky/SEPLNet.git.
Development of E-Module Assisted by Kvisoft Flipbook Maker on Work And Energy Materials for Senior High School Student Class X

This research is to develop an electronic module (e-module) for teaching materials using Kvisoft Flipbook Maker software in the work and energy topic for the high school students class X. This e-module was developed as an idea for the solution of difficulties in learning of physics. This research used the research and development (R&D) and employed the ADDIE instructional model which is consisted of five stages, analysis, design, development, and implementation and evaluation stages. To make sure this e-module is good for the student, the e-module was assessed by three validators. The validation data was scored using the validity index of the Aiken's V formula. After validation process, the results showed that the e-module met the valid criteria. It valid in the aspect of content feasibility, learning design aspects, linguistic aspects, aspects of program operation ease, aspects of consistency and animation aspect with Aiken's V validity index of 0.87. Finally, it can be said that the e-module that was developed using Kvisoft Flipbook Maker on work and energy materials for class X high school students can be used in the real teaching for the class X the high school students.
A Noise-Aware Framework for Blind Image Super-Resolution

The real-world image degradation in the super-resolution task is recently considered as a combination of Gaussian blur, down-sampling, and additional white Gaussian noise. To han-dle this degradation, previous methods estimate the Gaussian blur kernel or model the degradation based on a randomly selected image patch. However, these methods cannot han-dle degradations with high-level noise well as they ignore the spatial variability or even the existence of noise. Moreover, using image denoising networks to preprocess low-resolution images also fails due to the loss of important high-frequency information. In this paper, we propose a framework called EASE to flexibly handle real-world degradations. Specifi-cally, we develop a lightweight module to erase noise and blur simultaneously by learning from an image denoising and an image restoration network, which adapts to existing net-works that focus on handling bicubic down-sampling. Exten-sive experiments prove the superiority of our method, espe-cially when handling degradations with high-level noise.
Discretized Gaussian Mixture Hyperprior for Learned Image Compression with Mask Module

Learned image compression approaches have shown great potential with promising results. However, according to the commonly used measurement methods, there still lies a performance gap between learned compression methods and the latest compression standard versatile video coding (VVC), because of the remaining redundancy existing in contemporary algorithms. To obtain a more accurate entropy model for rate estimation, discretized Gaussian mixture hyperprior is proposed in this work to parameterize the distribution of latent codes. In addition, a proposed mask module is exploited to enhance the feature extraction ability of the encoder and adaptively allocate bit rates. In this case, the proposed learned image compression model achieves the state-of-the-art performance among the existing learned compression methods and most compression standards on both Kodak24 and CLIC Mobile Validation datasets. The proposed model also outperforms VVC under several bitrate scenarios.
Face Inverse Rendering from Single Images in the Wild

Face inverse rendering, an important and challenging task in computer vision and computer graphics, attempts to decompose face image into shape, reflectance, and illuminance. This problem becomes fundamentally difficult under non-laboratory conditions without controlled illumination. Though recent works have produced compelling results, most of these techniques rely on multiple lighting images captured under contronlled lighting by complex equipment, such as Light Stage, which is not flexible and applicable to common users. In this paper, we propose a novel face inverse rendering framework, which neither relies on complex devices nor labeled training data. Instead, it learns reflectance, shape, and illuminance from its physical constraints. Extensive experiments on both synthetic and real image datasets demonstrate consistently superior performance of the proposed method. Our code will be made publicly available.
Discrete-time Recurrent Neural Network Algorithm with Different Discretization Formulas for Finding Solution of Discrete-Time Complex Division

In this paper, in order to solve discrete-time complex division, we propose several discrete-time recurrent neural network (DT-RNN) algorithms. First of all, the continuous-time complex division is transformed into matrix equation equivalently, which can be solved by the continuous-time recurrent neural network (CT-RNN) algorithm. Secondly, several discretization formulas developed by Taylor formula are used to establish the corresponding DT-RNN algorithms. Finally, numerical experiments are carried out, which present the feasibility and effectiveness of these DT-RNN algorithms.
ROGC: Role-Oriented Graph Convolution Based Multi-Agent Reinforcement Learning

The role-oriented learning approach could improve the performance of multi-agent reinforcement learning by decomposing complex multi-agent tasks into different roles. However, due to the dynamic environment and interactions among agents, the role undertaken by an agent changes rapidly with time going on. Therefore, the roles of agents should be adapted to the varying situation during the learning process. In this paper, we propose a role-oriented graph convolution based multi-agent reinforcement learning framework (ROGC). Firstly, we design a role assigner based on samples generated from the environment to learn roles for classifying agents into different groups. To further enhance cooperation among agents in the same group for higher performance, we design a graph convolutional module to achieve intra-role communications based on discovered roles. With roles and extracted role features, we design a role-oriented policy learning module that embeds the role information into the algorithm and generates effective policies for individuals. Further, we introduce an auto-encoder to learn the intra-role cooperation knowledge in the graph convolutional module, which ensures our framework executes in a decentralized way. Extensive experiments show that our framework can learn dynamic roles and make full use of learned roles, which makes it outperform popular MARL methods.
Semi-Supervised Generative Learning with Extended Distribution Matching for Class-Conditional Image Synthesis

Generative Adversarial Network (GAN)-based models have made remarkable progress in high-fidelity image synthesis. However, the performance of class-conditional image synthesis may significantly deteriorate for the case where a limited number of labeled training samples are available. To reduce the dependence on labeled data, we propose a semi-supervised GAN with Extended Distribution Matching, and our model is referred to as EDM-GAN. To prevent a class-conditional discriminator from overfitting the limited labeled data, we perform a transformation of random regional replacement on both real and synthesized samples. By matching the extended distributions, the discriminator is encouraged to focus more on the spatial regions that contain certain objects, while at the same time a class-conditional generator is induced to capture precise class semantics. The adversarial training process can be effectively stabilized and converges to a better solution. Our experimental results on multiple standard benchmarks demonstrate consistent performance gains in synthesis quality and class-semantic accuracy.