Reviewerly/exHarmony huggingface dataset:
RCNet: Recurrent Collaboration Network Guided by Facial Priors for Face Super-Resolution

In this paper, we present a novel FSR method, called RCNet, which progressively improves the performance of FSR and Landmark Estimation (LE) via recurrent collaboration. In our approach, FSR and LE complement each other. Different from previous FSR methods that directly estimate the facial landmarks on the low-resolution face images, the proposed RCNet conducts LE on the super-resolution face image obtained through multiple iterations. Benefiting from the super-resolution face images, facial landmarks are precisely estimated, which boosts the performance of FSR in turn. Furthermore, we design a Component-Aware Fusion Module (CAFM) for better recovering facial details, which adaptively fine-tunes the estimated landmarks and groups them into face components to maximize the guiding role of the facial land-marks. In addition, the iterative feature aggregation is developed to preferably capture the information from the LR/SR face images. Experimental results show that the proposed RCNet outperforms the state-of-the-art methods in both quantitative and qualitative aspects for super-resolving very low-resolution faces.
Detecting Anomalous Events from Unlabeled Videos via Temporal Masked Auto-Encoding

Unsupervised video anomaly detection (UVAD) intends to discern anomalous events from fully unlabeled videos. However, existing UVAD methods suffer from poor performance. Inspired by recent masked autoencoder (MAE) [1], we propose Temporal Masked Auto-Encoding (TMAE) as an effective end-to-end UVAD method. Specifically, we first denote video events by spatial-temporal cubes (STCs), which are built by temporally consecutive foreground patches from unlabeled videos. Then, half of patches in an STC are masked along the temporal dimension, while a vision transformer (ViT) is trained to exploit unmasked patches to predict masked patches. The rare and unusual nature of anomaly will result in a poorer prediction for anomalous events, which enables us to discriminate anomalies from unlabeled videos and compute the anomaly scores. Furthermore, to utilize motion clues in videos, we also propose to apply TMAE on optical flow, which can further boost performance. Experiments show that TMAE significantly outperforms existing UVAD methods by a notable margin (3.9%–6.6% AUC).
On Optimal Resource Allocation of VAV Customized Cloud Platform Based on Bi-level Programming

Aiming at the optimal resource allocation of UAV customized cloud platform (UAV -CCM), this paper considers the sustainability of cloud platform at the operational level, and introduces the sustainability evaluation indicators of UA V -CCM from three aspects of economy, environment and society. We propose a multi-objective bi-level programming resource allocation (MOBPRA) model of UA V -CCM. The model maximizes the sustainability and quality of service of UA V -CCM from the perspective of the platform operator and multiple service demanders. In order to solve the established MOBPRA model, a new hybrid algorithm of particle swarm optimization and simulated annealing (PSO-SA) is proposed. Finally, the bi-level programming model and PSO_ SA algorithm is applied to UAV -CCM to further verify its feasibility. The results demonstrate the feasibility and effectiveness of the proposed model and algorithm.
Effective JPEG Steganalysis Using Non-Linear Pre-Processing and Residual Channel-Spatial Attention

Nowadays, convolutional neural network (CNN)-based JPEG steganalyzers have demonstrated much better performance than the conventional steganalysis methods based on hand-crafted feature sets. By incorporating the selection-channel aware (SCA) knowledge, the performance of the deep learning-based approach could be further improved. For prac-tical applications, however, the SCA knowledge is usually u-navailable to steganalyzers. In this paper, a novel CNN model is proposed by including extra non-linear kernels in the first network layer to enhance stego signal and exploring the resid-ual channel-spatial attention (CSA) module which plays the same role as SCA to further improve the performance. In addition, instead of global average pooling, spatial pyramid pooling (SPP) is adopted to better preserve the hierarchical feature representation at various scales for JPEG steganaly-sis. Experimental results show that the proposed CNN model with extra non-linear kernels, CSA and SPP outperforms other state-of-the-art deep learning-based approaches for JPEG steganalysis.
Distilldarts: Network Distillation for Smoothing Gradient Distributions in Differentiable Architecture Search

Recent studies show that differentiable architecture search (DARTS) suffers notable instability and collapse issue: skip-connect may gradually dominate the cell, leading to deteri-orating architectures. We conjecture that the domination of skip-connect is due to its superiority in gradient compen-sate. On this foundation, we propose a novel and stable method, called DistillDARTS, to stabilize DARTS by knowl-edge distillation and self-distillation scheme. Specifically, the distillation is able to serve as a substitute for skip-connect and smooth the back-propagated gradient distributions among layers of DARTS. By compensating gradients in shallow lay-ers, our method can relieve the dependence of gradient on skip-connect and hence mitigates the collapse issue. Exten-sive experiments on a range of benchmarks demonstrate that DistillDARTS can obtain sturdy architectures with few skip-connects without additional manual interventions, thus suc-cessfully improving the robustness of DARTS. Due to the im-proved stability, our proposed approach achieves the accuracy of 97.57% on CIFAR-10 and 75.8% on ImageNet.
Semantic-Driven Automatic Filter Pruning for Neural Networks

Filter pruning is widely used for neural network compression. However, existing methods mostly judge importance of filters by measuring magnitudes or distributions of data in weight parameters or feature maps, but neglect semantics of filters, which weakens interpretability. Meanwhile, layer-wise compression ratios are assigned automatically only under given FLOPs by architecture search algorithms or just manually. It takes a lot of time and is impractical. In this paper, a novel semantic-driven automatic filter pruning method is proposed to solve the above problems. Filters are treated as semantic detectors, and the semantic-driven importance criteria is developed by evaluating correlations between input tasks and feature maps. Besides, inspired by the phenomenon that each filter has deterministic impact on FLOPs and network parameters, we provide a new efficient adaptive compression ratio allocation strategy based on differentiable pruning approximation by considering FLOPs and parameters simultaneously. Our method is validated with extensive experiments on various neural networks.
Context Gating with Multi-Level Ranking Learning for Visual Dialog

Visual dialog aims to answer several consecutive questions based on image and dialog history. Most works resolve all questions with ambiguous references (e.g., “she”) by dialog history, which generates redundant information and gets in-accurate results. Also, they regard this task as a classification task, which ignores the diversity of response answers and results in poor generalization capability. To tackle these problems, we propose a novel Context Gating with Multi-level Ranking Learning (CGMRL). Specifically, the proposed context gating considers both question and image to adaptively determine whether the history is needed for question answering, which reduces the redundant or even noisy information generated by history. To improve the generalization capability of the model, a new constrained multi-level ranking learning is proposed to encourage the model to consider the correct semantic options rather than only choose the ground truth answer. Experimental validations on the VisDial v1.0 show the superiority of the proposed method compared with other methods. Implementation code is published in anonymous Github: https://github.com/sy742/CGMRL_.
Triple Disentangling Network for Unsupervised Domain Adaptation

Most existing unsupervised domain adaptation methods learn domain-invariant representations with entangled domain in-formation, semantic information, and instance information. Differently, in this paper, we propose a Triple Disentangling Network (TDN), to disentangle these three types of information and then predict the target labels merely using semantic information. Specifically, TDN consists of a reconstruction module and a disentanglement module. In the reconstruction module, TDN utilizes a variational auto-encoder to re-construct the domain, semantic, and instance latent variables behind the data. In the disentanglement module, adversar-ial learning, discriminative clustering, and instance separation are seamlessly integrated to disentangle these three sets of re-constructed latent variables. Significantly, TDN can not only effectively alleviate the negative transfer of outliers through disentangling instance information, but also disentangle se-mantic information more thoroughly by exploring discriminative structure knowledge. Experimental studies on two bench-mark datasets demonstrate the superiority of TDN.
Dyadic Human Relation Recognition

We introduce a new dataset and novel method for dyadic human relation (DHR) recognition. DHR recognition is a new image understanding task that concerns the recognition of the type (i.e., verb) and roles of a two-person interaction. Unlike past human action and dyadic interaction detection, our goal is to extract richer information regarding the roles of actors, i.e., which person is acting on whom. For this, we introduce the DHR-WebImages dataset which consists of a total of 22,046 images of 51 verb classes of DHR with per-image annotation of the verb and roles. We derive a novel model for DHR recognition by consolidating information from three streams corresponding to the visual, pose, and spatial relation features of the two people. Experimental results show that our model achieves accurate DHR recognition robust to severe occlusions and lacking visibility of actors. We believe these results will lead to richer visual information extraction.
Adaptive Bitrate Control Mechanism Based on Long-Term Evaluation

This paper addresses the improvement of the quality expe-rienced by users while using adaptive bitrate (ABR) video streaming by considering long-term aspects in the chunk se-lection process. Compared with traditional approaches that select chunks on the basis of predicted immediate throughput value, leveraging long-term throughput values can enable a long-term chunk selection strategy to be defined that enables perceived quality to be increased in challenging network con-ditions. In results for real-world throughput measurements, quality was improved by at least 0.5 units on a five-grade mean opinion score (MOS) in up to 60% of tested challenging network conditions thanks to the increase in the time frame in the chunk selection mechanism. Furthermore, using a long-term chunk selection mechanism was shown to enable higher quality to be maintained than short-term approaches when throughput prediction becomes noisy. The proposed approach was shown to benefit a quality-centric ABR control scheme but is also applicable to other ABR algorithms.
Pattern Matters: Hierarchical Correlated Strip Convolutional Network for Scene Text Recognition

Many state-of-the-art scene text recognition methods leverage a pre-designed backbone for general object recognition, where general objects are usually lacking of distinctive characteristics. However, unlike general objects, texts in images are usually composed of narrow strokes and their characters often share similar attributes, e.g. texture and intensity. Such useful patterns can be deteriorated if directly employing standard convolutional networks to the scene text recognition. To solve the problem, we introduce a novel operation Strip Convolution, a specially designed convolution for extracting features of narrow strokes. And we further apply a Hierarchical Correlation strategy, adopting multi-level attention mechanisms to capture common text attributes. Based on this, we design a novel framework, named as Hierarchical Correlated Strip Convolutonal network for scene text recognition. Extensive experiments demonstrate the superiority of the proposed HCSC network, improving the accuracy of text recognition effectively.
Frequency-Dividing Downsampling Module of the Lifting Scheme for Image Classification

Convolutional neural networks(CNNs) currently dominate the field of computer vision, where the pooling layer plays an important role in reducing computational effort and avoiding overfitting. However, the commonly used methods do not design the pooling layer from the perspective of frequency. In this paper, we propose a Lifting Scheme-based frequency-dividing downsampling framework and describe a pooling layer called frequency-dividing pooling (FDP). The two branches of the Lifting Scheme process the images by frequency, which not only enhances the interpretability of the neural network but also improves the classification accuracy of the neural network. We conduct experiments on three standard datasets and the results all demonstrate that our proposed FDP is effective.
Siamese Template Diffusion Networks for Robust Visual Tracking

In recent years, Siamese trackers have shown great potentials in visual tracking. Most of these trackers fix the template with the initial target representation during on-line tracking and the tracking performance heavily depends on the generalization of template matching network learnt off-line. In this paper, we propose novel Siamese Template Diffusion Networks for on-line adaption of target appearance variations during tracking. To be more specific, we embed new feature aggregation modules (FAMs) into a Siamese network to generate more accurate template and search region for better matching. The new FAMs can establish long-term temporal dependencies of templates at both channel level and spatial level. We plug the new FAMs into two Siamese trackers (i.e., SiamFC and SiamFC++), and propose two Siamese template diffusion trackers (i.e., SiamTDN and SiamTDN++). Experimental results on five challenging datasets show that the proposed trackers outperforms their baselines by a large margin (e.g., 19%/5.6% in terms of the EAO score on VOT2018).
Multimodal Crowd Counting with Mutual Attention Transformers

Crowd counting is a fundamental yet challenging task that aims to automatically estimate the number of people in crowded scenes. Nowadays, with the rapid development of thermal and depth sensors, thermal images and depth maps become more accessible, which are proven to be beneficial information in boosting the performance of crowd counting. Consequently, we propose a Mutual Attention Transformer (MAT) module to fully leverage the complementary information of different modalities. Specifically, our MAT employs a cross-modal mutual attention mechanism to utilize the features of one modality to enhance the features of the other. Moreover, to improve performance by learning better visual representation and further exploiting modality-wise comple-mentarity, we design a self-supervised pre-training method based on cross-modal image reconstruction. Extensive experiments on two standard benchmarks (i.e., RGBT-CC and ShanghaiTechRGBD) show that the proposed method is effective and universal for multimodal crowd counting, outper-forming previous state-of-the-art methods.
Domain Generalization Via Adversarially Learned Novel Domains

This paper focuses on the domain generalization task, which aims to learn a model that generalizes to unseen domains by utilizing multiple training domains. More specifically, we follow the idea of adversarial data augmentation, which aims to synthesize and augment training data with “hard” domains for improving the model's domain generalization ability. Previous works augment training data only with samples similar to the training data, resulting in limited generalization ability. We propose a novel adversarial data augmentation method, termed GADA (Generative Adversarial Domain Augmentation), which employs an image-to-image translation model to obtain a distribution of novel domains that are semantically different from the training domains, and, at the same time, hard to classify. Evaluation and further analysis suggest that adversarial data augmentation with semantically different samples leads to better domain generalization performance.
CoNet: Co-Embedding by Reinforcing Graph Feature and Topology Information

Sparsity and smoothness are two main factors that affect the performance of Graph Convolutional Networks (GCNs). Sparsity ensures that models have the first-class generalization ability, while smoothness benefits to reduce noise and make edges reliable. As real-world graphs are often incom-plete and noisy, most GCNs learn node embeddings only acting them as ground-truth information, which unavoidably lead to suboptimal solutions. This paper proposes a co-embedding network (CoNet), jointly learns embeddings by fusing the global and local dependencies to capture the uni-versal and intrinsic properties. We proposed NodeNet and EdgeNet modules, which aggregate global node information and refine local topology structure respectively. Moreover, we further introduce two piplines of variational auto-encoders to fuse the intermediate latent variables of each module to ob-tain co-embeddings via Knowledge Distillation strategy. Ex-tensive experiments on multiple benchmarks show that our proposed approach achieves better performance than existing methods on the graph node classification task.
Unified Multi-Modal Multi-Task Joint Learning for Language-Vision Relation Inference

The relationships between language and vision are valuable for natural language processing and computer vision research, where the text and image data are employed to develop computing techniques for image caption or visual grounding. Although the existing studies have been engaged in language- vision relation inference (LVRI), they are limited to low- resource of the task itself. In this paper, we mainly focus on LVRI upon text-image pairs in Twitter with a unified multi- modal multi-task joint learning approach. Different from the conventional multi-modal multi-task learning approach within the same multi-modal dataset, we leverage a relevant multi - modal task on the external dataset as an auxiliary task to facilitate the LVRI task. Systematic experiments demonstrate the effectiveness of our proposed multi-modal multi- taskjoint learning approach.
Map analysis of benchmarking in higher education using VOSViewer

This Quality assurance in higher education has become an essential issue for many policymakers. In order to improve quality and service, benchmarking serves as a solution for continuous improvement of service, including in higher education. Given the importance of study in this topic, the researchers conducted a map analysis using technology. We used publish and perish software to have the publication database from Google scholar, and then the data were analyzed using VOSviewer. The keywords used in this study are benchmarking and benchmarking in higher education. We collected 100 pieces of metadata from google scholar related to each keyword. After conducting the map analysis of network, overlay, and density visualization, the study discovered that several studies have been conducted related to benchmarking in higher education. However, some aspects, such as sustainability and its relationship to benchmarking, still need to be further investigated. Therefore, further studies need to consider, such as how benchmarking can support higher education sustainability.
Learning a Blind Quality Evaluator for UGC Videos in Perceptually Relevant Domains

The absence of reference videos with pristine quality and the complexity of authentic distortions pose great challenges for blind video quality assessment (BVQA) towards user-generated content (UGC) videos. Although it is straightfor-ward to leverage transfer learning techniques to learn effective BVQA models, it is nontrivial to explore how to bridge the domain shifts for better video representation learning. In this work, we propose to transfer meaningful knowledge from perceptually relevant domains, i.e., image quality assessment (IQA) with authentic distortions and video classification with rich motion patterns. We develop a promising strategy to use both groups of data to learn the feature extractors. We train the proposed model on the target VQA databases using a mixed list-wise ranking loss function. Extensive experiments on six VQA databases demonstrate that our method performs very competitively under both individual database and mixed database training settings. Codes and models are available at https://github.com/zwx8981/BVQA-2021.
RPITN: Review Based Preference Invariance Transfer Network for Cross-Domain Recommendation

Cross-domain recommendation is an effective way to cope with the cold-start problem in recommendation systems. Knowledge of the current, particularly reviews, is taken into account to improve user/item embedding to reduce the neg-ative transfer that occurs during mapping processes across the source and target domains. Traditional approaches, on the other hand, typically apply review information from the source and target domain independently without consideration of user preference divergence. In this paper, we propose a novel Review-based Preference Invariance Transfer Network (RPITN) to minimize negative transfer by combining reviews from two domains. We first build a review preference invari-ance (RPI) embedding procedure to express user/item review correlations between two domains. Then, to improve the gen-eralization ability of user/item embedding and prevent negative transfer across domains, we carefully insert RPI into the embedding learning and mapping process. Extensive exper-iments on real-world datasets demonstrate the superiority of RPITN compared with other recommendation methods.
On-Road Pedestrian Tracking Across Multiple Moving Cameras

With the rapid development of autonomous driving, tracking on-road pedestrians raises more attention in the public. Currently, most researches focus on single camera based tracking or tracking across multiple static cameras. Tracking across multiple moving cameras has not been well studied yet. In this paper, we propose a workflow for tracking pedestrians across multiple moving cameras, leveraging the state-of-the-art single camera based tracking method of FairMOT. We consider different factors such as appearance features, motion information, and camera spatial distribution to improve the tracking performance. The experimental results carried on a multi-target multi-moving camera tracking dataset show the feasibility of the proposed scheme in solving the tracking issue in a complex environmental setting.
Multi-Definition Video Deepfake Detection via Semantics Reduction and Cross-Domain Training

The recent development of Deepfake videos directly threatens our information security and personal privacy. Although lots of previous works have made much progress on the Deepfake detection, we empirically find that the existing approaches do not perform well on the low definition (LD) and crossdefinition (high and low) videos. To address this problem, in this paper, we follow two motivations: (1) high-level semantics reduction and (2) cross-domain training. For (1), we propose the Facial Structure Destruction and Adversarial Jigsaw Loss to reduce our model to learn high-level semantics and focus on learning low-level discriminative information; For (2), we propose a domain generalization method based on adversarial learning. We conduct extensive experiments on the FaceForensics++ dataset. Results show the great effectiveness of our method and we also achieve very competitive performance against state-of-the-art methods.