Reviewerly/exHarmony huggingface dataset:
LPGN: Language-Guided Proposal Generation Network for Referring Expression Comprehension

Referring expression comprehension(REC) aims to ground the referring expression in an image. Many mainstream frameworks are implemented in a two-stage process: the first-stage model generates candidate region proposals, and the second-stage model locates the referent among the proposals. Existing proposal generators create proposals entirely based on images, so there is a gap between generated proposals and referring expression, which leads to a bottleneck limiting the performance of the whole model. In order to break the bottle-neck, we introduce a novel language-guided proposal generation network: LPGN. Moreover, we introduce an uncertainty-aware proposal generation strategy to tackle the vagueness of language, so as to improve the training effectiveness. LPGN is convenient to integrate it into the existing two-stage REC models, because it is agnostic to the second stage model. Through extensive experiments on benchmark datasets, we demonstrate that our LPGN can generate proposals of higher quality than existing proposal generators and effectively alleviate the proposal bottleneck of the existing two-stage REC model.
Meta Descent Learning for Class Imbalanced Age Estimation

In this paper, we propose a meta descent learning method (MDL) for class imbalanced age estimation while preserving the relative ordinal information. The class imbalanced problem causes head classes with enough samples dominant the gradient descent process, thus suppressing the performance of tail classes. Due to its superiority for quick adaptive descent optimization, we utilize meta learning to adjust the learning gradient iteratively for balancing training. Specifically, we reweight the sample loss using meta descent learning to prevent the dominance of head classes in feature space. Furthermore, we propose an order-consistent loss to explicitly constrain the ordinal information on the output logits, helping the exploit of semantic correlation in aging images. Experimental results on several datasets including Morph II and ChaLearn LAP demonstrate the effectiveness of our method.
Gradual Domain Adaptation with Sample Transferability Exploitation for Person Re-Identification

In this paper, we propose a novel gradual domain adaptation method with sample transferability exploitation to tackle the unsupervised domain adaptation (UDA) for person re-identification (re-id). Due to the direct but rough adaptation scheme, existing UDA for person re-id methods usually suffer from source domain-specific characteristics. To filter out the source domain-specific characteristics, motivated by the curriculum learning strategy, we conduct gradual domain adaptation by domain-level re-weighting with polynomial weight decay. Furthermore, we exploit sample transferability via maximum mean discrepancy based sample-level re-weighting strategy to diminish the domain gap. The sample transferability exploitation spotlights samples with higher importance to the adaptation process in each domain, hence enhance the adaptation performance. By combining the gradual domain adaptation with the sample transferability exploitation, our method achieves the state-of-the-art performance on transferring between two common person re-id datasets.
SADG-Net: Sparse Adaptive Dynamic Guidance Network for Depth Completion

Existing depth completion methods with standard CNN and fixed guidance information often produce invalid value diffusion and mismatch between the guidance and the depth. To address this issue, we propose a SADG-Net for depth completion. Specifically, a sparse adaptive module is designed to infer the initial dense depth map and its confidence, as well as the affinity between pixels. Then we develop a dynamic guidance spatial propagation network to refine the initial depth map and dynamically update the guidance information with the inferred depth. In contrast to previous algorithms, our method effectively optimizes the processing for sparse depth and significantly alleviates the error accumulation issues in spatial propagation. Extensive experiments demonstrate that our model improves upon the state-of-the-art performance on NYUv2 and BIDCD datasets.
Unsupervised Image-to-Image Translation with Patch Pyramid Dual Contrastive Learning for Cross Domain Object Detection

Unsupervised image-to-image (UI2I) translation methods aim to learn a mapping between different visual domains with well-preserved content and consistent structure. It has been proven that the generated images are useful for enhancing the performance of computer vision tasks like object detection in a different domain with distribution discrepancies. Current UI2I translation techniques are highly relied on cycle-structure consistency and content/style disentanglement. In-tense focus has been paid on global translation over an en-tire image, instead of the stylistic variations and complex structure among multiple disparate object instances and back-grounds, resulting in unsatisfactory performance gain for the downstream vision tasks. In this work, we propose Patch Pyramid Dual Contrastive Learning Unsupervised Image Translation (PPDCLUIT) framework to strengthen the cross domain object detection performance. We present patch pyra-mid dual contrastive learning strategy for coherent associations at each specific location and propose identity loss to further enforce the object instances preservation. We im-plement extensive experiments to demonstrate the efficacy of our PPDCLUIT framework for cross domain object detection and achieve state-of-the-art performance on the benchmarks of Normal-to-Foggy, and Day-to-Night.
Face Landmark Calibration Based on 3D Reconstruction and Deep Learning

Face alignment is an important research field in machine vision, which have applied in face recognition, expression recognition, pose estimation, and face synthesis. However, many algorithms are design for faces in small poses and good lighting, they do not perform well in the wild. In this paper, we propose a face alignment algorithm to improve the accuracy of face landmark. This algorithm begins with a network to locate face landmark in general and employs a face normalization to reduce disturbing redundant information. Then a 3D facial reconstruction is introduced to get exact camera matrix. Euler angle based on 3D reconstruction eliminates the effect of pose for face landmark. Experiments on the IBUG dataset show that the algorithm achieves improvement over state-of-the-art algorithms.
Video Feature Compression for Machine Tasks

We consider the problem of transmitting video from a remote device to a cloud-based classification system in a bandwidth limited network. Our focus is on developing an end-to-end system that extracts features from the video data and compresses these features for transmission. In this paper, we consider approaches that operate on each video frame independently as well as exploiting the temporal correlation between frames. In both cases, the transmitted features can be used for object detection and instance segmentation tasks using existing, pre-trained networks. Results show the efficacy of the approach with improvements in coding efficiency ranging from 46.3% to 92.8% when compared to compressing the video data using state-of-the-art video compression standards.
GRDN: Graph Relation Decision Network for Object Detection

Most object detection methods learn regional features in visual space while neglecting the relationship between targets, limiting their performance. We introduce the Graph Relation Decision Network (GRDN) to handle this problem, which includes the graph decision network, decision coefficient, and step-wise relation deduction module (SWRD). Among them, graph decision network contains edge decision network and node decision network. To obtain the implicit relationship between the labels in the dataset, we employ a data-driven technique. Through a graph decision network, the implicit relationship generates a dynamic graph, which can adaptively strengthen and enrich the relationship between the objects. We re-encode the dynamic graph. This process introduces decision coefficient, which can enhance semantic information. In the SWRD, we use semantic information to guide the basic visual features to prevent the visual features from being distracted. Experiments on the COCO dataset show that GRDN has improved performance in object detection and instance segmentation.
Boundary-Guided Probability Hashing

Deep supervised hashing for Hamming space retrieval has recently attracted increasing attention because it enables large-scale image retrieval with constant-time cost. However, the existing Hamming space retrieval methods cannot effectively focus on different pairs simultaneously inside and outside the Hamming ball, making it difficult to push dissimilar pairs outside or pull similar pairs inside the Hamming ball. We propose a novel Boundary-Guided Probability Hashing (BGPH) method that introduces a boundary to guide probability distribution. It makes the probability of similar pairs within the Hamming ball greater than dissimilar pairs and vice versa, which fits the purpose of Hamming space retrieval well. Moreover, we propose a threshold weighting method to indicate when optimization should be stopped to avoid the problem that dissimilar data are pulled into the ball caused by over-optimization in multi-label retrieval scenarios. Comprehensive experiments on three benchmark datasets demonstrate that BGPH yields state-of-the-art retrieval performance.
Research on Self-Adaptive Consensus Method Based on Blockchain

The blockchain is an emerging and hot field in recent years, and one of the core is how to choose a suitable consensus mechanism to reach a consensus reliably and quickly. Each consensus mechanism has its advantages and disadvantages. Aiming at the problem that one consensus mechanism in the current blockchain cannot meet a variety of application scenarios and user needs, this paper proposes a self-adaptive consensus method. First, this paper analyzes the deficiencies of the existing consensus algorithm Raft and proposes some improvements. Then this paper combines multiple consensus algorithms to provide a self-adaptive consensus method. This method can dynamically select the corresponding consensus algorithm according to changes in the application scenarios and user needs, so as to ensure that the consensus process is efficient and reliable.
HPC enabled a Novel Deep Fuzzy Scalable Clustering Algorithm and its Application for Protein Data

Fuzzy clustering is a common way to divide data into groups. Even though it has been improved a lot, fuzzy clustering still has problems while clustering real high-dimensional Big Data with complicated latent distributions. To solve this problem, this study comes up with a way to represent the data in a feature space that was built from a scalable deep neural network using Apache Spark on HPC. In this paper, we proposed SDnnRSIO-FCM, a Scalable Deep Neural Network Random Sampling Iterative Optimization-FCM clustering algorithm, and the SDnnLFCM, a scalable version of the Deep Neural Network Literal Fuzzy c-Means algorithm. We focus on the design and implementation of the proposed SDnnRSIO-FCM and SDnnLFCM algorithms using the Apache Spark cluster in a High-Performance Computing (HPC) environment by representing the data in a feature space produced by the neural network to handle Big Data. First, data is mapped into new feature space to aid in the reconstruction of the original data by providing a good representation. Second, scalable fuzzy clustering is embedded with neural networks to propose deep fuzzy clustering methods. The experimental results conducted on two huge benchmark datasets show that the SDnnRSIO-FCM algorithm outperforms the SDnnLFCM algorithm in terms of Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F-score. Furthermore, the proposed SDnnRSIO-FCM applied to huge soybean protein sequences in comparison with SDnnLFCM shows a significant improvement in terms of Silhouette index (SI), Davies-Bouldin index (DBI), and Calinski-Harabasz index (CHI).
Relational Graph Reasoning Transformer for Image Captioning

The current published methods of image captioning are directly inputting the features of objects in image into model, and introduced a variety of attention mechanisms to capture the associations between the objects and specific words. But the relationships of vision and semantic between objects are not sufficiently concerned. In this paper, we propose a relational graph reasoning Transformer which explicitly incorporates the relationships of vision and semantic between objects to construct an object relational graph in Transformer. Specifically, besides the detected object features, the global spatial relationships and the semantic context between different objects is attended. Meanwhile, a graph structures feature which correlates object features, their spatial and semantic information is reasoned by a learned grafting mechanism. Finally, the contextual graph feature is integrated into the proposed Transformer decoder. Experimental results demonstrate the significance of our relationship reasoning Transformer model.
Source Free Domain Adaptation for Semantic Segmentation via Distribution Transfer and Adaptive Class-Balanced Self-Training

Unsupervised Domain Adaptation (UDA) for semantic seg-mentation aims to transfer the knowledge learned from the source domain to the target domain. Unlike the source-available UDA setting, Source-Free Domain Adaptation (SFDA) has no access to the source data and rely solely on the well-trained source model for adaptation. Without the source data for reference, SDFA often leads to unstable adaptation and mostly focuses on common semantic classes. In this pa-per, we propose a Distribution Transfer and Adaptive Class-balanced self-training (DTAC) framework to tackle the issues of SFDA for semantic segmentation. First, in the distribution transfer stage, we propose to narrow the domain gap by aligning the implicit feature characteristics of source model with the feature statistics of the target data. Next, in the self-training stage, we propose a multi-class negative learning method with adaptive thresholding to dynamically select per-class pseudo labels for self-supervision. Experimental re-sults on urban scene benchmarks show that DTAC outper-forms other SFDA baselines and even achieves competitive results with source-available UDA methods.
KPI Anomaly Detection Based on LSTM with Phase Space

Anomaly detection based on Key Performance Indicator (KPI) plays a crucial role in the cloud application monitoring platform. The methods for anomaly detection based on thresholds and rules are commonly used to monitor KPI. The key factor affecting the accuracy of these methods is the fitting accuracy of normal patterns. The Long Short-Term Memory (LSTM) Network can learn normal patterns and predict future values. However, according to the low-dimensional and non-stationary characteristics of KPI data, the prediction model merely on the basis of time dimension is limited. More specifically, the error occurs in the amplitude dimension and the time lag is present in the trend dimension, which further interfere with the accuracy of anomaly detection. In view of the problem mentioned above, we propose a model of anomaly detection based on LSTM with phase space (PSR-LSTM-AD). With the theory of phase space reconstruction, the motion trajectory of time series is recreated in the high-dimensional phase space, and the hidden spatial features of the original low-dimensional sequence are extracted. The model can extract features in both time and space dimensions. The accuracy of normal pattern prediction is improved in both amplitude and trend dimensions, thus enhancing the accuracy of anomaly detection. This model plays an effective role in the KPI dataset collected from the real environment, with average F1-score over 0.79. Besides, compared with another related model (KPI-TSAD) on this dataset, the proposed model works better, with average accuracy increasing by 7% and average F1-score increasing by 7%.
Spatial-Temporal Correlation Modeling for Motion Prediction

Human motion prediction is fundamental for many applications in computer vision. Current methods typically handle motion prediction with seqential models, which ignore the fact that joint movement is driven by forces. In this paper, we provide a novel mechanical view to decompose force into magnitude and direction, which contributes to modeling the temporal evolution of joints. Moreover, existing graph convolution-based methods merely utilize the deep-level features, which is difficult to capture the complex spatial dependencies contexts. We introduce a novel spatial connections encoding model to capture the multi-level spatial dependencies between joints. Finally, to encode abundant temporal dependencies, we present a multi-head temporal encoding module. Comprehensive experiments show that our model sets the state-of-the-art performance on the largest human motion benchmark datasets.
They Like Comedy, Don't You? A Cluster-Based Meta-Learning for Cold-Start Recommendation

Cold start has always been a challenging problem due to the sparse user-item interaction. Recently meta-learning models have performed outstandingly in solving this problem, which train an optimal initialization parameter by sharing the knowledge of all users. However, knowledge sharing between users with different preferences is negatively affected. In this paper, we try to share knowledge only among users with similar interests. Specifically, we first propose a user cluster method in the cold start scenario. Then, with the user's cluster information we design a conversion network to transform the global initialization parameters learned by meta learning into the cluster optimal initialization parameters. In this way, we can reduce the negative impact among users with large differences in preferences, and improve the performance of the meta-learning model. Extensive experiments on MovieLens and Yelp demonstrate that our method significantly outperforms the state of the arts in both warm up and cold-start scenarios.
The Development of Interactive Learning Media Based on Macromedia Flash for Economics Lessons in High School

The aim of this research is to develop interactive learning media with macromedia flash-based for economics lessons in high school. Subject focusing on interactive learning media with problem-based learning models on "Balance of Payments" subject. This research wants to find out the process passed in the development of learning media, the validity and feasibility of it. The method used is Research and Development by using procedures from Borg and Gall. This study implemented at SMA Negeri 12 Banjarmasin. The instruments used are validation sheets and questionnaires. The data collection methods used are interviews and validations. The results showed that overall, the learning media got a score of 4,27 (very valid). The initial field trial gets a media expert validation score of 4,1 (valid) and practitioners score of 4,31 (very valid). Field trials with small-scale student responses to learning media with validity score of 4.45 (very valid). The field execution test resulted in a larger-scale student response score with validity score of 4.23 (very valid). It means that interactive learning media with macromedia flash-based is valid and feasible to implemented for economics lessons in high school on "Balance of Payments" subject.
Detecting Anomalies in Satellite Telemetry Data based on Causal Multivariate Temporal Convolutional Network

Prediction-based methods are widely used in satellite telemetry data anomaly detection. However, pursuit to detect as many anomalies as possible often leads to unsatisfactory false positives. Moreover, the interdependence (causality) between telemetry parameters is often overlooked as an important tool for data analysis, resulting in poor model interpretability. We propose a Causal Multivariate Temporal Convolutional Network (CMTCN) model, which first discovers anomaly propagation causality among parameters by constructing a causal network based on Anomalous Transfer Entropy (ATE), then applies a multi-feature input Temporal Convolutional Network (TCN) to predict telemetry data. We also propose a Peaks Over Thresholds (POT) -based anomaly score calculation method and anomaly identification criterion for the residuals. CMTCN can effectively capture and exploit the interdependence between features, which not only increases the interpretability of the model, but also is more sensitive to anomalies. Case studies on two public telemetry datasets and a multivariate time series dataset show that our method outperforms all baselines in F1-score.
Fine-Grained Cross-Modal Retrieval with Triple-Streamed Memory Fusion Transformer Encoder

Recently, the powerful attention mechanism has been wildly used to learn the fine-grained cross-modal correspondences. However, the trade-off between effectiveness and efficiency sometimes bothers existing attention-mechanism-based meth-ods. To address this deficiency, we propose a novel Triple-streamed architecture with a newly designed Memory fusion Transformer Encoder (Tri-MTE) for fine-grained cross-modal retrieval. Specifically, the whole model reserves the “late fusion” strategy thus ensuring efficiency. To strengthen the inter-modality interaction and improve the effectiveness, a memory fusion stream is designed and inserted between the modality streams to remember the modality-irrelevant infor-mation. Encoding such information to the modality represen-tation would significantly enhance the cross-modal retrieval performance. Finally, a bionic memory activation constrain-t is proposed to aid the learning procedure. Extensive ex-periments on two benchmark datasets show that the proposed method achieves promising results.
Mixed Color Channels (MCC): A Universal Module for Mixed Sample Data Augmentation Methods

Color invariance is critical for computer vision systems since it significantly increases the robustness and effectiveness of the system. MSDA approaches (e.g., FMix and CutMix) have attracted considerable attention in recent years since they are simple, effective, and do not require extra computation con-sumption. By mixing samples, these approaches extend the distribution of training samples. However, the color information of these mixed samples is not changed, which makes it still difficult for trained models to achieve color invariance. To address this issue, we propose a universal module called Mixed Color Channels (MCC) that implements color changes by mixing the sample and its color variants, which enables trained models to achieve color invariance. In the experimen-tal section, we insert MCC into four state-of-the-art MSDA approaches, evaluate its effectiveness, and embed MCC into a non-MSDA method to demonstrate its extensibility.
Attention Based Adversarial Attacks with Low Perturbations

Deep neural networks are vulnerable to adversarial examples generated by black-box attacks with tiny perturbations. However, black-box based transferability attacks usually add perturbation to the whole image. It is easy for the defender to detect the adversarial examples. Inspired by attention modules, we propose a method named Gradient-mask and Attention-whey (GM&AW) to reduce redundant noise and maintain attack effect of them in this work. During Gradient-mask iterations, we only choose the regions with greater gradient and update them along the direction of gradient. Then, we utilize Attention-whey optimization combining attention mechanism and query to further decrease noise. Extensive experiments on Imagenet demonstrate that our approach can achieve enormous decrease on noise in <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{2}$</tex> norm and keep attack success rate for black models.