Reviewerly/exHarmony huggingface dataset:
Skeletal Twins: Unsupervised Skeleton-Based Action Representation Learning

In this paper, we investigate unsupervised representation learning for skeleton action recognition, and develop a simple yet effective framework: SKeletal Twins (SKT), which is capable of learning representations from unlabeled skeleton data. To be specific, we choose skeleton-specific spatial and temporal augmentations for spatio-temporal dynamics learning, then the augmented skeleton sequence is represented as a graph with both spatial and temporal edges so that the GCN-based twin encoders are able to encode human pose and joint's temporal motion. Barlow Twins' objective function is used to minimize the redundancy and keep similarity of different skeleton augmentations. However it ignores the instance-level consistency of the skeleton instance from different augmentations, thus an instance-level consistency-enhanced objective function is designed and jointly optimized, which boosts the representation learning. Extensive experiments verify that the proposed framework obtains the state-of-the-art results on the challenging NTU-60 and NTU-120 datasets.
Locality-Aware Attention Network with Discriminative Dynamics Learning for Weakly Supervised Anomaly Detection

Video anomaly detection is recently formulated as a multiple instance learning task under weak supervision, in which each video is treated as a bag of snippets to be determined whether contains anomalies. Previous efforts mainly focus on the discrimination of the snippet itself without modeling the temporal dynamics, which refers to the variation of adjacent snippets. Therefore, we propose a Discriminative Dynamics Learning (DDL) method with two objective functions, i.e., dynamics ranking loss and dynamics alignment loss. The former aims to enlarge the score dynamics gap between positive and negative bags while the latter performs temporal alignment of the feature dynamics and score dynamics within the bag. Moreover, a Locality-aware Attention Network (LA-Net) is constructed to capture global correlations and re-calibrate the location preference across snippets, followed by a multilayer perceptron with causal convolution to obtain anomaly scores. Experimental results show that our method achieves significant improvements on two challenging benchmarks, i.e., UCF-Crime and XD-Violence.
Alfie: Neural-Reinforced Adaptive Prefetching for Short Videos

Short videos have received extraordinary success in recent years. To provide smooth playback and avoid rebuffering delay, prefetching upcoming videos is commonly used in cellular networks. Current prefetching designs fall short in dealing with bandwidth overhead, especially the exit overhead of downloaded but unconsumed chunks due to user exit. Measurement from a large short video platform shows that exit overhead accounts for up to 43.5% of bandwidth overhead. Thus we build Alfie, a bandwidth-efficient short video prefetching algorithm via reinforcement learning. Essentially Alfie adjusts prefetching based upon user viewing patterns in addition to network conditions. We demonstrate that Alfie outperforms the state of the art by up to 26.8% in overall performance while reducing the exit overhead by up to 84.9%.
GTLR: Graph-Based Transformer with Language Reconstruction for Video Paragraph Grounding

Video Paragraph Grounding aims at retrieving multiple relevant moments from an untrimmed video with a given natural language paragraph query. However, the complex paragraph query brings more challenges to the multimodal fusion and context modeling, which limited the performance of existing VPG methods. To this end, we propose a novel framework for VPG in this paper, termed Graph-based Transformer with Language Reconstruction (GTLR). It consists of three components: (1) Multimodal Graph Encoder conducting the graph reasoning for video-text fusion. (2) Event-wise Decoder predicting the timestamps based on multiple sentence-level features. (3) Language Reconstructor rebuilding the paragraph queries and making our model explainable. We adopt two benchmarks, i.e., ActivityNet-Caption and Charades-STA, to evaluate our model and conduct comprehensive experiments to analyze the effectiveness of each component. The experimental results show that our GTLR method outperforms recent state-of-the-art methods.
Discovering Implicit Classes Achieves Open Set Domain Adaptation

In Open Set Domain Adaptation (OSDA), large amounts of target samples are drawn from the implicit categories that never appear in the source domain. Due to the lack of their specific belonging, existing methods indiscriminately regard them as a single class “unknown”. We challenge this broadly-adopted practice that may arouse unexpected detrimental ef-fects because the decision boundaries between the implicit categories have been fully ignored. Instead, we propose Self-supervised Class-Discovering Adapter (SCDA) that attempts to achieve OSDA by gradually discovering those implicit classes, then incorporating them to restructure the classifier and update the domain-adaptive features iteratively. SCDA performs two alternate steps to achieve implicit class discov-ery and self-supervised OSDA, respectively. By jointly op-timizing for two tasks, SCDA achieves the state-of-the-art in OSDA and shows a competitive performance to unearth the implicit target classes.
Synthesizing Audio and Video Bitrate Selections via Learning from Actual Requirements

Adaptive bitrate (ABR) algorithms are routinely adopted for transmitting media contents across dynamic networks. State-of-the-art ABR algorithms only adapt to video bitrate without considering audio bitrate adaption as they consider the im-pact on the video to be negligible due to the small size of the audio. However, to bring users an immersive experience, more and more content providers have applied high-quality audio with large sizes, like stereophonic and surround (Dolby Atmos). Therefore, improper audio bitrate selection will ad-versely affect video bitrate selection, leading to undesirable audio/video combinations (the highest video quality with the lowest audio quality, vice versa) and frequent playback inter-ruptions. To address these inefficiencies, we propose a Self-Play reinforcement learning-based Audio-aware ABR algorithm named SPA to learn strategies for audio and video bi-trate selections. Experimental results demonstrate SPA's con-siderable superiority as compared with existing approaches.
Deep Dictionary Learning with an Intra-Class Constraint

In recent years, deep dictionary learning (DDL)has attracted a great amount of attention due to its effectiveness for represen-tation learning and visual recognition. However, most existing methods focus on unsupervised deep dictionary learning, failing to further explore the category information. To make full use of the category information of different samples, we pro-pose a novel deep dictionary learning model with an intra-class constraint (DDLIC) for visual classification. Specif-ically, we design the intra-class compactness constraint on the intermediate representation at different levels to encour-age the intra-class representations to be closer to each other, and eventually the learned representation becomes more dis-criminative. Unlike the traditional DDL methods, during the classification stage, our DDLIC performs a layer-wise greedy optimization in a similar way to the training stage. Experi-mental results on four image datasets show that our method is superior to the state-of-the-art methods.
Generative Steganographic Flow

Generative steganography (GS) is a new data hiding manner, featuring direct generation of stego media from secret data. Existing GS methods are generally criticized for their poor performances. In this paper, we propose a novel flow based GS approach - Generative Steganographic Flow (GSF), which provides direct generation of stego images without cover image. We take the stego image generation and secret data recovery process as an invertible transformation, and build a reversible bijective mapping between input secret data and generated stego images. In the forward mapping, secret data is hidden in the input latent of Glow model to generate stego images. By reversing the mapping, hidden data can be extracted exactly from generated stego images. Furthermore, we propose a novel latent optimization strategy to improve the fidelity of stego images. Experimental results show our proposed GSF has far better performances than SOTA works.
Rethinking the Grouping Process in Corner-Based Detectors

The grouping process of corner-based detectors still faces two challenges: 1) Hard-grouping. If one of the paired corners is wrongly estimated, the grouping goes wrong. 2) Complex pipeline. Vanilla methods regard corner grouping as an additional stage, complicating the post-processing. To eliminate these issues, we propose a novel grouping algorithm, termed as Soft-Grouping Non-Maximum Suppression (SG-NMS), which merges grouping with NMS into a whole to simplify the pipeline and lift the efficiency. SG-NMS is flexible to match the varied number of detected corners. Accordingly, we propose a multi-corners context enhanced module, Corner Visual Reasoning (CVR), as a grouping helper. Equipped with the proposed SG-NMS, a new multi-corners detector, SGCDet (Soft Grouping Corner-based Detector), is proposed. Experiments show that the inference speed of SGCDet is more than two times faster than state-of-the-art corner-based models with much higher accuracy.
Attentive Decoupling Network for Cloth-Changing Re-Identification

Recently, Cloth-Changing person Re-IDentification (CC-ReID) plays a vital role in the public security system and social livelihood, and suffers the problem of considerable intra-class variation. This paper demonstrates that coarse-grained appearance and body shape features are helpful for CC-ReID. We propose an Attentive DeCoupling (ADC) Network for CC-ReID without auxiliary information. The proposed network is built on two core designs. First, a joint identification structure is proposed to retain ID-relevant information at appearance and shape levels. Second, Competitive Attention (CA) is adopted, where the model progressively updates attention to accumulate sound cues for discriminating identity (ID). The proposed decoupling process is continuously improved through constant self-defeating competition of the network. Experimental results on the public cloth-changing dataset show the proposed method's effectiveness and generalizability.
Pruning Dynamic Group Convolution with Static Substitute

Deep learning networks are gradually deployed in edge applications, such as phones and cameras, which has a restriction of the computational resources. Thus, to improve the computational efficiency, numerous types of group convolution-based frameworks have been studied, including the dynamic group convolution (DGC). However, it is worth noting that sometimes the same channels are selected in different dynamic heads in DGC, which violates its original intention. It indicates the number of dynamic heads exceeds what is really needed. In this paper, a novel model by pruning dynamic group convolution with static substitute is proposed. Specifically, those channels pruned in the dynamic head can be reselected in the static head of the group convolution in the proposed model. In addition, a new polarization regularization is introduced to prune more useless channels with less accuracy loss. The experiment results on two image classification benchmarks (CIFAR-I00 and TinylmageNet-200) show promising performance.
Dual Reasoning Based Pairwise Representation Network for Document Level Relation Extraction

Relation extraction is the task of extracting relational facts between entities from plain text. When the extraction scope is extended to the document level, entities may exist in dif-ferent sentences. This requires the model to consider the in-teraction between multiple sentences comprehensively. Thus, document-level relation extraction becomes especially chal-lenging. Most existing models adopt entity representation learning to tackle this challenge. Nevertheless, they gener-ally perform representations of individual entities rather than directly modeling the dependencies between entities reflect target relation, which may cause wrong relation reasoning. To solve this issue, we propose a novel dual reasoning-based pairwise representation network. Specifically, we perform the semantic and syntactic-based reasoning to model the associ-ation between entities, and towards the capture of relational information in the document through an unique contextual se-lection for each entity pair. Experiments on three benchmark datasets demonstrate the inspiring performance improvement over state-of-the-art relation extraction models.
A Hybird Alignment Loss for Temporal Moment Localization with Natural Language

This paper addresses the problem of temporal moment localization with natural language, which aims to localize a target video moment according to a language description. A key challenge in this task is how to learn effective alignment between vision and language features extracted from an untrimmed video and a sentence description. Traditional methods generally utilize the vanilla attention mechanism to simulate soft alignment between extracted video and sentence features, however, they lack the training signal to alleviate the domain gap between video and sentence domains. In this paper, we propose a novel cross-domain alignment module to deeply align the features in two domains, yielding to learn better representations. We evaluate our model on three public datasets: Charades-STA, ActivityNet Captions and TACoS, and the experiments prove that our proposed cross-domain alignment module can bring significant improvement, and can be also applied to other baseline models.
Just Noticeable Learning for Unsupervised Anomaly Localization and Detection

Anomaly detection targets to detect unseen defective ap-pearances that are different from the seen non-anomalous instances. Recent unsupervised learning methods generate anomalous instances to inspire the anomaly detector to learn discriminative features. However, the generated anomalous instances are more easily noticed than real defects and may cause inconsistency between normal and anomalous. It lim-its the reconstruction of anomaly-free images and the seg-mentation of diversity anomalies. To conquer these prob-lems, we propose a Just Noticeable learning for unsupervised anomaly Localization and Detection(JNLD). The method simulates multi-scale noticeable defects to enrich generaliz-able representation. It embeds just noticeable learning in the sub-networks of reconstruction and segmentation to di-rectly localize the defects without any complex additional post-processing. With further equipment of label inconsis-tency correction, our JNLD achieves 73.8 for unseen anomaly localization average precision that surpasses the current state-of-the-art by a large margin of 5.4 percentage on the challenging dataset MVTecAD.
GNC: Geometry Normal Consistency Loss for 3D Face Reconstruction and Dense Alignment

In this work, we propose Geometry Normal Consistency Loss (GNC) for 3D face reconstruction and dense alignment. The existing methods based on the strong constraints of the 3DMM parameter regression only consider reducing the error between 68 landmarks, while they rarely consider the geometric contour structure relation of the face. Instead, we take into account the discrete 68 landmarks as loss constrain by introducing geometry area and normal consistency loss, which naturally defines the holistic and local geometric contour structure of the face. In detail, we select the inverted triangle formed by the leftmost and rightmost landmarks on the cheek and the lowest point of the chin to globally constrain the entire facial feature. In addition, we triangulate the facial landmarks to construct triangular patches, and calculate the normals of the patches, which aims to make use of normal consistency between the local patch and the corresponding ground truth to reconstruct rich local details. Extensive experimental results on AFLW2000-3D and AFLW datasets demonstrate that our GNC achieves compelling performance compared to the state of the arts.
Unlabeled Backdoor Poisoning in Semi-Supervised Learning

Different from supervised learning which requires all training examples to be labeled, Semi-Supervised Learning (SSL) learns from a few labeled training examples and a large number of unlabeled training examples. Recently, studies have shown that SSL is also vulnerable to backdoor attacks. However, their performance is poor. In this paper, we propose a novel unlabeled backdoor poisoning attack against SSL, where only poisoning unlabeled examples in the training set to inject the backdoor into the network. Specifically, our attack exploits the vulnerability of SSL algorithms in guessing pseudo labels of unlabeled examples. We propose a backdoor generation network to generate poisoned examples with both the backdoor property and misleading function, thus inducing the victim model itself to mislabel the poisoned examples as the target class and causing the backdoor to be injected. Our attack achieves favorable attack success rates on the SSL algorithm while bypassing backdoor defenses.
STAR-GNN: Spatial-Temporal Video Representation for Content-Based Retrieval

We propose a video feature representation learning frame-work called STAR-GNN, which applies a pluggable graph neural network component on a multi-scale lattice feature graph. The essence of STAR-GNN is to exploit both the temporal dynamics and spatial contents as well as vi-sual connections between regions at different scales in the frames. It models a video with a lattice feature graph in which the nodes represent regions of different granularity, with weighted edges that represent the spatial and temporal links. The contextual nodes are aggregated simultaneously by graph neural networks with parameters trained with re-trieval triplet loss. In the experiments, we show that STAR-GNN effectively implements a dynamic attention mechanism on video frame sequences, resulting in the emphasis for dy-namic and semantically rich content in the video, and is robust to noise and redundancies. Empirical results show that STAR-GNN achieves state-of-the-art performance for Content-Based Video Retrieval.
Targeted Anonymization: A Face Image Anonymization Method for Unauthorized Models

As an important biometric feature of every person, face data has faced serious risks of leakage in recent years. Lawbreakers can use face recognition systems (FRS) to analyze the leaked face data and then correlate other private information, causing serious privacy leaks. For security reasons, we hope our face images can only be recognized by the organizations' authorized models. To achieve this goal, this work proposes a targeted face image anonymization method that only enables anonymization for unauthorized facial recognition models, whilst authorized models, human eyes can still accurately recognize faces. Our method mainly uses transfer-based adversarial attacks to achieve anonymization. On this basis, we propose constraints for generating targeted anonymization samples and boundary walking strategy, focusing on improving the anonymization for unauthorized models while guaranteeing the accurate recognition of authorized models. Local experiments prove that our method can reduce the recognition probability of unauthorized models while guaranteeing the correctness of authorized models. Finally, we apply our approach to an online face recognition API and experimentally demonstrate that our approach can significantly reduce the recognition accuracy of the commercial face recognition model.
Adaptive Multi-Feature Extraction Graph Convolutional Networks for Multimodal Target Sentiment Analysis

The multi-modal target-oriented sentiment analysis aims at predicting the sentiment polarities for target entities in a sentence by combining vision and language information. However, most existing deep learning approaches fail to extract valuable information from the visual modality and ignore the usability of syntactic dependency information embedded in the text modality. In this paper, we propose a two-stream adaptive multi-feature extraction graph convolutional networks (AME-GCN), which translates the image into a textual caption and dynamically fuses the semantic and syntactic feature from the given sentence and generated caption to model the inter/intra-modality dynamics. Extensive experiments on two multi-modal Twitter datasets show the effectiveness of the proposed model against popular textual and multi-modal approaches, demonstrating that AME-GCN is a best alternative for this task.
Learning to Mitigate Extreme Distribution Bias for Few-Shot Object Detection

Few-shot object detection is an important but challenging task where only a few instances of novel categories are available. The widely used approach is to pretrain a detector on base classes with abundant samples and then fine-tune it for novel classes. Due to the extreme data imbalance between base and novel classes, the detection performance of novel classes degrades with the distribution bias. To overcome this limitation, we propose a distribution calibration strategy and a class discrimination regularization method for better few-shot detection. Based on theoretical analysis on decision margins of base and novel classes, the decision area of novel classes is enlarged to balance the prediction probability. On the other hand, to increase the separability of inter-class distributions, the similarity between class-specific representations is minimized. Extensive experiments on PASCAL VOC and MS COCO datasets verify the effectiveness and generalization ability of our method to improve few-shot object detection.
Location-Aware Feature Selection Network for Multi-Oriented Scene Text Detection

Direct Regression-based text detection methods have already achieved promising performances with simple network structure. However, there still leaves improvement room in terms of regression accuracy, especially for long and large text instances, which hinders their applicability in some realistic scenarios that need recognition of the detected text. To address this issue, we propose a novel Location Aware feature Selection text detection Network (LASNet) which selects suitable features from different locations to separately predict each component of a bounding box and gets the final bounding box through the combination of them. 1As a result, LASNet predicts the more accurate bounding boxes by effectively making use of features with a learnable feature selection way. The experimental results demonstrate that our LASNet achieves state-of-the-art performance with single model and single-scale testing.
STDNet: Spatio-Temporal Decomposed Network for Video Grounding

Previous methods for video grounding treated either the query or the video as a whole, while neglecting their respective semantics in the orthogonal space and time dimensions. Since spatial semantics appears frequently in a video, temporal semantics is more discriminative and deserves more attention. Based on such considerations, we propose a novel Spatio-Temporal Decomposed Network (STDNet) which decomposes the query and the video into their spatial and temporal semantics, respectively. Specifically, spatial and temporal words are selected from the query, and the video is split into two pathways. Spatial cross-modal attention is computed first and serves as prior knowledge for temporal attention. A new localization strategy is also devised which regresses the segment's start conditioned on the end and essentially breaks the independence assumption made in previous methods. Experimental results on three public benchmark datasets show that our STDNet outperforms the state-of-the-art methods.