Reviewerly/exHarmony huggingface dataset:
A Sampling-Based Density Peaks Clustering Algorithm for Large-Scale Data

With the rapid development of information technology, massive amount of data is generated. How to discover useful information to support decision-making has become one of the focuses of scholar's research. Clustering is thought to be one of the main means to deal with large-scale data. Density peaks clustering (DPC) is an effective density-based clustering algorithm which is widely applied in numerous fields because of its satisfactory performance. However, the computational complexity of DPC is O(N2) which is not friendly to large-scale data. To solve this issue, a sampling-based density peaks clustering algorithm for large-scale data (SDPC) is proposed. Firstly, a sampling method is used to reduce the distance calculations. Secondly, approximate representatives are identified by an improved TI search strategy which further accelerates the clustering process. Afterwards, the approximate representatives are clustered by DPC. Finally, the remaining points are allocated to the same cluster as its nearest representatives. Experimental results on both synthetic datasets and real-world datasets illustrate that SDPC is more efficient than DPC, while its clustering performance maintains the same level as DPC.
AES Based White Box Cryptography in Digital Signature Verification

According to the standard paradigm, white box cryptographic primitives are used to block black box attacks and protect sensitive information. This is performed to safeguard the protected information and keys against black box assaults. An adversary in such a setting is aware of the method and can analyze many system inputs and outputs, but is blind to the specifics of how a critical instantiation primitive is implemented. This is the focus of white-box solutions, which are designed to withstand attacks that come from the execution environment. This is significant because an attacker may obtain unrestricted access to the program's execution in this environment. The purpose of this article is to assess the efficiency of white-box implementations in terms of security. Our contribution is twofold: first, we explore the practical implementations of white-box approaches, and second, we analyze the theoretical foundations upon which these implementations are built. First, a research proposal is crafted that details white-box applications of DES and AES encryption algorithms. To begin, this preparation is necessary. The research effort planned for this project also includes cryptanalysis of these techniques. Once the general cryptanalysis results have been examined, the white-box design approaches will be covered. We have decided to launch an investigation into creating a theoretical model for white box, since no prior formal definitions have been offered, and suggested implementations have not been accompanied by any assurance of security. This is due to the fact that no formal definition of "white box" has ever been provided. In this way lies the explanation for why this is the situation. We define WBC to encompass the security requirements of WBC specified over a white box cryptography technology and a security concept by studying formal models of obfuscation and shown security. This definition is the product of extensive investigation. This state-of-the-art theoretical model provides a setting in which to investigate the security of white-box implementations, leading to a wide range of positive and negative conclusions. As a result, this paper includes the results of a Digital Signature Algorithm (DSA) study which may be put to use in the real world with signature verification. Possible future applications of White Box Cryptography (WBC) research findings are discussed in light of these purposes and areas of investigation.
Improving aspect-based sentiment analysis with Knowledge-aware Dependency Graph Network

Aspect-based sentiment analysis (ABSA) aims to mine multiple sentiment–target pairs contained in a review sentence. The main challenge of this task is how to extract the sentiment polarity of a specific sentiment item efficiently. Earlier research focused on recurrent neural networks (RNNs), which implicitly associate the sentiment items with sentiment polarities through an attention mechanism. However, due to the complexity of language and the fact that a sentence contains multiple sentiment pairs, these models often fail to capture sentiment pairs accurately. Most recent efforts have applied syntactic information, especially dependency information, to construct structured models (e.g., tree-based models or graph neural networks) for sentiment analysis. Although these structured models achieve better results, they ignore the domain knowledge related to the entities of the comment sentences. This domain knowledge (e.g., brand reputation, influence) significantly impacts the sentiment polarity. Hence, this paper proposes a Knowledge-aware Dependency Graph Network (KDGN) based on the dependency graph incorporating domain knowledge, dependency labels, and syntax path. Experimental results on the benchmarking datasets demonstrate that our KDGN significantly outperforms previous state-of-the-art methods on the ABSA task, further illustrating that the domain knowledge, dependency labels, and syntax path are crucial for the ABSA task.
Hierarchical belief rule-based model for imbalanced multi-classification

Classification tasks are of great importance in machine learning. However, class imbalance is a universal problem that needs to be solved in classification and can greatly affect the performance of machine learning classifiers. Developing from the basic belief rule base (BRB) system, the hierarchical belief rule-based system can integrate expert knowledge and has the potential to alleviate the negative effect of class imbalance. To utilize BRB to solve the imbalanced multi-classification task and avoid the combinational explosion problem, a novel hierarchical BRB structure based on the extreme gradient boosting (XGBoost) feature selection method, abbreviated as HFS-BRB is proposed in this paper in order to deal with any number of classes. In the hierarchical BRB structure, there is one main-BRB in the first level and several sub-BRBs in the second level. The XGBoost technique is used for feature selection in the modelling process of each abovementioned BRB model. The output of the main BRB represents the approximated classification between confusable classes. Then, these samples were transmitted to a certain sub-BRB for binary classification to make a precise prediction. Thus, a multi-classification problem can be transformed into several binary classification problems. The class imbalance is alleviated. To test the effectiveness of the proposed method, seven classical benchmark problems for imbalanced classification and a real asteroid orbit classification were performed.
Review on Deep Learning Approaches for Anomaly Event Detection in Video Surveillance

In the last few years, due to the continuous advancement of technology, human behavior detection and recognition have become important scientific research in the field of computer vision (CV). However, one of the most challenging problems in CV is anomaly detection (AD) because of the complex environment and the difficulty in extracting a particular feature that correlates with a particular event. As the number of cameras monitoring a given area increases, it will become vital to have systems capable of learning from the vast amounts of available data to identify any potential suspicious behavior. Then, the introduction of deep learning (DL) has brought new development directions for AD. In particular, DL models such as convolution neural networks (CNNs) and recurrent neural networks (RNNs) have achieved excellent performance dealing with AD tasks, as well as other challenging domains like image classification, object detection, and speech processing. In this review, we aim to present a comprehensive overview of those research methods using DL to address the AD problem. Firstly, different classifications of anomalies are introduced, and then the DL methods and architectures used for video AD are discussed and analyzed, respectively. The revised contributions have been categorized by the network type, architecture model, datasets, and performance metrics that are used to evaluate these methodologies. Moreover, several applications of video AD have been discussed. Finally, we outlined the challenges and future directions for further research in the field.
FedPOIRec: Privacy-preserving federated poi recommendation with social influence

With the growing number of Location-Based Social Networks, privacy-preserving point-of-interest (POI) recommendation has become a critical challenge when helping users discover potentially interesting new places. Traditional systems take a centralized approach that requires the transmission and collection of private user data. In this work, we present FedPOIRec, a privacy-preserving federated learning approach enhanced with features from user social circles to generate top-N POI recommendations. First, the FedPOIRec framework is built on the principle that local data never leave the owner’s device, while a parameter server blindly aggregates the local updates. Second, the local recommender results are personalized by allowing users to exchange their learned parameters, enabling knowledge transfer among friends. To this end, we propose a privacy-preserving protocol for integrating the preferences of the user’s friends, after the federated computation, by exploiting the properties of the Cheon-Kim-Kim-Song (CKKS) fully homomorphic encryption scheme. To evaluate FedPOIRec, we apply our approach to five real-world datasets using two recommendation models. Extensive experiments demonstrate that FedPOIRec achieves comparable recommendation quality to centralized approaches, while the social integration protocol incurs low computation and communication overhead on the user device.
CAFE: Learning to Condense Dataset by Aligning Features

Dataset condensation aims at reducing the network training effort through condensing a cumbersome training set into a compact synthetic one. State-of-the-art approaches largely rely on learning the synthetic data by matching the gradients between the real and synthetic data batches. Despite the intuitive motivation and promising results, such gradient-based methods, by nature, easily overfit to a biased set of samples that produce dominant gradients, and thus lack a global supervision of data distribution. In this paper, we propose a novel scheme to Condense dataset by Aligning FEatures (CAFE), which explicitly attempts to preserve the real-feature distribution as well as the discriminant power of the resulting synthetic set, lending itself to strong generalization capability to various architectures. At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales, while accounting for the classification of real samples. Our scheme is further backed up by a novel dynamic bi-level optimization, which adaptively adjusts parameter updates to prevent over-/under-fitting. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art: on the SVHN dataset, for example, the performance gain is up to 11%. Extensive experiments and analysis verify the effectiveness and necessity of proposed designs.
Efficient Multi-view Stereo by Iterative Dynamic Cost Volume

In this paper, we propose a novel iterative dynamic cost volume for multi-view stereo. Compared with other works, our cost volume is much lighter, thus could be processed with 2D convolution based GRU. Notably, the every-step output of the GRU could be further used to generate new cost volume. In this way, an iterative GRU-based optimizer is constructed. Furthermore, we present a cascade and hierarchical refinement architecture to utilize the multiscale information and speed up the convergence. Specifically, a lightweight 3D CNN is utilized to generate the coarsest initial depth map which is essential to launch the GRU and guarantee a fast convergence. Then the depth map is refined by multi-stage GRUs which work on the pyramid feature maps. Extensive experiments on the DTU and Tanks & Temples benchmarks demonstrate that our method could achieve state-of-the-art results in terms of accuracy, speed and memory usage. Code will be released at https://github.com/bdwsq1996/Effi-MVS.
KeypointNeRF: Generalizing Image-Based Volumetric Avatars Using Relative Spatial Encoding of Keypoints

Image-based volumetric humans using pixel-aligned features promise generalization to unseen poses and identities. Prior work leverages global spatial encodings and multi-view geometric consistency to reduce spatial ambiguity. However, global encodings often suffer from overfitting to the distribution of the training data, and it is difficult to learn multi-view consistent reconstruction from sparse views. In this work, we investigate common issues with existing spatial encodings and propose a simple yet highly effective approach to modeling high-fidelity volumetric humans from sparse views. One of the key ideas is to encode relative spatial 3D information via sparse 3D keypoints. This approach is robust to the sparsity of viewpoints and cross-dataset domain gap. Our approach outperforms state-of-the-art methods for head reconstruction. On human body reconstruction for unseen subjects, we also achieve performance comparable to prior work that uses a parametric human body model and temporal feature aggregation. Our experiments show that a majority of errors in prior work stem from an inappropriate choice of spatial encoding and thus we suggest a new direction for high-fidelity image-based human modeling.
XYDeblur: Divide and Conquer for Single Image Deblurring

Many convolutional neural networks (CNNs) for single image deblurring employ a U-Net structure to estimate latent sharp images. Having long been proven to be effective in image restoration tasks, a single lane of encoder-decoder architecture overlooks the characteristic of deblurring, where a blurry image is generated from complicated blur kernels caused by tangled motions. Toward an effective network architecture for single image deblurring, we present complemental sub-solution learning with a one-encoder-two-decoder architecture. Observing that multiple decoders successfully learn to decompose encoded feature information into directional components, we further improve both the network efficiency and the deblurring performance by rotating and sharing kernels exploited in the decoders, which prevents the decoders from separating unnecessary components such as color shift. As a result, our proposed network shows superior results compared to U-Net while preserving the network parameters, and using the proposed network as the base network can improve the performance of existing state-of-the-art deblurring networks.
Homography Loss for Monocular 3D Object Detection

Monocular 3D object detection is an essential task in autonomous driving. However, most current methods consider each 3D object in the scene as an independent training sample, while ignoring their inherent geometric relations, thus inevitably resulting in a lack of leveraging spatial constraints. In this paper, we propose a novel method that takes all the objects into consideration and explores their mutual relationships to help better estimate the 3D boxes. More-over, since 2D detection is more reliable currently, we also investigate how to use the detected 2D boxes as guidance to globally constrain the optimization of the corresponding predicted 3D boxes. To this end, a differentiable loss function, termed as Homography Loss, is proposed to achieve the goal, which exploits both 2D and 3D information, aiming at balancing the positional relationships between different objects by global constraints, so as to obtain more ac-curately predicted 3D boxes. Thanks to the concise design, our loss function is universal and can be plugged into any mature monocular 3D detector, while significantly boosting the performance over their baseline. Experiments demon-strate that our method yields the best performance (Nov. 2021) compared with the other state-of-the-arts by a large margin on KITTI3D datasets.
Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Images

Learning 3D generative models from a dataset of monocular images enables self-supervised 3D reasoning and controllable synthesis. State-of-the-art 3D generative models are GANs that use neural 3D volumetric representations for synthesis. Images are synthesized by rendering the volumes from a given camera. These models can disentangle the 3D scene from the camera viewpoint in any generated image. However, most models do not disentangle other factors of image formation, such as geometry and appearance. In this paper, we design a 3D GAN which can learn a disentangled model of objects, just from monocular observations. Our model can disentangle the geometry and appearance variations in the scene, i.e., we can independently sample from the geometry and appearance spaces of the generative model. This is achieved using a novel non-rigid deformable scene formulation. A 3D volume that represents an object instance is computed as a non-rigidly deformed canonical 3D volume. Our method learns the canonical volume, as well as its deformations, jointly during training. This formulation also helps us improve the disentanglement between the 3D scene and the camera viewpoints using a novel pose regularization loss defined on the 3D deformation field. In addition, we model the inverse deformations, enabling the computation of dense correspondences between images generated by our model. Finally, we design an approach to embed real images into the latent space of our model, enabling editing of real images.
Towards Weakly-Supervised Text Spotting using a Multi-Task Transformer

Text spotting end-to-end methods have recently gained attention in the literature due to the benefits of jointly optimizing the text detection and recognition components. Existing methods usually have a distinct separation between the detection and recognition branches, requiring exact annotations for the two tasks. We introduce TextTranSpotter (TTS), a transformer-based approach for text spotting and the first text spotting framework which may be trained with both fully- and weakly-supervised settings. By learning a single latent representation per word detection, and using a novel loss function based on the Hungarian loss, our method alleviates the need for expensive localization annotations. Trained with only text transcription annotations on real data, our weakly-supervised method achieves competitive performance with previous state-of-the-art fully-supervised methods. When trained in a fully-supervised manner, TextTranSpotter shows state-of-the-art results on multiple benchmarks.
Personalized Image Aesthetics Assessment with Rich Attributes

Personalized image aesthetics assessment (PIAA) is challenging due to its highly subjective nature. People's aesthetic tastes depend on diversified factors, including image characteristics and subject characters. The existing PIAA databases are limited in terms of annotation diversity, especially the subject aspect, which can no longer meet the increasing demands of PIAA research. To solve the dilemma, we conduct so far, the most comprehensive subjective study of personalized image aesthetics and introduce a new Personalized image Aesthetics database with Rich Attributes (PARA), which consists of 31,220 images with annotations by 438 subjects. PARA features wealthy annotations, including 9 image-oriented objective attributes and 4 human-oriented subjective attributes. In addition, desensitized subject information, such as personality traits, is also provided to support study of PIAA and user portraits. A comprehensive analysis of the annotation data is provided and statistic study indicates that the aesthetic preferences can be mirrored by proposed subjective attributes. We also propose a conditional PIAA model by utilizing subject information as conditional prior. Experimental results indicate that the conditional PIAA model can outperform the control group, which is also the first attempt to demonstrate how image aesthetics and subject characters interact to produce the intricate personalized tastes on image aesthetics. We believe the database and the associated analysis would be useful for conducting next-generation PIAA study. The project page of PARA can be found at: https://cv-datasets.institutecv.com/#/data-sets.
Learning to represent programs with heterogeneous graphs

Code representation, which transforms programs into vectors with semantics, is essential for source code processing. We have witnessed the effectiveness of incorporating structural information (i.e., graph) into code representations in recent years. Specifically, the abstract syntax tree (AST) and the AST-augmented graph of the program contain much structural and semantic information, and most existing studies apply them for code representation. The graph adopted by existing approaches is homogeneous, i.e., it discards the type information of the edges and the nodes lying within AST. That may cause plausible obstruction to the representation model. In this paper, we propose to leverage the type information in the graph for code representation. To be specific, we propose the heterogeneous program graph (HPG), which provides the types of the nodes and the edges explicitly. Furthermore, we employ the heterogeneous graph transformer (HGT) architecture to generate representations based on HPG, considering the type of information during processing. With the additional types in HPG, our approach can capture complex structural information, produce accurate and delicate representations, and finally perform well on certain tasks. Our in-depth evaluations upon four classic datasets for two typical tasks (i.e., method name prediction and code classification) demonstrate that the heterogeneous types in HPG benefit the representation models. Our proposed HPG+HGT also outperforms the SOTA baselines on the subject tasks and datasets.
Weakly Supervised Temporal Sentence Grounding with Gaussian-based Contrastive Proposal Learning

Temporal sentence grounding aims to detect the most salient moment corresponding to the natural language query from untrimmed videos. As labeling the temporal boundaries is labor-intensive and subjective, the weakly- supervised methods have recently received increasing attention. Most of the existing weakly-supervised methods gen-erate the proposals by sliding windows, which are content- independent and of low quality. Moreover, they train their model to distinguish positive visual-language pairs from negative ones randomly collected from other videos, ignoring the highly confusing video segments within the same video. In this paper, we propose Contrastive Proposal Learning(CPL) to overcome the above limitations. Specifi-cally, we use multiple learnable Gaussian functions to gen-erate both positive and negative proposals within the same video that can characterize the multiple events in a long video. Then, we propose a controllable easy to hard neg-ative proposal mining strategy to collect negative samples within the same video, which can ease the model opti-mization and enables CPL to distinguish highly confusing scenes. The experiments show that our method achieves state-of-the-art performance on Charades-STA and Activi-tyNet Captions datasets. The code and models are available at https://github.com/minghangz/cpl.