Reviewerly/exHarmony huggingface dataset:
Deep Learning Based One-Class Detection System for Fake Faces Generated by GAN Network

Recently, the dangers associated with face generation technology have been attracting much attention in image processing and forensic science. The current face anti-spoofing methods based on Generative Adversarial Networks (GANs) suffer from defects such as overfitting and generalization problems. This paper proposes a new generation method using a one-class classification model to judge the authenticity of facial images for the purpose of realizing a method to generate a model that is as compatible as possible with other datasets and new data, rather than strongly depending on the dataset used for training. The method proposed in this paper has the following features: (a) we adopted various filter enhancement methods as basic pseudo-image generation methods for data enhancement; (b) an improved Multi-Channel Convolutional Neural Network (MCCNN) was adopted as the main network, making it possible to accept multiple preprocessed data individually, obtain feature maps, and extract attention maps; (c) as a first ingenuity in training the main network, we augmented the data using weakly supervised learning methods to add attention cropping and dropping to the data; (d) as a second ingenuity in training the main network, we trained it in two steps. In the first step, we used a binary classification loss function to ensure that known fake facial features generated by known GAN networks were filtered out. In the second step, we used a one-class classification loss function to deal with the various types of GAN networks or unknown fake face generation methods. We compared our proposed method with four recent methods. Our experiments demonstrate that the proposed method improves cross-domain detection efficiency while maintaining source-domain accuracy. These studies show one possible direction for improving the correct answer rate in judging facial image authenticity, thereby making a great contribution both academically and practically.
An NLP-guided ontology development and refinement approach to represent and query visual information

The ubiquitous presence of surveillance systems generates massive amounts of video data. Storage and analysis of this data in real-time is a substantial challenge. There is huge potential in representing data in machine-readable and machine-interpretable format due to the presence of hidden semantics in images and videos. However, such representation requires ontology, which calls for expert domain knowledge. In this paper, a novel NLP-guided approach to generate an ontology for multimedia representation and information retrieval is proposed. A semi-automatic NLP-guided framework, which extracts all possible relations among objects is presented. This framework leverages the textual data of the domain to generate possible descriptions and actions within the domain. Relations among objects get embedded as object properties, whereas the category of an object as a class. Features and attributes of objects encode the data properties of the ontology. The proposed ontology is compared with existing multimedia ontologies and evaluated with regard to its capability to represent relations occurring in benchmark datasets, demonstrating the completeness and thorough coverage of the domain concepts. Spatial reasoning rules are established using Semantic Web Rule Language (SWRL) rules, and information retrieval is demonstrated using Description Logic (DL) and SPARQL queries. The proposed NLP-guided ontology generation approach is general enough to help in the development of ontologies for other domains as well, by providing video and textual data of the domain of interest, with limited human involvement.
Cryptocurrencies and the promise of individual economic sovereignty in an age of digitalization: A critical appraisal

Cryptocurrencies have fuelled an ideological bifurcation between utopian imaginaries of borderless individual economic sovereignty and egalitarianism among libertarian sympathizers since Hayek and more recent dystopian admonitions against financial disruption and inequality by state actors. Providing a state-of-the-art sociological account of this debate, this article conducts the first empirical study on the links between cryptocurrency adoption, individual economic sovereignty and wealth inequality using a novel 2020 cross-societal database of the population-level adoption of the six largest cryptocurrencies. Analyses show that cryptocurrency adoption neither ameliorates nor worsens wealth inequality, but that the geographical expansion of cryptocurrencies is dependent on the very political economic conditions and state permissions that their libertarian enthusiasts spurn. This article concludes by invigorating a programmatic call and agenda for the empirical, sociological study of cryptocurrencies and the methodological designs required to sustain it.
Development of Deep Learning Technique of Features for the Analysis of Clinical Images Integrated with CANN

Computer tomography is an extensively used method for the detection of the disease in the subjects. Basically, computer-aided tomography depending on the artificial intelligence reveals its significance in smart health care monitoring system. Owing to its security and the private issue, analyzing the computed tomography dataset has become a tedious process. This study puts forward the convolutional autoencrypted deep learning neural network to assist unsupervised learning technique. By carrying out various experiments, our proposed method produces better results comparative to other traditional methods, which efficaciously solves the issues related to the artificial image description. Hence, the convolutional autoencoder is widely used in measuring the lumps in the bronchi. With the unsupervised machine learning, the extracted features are used for various applications.
Global trends and hotspots in research on extended reality in sports: A bibliometric analysis from 2000 to 2021

Extended reality technologies (e.g. virtual reality (VR), augmented reality (AR) and mixed reality (MR)) are gaining popularity in sports owing to their unique advantages. This study aims to analyse the progress of the application of extended reality technology in sports and reveal its cooperative features, research hotspots and development trends.We searched the literature in the Web of Science Core Collection (WoSCC) database within the period 2000 to 2021 and conducted a bibliometric analysis. The analysis methods included statistical, co-occurrence, hierarchical clustering and social network analyses.A total of 340 articles were gathered. The literature related to its research showed an increasing trend over time. The paper collaboration rate was 90.88% (309/340 papers), and the degree of author collaboration was 3.96 (1345/340). VR was found to be the most productive journal, and Queen's University Belfast was the most productive institution. The United States, China and the United Kingdom were the three main contributors to the field. The foundational themes in sports extended reality research were (i) sports games and extended reality systems, (ii) virtual simulation devices and artificial intelligence, (iii) sports training and performance and (iv) age-appropriate physical activity, sports rehabilitation and physical education.The level of author collaboration was low, but the degree of author collaboration is largely on the rise. The closeness of the collaboration between institutions and countries was also low. In addition, the subject of sport extended reality is relatively fragmented. Therefore, more research is needed to strengthen it in the future.
Graph Node Based Interpretability Guided Sample Selection for Active Learning

While supervised learning techniques have demonstrated state-of-the-art performance in many medical image analysis tasks, the role of sample selection is important. Selecting the most informative samples contributes to the system attaining optimum performance with minimum labeled samples, which translates to fewer expert interventions and cost. Active Learning (AL) methods for informative sample selection are effective in boosting performance of computer aided diagnosis systems when limited labels are available. Conventional approaches to AL have mostly focused on the single label setting where a sample has only one disease label from the set of possible labels. These approaches do not perform optimally in the multi-label setting where a sample can have multiple disease labels (e.g. in chest X-ray images). In this paper we propose a novel sample selection approach based on graph analysis to identify informative samples in a multi-label setting. For every analyzed sample, each class label is denoted as a separate node of a graph. Building on findings from interpretability of deep learning models, edge interactions in this graph characterize similarity between corresponding interpretability saliency map model encodings. We explore different types of graph aggregation to identify informative samples for active learning. We apply our method to public chest X-ray and medical image datasets, and report improved results over state-of-the-art AL techniques in terms of model performance, learning rates, and robustness.
A Comparative Study among Handwritten Signature Verification Methods Using Machine Learning Techniques

Nowadays, the verification of handwritten signatures has become an effective research field in computer vision as well as machine learning. Signature verification is naturally formulated as a machine-learning task. This task is performed by determining if the signature is genuine or forged. Therefore, it is considered a two‐class classification issue. Since handwritten signatures are widely used in legal documents and financial transactions, it is important for researchers to select an efficient machine-learning technique for verifying these signatures and to avoid forgeries that may cause many losses to customers. So far, great outcomes have been obtained when using machine learning techniques in terms of equal error rates and calculations. This paper presents a comprehensive review of the latest studies and results in the last 10 years in the field of online and offline handwritten signature verification. More than 20 research papers were used to make a comparison between datasets, feature extraction, and classification techniques used in each system, taking into consideration the problems that occur in each. In addition, the general limitations and advantages of machine-learning techniques that are used to classify or extract signature features were summarized in the form of a table. We also present the general steps of the verification system and a list of the most considerable datasets available in online and offline fields.
Knowledge-Sensed Cognitive Diagnosis for Intelligent Education Platforms

Cognitive diagnosis is a fundamental issue of intelligent education platforms, whose goal is to reveal the mastery of students on knowledge concepts. Recently, certain efforts have been made to improve the diagnosis precision, by designing deep neural networks-based diagnostic functions or incorporating more rich context features to enhance the representation of students and exercises. However, how to interpretably infer the student's mastery over non-interactive knowledge concepts (i.e., knowledge concepts not related to his/her exercising records) still remains challenging, especially when not giving relations between knowledge concepts. To this end, we propose a Knowledge-Sensed Cognitive Diagnosis (KSCD) framework, aiming at learning intrinsic relations among knowledge concepts from student response logs and incorporating them for inferring students' mastery over all knowledge concepts in an end-to-end manner. Specifically, we firstly project students, exercises and knowledge concepts into embedding representation matrices, where the intrinsic relations among knowledge concepts are reflected in the knowledge embedding representation matrix. Then, the knowledge-sensed student knowledge mastery vector and exercise factor vectors are obtained by the multiply product of their embedding representations and the knowledge embedding representation matrix, which make the student's mastery of non-interactive knowledge concepts be interpretably inferred. Finally, we can utilize classical student-exercise interaction functions to predict student's exercising performance and jointly train the model. In additional, we also design a new function to better model the student-exercise interactions. Extensive experimental results on two real-world datasets clearly show the significant performance gain of our KSCD framework, especially in predicting students' mastery over non-interactive knowledge concepts, by comparing to state-of-the-art cognitive diagnosis models (CDMs).
Multimodal Meta-Learning for Cold-Start Sequential Recommendation

In this paper, we study the task of cold-start sequential recommendation, where new users with very short interaction sequences come with time. We cast this problem as a few-shot learning problem and adopt a meta-learning approach to developing our solution. For our task, a major obstacle of effective knowledge transfer that is there exists significant characteristic divergence between old and new interaction sequences for meta-learning. To address the above issues, we purpose a Multimodal MetaLearning (denoted as MML) approach that incorporates multimodal side information of items (e.g., text and image) into the meta-learning process, to stabilize and improve the meta-learning process for cold-start sequential recommendation. In specific, we design a group of multimodal meta-learners corresponding to each kind of modality, where ID features are used to develop the main meta-learner and the rest text and image features are used to develop auxiliary meta-learners. Instead of simply combing the predictions from different meta-learners, we design an adaptive, learnable fusion layer to integrate the predictions based on different modalities. Meanwhile, we design a cold-start item embedding generator, which utilize multimodal side information to warm up the ID embeddings of new items. Extensive offline and online experiments demonstrate that MML can significantly improve the recommendation performance for cold-start users compared with baseline models. Our code is released at https://github.com/RUCAIBox/MML.
Are Gradients on Graph Structure Reliable in Gray-box Attacks?

Graph edge perturbations are dedicated to damaging the prediction of graph neural networks by modifying the graph structure. Previous gray-box attackers employ gradients from the surrogate model to locate the vulnerable edges to perturb the graph structure. However, unreliability exists in gradients on graph structures, which is rarely studied by previous works. In this paper, we discuss and analyze the errors caused by the unreliability of the structural gradients. These errors arise from rough gradient usage due to the discreteness of the graph structure and from the unreliability in the meta-gradient on the graph structure. In order to address these problems, we propose a novel attack model with methods to reduce the errors inside the structural gradients. We propose edge discrete sampling to select the edge perturbations associated with hierarchical candidate selection to ensure computational efficiency. In addition, semantic invariance and momentum gradient ensemble are proposed to address the gradient fluctuation on semantic-augmented graphs and the instability of the surrogate model. Experiments are conducted in untargeted gray-box poisoning scenarios and demonstrate the improvement in the performance of our approach.
Causal Intervention for Sentiment De-biasing in Recommendation

Biases and de-biasing in recommender systems have received increasing attention recently. This study focuses on a newly identified bias, i.e., sentiment bias, which is defined as the divergence in recommendation performance between positive users/items and negative users/items. Existing methods typically employ a regularization strategy to eliminate the bias. However, blindly fitting the data without modifying the training procedure would result in a biased model, sacrificing recommendation performance.
Pre-training Tasks for User Intent Detection and Embedding Retrieval in E-commerce Search

BERT-style models pre-trained on the general corpus (e.g., Wikipedia) and fine-tuned on specific task corpus, have recently emerged as breakthrough techniques in many NLP tasks: question answering, text classification, sequence labeling and so on. However, this tech- nique may not always work, especially for two scenarios: a corpus that contains very different text from the general corpus Wikipedia, or a task that learns embedding spacial distribution for a specific purpose (e.g., approximate nearest neighbor search). In this paper, to tackle the above two scenarios that we have encountered in an industrial e-commerce search system, we propose customized and novel pre-training tasks for two critical modules: user intent detec- tion and semantic embedding retrieval. The customized pre-trained models after fine-tuning, being less than 10% of BERT-base's size in order to be feasible for cost-efficient CPU serving, significantly improve the other baseline models: 1) no pre-training model and 2) fine-tuned model from the official pre-trained BERT using general corpus, on both offline datasets and online system. We have open sourced our datasets 1 for the sake of reproducibility and future works.
Sampling Enclosing Subgraphs for Link Prediction

Link prediction is a fundamental problem for graph-structured data (e.g., social networks, drug side-effect networks, etc.). Graph neural networks have offered robust solutions for this problem, specifically by learning the representation of the subgraph enclosing the target link (i.e., pair of nodes). However, these solutions do not scale well to large graphs as extraction and operation on enclosing subgraphs are computationally expensive. This paper presents a scalable link prediction solution, that we call ScaLed, which utilizes sparse enclosing subgraphs to make predictions. To extract sparse enclosing subgraphs, ScaLed takes multiple random walks from a target pair of nodes, then operates on the sampled enclosing subgraph induced by all visited nodes. By leveraging the smaller sampled enclosing subgraph, ScaLed can scale to larger graphs with much less overhead while maintaining high accuracy. Through comprehensive experiments, we have shown that ScaLed can produce comparable accuracy to those reported by the existing subgraph representation learning frameworks while being less computationally demanding.
Modeling Dynamic Heterogeneous Graph and Node Importance for Future Citation Prediction

Accurate citation count prediction of newly published papers could help editors and readers rapidly figure out the influential papers in the future. Though many approaches are proposed to predict a paper's future citation, most ignore the dynamic heterogeneous graph structure or node importance in academic networks. To cope with this problem, we propose a Dynamic heterogeneous Graph and Node Importance network (DGNI) learning framework, which fully leverages the dynamic heterogeneous graph and node importance information to predict future citation trends of newly published papers. First, a dynamic heterogeneous network embedding module is provided to capture the dynamic evolutionary trends of the whole academic network. Then, a node importance embedding module is proposed to capture the global consistency relationship to figure out each paper's node importance. Finally, the dynamic evolutionary trend embeddings and node importance embeddings calculated above are combined to jointly predict the future citation counts of each paper, by a log-normal distribution model according to multi-faced paper node representations. Extensive experiments on two large-scale datasets demonstrate that our model significantly improves all indicators compared to the SOTA models.
An image enhancement approach for coral reef fish detection in underwater videos

Coral reefs are rich in fisheries and aquatic resources, and the study and monitoring of coral reef ecosystems are of great economic value and practical significance. Due to complex backgrounds and low-quality videos, it is challenging to identify coral reef fish. This study proposed an image enhancement approach for fish detection in complex underwater environments. The method first uses a Siamese network to obtain a saliency map and then multiplies this saliency map by the input image to construct an image enhancement module. Applying this module to the existing mainstream one-stage and two-stage target detection frameworks can significantly improve their detection accuracy. Good detection performance was achieved in a variety of scenarios, such as those with luminosity variations, aquatic plant movements, blurred images, large targets and multiple targets, demonstrating the robustness of the algorithm. The best performance was achieved on the LCF-15 dataset when combining the proposed method with the cascade region-based convolutional neural network (Cascade-RCNN). The average precision at an intersection-over-union (IoU) threshold of 0.5 (AP50) was 0.843, and the F1 score was 0.817, exceeding the best reported results on this dataset. This study provides an automated video analysis tool for marine-related researchers and technical support for downstream applications.
Efficient graph attentional network for 3D object detection from Frustum-based LiDAR point clouds

LiDAR-based 3D object detection is important for autonomous driving scene perception, but point clouds produced by LiDAR are irregular and unstructured in nature, and cannot be adopted by the conventional Convolutional Neural Networks (CNN). Recently, Graph Convolutional Networks (GCN) has been proved as an ideal way to handle non-Euclidean structure data, as well as for point cloud processing. However, GCN involves massive computation for searching adjacent nodes, and the heavy computational cost limits its applications in processing large-scale LiDAR point cloud in autonomous driving. In this work, we adopt a frustum-based point cloud-image fusion scheme to reduce the amount of LiDAR point clouds, thus making the GCN-based large-scale LiDAR point clouds feature learning feasible. On this basis, we propose an efficient graph attentional network to accomplish the goal of 3D object detection in autonomous driving, which can learn features from raw LiDAR point cloud directly without any conversions. We evaluate the model on the public KITTI benchmark dataset, the 3D detection mAP is 63.72% on KITTI Cars, Pedestrian and Cyclists, and the inference speed achieves 7.9 fps on a single GPU, which is faster than other methods of the same type.