Reviewerly/exHarmony huggingface dataset:
PRaNA: PRNU-based Technique to Tell Real and Deepfake Videos Apart

Videos are a powerful source of communication adopted in several contexts and used for both benign and malicious purposes (e.g., education vs. reputation damage). Nowadays, realistic video manipulation strategies like deepfake generators constitute a severe threat to our society in term of misinformation. While the wide range of the current research focuses on deepfake detection as a binary task, the identification of a real video among a pool of deepfakes sharing the same origin is not widely investigated. While the pool task might be more rare in real-life compared to the binary one, outcomes that derives from these analyses might let us better understand deepfake behaviours, benefiting binary deep fake detection as well. In this paper, we address the less investigated scenario by investigating the role of Photo Response Non-Uniformity (PRNU) in deepfake detection. Our analysis, in agreement with prior studies, shows that PRNU can be a valuable source to identify deepfake videos. In particular, we found that unique PRNU characteristics exist to distinguish real videos from their deepfake versions: real video autocorrelations tend to be lower compared to their deepfakes versions. Motivated by this, we propose PRaNA, a training-free strategy that leverages PRNU autocorrelation. Our results on three well-known datasets confirm our algorithm's robustness and transferability, with accuracy up to 66% when considering one real video in a pool of four deepfakes using the real video as a source, and up to 80% when only one deepfake is considered. Our work aims to open different strategies to counter deepfake diffusion.
High Precision and Light-Weight Network for Low Resolution SAR Image Detection

Target areas of low resolution SAR images usually have blurred edge and large background noise, so most common object detection methods based on deep learning have obvious errors in this occasion. In this paper, we propose a high precision and lightweight network for low resolution SAR image detection. We take generalized distribution to model bounding box in training and predicting to better indicate target area boundaries in low resolution SAR images, improving detection accuracy. Moreover, we introduce “teacher-student” knowledge distilling method, which greatly reduces model parameters and further enhances the detection accuracy. Compared with conventional deep learning net-works(Faster R-CNN, SSD, CenterNet, FCOS and YOLOv3) on low resolution SAR images, the results show that our method has not only the best performance in target area extractionn, but rather light weight.
Contract Cheat Detection using Biometric Keystroke Dynamics

Contract cheating has become a profound issue in academics with the onset of the COVID-19 pandemic as digitised evaluation has become common practice. This evaluation method opens up for examining students remotely, either by online home exams or longer written assessments done away from the classroom. Contract cheating refers to a problem where the students hire a third party to complete their assignment and submit it for grading as their own. Manually dealing with contract cheating is a cumbersome task and tools for plagiarism detection are not able to detect contract cheaters as students do not use the work of other authors without consent. In this paper, a machine learning based system is designed to specifically detect the cases of contract cheating in academics. The system uses keystroke biometric behaviour where typing style is analysed to discriminate cheaters from genuine students. The experiments are conducted on two datasets where one is existing and another is designed by performing data collection in a university for recording the keystroke features. Two categories of keystroke dynamics, namely duration and latency-based features are studied for designing the various machine learning-based systems for investigating the efficient one. Furthermore, the performance of the systems are evaluated under the setting of zero false accusations in order to avoid genuine students being charged as imposters.
Application of frequency-domain features and high-order cumulants in ANN-based communication modulation recognition

The channel environment of wireless communication is growing more and more complex as communication technology advances, and modulation schemes are becoming more and more diverse. Traditional modulation recognition approaches will struggle to fulfill the future requirement for highly reliable and low latency wireless communication. Based on previous studies, this paper extracts the frequency domain features and high-order cumulants (HOC) of digitally modulated signals and analog modulated signals, and further builds the artificial neural networks (ANN) based communication modulation identification model. Then use it to classify the modulated signals to better cope with the diversity of communication systems. When the signal-to-noise ratio is 20 dB, the proposed method improves recognition accuracy by 18.82 percent compared to ANN-based signal modulation recognition without feature extraction.
Ensemble Policy Distillation with Reduced Data Distribution Mismatch

Policy distillation is a method for model compression for deep reinforcement learning, which is typically applied onto mobile devices to reduce power consumption and inference time. However, achieving full and stable distilled policies is challenging, which impedes higher compression ratios. In this work, we develop two policy distillation algorithms to address this problem. Our first algorithm, Ensemble Policy Distillation (EPD), incorporates the idea from supervised learning distillation that uses an ensemble of teacher networks to provide diverse supervision for a compact student policy network. In the Deep Q-Network (DQN) framework, our experiments verify that highly compressed student networks distilled using EPD even outperform teachers for numerous Atari games. Additionally, we analyze how the issue of data distribution mismatch caused by the teacher ensemble in EPD negatively impacts teachers' learning, and introduce the second algorithm, Double Policy Distillation (DPD), as a novel method to mitigate the distribution mismatch. Empirical results show that DPD improves both the teachers' learning and the student's distillation in Atari games and continuous control tasks.
A Robust Lightweight Deepfake Detection Network Using Transformers

Deepfake detection attracts widespread attention in the computer vision field. Existing efforts achieve outstanding progress, but there are still significant unresolved issues. Coarse-grained local and global features are insufficient to capture subtle forgery traces from various inputs. Moreover, the detection efficiency is not powerful enough in practical applications. In this paper, we propose a robust and efficient transformer-based deepfake detection (TransDFD) network, which learns more discriminative and general manipulation patterns in an end-to-end manner. Specifically, a robust transformer module is designed to study fine-grained local and global features based on intra-patch locally-enhanced relations as well as inter-patch locally-enhanced global relationships in face images. A novel plug-and-play spatial attention scaling (SAS) module is proposed to emphasize salient features while suppressing less important representations, which can be integrated into any transformer-based models without increasing computational complexity. Extensive experiments on several public benchmarks demonstrate that the proposed TransDFD model outperforms the state-of-the-art in terms of robustness and computational efficiency.
Gaussian Variational Inference with Covariance Constraints Applied to Range-only Localization

Accurate and reliable state estimation is becoming increasingly important as robots venture into the real world. Gaussian variational inference (GVI) is a promising alternative for nonlinear state estimation, which estimates a full probability density for the posterior instead of a point estimate as in maximum a posteriori (MAP)-based approaches. GVI works by optimizing for the parameters of a multivariate Gaussian (MVG) that best agree with the observed data. However, such an optimization procedure must ensure the parameter constraints of a MVG are satisfied; in particular, the inverse covariance matrix must be positive definite. In this work, we propose a tractable algorithm for performing state estimation using GVI that guarantees that the inverse covariance matrix remains positive definite and is well-conditioned throughout the optimization procedure. We evaluate our method extensively in both simulation and real-world experiments for range-only localization. Our results show GVI is consistent on this problem, while MAP is over-confident.
Scraping Textures from Natural Images for Synthesis and Editing

Existing texture synthesis methods focus on generating large texture images given a small texture sample. But such samples are typically assumed to be highly curated: rectangular, clean, and stationary. This paper aims to scrape textures directly from natural images of everyday objects and scenes, build texture models, and employ them for texture synthesis, texture editing, etc. The key idea is to jointly learn image grouping and texture modeling. The image grouping module discovers clean texture segments, each of which is represented as a texture code and a parametric sine wave by the texture modeling module. By enforcing the model to reconstruct the input image from the texture codes and sine waves, our model can be learned via self-supervision on a set of cluttered natural images, without requiring any form of annotation or clean texture images. We show that the learned texture features capture many natural and man-made textures in real images, and can be applied to tasks like texture synthesis, texture editing and texture swapping.
Robustness through Cognitive Dissociation Mitigation in Contrastive Adversarial Training

In this paper, we introduce a novel neural network training framework that increases model’s adversarial robustness to adversarial attacks while maintaining high clean accuracy by combining contrastive learning (CL) with adversarial training (AT). We propose to improve model robustness to adversarial attacks by learning feature representations that are consistent under both data augmentations and adversarial perturbations. We leverage contrastive learning to improve adversarial robustness by considering an adversarial example els another positive example, and aim to maximize the similarity between random augmentations of data samples and their adversarial example, while constantly updating the classification head in order to avoid a cognitive dissociation between the classification head and the embedding space. This dissociation is caused by the fact that CL updates the network up to the embedding space, while freezing the classification head which is used to generate new positive adversarial examples. We validate our method, Contrastive Learning with Adversarial Features (CLAF), on the CIFAR-10 dataset on which it outperforms both robust accuracy and clean accuracy over alternative supervised and self-supervised adversarial learning methods.
A Stumble Detection Method for Programming with Multi-modal Information

Various stumbles occur when learning programming, which leads to lower learning efficiency and motivation. If such stumbles could be detected automatically, teachers could monitor learners’ progress and support passive students. Stumbles in learning programming can be classified into several types, which could be divided into those that are evident from the source code the learner wrote and those that are expressed in their psychological state. These stumbles could be detected by combining biometric data with code-related metrics. In this study, we propose a method to detect stumbles in learning programming by combining the learner’s heart rate information with code-related metrics. We compared the accuracy of models using only code-related metrics, using only heart rate information, and using a combination of both. The results showed that the code-related model and the multimodal model had the highest accuracy and the multimodal model can detect the most variety of stumbles.
Semi-Supervised Online Kernel Extreme Learning Machine for Multi-Label Data Stream Classification

Semi-supervised multi-label data stream classification serves a practical yet challenging task since only a small number of labeled instances are available in real streaming environments. However, the mainstream of existing semi-supervised multi-label classification technique is focused on the batch process. Meanwhile, many data stream classification approaches have been proposed, and one of popularly used base models is ELM (extreme learning machine). But only few ELM-based algorithms are proposed for the multi-label data stream classification. Therefore, in this paper, we present a novel Semi-supervised Online Extreme Learning Machine with Kernel function for multi-label data stream classification, called SSO-KELM. Specifically, we firstly introduce the kernel function to output the multi-dimensional vector, for adapting to the multi-label data. Secondly, to make full use of labeled and unlabeled data in a data stream, we derive a novel online semi-supervised ELM algorithm, which can adapt to the stream setting and achieve a higher classification accuracy. Finally, extensive experiments conducted on six benchmark multi-label data sets demonstrate the effectiveness of our approach compared to state-of-the-art approaches.
Students’ Attitude to Digital Entrepreneurship

Research relevance of students' attitudes towards digital entrepreneurship is due to a sharp reduction in the number of the most working age (20–29 years) in Russia. According to a survey by the audit and consulting company Fin Expertiza, the number of the most working age was decreased by 460.5 thousand people for the period from September 2020 to September 2021. By Sberbank public joint stock company research, approximately 3% of Russian Federation population, or 4.2 million people are engaged in entrepreneurial activities. Identification of students' attitudes towards entrepreneurial activity in the context of the latest statistics showing a negative trend will allow predicting possible negative consequences for business entities. Research purpose is to identify students' attitudes towards digital entrepreneurship. The research objectives were influence factors analysis of students' readiness potential for digital entrepreneurship and representation definition of an entrepreneur, which is formed in students' minds in the learning process. Survey results conducted among students of Peter the Great St. Petersburg Polytechnic University (September-October 2021, methods—online survey and focus group) made it possible to establish students' readiness potential, associated with the growing students' interest in digital entrepreneurship, the need to obtain relevant business competencies. The poll showed that almost half of students plans to do business, a quarter of those surveyed already had experience in running own online business. University actively promotes the skills development in working with information, stress resistance, diligence and purposefulness, the ability to be responsible for one's own actions in the opinion of students. However, university contribution to stimulating such qualities as the ability to persuade, readiness to take risks and foresight seems to be minimal.
Convolutive Attention for Image Registration

Elastic registration of deformed images is a vital component of many computer vision tasks, especially when considering medical image data. Deep learning techniques, par-ticularly U-Nets, offer state-of-the-art performance, but do not yet use the rich spatial information context available in natural images. We propose an augmentation based on the recently introduced attention mechanism to allow a U - Net to use spatial image context. A dedicated convolutive attention scheme has been developed by calculating local similarity scores of the multidimensional inputs. Additionally, a dedicated composite error function based on common image similarity measures is introduced in order to further improve the registration results. To evaluate our approach, we conducted several experiments on an augmented real-world dataset containing cardiac cine MRI scans. The comparison with state-of-the-art registration schemes highlights the potential of our approach.
A Transformer-based Medical Visual Question Answering Model

While the Transformer architecture has been widely used in natural language processing tasks and computer vision tasks, its application in medical visual question answering is still limited. Most current methods rely on an image extractor to obtain visual features and a text extractor to capture semantic features, and then a fusion module to merge the information from the two modalities to predict the final result. In contrast, this paper proposes a novel Transformer-based medical vision question answering model, called MQAT, in which an improved Transformer structure is used for feature extraction and modal fusion to achieve better performance. Experimental results demonstrate that our Transformer structure not only ensures the stability of the model performance, but also accelerates its convergence, and the MQAT model outperforms the existing state-of-the-art methods.
A reimagined catalogue of software security patterns

Since their introduction, security patterns had the promise to aid non-security experts to design secure software, yet in practice adoption and thus impact of security patterns remains limited. We believe one of the reasons is that existing security patterns are a mixture of security advice and advice on software design, such as encapsulation for maintainability. To address this, we propose a new security pattern catalogue in which we approach patterns from a security-centric perspective instead of a generic software engineering perspective. More specifically, we treat security as a first-class citizen while relying as much as possible on the vast body of knowledge from the security domain. Furthermore, our catalogue is structured to enable easy navigation for identifying relevant security problems and selecting appropriate solutions. In order to ensure a consistent level of abstraction and allow easier combination of multiple patterns, we describe our catalogue in a uniform description language and metamodel. An initial evaluation shows that our catalogue has good coverage of common security problems and solutions, indicating the catalogue's potential, but further evaluation is required to evaluate its impact in practice.
A Survey on Techniques and Methods of Recommender System

As prevalence is growing for social media, the value of its content is becoming paramounting. This data can reveal about a person’s personal and professional life. The behaviors done on social media either frequent or periodic can comprehend fondness and attentiveness of users on certain matters. But accompanying it, diverse data from multiple sources with high volumes sets its foot in. Here becomes operational the usage of recommender systems. These are capable of providing customized assistance to users based on respective quondam behavior and preferences. Machine Learning and deep learning methods have proven to be a boon in these tasks of predictions with notable accuracy. This paper discusses existing techniques with its fors and againsts, concerns and issues, extrapolating the results and solutions, which in turn can help in better interpretation of current developments to pave a path for pioneering researches.
Visualwind: a Novel Video Dataset for Cameras to Sense the Wind

The goal of this paper is to empower cameras to sense the wind from videos by capturing the motion information using optical flow and machine learning models, to potentially revolutionise the spatiotemporal resolution of existing professional wind records that are often at the city scale. To this end, we build a novel video dataset of over 6000 labeled video clips, covering eleven wind classes of the Beaufort scale. The videos are collected from social media, public cameras, and self-recording. Every video clip has a fixed 10 seconds length with varied frame rates, and contains scenes of various trees swaying in different scales of wind. We describe the key statistics of the dataset, how it was collected and annotated, and evaluate both one-stage and two-stage models trained and tested for wind scale estimation on this dataset to give some baseline performance figures. The dataset is publicly accessible <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://sme.uds.exeter.ac.uk/folders/48caf5102d6196b9645fab1f46e494ec. Please contact the authors to get the access key due to the server protection policy..
NAM: What Does a Neural Network See?

Deep Convolutional Neural Networks (CNNs) still lack interpretability and are often treated as miraculous blackbox machines. Therefore, when an intelligent system fails, it is usually difficult to troubleshoot the problems. Among others, these issues can be caused by incorrect decisions of the CNN classifier. The other reason can be selective “blindness” of the CNN - caused by an insufficient generalization of the convolutional feature extractor. To better understand the CNN decisions, methods from the Class Activation Mapping (CAM) family have been introduced. In contrast to CAM techniques, which focus on the model's predictions (thus a classifier), we propose a simple yet informative way to visualize network activation - Network Activation Mapping (NAM). Our method targets the most important part of the CNN - a convolutional feature extractor. Opposed to CAM methods, NAM is class-and classifier-independent and provides insight into what the neural network focuses on during the feature extraction process and what features it finds the most prominent in the examined image. Due to the classifier-independence, it can be used with all CNN models. In our experiments, we demonstrate how the performance of a convolutional feature extractor can be preliminarily evaluated using NAM. We also present results obtained for a simple NAM-based visual attention mechanism, which allows us to filter out less informative regions of the image and facilitates the decision making process.
GAT-CADNet: Graph Attention Network for Panoptic Symbol Spotting in CAD Drawings

Spotting graphical symbols from the computer-aided design (CAD) drawings is essential to many industrial applications. Different from raster images, CAD drawings are vector graphics consisting of geometric primitives such as segments, arcs, and circles. By treating each CAD drawing as a graph, we propose a novel graph attention network GAT-CADNet to solve the panoptic symbol spotting problem: vertex features derived from the GAT branch are mapped to semantic labels, while their attention scores are cascaded and mapped to instance prediction. Our key contributions are three-fold: 1) the instance symbol spotting task is formulated as a subgraph detection problem and solved by predicting the adjacency matrix; 2) a relative spatial encoding (RSE) module explicitly encodes the relative positional and geometric relation among vertices to enhance the vertex attention; 3) a cascaded edge encoding (CEE) module extracts vertex attentions from multiple stages of GAT and treats them as edge encoding to predict the adjacency matrix. The proposed GAT-CADNet is intuitive yet effective and manages to solve the panoptic symbol spotting problem in one consolidated network. Extensive experiments and ablation studies on the public benchmark show that our graph-based approach surpasses existing state-of-the-art methods by a large margin.