Reviewerly/exHarmony huggingface dataset:
Model Attribution of Face-Swap Deepfake Videos

AI-created face-swap videos, commonly known as Deepfakes, have attracted wide attention as powerful impersonation attacks. Existing research on Deepfakes mostly focuses on binary detection to distinguish between real and fake videos. However, it is also important to determine the specific generation model for a fake video, which can help attribute it to the source for forensic investigation. In this paper, we fill this gap by studying the model attribution problem of Deepfake videos. We first introduce a new dataset with DeepFakes from Different Models (DFDM) based on several Autoencoder models. Specifically, five generation models with variations in encoder, decoder, intermediate layer, input resolution, and compression ratio have been used to generate a total of 6, 450 Deepfake videos based on the same input. Then we take Deepfakes model attribution as a multiclass classification task and propose a spatial and temporal attention based method to explore the differences among Deep-fakes in the new dataset. Experimental evaluation shows that most existing Deepfakes detection methods failed in Deep-fakes model attribution, while the proposed method achieved over 70% accuracy on the high-quality DFDM dataset <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Self-Supervised Domain Adaptation in Crowd Counting

Self-training crowd counting has not been attentively explored though it is one of the important challenges in computer vision. In practice, the fully supervised methods usually require an intensive resource of manual annotation. In order to address this challenge, this work introduces a new approach to utilize existing datasets with ground truth to produce more robust predictions on unlabeled datasets, named domain adaptation, in crowd counting. While the network is trained with labeled data, samples without labels from the target domain are also added to the training process. In this process, the entropy map is computed and minimized in addition to the adversarial training process designed in parallel. Experiments on Shanghaitech, UCF_CC_50, and UCF-QNRF datasets prove a more generalized improvement of our method over the other state-of-the-arts in the cross-domain setting.
Towards Generalizable DEEPFAKE Face Forgery Detection with Semi-Supervised Learning and Knowledge Distillation

Existing methods for deepfake face forgery detection have already achieved tremendous progress in well-controlled laboratory conditions. However, under wild scenarios where the training and testing forgeries are synthesized by different algorithms and when labeled data are insufficient, the performance always drops greatly. In this work, we present a Semi-supervised Contrastive Learning and Knowledge Distillation-based framework (SCL-KD) for deepfake detection to reduce the aforementioned performance gap. Our proposed framework contains three stages: self-supervised pre-training, supervised training, and knowledge distillation. Specifically, a feature encoder is firstly trained in a self-supervised manner with a large number of unlabeled samples through a momentum contrastive mechanism. Secondly, a fully-connected classifier on top of the feature encoder is trained in a supervised manner with a small amount of labeled samples to build a teacher model. Finally, a compact student model is trained with the help of the teacher model using knowledge distillation, in order to avoid overfitting to labeled data and have better generalizability on mismatched datasets. Evaluations on several benchmark datasets corroborate the good performance of our approach in cross-dataset situations and few labeled data scenarios. It reveals the potential of our proposed method for real-world deepfake detection.
I Saw: A Self-Attention Weighted Method for Explanation of Visual Transformers

Recently, visual transformers have shown promising results in tasks such as image classification, segmentation, object detection, etc. The explanation of their decision remains a challenge. This paper focuses on exploiting self-attention for an explanation. We propose a generalized interpretation of the transformers i.e model agnostic but class-specific explanations. The main principle is in the use and weighting self-attention maps of a visual transformer. To evaluate it, we use the popular hypothesis that an explanation is good if it correlates with human perception of a visual scene. Thus, the method has been evaluated against the Gaze Fixation Density Maps obtained in a psycho-visual experiment on a public database. It has been compared with other popular explainers such as Grad-Cam, LRP, Rollout, and Adaptive Relevance methods. The proposed method outperforms the best baseline by 2% in a standard Pearson Correlation Coefficient (PCC) metric.
Prohibited Object Detection in X-ray Images with Dynamic Deformable Convolution and Adaptive IoU

Due to the variety and complexity of objects in X-ray images, how to detect the prohibited items automatically and accurately is a challenging problem. In this paper, an X-ray image prohibited object detection method based on Dynamic Deformable Convolution (DyDC) and adaptive Intersection over Union (IoU) is proposed based on Cascade R-CNN framework. The main contributions are as follows. First, DyDC is proposed to cope with the diversity of the prohibited objects in X-ray images and to improve the feature representation capability. Then, adaptive IoU mechanism is proposed, which can dynamically adjust the IoU threshold during the training process to generate high quality proposals. The proposed method is extensively evaluated on two publicly available benchmark datasets, namely SIXray and OPIXray, and the experimental results show that it can achieve the state-of-the-art detection accuracy, compared with other existing methods.
ATCA: An ARC Trajectory Based Model with Curvature Attention for Video Frame Interpolation

Video frame interpolation is a classic and challenging low-level computer vision task. Recently, deep learning based methods have achieved impressive results, and it has been proven that optical flow based methods can synthesize frames with higher quality. However, most flow-based methods assume a line trajectory with a constant velocity between two input frames. Only a little work enforces predictions with curvilinear trajectory, but this requires more than two frames as input to estimate the acceleration, which takes more time and memory to execute. To address this problem, we propose an arc trajectory based model (ATCA), which learns motion prior from only two consecutive frames and also is lightweight. Experiments show that our approach performs better than many SOTA methods with fewer parameters and faster inference speed.
Multi Label Image Classification using Adaptive Graph Convolutional Networks (ML-AGCN)

In this paper, a novel graph-based approach for multi-label image classification called Multi-Label Adaptive Graph Convolutional Network (ML-AGCN) is introduced. Graph-based methods have shown great potential in the field of multi-label classification. However, these approaches heuristically fix the graph topology for modeling label dependencies, which might be not optimal. To handle that, we propose to learn the topology in an end-to-end manner. Specifically, we incorporate an attention-based mechanism for estimating the pairwise importance between graph nodes and a similarity-based mechanism for conserving the feature similarity between different nodes. This offers a more flexible way for adaptively modeling the graph. Experimental results are reported on two well-known datasets, namely, MS-COCO and VG-500. Results show that ML-AGCN outperforms state-of-the-art methods while reducing the number of model parameters.
Localization and Classification of Parasitic Eggs in Microscpic Images Using An Efficientdet Detector

IPIs caused by protozoan and helminth parasites are among the most common infections in humans in LMICs. They are regarded as a severe public health concern, as they cause a wide array of potentially detrimental health conditions. Researchers have been developing pattern recognition techniques for the automatic identification of parasite eggs in microscopic images. Existing solutions still need improvements to reduce diagnostic errors and generate fast, efficient, and accurate results. Our paper addresses this and proposes a multi-modal learning detector to localize parasitic eggs and categorize them into 11 categories. The experiments were conducted on the novel Chula-ParasiteEgg-11 dataset that was used to train both EfficientDet model with EfficientNet-v2 backbone and EfficientNet-B7+SVM. The dataset has 11,000 microscopic training images from 11 categories. Our results show robust performance with an accuracy of 92%, and an F1 score of 93%. Additionally, the IOU distribution illustrates the high localization capability of the detector.
DPNET: Dual-Path Network for Efficient Object Detection with Lightweight Self-Attention

Object detection often costs a considerable amount of computation to get satisfied performance, which is unfriendly to be deployed in edge devices. To address the trade-off be-tween computational cost and detection accuracy, this paper presents a dual path network, named DPNet, for efficient object detection with lightweight self-attention. In backbone, a single input/output lightweight self-attention module (LSAM) is designed to encode global interactions between different positions. LSAM is also extended into a multiple-inputs version in feature pyramid network (FPN), which is employed to capture cross-resolution dependencies in two paths. Extensive experiments on the COCO dataset demonstrate that our method achieves promising detection results. More specifically, DPNet obtains 29.0% AP on COCO test-dev, with only 1.14 GFLOPs and 2.27M model size for a 320 × 320 image.
Thermal to Visible Image Synthesis Under Atmospheric Turbulence

In many practical applications of long-range imaging such as biometrics and surveillance, thermal imagining modalities are often used to capture images in low-light and nighttime conditions. However, such imaging systems often suffer from atmospheric turbulence, which introduces severe blur and deformation artifacts to the captured images. Such an issue is unavoidable in long-range imaging and significantly decreases the face verification accuracy. In this paper, we first investigate the problem with a turbulence simulation method on real-world thermal images. An end-to-end reconstruction method is then proposed which can directly transform thermal images into visible-spectrum images by utilizing natural image priors based on a pre-trained StyleGAN2 network. Compared with the existing two-steps methods of consecutive turbulence mitigation and thermal to visible image translation, our method is demonstrated to be effective in terms of both the visual quality of the reconstructed results and face verification accuracy. Moreover, to the best of our knowledge, this is the first work that studies the problem of thermal to visible image translation under atmospheric turbulence.
Analyzing tourism reviews using an LDA topic-based sentiment analysis approach

It has become increasingly necessary to automate systems for organizing and classifying user reviews according to their domain-specific aspects and sentiment polarities, as online customer opinions have increased on specialized platforms and social networks. This study's methodology employs a combination of topic modeling and sentiment analysis, as well as human validation techniques of topic labels, to extract valuable insights about Marrakech city from TripAdvisor reviews. Through this technique, tourism practitioners and field specialists may dive deeper into online users generated content, leveraging aspect-based sentiment analysis to explore each destination's weaknesses and strengths.•Data collection and pre-processing.•Extracting latent topics using LDA algorithm (Latent Dirichlet Allocation) on collected reviews.•Applying sentiment analysis to each topic.
EEPPDA—Edge‐enabled efficient privacy‐preserving data aggregation in smart healthcare Internet of Things network

Abstract The Internet of Things‐based smart healthcare provides numerous facilities to patients and medical professionals. Medical professionals can monitor the patient's real‐time medical data and diagnose diseases through the medical health history stored in the cloud database. Any kind of attack on the cloud database will result in misdiagnosis of the patients by medical professionals. Therefore, it becomes a primary concern to secure private data. On the other hand, the conventional data aggregation method for smart healthcare acquires immense communication and computational cost. Edge‐enabled smart healthcare can overcome these limitations. The paper proposes an edge‐enabled efficient privacy‐preserving data aggregation (EEPPDA) scheme to secure health data. In the EEPPDA scheme, captured medical data have been encrypted by the Paillier homomorphic cryptosystem. Homomorphic encryption is engaged in the assurance of secure communication. For data transmission from patients to the cloud server (CS), data aggregation is performed on the edge server (ES). Then aggregated ciphertext data are transmitted to the CS. The CS validates the data integrity and analyzes and processes the authenticated aggregated data. The authorized medical professional executes the decryption, then the aggregated ciphertext data are decrypted in plaintext. EEPPDA utilizes the batch verification process to reduce communication costs. Our proposed scheme maintains the privacy of the patient's identity and medical data, resists any internal and external attacks, and verifies the health data integrity in the CS. The proposed scheme has significantly minimized computational complexity and communication overhead concerning the existing approach through extensive simulation.
Online machine learning for auto-scaling in the edge computing

The evolution of edge computing devices has enabled machine intelligence techniques to process data close to its producers (the sensors) and end-users. Although edge devices are usually resource-constrained, the distribution of processing services among several nodes enables a processing capacity similar to cloud environments. However, the edge computing environment is highly dynamic, impacting the availability of nodes in the distributed system. In addition, the processing workload for each node can change constantly. Thus, the scaling of processing services needs to be rapidly adjusted, avoiding bottlenecks or wasted resources while meeting the applications' QoS requirements. This paper presents an auto-scaling subsystem for container-based processing services using online machine learning. The auto-scaling follows the MAPE-K control loop to dynamically adjust the number of containers in response to workload changes. We designed the approach for scenarios where the number of processing requests is unknown beforehand. We developed a hybrid auto-scaling mechanism that behaves reactively while a prediction online machine learning model is continuously trained. When the prediction model reaches a desirable performance, the auto-scaling acts proactively, using predictions to anticipate scaling actions. An experimental evaluation has demonstrated the feasibility of the architecture. Our solution achieved fewer service level agreement (SLA) violations and scaling operations to meet demand than purely reactive and no scaling approaches using an actual application workload. Also, our solution wasted fewer resources compared to the other techniques.
Quaternion with quantum walks for designing a novel color image cryptosystem

Data confidentiality plays a paramount part in our everyday lives. While most modern cryptosystems perform skillfully on digital resources, the rapid development of quantum resources may provide sufficient capability in the near future to hack most current cryptosystems because their construction is mathematical model-based. Although quantum algorithms will not be effective until quantum computers are physically realized and have sufficient capability to outperform digital computation, new cryptosystems are required to be designed using quantum technologies and have the capability to be implemented in both digital and quantum resources besides resisting the probable attacks from those resources. Quantum walk is a computational paradigm that can be implemented as a quantum-inspired paradigm and serves as a chaotic dynamical model employed for developing modern cryptosystems. This work presented a new color image cryptosystem using quaternion, quantum walks, and logistic-sine map. The aim of utilizing quaternion is to increase the key space of the presented cryptosystem, in addition to integrating the sequence originating from the logistic-sine map with the probability distribution originating from quantum walks for generating inseparable chaotic sequences utilized in the permutation and substitution phases of the presented image cryptosystem. Experimental outcomes were performed on 21 color images and the average values of UACI is 33.45969%, NPCR is 99.61070%, correlation is -0.00003, Chi-square is 255.904, global entropy is 7.999837, local entropy is 7.902542, and the encryption speed is 0.725868 Megabytes/second, in which the outcomes demonstrated that the offered encryption technique is both efficient and secure.
4D chaotic system-based secure data hiding method to improve robustness and embedding capacity of videos

Chaotic systems are used in various applications due to their high security and complexity. Because minor changes in parameters have a significant effect on the output values and the dynamic behavior of the system. Besides, security and robustness are the most important requirements of data hiding. Therefore, in this paper, it is proposed to use chaos-based data hiding for videos. The merits of the proposed method are selection of embedding location with robust system, increasing the secret data capacity, and keep the visual quality, which are in trade-off for existing methods. Because, in the existing methods the secret message is embedded into pixels of a consecutive, a selected or a random frame. Another deficiency of existing methods is embedding capacity. They embed fewer secret data to keep the visual quality of the stego object. However, compared to the drawback of existing literature, in the proposed method the secret message is embedded in entire video with 4D chaotic system, which is also an important novelty of this paper. The proposed method evaluated by objective criteria and achieved valuable results. The histogram analysis proves our method is undetectable. For different high ratios of embedding capacity, the visual quality of the stego videos is high where the average PSNR is about 55 dB, and MS-SSIM is closed to 1. The proposed method's results compared chaotic and non-chaotic video steganography methods. Our methods achieve 54.92 dB for embedding ratio 8.08% where the results are 53.57 dB and 5% for the best existing non-chaotic method and 49.99 dB and 0.001% for the best chaotic method, for similar capacity the proposed method gives superior visual performance over the chaotic literature. Finally, the robustness analysis with pixel value attack and lsb attack shows the proposed method appears to be resistant to these types of attacks.
Accelerating Federated Learning on Non-IID Data Against Stragglers

In large-scale federated learning systems, it is common to observe straggler effect from those clients with slow speed to delay the overall learning. However, in the standard federated learning frameworks (e.g., FedAvg) on non-iid data distribution among heterogeneous clients, we need to wait for all the clients' updates in each iteration as stopping each iteration in the meantime will change the original data distribution for the learning result. To make up the missing updates from straggler clients and accelerate the federated learning on non-iid data, we are the first to propose a novel method called stopping-and-weight-balancing (SAWB) to stop strategically each iteration and balance the update weights across the non-straggler clients. We prove that this method can keep the original data distribution, and controls weight divergence well with limited accuracy loss from FedAvg and the centralized learning. Finally, using different datasets (MNIST and CIFAR-10) for federated learning experiments, we show that our method can greatly save training time for a large-scale system while preserving the accuracy of the learning result.
Understanding Sarcasm from Reddit texts using Supervised Algorithms

The use of satirical or ironic language for conveying a message is referred to as Sarcasm. Social networks such as Reddit, Twitter, etc. usually uses Sarcasm. Reddit which is an American website contains social aggregations of news, ratings of the content and discussions. These resources, which include links, text articles, photographs, and videos, are published to the platform by registered users and can be voted up or down. Posts that cover topics related to books, cooking, pets, news, politics, movies, religions, science, sports, fitness, video games, music, and image-sharing are organized as “communities” or “subreddits”. The submissions receiving enough outvotes appear on the front page of the site and towards the top of the subreddit. This paper is about classifying a Reddit comment as sarcastic or non-sarcastic with the help of machine learning techniques. The dataset used in this study named as ‘Sarcasm on Reddit’ for classi-fication according to genre-based and have followed some basic steps using supervised machine learning and deep learning algorithms for the classification of texts including pre-processing, feature extraction and modeling. In this approach, we have achieved 71%, 76% and 70% accuracy for LSTM, CNN, and Logistic Regression algorithms respectively.
Transliterated Bengali Comment Classification from Social Media

In the era of technological advancement, the internet acts as an essential part of our daily life. People express their opinions on social media through different types of comments. In this paper, machine learning (ML) and deep learning (DL) models have been used to classify transliterated Bengali comments. Due to the lack of a large publicly available transliterated Bengali corpus, we have created our own dataset, consisting of 1,300 transliterated Bengali comments, which is publicly available in Mendeley Data. Moreover, we have applied several ML and DL algorithms, e.g., multinomial naive bayes (MNB), logistic regression (LR), linear SVM, decision Tree (DT), AdaBoost, random forest (RF), RBF SVM, gradient boosting, recurrent neural network (RNN), gated recurrent units (GRU), and long short-term memory (LSTM) for classifying comments. We have implemented different feature extraction techniques to compare the results. Among all these algorithms, logistic regression with countVectorizer performed best with 85.76% accuracy and 85.70% F1 score.