Reviewerly/exHarmony huggingface dataset:
IncepForest: A Novel Approach for Sign Language Recognition with Transfer and Ensemble Learning

Recent advances in the field of the human-computer interface have sparked a surge of interest in hand gesture detection due to its inherent flexibility and user-friendliness. Gesture recognition techniques have proven invaluable in creating systems that facilitate seamless information exchange among impaired individuals or device operations. As ubiquitous computing continues to progress, traditional user interaction methods like keyboards, mice, and pens are proving inadequate, thus necessitating more natural input methods such as direct hand input. In this study, the application of transfer learning methods for hand gesture detection in Indian Sign Language (ISL) and Bengali Sign Language (BSL) is investigated. To this end, several transfer learning techniques including InceptionResNetV2, VGG16, InceptionV3, and EfficientNet were employed on both datasets. Results demonstrate that InceptionResNetV2 outperformed the other transfer learning approaches, achieving an accuracy of 99.72 % for ISL and 91.00% for BSL. To further enhance the prediction accuracy of hand gesture recognition, several hybrid approaches were implemented by integrating InceptionResNetV2 with ensemble learning models like Bagging, Boosting, and Random Forest Algorithms. This study proposes a novel hybrid model, named IncepForest, that combines InceptionResNetV2 with Random Forest and yields the highest accuracy of 96.00% for BSL.
Multimodal Sarcasm Detection (MSD) in Videos using Deep Learning Models

Every day, individuals all over the globe use video-sharing websites to broadcast their thoughts, experiences, and recommendations to the world. There has been a huge rise in educational interest in the area of sarcasm detection for these opinionated videos. Although sarcasm has proven effective for written text, it remains an unexplored area of research when applied to video and other forms of multimodal data. Numerous verbal and nonverbal signs, such as a shift in voice, an overemphasis in a phrase, a stretched pronunciation, or a stiff-looking face, are often used to convey sarcasm. The majority of current research on sarcasm detection has heavily relied on textual content. Hence, in this paper, we suggest that the use of multiple modalities can help with more accurate sarcasm identification. The results of our studies on one of the multimodal sarcasm detection (MSD) benchmark datasets “MUSTARD” reveal that our strategy outperforms other algorithms in sarcasm classification.
Attention-free based dual-encoder mechanism for Aspect-based Multimodal Sentiment Recognition

Multimodal aspect-based sentiment recognition (MABSR) is a recently developed task in sentiment recognition that tries to assess the sentiment associated with text and image pairings by generally extracting the polarity terms from the pairs. Both the pipeline and the unified transformer based technique, which employs the cross-attention only mechanism, have been widely utilized in recent works. However, the alignment between text and picture is not openly and reliably included in these approaches. There is still a minimum threshold of aligned image-text pairings needed for supervised fine-tuning of said universal transformers for MABSR. Motivated by this observation and inspired by the various attention-only mechanisms, we analyze MABSR and propose an attention-free encoder-based transformer architecture. Dual attention-free based backbone encoder models with cross-modal symmetry are utilized in this work. To improve cross-modal performance, we include two new subtasks: aspect-only extraction and polarity feature representation alignment. This motivates both encoders to provide more precise depictions of multiple modalities.
Automated Tweet Sentiment Analysis Using Machine Learning models

The work adds towards sentiment analysis by introducing an understanding lexicon and BERT based NLP model XLNet. Based on fundamental research and systematic practice in Design Phase, the lexicon describes the benefits and drawbacks of products as a result of the interaction between artifacts and users. The study collects data from Twitter API that represent customer interactions about the items in various domains as per filters given by the consumers. It compares the findings of a typical sentiment analysis with the results filtered using the lexicon and delivers visual statistics for the same. The initial product and brand comparison results show the value of sentiment analysis and text analysis on publicly available data, and the utilization of deep learning classification models for forecasting sentiment polarity gives consumers, product manufacturers and distributors, regulatory and monitoring agencies a great tool for tracking brand or product perception trends and taking action in the event of a sudden or significant upsurge in negative sentiment.
Handwritten signature forgery detection using Deep Neural Network

A common biometric for establishing a person's identity in document forensics is verification of handwritten signatures. Signature plays an important role in banking, financial, commercial and so on. However, issues associated with signatures are numerous since any two signatures may look very similar with little to no differences written by the same person. Despite the enormous research efforts, offline signature verification is still difficult, especially when attempting to differentiate between competent forgeries and authentic signatures because the visual differences between the two can sometimes be less noticeable than between two authentic ones. Due to the expanding usage of digital signatures and electronic documents, there has been an increase in the demand for effective and precise signature forgery detection systems (SFDS). In order to effectively combat handwritten signature forgery, this study intends to create an automated system utilizing a deep neural network (DNN) especially a simple and efficient Convolutional Neural Network (CNN) that can determine if a given signature is real or faked. The model will take signature image as input and extract important features through different layers of CNN that will helpful in distinguishing genuine and forged signatures. The model performance is also accessed with different image resizing techniques and optimizers used in training of CNN. The model's performance in terms of a set of assessment metrics, such as accuracy, precision, recall, and Fl-score are provided in this paper after extensive tests are conducted across a huge dataset of handwritten signatures including both genuine and various types of forged signatures. Additionally, the model's performance is also contrasted with a cutting-edge deep learning model. The simulation results conclusively demonstrate the appropriateness of the suggested methodology for automatically detecting forgery in handwritten signature images.
Short Review on Contrastive Learning-based Segmentation Techniques for Medical Image Processing

Due to more advancements in deep learning approaches, medical image analysis has become more popular in research. Image segmentation plays an indispensable role in image processing. Digital images are classified as segments, and segmentation approaches are used to analyze the important features and data presented in the input digital images. Segmentation is mainly performed to recover the essential features easily from the region of interest. The segmentation process generates a meaningful digital image, and they are easy to analyze. Recently, segmentation approaches have become more popular in a medical environment, and also they secured more numbers of successful applications in neutrosopy. Therefore, it is decided to make the comparative analysis of the medical image segmentation techniques based on the deep learning concept. This survey encloses various existing contrastive learning-based segmentation techniques for performing algorithmic classification in the medical domain. These surveys also compare different performance measures, datasets utilized, and tools used for the implementation. Then, upcoming research and current research gaps in medical image segmentation are analyzed. This review on state-of-the-art medical image segmentation tools has shown their potential in clinical practices for effectively diagnosing diseases with better segmentation approaches using contrastive learning.
A Comprehensive Review of Metaheuristic Algorithms Inspired by Quantum Mechanics

This article comprehensively reviews the recently developed quantum mechanics-based optimization techniques. The quantum theory has been employed to speed up the evolutionary method and enhance the possibility of finding the optimal solution of conventional optimization techniques. Current developments in quantum computing have demonstrated that quantum theory can offer significant benefits over conventional theory for certain optimization techniques. The future prospective and benefits and drawbacks of this fresh category of quantum optimization techniques have been presented in this review. It also enables new researchers and algorithm developers to use these simple but extremely effective algorithms for problem-solving. This paper aims to give readers an overview of the fundamental elements and recent advances in optimization techniques so that they can develop and implement these for various applications. Finally, a few findings are reached, and future study on quantum-based optimization techniques is discussed.
SHATTER: Control and Defense-Aware Attack Analytics for Activity-Driven Smart Home Systems

Modern smart home control systems utilize realtime occupancy and activity monitoring to ensure control efficiency, occupants' comfort, and optimal energy consumption. Moreover, adopting machine learning-based anomaly detection models (ADMs) enhances security and reliability. However, sufficient system knowledge allows adversaries/attackers to alter sensor measurements through stealthy false data injection (FDI) attacks. Although ADMs limit attack scopes, the availability of information like occupants' location, conducted activities, and alteration capability of smart appliances increase the attack surface. Therefore, performing an attack space analysis of modern home control systems is crucial to design robust defense solutions. However, state-of-the-art analyzers do not consider contemporary control and defense solutions and generate trivial attack vectors. To address this, we propose a control and defense-aware novel attack analysis framework for a modern smart home control system, efficiently extracting ADM rules. We verify and validate our framework using a state-of-the-art dataset and a prototype testbed.
Adaptive Webpage Fingerprinting from TLS Traces

In webpage fingerprinting, an on-path adversary infers the specific webpage loaded by a victim user by analysing the patterns in the encrypted TLS traffic exchanged between the user's browser and the website's servers. This work studies modern webpage fingerprinting adversaries against the TLS protocol; aiming to shed light on their capabilities and inform potential defences. Despite the importance of this research area (the majority of global Internet users rely on standard web browsing with TLS) and the potential real-life impact, most past works have focused on attacks specific to anonymity networks (e.g., Tor). We introduce a TLS-specific model that: 1) scales to an unprecedented number of target webpages, 2) can accurately classify thousands of classes it never encountered during training, and 3) has low operational costs even in scenarios of frequent page updates. Based on these findings, we then discuss TLS-specific countermeasures and evaluate the effectiveness of the existing padding capabilities provided by TLS 1.3.
Get Your Cyber-Physical Tests Done! Data-Driven Vulnerability Assessment of Robotic Aerial Vehicles

The rapid growth of robotic aerial vehicles (RAVs) has attracted extensive interest in numerous public and civilian applications, from flying drones to quadrotors. Security of RAV systems is posting greater challenges as RAV controller software becomes more complex and exposes a growing attack surface. Memory isolation techniques, which virtually separate the memory space and conduct hardware-based memory access control, are believed to prevent the attacker from compromising the entire system by exploiting one memory vulnerability. In this paper, we propose Ares, a new variable-level vulnerability assessment framework to explore deeper bugs from a combined cyber-physical perspective. We present a data-driven method to illustrate that, despite state-of-the-art memory isolation efforts, RAV systems are still vulnerable to physics-aware data manipulation attacks. We augment RAV control states with intermediate state variables by tracing accessible control parameters and vehicle dynamics within the same isolated memory region. With this expanded state variable space, we apply multivariate statistical analysis to investigate inter-variable quantitative data dependencies and search for vulnerable state variables. Ares utilizes a reinforcement learning-based method to show how an attacker can exploit memory bugs and parameter defects in a legitimate memory view and elaborately craft adversarial variable values to disrupt a RAV's safe operations. We demonstrate the feasibility and capability of Ares on the widely-used ArduPilot RAV framework. Our extensive empirical evaluation shows that the attacker can leverage these vulnerable state variables to achieve various RAV failures during real-time operation, and even evade existing defense solutions.
On Adversarial Robustness of Point Cloud Semantic Segmentation

Recent research efforts on 3D point cloud semantic segmentation (PCSS) have achieved outstanding performance by adopting neural networks. However, the robustness of these complex models have not been systematically analyzed. Given that PCSS has been applied in many safety-critical applications like autonomous driving, it is important to fill this knowledge gap, especially, how these models are affected under adversarial samples. As such, we present a comparative study of PCSS robustness. First, we formally define the attacker's objective under performance degradation and object hiding. Then, we develop new attack by whether to bound the norm. We evaluate different attack options on two datasets and three PCSS models. We found all the models are vulnerable and attacking point color is more effective. With this study, we call the attention of the research community to develop new approaches to harden PCSS models.
Fabricated Flips: Poisoning Federated Learning without Data

Attacks on Federated Learning (FL) can severely reduce the quality of the generated models and limit the usefulness of this emerging learning paradigm that enables on-premise decentralized learning. However, existing untargeted attacks are not practical for many scenarios as they assume that i) the attacker knows every update of benign clients, or ii) the attacker has a large dataset to locally train updates imitating benign parties. In this paper, we propose a data-free untargeted attack (DFA) that synthesizes malicious data to craft adversarial models without eavesdropping on the transmission of benign clients at all or requiring a large quantity of task-specific training data. We design two variants of DFA, namely DFA-R and DFA-G, which differ in how they trade off stealthiness and effectiveness. Specifically, DFA-R iteratively optimizes a malicious data layer to minimize the prediction confidence of all outputs of the global model, whereas DFA-G interactively trains a malicious data generator network by steering the output of the global model toward a particular class. Experimental results on Fashion-MNIST, Cifar-10, and SVHN show that DFA, despite requiring fewer assumptions than existing attacks, achieves similar or even higher attack success rate than state-of-the-art untargeted attacks against various state-of-the-art defense mechanisms. Concretely, they can evade all considered defense mechanisms in at least 50% of the cases for CIFAR-10 and often reduce the accuracy by more than a factor of 2. Consequently, we design REFD, a defense specifically crafted to protect against data-free attacks. REFD leverages a reference dataset to detect updates that are biased or have a low confidence. It greatly improves upon existing defenses by filtering out the malicious updates and achieves high global model accuracy.
Fortifying Federated Learning against Membership Inference Attacks via Client-level Input Perturbation

Membership inference (MI) attacks are more diverse in a Federated Learning (FL) setting, because an adversary may be either an FL client, a server, or an external attacker. Existing defenses against MI attacks rely on perturbations to either the model's output predictions or the training process. However, output perturbations are ineffective in an FL setting, because a malicious server can access the model without output perturbation while training perturbations struggle to achieve a good utility. This paper proposes a novel defense, called CIP, to fortify FL against MI attacks via a client-level input perturbation during training and inference procedures. The key insight is to shift each client's local data distribution via a personalized perturbation to get a shifted model. CIP achieves a good balance between privacy and utility. Our evaluation shows that CIP causes accuracy to drop at most 0.7% while reducing attacks to random guessing.
A New Feature Fusion Method Based on Pre-Training Model for Sequence Labeling

To fuse vocabulary features into the pre-training model is the mainstream data feature processing method for sequence labelling tasks. In general, the feature fusion methods that have been proposed at present are direct fusion outside the pre-training model or fusion of lexical features using attention mechanism. However, the study found that this way of vocabulary enhancement does not conform to the word formation rules of modern Chinese. In the Chinese language, it is easy to fuse irrelevant or even incorrect lexical features into the sequence using the above feature processing methods, which is bad for the experimental results of the Chinese sequence labelling task. To solve these problems, we propose to use Cosine Similarity Adapter to process lexical features in Chinese sequence labelling tasks. CSBERT is a hybrid model using this structure based on BERT, which conforms to the word formation rules of modern Chinese to a certain extent. It can fuse the features of the word into the character or eliminate the features of the word in the character according to the cosine similarity between the character vector and a word vector. The experimental results show that CSBERT has better ability to label Chinese sequences than the benchmark model. CSBERT has achieved the best experimental results such as F1-Score on 7 open datasets and the best ability of multi-label classification, which proves that the model has good practical value.
Graph Embedding Method for Node Classification Guided by Structure and Attributes

The node category in the graph usually has a strong correlation with the node attribute and the domain. The embedding vector learned by the random walk method only considers the topology of the node, ignoring the category attribute of the node, which reduces the accuracy of the node classification task. This paper proposes a network representation learning method that integrates node attributes and structural features. In the process of node sequence sampling, the method selects one node as a follow-up from the neighbor nodes whose similarity with the current node is higher than the threshold. And finally submits a sequence of a certain length to the model for training, and gets the embedding vector for each node. The experimental results of node classification on public datasets show that this method has achieved good results on training sets of different sizes. The macro-F1 value of node classification is 5.1% and 3.6% higher than the optimal baseline method, respectively.
Entity Relation Interactive Graph Convolutional Network for Knowledge Embedding

There are several research grown up in recent years, which focused on GCN methods for complex relational graphs. The knowledge graph with rich semantic information is a typical complex relational graph and has become a major research in the field. Many existing GCN methods for complex relational graphs try to combine knowledge embedding methods to capture the rich semantic, but they are limited to simply propagating and updating the embeddings of entities and relations, ignoring the interactions between entity and relation information. Therefore, this paper proposes a novel framework for knowledge embedding, named as entity relation interactive graph convolutional network (ER-GCN), which combines graph convolution operations and knowledge embedding methods. Specifically, ER-GCN incorporates the multi-hop neighbor nodes by constructing auxiliary edges for entity embedding, and introduces the relation information semantically similar to the given relation for relation embedding. Experimental results on three datasets show the well performance of ER-GCN on link prediction and node classification.
Evaluating the Public Perception of the National Academy of Science and Technology, Philippines (NAST PHL) through Analytics of its Social Media Accounts

The increasing popularity of social media in all sectors of society has prompted more companies, including government agencies, to create and manage social media accounts to engage their clients online. For most government agencies, the use of Facebook deals not only with information dissemination but also, with managing reputation, communicating crises, and acting as a communication channel. This paper aims to: a) determine the sentiment of the stakeholders based on their feedback and comments on the NAST PHL Facebook pages from 01 April 2020 to 31 October 2021; b) develop and determine the performance of the different models for sentiment analysis; and c) determine the best model in terms of accuracy, precision, recall, and F1 Score. The researchers used RapidMiner to build a model while the classification process was initiated using Naive Bayes, K-nearest neighbor, and Decision Tree. The performance of each classifier was measured using accuracy, precision, recall, and F1 score. Results showed that the public has a positive sentiment (66 percent) on the various virtual events under the NAST core functions. On the other hand, DT gave the highest values for three out of the four performance measures: precision, recall, and F1 score. However, the values from DT and KNN for accuracy and F1 score were remarkably close, i.e., difference of 0.99 and 0.08, respectively. Based on the results, DT is the best among the three classifiers to use when evaluating the sentiment using Facebook data.
Hydropower Signal Detection Via Knowledge Graph Enhaced by Frequent Term Sets

Anomaly detection of hydropower system signals is an important and challenging task, and the difficulty lies in the complexity of the relationship between the signals. There are few existing works on anomaly detection of hydropower signals, and the existing works only build detection methods based on simple historical states or associations. In this paper, we believe that it is better to consider the correlation between the signals in the monitoring process than the associations. Therefore, a knowledge graph for hydroelectric signals is constructed, based on which anomaly monitoring of signals is realized. Among them, three relationships between signals are proposed: causal relationship, accompanying relationship and symbiotic relationship. Finally, we verified the effectiveness of the detection method by conducting experiments on the briefing historical data of waterfall ditch hydropower Unit.