pwc-archive/papers-with-abstracts huggingface dataset:
A Malware Obfuscation AI Technique to Evade Antivirus Detection in Counter Forensic Domain
Authors: Ahmed A. Mawgoud, Hussein M. Rady, Benbella S. Tawfik
Date: 2020-09-24 00:00:00
Proceeding: Enabling AI Applications in Data Science. Studies in Computational Intelligence, vol 911. Springer, Cham. 2020 9

Data obfuscation is the process of converting a constant or a variable into computational results of several constants to make it hard for the files to be detected or analyzed by anti-malware engines. In recent years, malicious based attacks are considered as one of the highest internet threats; as the majority of internet users depend on antivirus software as a protection tool, attackers use obfuscation techniques to achieve high evasion rates against different antiviruses. In this paper, we introduce multiple techniques consists of four stages that aid a malware; to avoid anti-malware tools, these techniques were mainly developed to provide a high evasion rate against anti-malware systems via dynamic analysis techniques. The evasion rate success of our samples were tested through (Kaspersky, Virustotal and Virusscan), then the result of our experiment were compared with other obfuscation techniques to stand on the success level of the experiment as well as extracting the strength and weakness points for any possible future works.
Uncertainty Quantification to Enhance Probabilistic Fusion Based User Identification Using Smartphones
Authors: Rouhollah Ahmadian, Mehdi Ghatee, Johan Wahlström, Hadi Zare
Date: 2024-07-15 00:00:00
Proceeding: journal 2024 7
Tasks: Uncertainty Quantification, User Identification

User identification through smartphones and wearable sensors holds promise but faces challenges from variability in user activities and sampling windows. This paper presents a method that takes into account uncertainties to enhance the performance of user identification. The conventional sliding window technique is commonly employed for segmenting data, but using small windows can decrease classification performance by creating similar instances with different labels. Conversely, larger windows introduce concept drift resulting from mixed activity patterns. To tackle these challenges, the proposed method assesses the prediction uncertainty of a CNN classifier trained on small window sizes using the Monte Carlo Dropout technique and combines the predictions within a decision window. Uncertainty scores assist in discounting uncertain predictions during the final decision-making. Through experiments on five real-world datasets, the study demonstrates improved performance in identifying users across a range of activities compared to existing methods. It was also directly compared to state-of-the-art methods using two well-known datasets, improving accuracy by 1.29% in one case and 7.98% in the other. These findings validate the effectiveness of the new approach for continuous user identification, even when faced with unpredictable user behavior.
Driver Identification by an Ensemble of CNNs Obtained from Majority-Voting Model Selection
Authors: Rouhollah Ahmadian, Mehdi Ghatee, Johan Wahlström
Date: 2023-10-05 00:00:00
Proceeding: Conference 2023 10
Tasks: Driver Identification, Model Selection

Driver identification refers to the task of identifying the driver behind the wheel among a set of drivers. It is applicable in intelligent insurance, public transportation control systems, and the rental car business. An critical issue of these systems is the level of privacy, which encourages a lot of research using non-visual data. This paper proposes a novel method based on IMU sensors’ data of smartphones. Also, an ensemble of convolutional neural networks (CNNs) is applied to classify drivers. Furthermore, the final prediction is obtained by a majority vote mechanism. This paper demonstrates that model selection using a majority vote significantly improves the accuracy of the model. Finally, the performance of this research in terms of the accuracy, precision, recall, and f1-measure are 93.22%, 95.61%, 93.22%, and 92.80% respectively when the input length is 5 min.
Calibrated SVM for Probabilistic Classification of In-Vehicle Voices into Vehicle Commands via Voice-to-Text LLM Transformation
Authors: Mobina Moeini, Rouhollah Ahmadian, Mehdi Ghatee
Date: 2024-06-28 00:00:00
Proceeding: Conference 2024 6
Tasks: Speech-to-Text, text-classification, Text Classification, Voice Query Recognition

With the rapid advancement of artificial intelligence technologies in human life, particularly within the automotive industry, the popularity of smart vehicles has increased. Designing an intelligent car cabin capable of seamless interactions with both the driver and the vehicle emerges as a crucial solution to address human-vehicle interaction challenges. This project aims to implement a digital voice assistant that recognizes vehicle commands. It utilizes three main techniques: speech-to-text conversion, text classification, and calibrating the classifier in order to detect out-of-distribution (OOD) sentences. Using Vosk, an LLM model, voices in the vehicle environment are converted into text format. Then, after pre-processing the text, an SVM classifies them. Using Platt scaling the SVM classifier outputs become calibrated, which makes them probabilistic. The experimental result shows that the proposed model achieves command recognition with 96.43% accuracy, 96.83% precision, 96.43% recall, 96.39% F1-score, and 0.1437 cross-entropy loss. Furthermore, the optimal threshold for OOD sentences is 0.4.
Privacy-Enhanced Zero-Shot Learning via Data-Free Knowledge Transfer
Authors: Gao Rui, Wan Fan, Organisciak Daniel, Pu Jiyao, Duan Haoran, Zhang Peng, Hou Xingsong, Long Yang
Date: 2023-08-25 00:00:00
Proceeding: 2023 IEEE International Conference on Multimedia and Expo (ICME) 2023 8
Tasks: Transfer Learning, Zero-Shot Learning

Considering the increasing concerns about data copyright and sensitivity issues, we present a novel Privacy-Enhanced Zero-Shot Learning (PE-ZSL) paradigm. The key innovation is to involve a teacher model as the data safeguard to guide the PE-ZSL model training without data sharing. The PE-ZSL model consists of a generator and student network, which can achieve data-free knowledge transfer while maintaining the performance of teacher model. We investigate ‘black-’ and ‘white-box’ scenarios in PE-ZSL task as different levels of framework privacy. Besides, we provide the discussion of teacher model in both omniscient and quasi-omniscient settings according to the knowledge space. Despite simple implementations and data-missing disadvantages, our PE-ZSL framework can retain state-of-the-art ZSL and GZSL performance under the ‘white-box’ scenario. Extensive qualitative and quantitative analysis also demonstrates promising results when deploying the model under ‘black-box’ scenario.
Do You Act Like You Talk? Exploring Pose-based Driver Action Classification with Speech Recognition Networks
Authors: Pablo Pardo-Decimavilla, Luis M. Bergasa, Santiago Montiel-Marín, Miguel Antunes, Ángel Llamazares
Date: 2024-07-15 00:00:00
Proceeding: IEEE Intelligent Vehicles Symposium (IV) 2024 7
Tasks: Action Classification, Data Augmentation, Skeleton Based Action Recognition, speech-recognition, Speech Recognition

Recognizing distractions on the road is crucial to reduce traffic accidents. Video-based networks are typically used, but are limited by their computational cost and are vulnerable to viewpoint changes. In this paper, we propose a novel approach for pose-based driver action classification using speech recognition networks, which is lighter and more viewpoint invariant that video-based one. We leverage the similarity in the encoding of information between audio and pose data, representing poses as key points over time. Our architecture is based on Squeezeformer, an efficient attentionbased speech recognition network. We introduce a selection of data augmentation techniques to enhance generalization. Experiments on the Drive&Act dataset demonstrate superior performance compared to state-of-the-art methods. Additionally, we explore the integration of object information and the impact of viewpoint changes. Our results highlight the effectiveness and robustness of speech recognition networks in pose-based action classification.
Automatic Speech Interruption Detection: Analysis, Corpus, and System
Authors: Martin Lebourdais, Marie Tahon, Antoine Laurent, Sylvain Meignier
Date: 2024-05-20 00:00:00
Proceeding: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) 2024 5
Tasks: Speech Interruption Detection

Interruption detection is a new yet challenging task in the field of speech processing. This article presents a comprehensive study on automatic speech interruption detection, from the definition of this task, the assembly of a specialized corpus, and the development of an initial baseline system. We provide three main contributions: Firstly, we define the task, taking into account the nuanced nature of interruptions within spontaneous conversations. Secondly, we introduce a new corpus of conversational data, annotated for interruptions, to facilitate research in this domain. This corpus serves as a valuable resource for evaluating and advancing interruption detection techniques. Lastly, we present a first baseline system, which use speech processing methods to automatically identify interruptions in speech with promising results. In this article, we derivate from theoretical notions of interruption to build a simplification of this notion based on overlapped speech detection. Our findings can not only serve as a foundation for further research in the field but also provide a benchmark for assessing future advancements in automatic speech interruption detection.
Automatic classification of speech overlaps: Feature representation and algorithms
Authors: Shammur Absar Chowdhury, Evgeny A. Stepanov, Morena Danieli, Giuseppe Riccardi
Date: 2018-12-02 00:00:00
Proceeding: Computer Speech & Language 55 (2019) 145-167 2018 12
Tasks: Speech Interruption Detection, Word Embeddings

Overlapping speech is a natural and frequently occurring phenomenon in humanhuman conversations with an underlying
purpose. Speech overlap events may be categorized as competitive and non-competitive. While the former is an attempt to grab
the floor, the latter is an attempt to assist the speaker to continue the turn. The presence and distribution of these categories are
indicative of the speakers’ states during the conversation. Therefore, understanding these manifestations is crucial for conversational analysis and for modeling human-machine dialogs. The goal of this study is to design computational models to classify
overlapping speech segments of dyadic conversations into competitive vs. non-competitive acts using lexical and acoustic cues,
as well as their surrounding context. The designed overlap representations are evaluated in both linear - Support Vector
Machines (SVM) - and non-linear - feed-forward (FFNN), convolutional (CNN) and long short-term memory (LSTM) neural
network  models. We experiment with lexical and acoustic representations and their combinations from both speaker channels
in feature and hidden space. We observe that lexical word-embedding features significantly increase the overall F1-measure compared to both acoustic and bag-of-ngrams lexical representations, suggesting that lexical information can be utilized as a powerful
cue for overlap classification. Our comparative study shows that the best computational architecture is an FFNN along with a
combination of word embeddings and acoustic features.
 2018 Elsevier Ltd. All rights reserved.
Unsupervised recognition and clustering of speech overlaps in spoken conversations
Authors: Shammur Absar Chowdhury, Giuseppe Riccardi, Firoj Alam
Date: 2014-09-11 00:00:00
Proceeding: Workshop on Speech, Language and Audio in Multimedia (SLAM 2014) 2014 9
Tasks: Clustering, Speech Interruption Detection

We are interested in understanding speech overlaps and their function in human conversations. Previous studies on speech overlaps have relied on supervised methods, small corpora and controlled conversations. The characterization of overlaps based on timing, semantic and discourse function requires an analysis over a very large feature space. In this study, the corpus of overlapped speech segments was automatically extracted from human-human spoken conversations using a large vocabulary Automatic Speech Recognizer (ASR) and a turn segmenter. Each overlap instance is automatically projected onto a high dimensional space of acoustic and lexical features. Then, we used unsupervised clustering to find the distinct and well-separated clusters in terms of acoustic and lexical features. We have evaluated recognition and clustering algorithms over a large set of real human-human spoken conversations. The clusters have been comparatively evaluated in terms of feature distributions and their contribution to the automatic classification of the clusters.
Automatic text-based speech overlap classification: A novel approach using Large Language Models
Authors: J.H. Domhof
Date: 2023-06-25 00:00:00
Proceeding: A Thesis Submitted to EEMCS Faculty Delft University of Technology, In Partial Fulfilment of the Requirements For the Bachelor of Computer Science and Engineering 2023 6
Tasks: In-Context Learning, Speech Interruption Detection

Meetings are the keystone of a good company. They allow for quick decision making, multiple-perspective problem solving and effective communication. However, most employees and managers have a negative view on the efficiency and quality of their meetings. High quality meetings where every participant feels equally heard and respected is crucial for having positive meeting sentiment within a company. One of the most influential aspects of meetings are speech overlaps. Overlaps range from short utterances such as backchannels, to follow up questions and clarifications, to complete interruptions. In non-competitive cases, the overlapped speaker feels that the other participants are listening and actively engaging with them during the meeting. In competitive cases, the overlapped speaker can feel interrupted and unimportant. Therefore, competitive overlaps often have a negative impact on the course of the discussion and the overlappee's meeting sentiment. In problematic cases, these overlaps should be reduced to a minimum. In order to do this, overlaps must be classified as either competitive or non-competitive. This paper proposes a novel approach to overlap classification, namely that of text-based classification through Large Language Models. Four different prompt designs are used and tested on the two best performing and publicly available models, GPT-3.5-turbo and GPT-4. The results show that the in-context learning approach using the GPT-4 model results in the most accurate classifications. When comparing the results to previous work, it is observed that the text-based GPT-4 model matches carefully engineered neural networks that even adopt a multi-modular approach.
Automated mapping of competitive and collaborative overlapping talk in video meetings.
Authors: Eleni Margariti, Sean Rintel, Brendan Murphy, Abigail Sellen
Date: 2022-04-28 00:00:00
Proceeding: CHI EA '22: CHI Conference on Human Factors in Computing Systems Extended Abstracts 2022 4
Tasks: Speech Interruption Detection

Video meetings are notorious for difficulties with conversational turn-taking, which has impacts on inclusion and outcomes. We present a scalable automatic process to categorize turn-taking patterns in remote meetings based on eyes-off analysis of meeting transcripts. Drawing on a series of remote meetings (10 series, 34 total meetings) recorded in July-August 2021 by employees of a global technology company, we identified and parametrized patterns of cooperative and competitive overlaps of turns. The results show initial success characterizing people's behaviours as either likely to continue or cede turns based on either the amount of overlap that they produce during other's turns or the amount of overlap they experience in their own turns. With further development and validation, this method could be used to measure inclusion in remote and hybrid meetings.
Ensemble learning for alcoholism classification using EEG signals
Authors: Seffi Cohen, Or Katz, Dan Presil, Ofir Arbili, Lior Rokach
Date: 2023-06-17 00:00:00
Proceeding: IEEE Sensors 2023 6
Tasks: Classification, Diagnostic, EEG, Ensemble Learning

Excessive drinking is a major risk factor that leads to many health complications. The diagnosis of alcoholism is challenging, especially when the standard diagnostic tests rely on blood tests and questionnaires that are subjective to the patient and the examiner. The study’s major goal is to find new electroencephalography (EEG) classification methods to improve past findings and construct a robust EEG classification algorithm to generate accurate predictions with explainable results. The EEG records were examined from two different perspectives and combined with an ensemble of classification models. The first approach was temporal data, and the second was images derived from the original signals. Using fast Fourier transform (FFT) and independent component analysis (ICA), we convert 64-channel temporal data into images along with applying the Symbolic Aggregate approXimation (SAX) technique. Our model combines input data in tabular, temporal, and image formats with an ensemble of linear neural networks, long short-term memory (LSTM), and efficient-net classification models. We have evaluated our method using a publicly available dataset for EEG classification of alcoholic and nonalcoholic subjects. Overall, our algorithm’s highest cross-validation classification accuracy is 85.52% compared to the state-of-the-art EEG-NET’s accuracy of 81.19%
Mini-Sequence Transformer: Optimizing Intermediate Memory for Long Sequences Training
Authors: Cheng Luo, Jiawei Zhao, Zhuoming Chen, Beidi Chen, Anima Anandkumar
Date: 2024-07-22 00:00:00

We introduce Mini-Sequence Transformer (MsT), a simple and effective methodology for highly efficient and accurate LLM training with extremely long sequences. MsT partitions input sequences and iteratively processes mini-sequences to reduce intermediate memory usage. Integrated with activation recomputation, it enables significant memory savings in both forward and backward passes. In experiments with the Llama3-8B model, with MsT, we measure no degradation in throughput or convergence even with 12x longer sequences than standard implementations. MsT is fully general, implementation-agnostic, and requires minimal code changes to integrate with existing LLM training frameworks. Integrated with the huggingface library, MsT successfully extends the maximum context length of Qwen, Mistral, and Gemma-2 by 12-24x.
QMViT: A Mushroom is worth 16x16 Words
Authors: Siddhant Dutta, Hemant Singh, Kalpita Shankhdhar, Sridhar Iyer
Date: 2024-05-11 00:00:00

Consuming poisonous mushrooms can have severe health consequences, even resulting in fatality and accurately distinguishing edible from toxic mushroom varieties remains a significant challenge in ensuring food safety. So, it's crucial to distinguish between edible and poisonous mushrooms within the existing species. This is essential due to the significant demand for mushrooms in people's daily meals and their potential contributions to medical science. This work presents a novel Quantum Vision Transformer architecture that leverages quantum computing to enhance mushroom classification performance. By implementing specialized quantum self-attention mechanisms using Variational Quantum Circuits, the proposed architecture achieved 92.33% and 99.24% accuracy based on their category and their edibility respectively. This demonstrates the success of the proposed architecture in reducing false negatives for toxic mushrooms, thus ensuring food safety. Our research highlights the potential of QMViT for improving mushroom classification as a whole.
FCNR: Fast Compressive Neural Representation of Visualization Images
Authors: Yunfei Lu, Pengfei Gu, Chaoli Wang
Date: 2024-07-23 00:00:00
Tasks: Image Compression

We present FCNR, a fast compressive neural representation for tens of thousands of visualization images under varying viewpoints and timesteps. The existing NeRVI solution, albeit enjoying a high compression ratio, incurs slow speeds in encoding and decoding. Built on the recent advances in stereo image compression, FCNR assimilates stereo context modules and joint context transfer modules to compress image pairs. Our solution significantly improves encoding and decoding speed while maintaining high reconstruction quality and satisfying compression ratio. To demonstrate its effectiveness, we compare FCNR with state-of-the-art neural compression methods, including E-NeRV, HNeRV, NeRVI, and ECSIC. The source code can be found at https://github.com/YunfeiLu0112/FCNR.
T3RD: Test-Time Training for Rumor Detection on Social Media
Authors: Huaiwen Zhang, Xinxin Liu, Qing Yang, Yang Yang, Fan Qi, Shengsheng Qian, Changsheng Xu
Date: 2024-05-13 00:00:00
Proceeding: WWW '24: Proceedings of the ACM on Web Conference 2024 2024 5
Tasks: Contrastive Learning, Fake News Detection, Self-Supervised Learning

With the increasing number of news uploaded to the internet daily, rumor detection has garnered significant attention in recent years. Existing rumor detection methods excel on familiar topics with sufficient training data (high resource) collected from the same domain. However, when facing emergent events or rumors propagated in different languages, the performance of these models is significantly degraded, due to the lack of training data and prior knowledge (low resource). To tackle this challenge, we introduce the Test-Time Training for Rumor Detection (T^3RD) to enhance the performance of rumor detection models on low-resource datasets. Specifically, we introduce self-supervised learning (SSL) as an auxiliary task in the test-time training. It consists of global and local contrastive learning, in which the global contrastive learning focuses on obtaining invariant graph representations and the local one focuses on acquiring invariant node representations. We employ the auxiliary SSL tasks for both the training and test-time training phases to mine the intrinsic traits of test samples and further calibrate the trained model for these test samples. To mitigate the risk of distribution distortion in test-time training, we introduce feature alignment constraints aimed at achieving a balanced synergy between the knowledge derived from the training set and the test samples. The experiments conducted on the two widely used cross-domain datasets demonstrate that the proposed model achieves a new state-of-the-art in performance. Our code is available at https://github.com/social-rumors/T3RD.