pwc-archive/papers-with-abstracts huggingface dataset:
Distributed Bayesian Inference for Large-Scale IoT Systems
Authors: Eleni Vlachou, Aristeidis Karras 1, Christos Karras 1, *, Leonidas Theodorakopoulos 2, Constantinos Halkiopoulos and Spyros Sioutas
Date: 2023-12-19 00:00:00
Proceeding: Article 2023 12
Tasks: Bayesian Inference, CPU, Management, Specificity

Abstract: In this work, we present a Distributed Bayesian Inference Classifier for Large-Scale Systems, where we assess its performance and scalability on distributed environments such as PySpark. The presented classifier consistently showcases efficient inference time, irrespective of the variations in the size of the test set, implying a robust ability to handle escalating data sizes without a proportional increase in computational demands. Notably, throughout the experiments, there is an observed increase in memory usage with growing test set sizes, this increment is sublinear, demonstrating the proficiency of the classifier in memory resource management. This behavior is consistent with the typical tendencies of PySpark tasks, which witness increasing memory consumption due to data partitioning and various data operations as datasets expand. CPU resource utilization, which is another crucial factor, also remains stable, emphasizing the capability of the classifier to manage larger computational workloads without significant resource strain. From a classification perspective, the Bayesian Logistic Regression Spark Classifier consistently achieves reliable performance metrics, with a particular focus on high specificity, indicating its aptness for applications where pinpointing true negatives is crucial. In summary, based on all experiments conducted under various data sizes, our classifier emerges as a top contender for scalability-driven applications in IoT systems, highlighting its dependable performance, adept resource management, and consistent prediction accuracy.
Keywords: Bayesian inference; big data; IoT data engineering; large-scale IoT systems; PySpark
R^2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding
Authors: Ye Liu, Jixuan He, Wanhua Li, Junsik Kim, Donglai Wei, Hanspeter Pfister, Chang Wen Chen
Date: 2024-04-02 00:00:00
Tasks: Highlight Detection, Moment Retrieval, Natural Language Queries, Transfer Learning, Video Summarization, Video Understanding

Video temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries. Most existing VTG models are built upon frame-wise final-layer CLIP features, aided by additional temporal backbones (e.g., SlowFast) with sophisticated temporal reasoning mechanisms. In this work, we claim that CLIP itself already shows great potential for fine-grained spatial-temporal modeling, as each layer offers distinct yet useful information under different granularity levels. Motivated by this, we propose Reversed Recurrent Tuning (R^2-Tuning), a parameter- and memory-efficient transfer learning framework for video temporal grounding. Our method learns a lightweight R^2 Block containing only 1.5% of the total parameters to perform progressive spatial-temporal modeling. Starting from the last layer of CLIP, R^2 Block recurrently aggregates spatial features from earlier layers, then refines temporal correlation conditioning on the given query, resulting in a coarse-to-fine scheme. R^2-Tuning achieves state-of-the-art performance across three VTG tasks (i.e., moment retrieval, highlight detection, and video summarization) on six public benchmarks (i.e., QVHighlights, Charades-STA, Ego4D-NLQ, TACoS, YouTube Highlights, and TVSum) even without the additional backbone, demonstrating the significance and effectiveness of the proposed scheme.
$R^2$-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding
Authors: Ye Liu, Jixuan He, Wanhua Li, Junsik Kim, Donglai Wei, Hanspeter Pfister, Chang Wen Chen
Date: 2024-03-31 00:00:00
Tasks: Highlight Detection, Moment Retrieval, Natural Language Queries, Transfer Learning, Video Summarization, Video Understanding

Video temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries. Most existing VTG models are built upon frame-wise final-layer CLIP features, aided by additional temporal backbones (e.g., SlowFast) with sophisticated temporal reasoning mechanisms. In this work, we claim that CLIP itself already shows great potential for fine-grained spatial-temporal modeling, as each layer offers distinct yet useful information under different granularity levels. Motivated by this, we propose Reversed Recurrent Tuning ($R^2$-Tuning), a parameter- and memory-efficient transfer learning framework for video temporal grounding. Our method learns a lightweight $R^2$ Block containing only 1.5% of the total parameters to perform progressive spatial-temporal modeling. Starting from the last layer of CLIP, $R^2$ Block recurrently aggregates spatial features from earlier layers, then refines temporal correlation conditioning on the given query, resulting in a coarse-to-fine scheme. $R^2$-Tuning achieves state-of-the-art performance across three VTG tasks (i.e., moment retrieval, highlight detection, and video summarization) on six public benchmarks (i.e., QVHighlights, Charades-STA, Ego4D-NLQ, TACoS, YouTube Highlights, and TVSum) even without the additional backbone, demonstrating the significance and effectiveness of the proposed scheme. Our code is available at https://github.com/yeliudev/R2-Tuning.
An RML-FNML module for Python user-defined functions in Morph-KGC
Authors: Julián Arenas-Guerrero, Paola Espinoza-Arias, José Antonio Bernabé-Diaz, Prashant Deshmukh, José Luis Sánchez-Fernández, Oscar Corcho
Date: 2024-04-01 00:00:00
Proceeding: SoftwareX 2024 4
Tasks: Data Integration, Knowledge Graphs, Knowledge Graphs Data Curation, MORPH

The RML mapping language declares schema transformations to map heterogeneous data into knowledge graphs. Although the schema transformations provided by RML are sufficient for simple use cases, large real-world ones typically involve diverse data and require complex computations. User-defined functions provide the flexibility to enable the creation and application of knowledge graphs for these use cases and also allow reusing existing software packages for data processing in multiple domains. In this work, we present an implementation for data transformations in Morph-KGC. This implementation has the following benefits: (i) it conforms to RML-FNML, the standard module for data transformations in RML, (ii) it can handle user-defined functions written in Python, a widely used programming language for data processing, (iii) it includes support for YARRRML, a user-friendly syntax of RML to maximize usability. The implementation is currently being used by BASF, a large multinational chemical company, to semantically integrate its industrial data in a maintainable and reproducible manner.
Fingerprinting web servers through Transformer-encoded HTTP response headers
Authors: Patrick Darwinkel
Date: 2024-03-26 00:00:00
Tasks: Language Modeling, Language Modelling, Masked Language Modeling, Specificity

We explored leveraging state-of-the-art deep learning, big data, and natural language processing to enhance the detection of vulnerable web server versions. Focusing on improving accuracy and specificity over rule-based systems, we conducted experiments by sending various ambiguous and non-standard HTTP requests to 4.77 million domains and capturing HTTP response status lines. We represented these status lines through training a BPE tokenizer and RoBERTa encoder for unsupervised masked language modeling. We then dimensionality reduced and concatenated encoded response lines to represent each domain's web server. A Random Forest and multilayer perceptron (MLP) classified these web servers, and achieved 0.94 and 0.96 macro F1-score, respectively, on detecting the five most popular origin web servers. The MLP achieved a weighted F1-score of 0.55 on classifying 347 major type and minor version pairs. Analysis indicates that our test cases are meaningful discriminants of web server types. Our approach demonstrates promise as a powerful and flexible alternative to rule-based systems.
Edinburgh Clinical NLP at SemEval-2024 Task 2: Fine-tune your model unless you have access to GPT-4
Authors: Aryo Pradipta Gema, Giwon Hong, Pasquale Minervini, Luke Daines, Beatrice Alex
Date: 2024-03-30 00:00:00
Tasks: In-Context Learning, Language Modelling, Natural Language Inference, parameter-efficient fine-tuning, Task 2, Triplet

The NLI4CT task assesses Natural Language Inference systems in predicting whether hypotheses entail or contradict evidence from Clinical Trial Reports. In this study, we evaluate various Large Language Models (LLMs) with multiple strategies, including Chain-of-Thought, In-Context Learning, and Parameter-Efficient Fine-Tuning (PEFT). We propose a PEFT method to improve the consistency of LLMs by merging adapters that were fine-tuned separately using triplet and language modelling objectives. We found that merging the two PEFT adapters improves the F1 score (+0.0346) and consistency (+0.152) of the LLMs. However, our novel methods did not produce more accurate results than GPT-4 in terms of faithfulness and consistency. Averaging the three metrics, GPT-4 ranks joint-first in the competition with 0.8328. Finally, our contamination analysis with GPT-4 indicates that there was no test data leakage.
Crystal ball: From innovative attacks to attack effectiveness classifier
Authors: Berger, Hajaj, Mariconti, Dvir
Date: 2024-12-24 00:00:00
Proceeding: IEEE Access 2024 12
Tasks: Android Malware Detection, Malware Detection

Android OS is one of the most popular operating systems worldwide, making it a desirable
target for malware attacks. Some of the latest and most important defensive systems are based on machine
learning (ML) and cybercriminals continuously search for ways to overcome the barriers posed by these
systems. Thus, the focus of this work is on evasion attacks in the attempt to show the weaknesses of state of
the art research and how more resilient systems can be built. Evasion attacks consist of manipulating either
the actual malicious application (problem-based) or its extracted feature vector (feature-based), to avoid
being detected by ML systems. This study presents a set of innovative problem-based evasion attacks against
well-known Android malware detection systems, which decrease their detection rate by up to 97%. Moreover,
an analysis of the effectiveness of these attacks against VirusTotal (VT) scanners was conducted, empirically
showing their efficiency against well-known scanners (e.g., McAfee and Comodo) as well. The VT system
proved to be a great candidate for the attacks, as in 98% of the apps, less scanners detected the manipulated
apps than the original malicious apps. As not all the attacks are effective in the same manner against the VT
scanners, the attack efficiency classifiers are advised. Each classifier predicts the applicability of one of the
attacks. The set of classifiers creates an ensemble, which shows high success rates, allowing the attacker to
decide which attack is best to use for each malicious app and defense system.
Cloning and Beyond: A Quantum Solution to Duplicate Code
Authors: Samyak Jhaveri, Cristina V. Lopes, Alberto Krone-Martins
Date: 2023-10-19 00:00:00
Proceeding: Onward!: ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Pages 32–49 2023 10
Tasks: Clone Detection

Quantum computers are becoming a reality. The advantage of quantum computing is that it has the potential to solve computationally complex problems in a fixed amount time, independent of the size of the problem. However, the kinds of problems for which these computers are a good fit, and the ways to express those problems, are substantially different from the kinds of problems and expressions used in classical computing. Quantum annealers, in particular, are currently the most promising and available quantum computing devices in the short term. However, they are also the most foreign compared to classical programs, as they require a different kind of computational thinking. In order to ease the transition into this new world of quantum computing, we present a novel quantum approach to a well-known software problem: code clone detection. We express code clone detection as a subgraph isomorphism problem that is mapped into a quadratic optimization problem, and solve it using a DWave quantum annealing computer. We developed a quantum annealing algorithm that compares Abstract Syntax Trees (AST) and reports an energy value that indicates how similar they are.
The motivation behind this research goes well beyond code duplicate detection: our approach paves the way into how to express software engineering problems as optimization problems that can be solved by quantum annealers.
Generation and Detection of Sign Language Deepfakes - A Linguistic and Visual Analysis
Authors: Shahzeb Naeem, Muhammad Riyyan Khan, Usman Tariq, Abhinav Dhall, Carlos Ivan Colon, Hasan Al-Nashash
Date: 2024-04-01 00:00:00
Tasks: Face Swapping

A question in the realm of deepfakes is slowly emerging pertaining to whether we can go beyond facial deepfakes and whether it would be beneficial to society. Therefore, this research presents a positive application of deepfake technology in upper body generation, while performing sign-language for the Deaf and Hard of Hearing (DHoH) community. The resulting videos are later vetted with a sign language expert. This is particularly helpful, given the intricate nature of sign language, a scarcity of sign language experts, and potential benefits for health and education. The objectives of this work encompass constructing a reliable deepfake dataset, evaluating its technical and visual credibility through computer vision and natural language processing models, and assessing the plausibility of the generated content. With over 1200 videos, featuring both previously seen and unseen individuals for the generation model, using the help of a sign language expert, we establish a deepfake dataset in sign language that can further be utilized to detect fake videos that may target certain people of determination.
OVFoodSeg: Elevating Open-Vocabulary Food Image Segmentation via Image-Informed Textual Representation
Authors: Xiongwei Wu, Sicheng Yu, Ee-Peng Lim, Chong-Wah Ngo
Date: 2024-04-01 00:00:00
Proceeding: CVPR 2024 1
Tasks: Image Segmentation, Image to text, Segmentation, Semantic Segmentation

In the realm of food computing, segmenting ingredients from images poses substantial challenges due to the large intra-class variance among the same ingredients, the emergence of new ingredients, and the high annotation costs associated with large food segmentation datasets. Existing approaches primarily utilize a closed-vocabulary and static text embeddings setting. These methods often fall short in effectively handling the ingredients, particularly new and diverse ones. In response to these limitations, we introduce OVFoodSeg, a framework that adopts an open-vocabulary setting and enhances text embeddings with visual context. By integrating vision-language models (VLMs), our approach enriches text embedding with image-specific information through two innovative modules, eg, an image-to-text learner FoodLearner and an Image-Informed Text Encoder. The training process of OVFoodSeg is divided into two stages: the pre-training of FoodLearner and the subsequent learning phase for segmentation. The pre-training phase equips FoodLearner with the capability to align visual information with corresponding textual representations that are specifically related to food, while the second phase adapts both the FoodLearner and the Image-Informed Text Encoder for the segmentation task. By addressing the deficiencies of previous models, OVFoodSeg demonstrates a significant improvement, achieving an 4.9\% increase in mean Intersection over Union (mIoU) on the FoodSeg103 dataset, setting a new milestone for food image segmentation.
OpenChemIE: An Information Extraction Toolkit For Chemistry Literature
Authors: Vincent Fan, Yujie Qian, Alex Wang, Amber Wang, Connor W. Coley, Regina Barzilay
Date: 2024-04-01 00:00:00

Information extraction from chemistry literature is vital for constructing up-to-date reaction databases for data-driven chemistry. Complete extraction requires combining information across text, tables, and figures, whereas prior work has mainly investigated extracting reactions from single modalities. In this paper, we present OpenChemIE to address this complex challenge and enable the extraction of reaction data at the document level. OpenChemIE approaches the problem in two steps: extracting relevant information from individual modalities and then integrating the results to obtain a final list of reactions. For the first step, we employ specialized neural models that each address a specific task for chemistry information extraction, such as parsing molecules or reactions from text or figures. We then integrate the information from these modules using chemistry-informed algorithms, allowing for the extraction of fine-grained reaction data from reaction condition and substrate scope investigations. Our machine learning models attain state-of-the-art performance when evaluated individually, and we meticulously annotate a challenging dataset of reaction schemes with R-groups to evaluate our pipeline as a whole, achieving an F1 score of 69.5%. Additionally, the reaction extraction results of \ours attain an accuracy score of 64.3% when directly compared against the Reaxys chemical database. We provide OpenChemIE freely to the public as an open-source package, as well as through a web interface.
Robustly estimating heterogeneity in factorial data using Rashomon Partitions
Authors: Aparajithan Venkateswaran, Anirudh Sankar, Arun G. Chandrasekhar, Tyler H. McCormick
Date: 2024-04-02 00:00:00

Many statistical analyses, in both observational data and randomized control trials, ask: how does the outcome of interest vary with combinations of observable covariates? How do various drug combinations affect health outcomes, or how does technology adoption depend on incentives and demographics? Our goal is to partition this factorial space into "pools" of covariate combinations where the outcome differs across the pools (but not within a pool). Existing approaches (i) search for a single "optimal" partition under assumptions about the association between covariates or (ii) sample from the entire set of possible partitions. Both these approaches ignore the reality that, especially with correlation structure in covariates, many ways to partition the covariate space may be statistically indistinguishable, despite very different implications for policy or science. We develop an alternative perspective, called Rashomon Partition Sets (RPSs). Each item in the RPS partitions the space of covariates using a tree-like geometry. RPSs incorporate all partitions that have posterior values near the maximum a posteriori partition, even if they offer substantively different explanations, and do so using a prior that makes no assumptions about associations between covariates. This prior is the $\ell_0$ prior, which we show is minimax optimal. Given the RPS we calculate the posterior of any measurable function of the feature effects vector on outcomes, conditional on being in the RPS. We also characterize approximation error relative to the entire posterior and provide bounds on the size of the RPS. Simulations demonstrate this framework allows for robust conclusions relative to conventional regularization techniques. We apply our method to three empirical settings: price effects on charitable giving, chromosomal structure (telomere length), and the introduction of microfinance.
Deep learning phase recovery: data-driven, physics-driven, or combining both?
Authors: Kaiqiang Wang, Edmund Y. Lam
Date: 2024-04-01 00:00:00
Tasks: Deep Learning, Self-Supervised Learning

Phase recovery, calculating the phase of a light wave from its intensity measurements, is essential for various applications, such as coherent diffraction imaging, adaptive optics, and biomedical imaging. It enables the reconstruction of an object's refractive index distribution or topography as well as the correction of imaging system aberrations. In recent years, deep learning has been proven to be highly effective in addressing phase recovery problems. Two most direct deep learning phase recovery strategies are data-driven (DD) with supervised learning mode and physics-driven (PD) with self-supervised learning mode. DD and PD achieve the same goal in different ways and lack the necessary study to reveal similarities and differences. Therefore, in this paper, we comprehensively compare these two deep learning phase recovery strategies in terms of time consumption, accuracy, generalization ability, ill-posedness adaptability, and prior capacity. What's more, we propose a co-driven (CD) strategy of combining datasets and physics for the balance of high- and low-frequency information. The codes for DD, PD, and CD are publicly available at https://github.com/kqwang/DLPR.
Unlocking the Emotional States of High-Risk Suicide Callers through Speech Analysis
Authors: Alaa Nfissi, Wassim Bouachir, Nizar Bouguila, Brian Mishara
Date: 2024-03-22 00:00:00
Proceeding: IEEE 18th International Conference on Semantic Computing (ICSC) 2024 3
Tasks: Emotion Recognition, Speech Emotion Recognition

Suicide remains a major public health concern worldwide, and early detection of suicidal ideation is crucial for prevention. One promising approach for monitoring symptoms is through the prediction of suicidal speech, as speech can be passively collected and may provide insight into changes in risk. However, identifying suicidal speech is challenging due to the rapid variability in speech characteristics and the association of suicidal ideation with emotion dysregulation. In light of these challenges, we present a novel end-to-end (E2E) method for speech emotion recognition (SER) as a mean of detecting changes in emotional state, that may indicate a high risk of suicide. Our method incorporates the use of Convolutional Neural Networks (CNNs) and Gated Recurrent Units (GRUs) to analyze raw waveform signals. Firstly, our model is designed to train the CNN component to identify higher-level speech representations directly from raw waveform data, rather than relying on manually crafted features or spectrograms. This enables the network to effectively capture specific emotion-related features within a narrow frequency range, while also handling speech of varying lengths without the need for segmentation. Secondly, the GRU component is capable of learning temporal patterns, enhancing the network’s ability to capture time-dependent features in signal. Furthermore, we validate our approach on the NSPL- CRISE emotion dataset that we recently created. This dataset contains phone call recordings from lifeline frequent callers with psychological problems, and potentially with a history of suicidal ideation and previous attempts. Our experimental results show that our method outperforms other state-of-the-art techniques for SER.