OpenRaiser/Intern-Atlas huggingface dataset:
Hybrid diffusion models: combining supervised and generative pretraining for label-efficient fine-tuning of segmentation models
Venue: arXiv cs.CV 2024
Year: 2024

We are considering in this paper the task of label-efficient fine-tuning of segmentation models: We assume that a large labeled dataset is available and allows to train an accurate segmentation model in one domain, and that we have to adapt this model on a related domain where only a few samples are available. We observe that this adaptation can be done using two distinct methods: The first method, supervised pretraining, is simply to take the model trained on the first domain using classical supervised learning, and fine-tune it on the second domain with the available labeled samples. The second method is to perform self-supervised pretraining on the first domain using a generic pretext task in order to get high-quality representations which can then be used to train a model on the second domain in a label-efficient way. We propose in this paper to fuse these two approaches by introducing a new pretext task, which is to perform simultaneously image denoising and mask prediction on the first domain. We motivate this choice by showing that in the same way that an image denoiser conditioned on the noise level can be considered as a generative model for the unlabeled image distribution using the theory of diffusion models, a model trained using this new pretext task can be considered as a generative model for the joint distribution of images and segmentation masks under the assumption that the mapping from images to segmentation masks is deterministic. We then empirically show on several datasets that fine-tuning a model pretrained using this approach leads to better results than fine-tuning a similar model trained using either supervised or unsupervised pretraining only.
Probabilistic Surrogate Model for Accelerating the Design of Electric Vehicle Battery Enclosures for Crash Performance
Venue: arXiv cs.LG 2024
Year: 2024

This paper presents a probabilistic surrogate model for the accelerated design of electric vehicle battery enclosures with a focus on crash performance. The study integrates high-throughput finite element simulations and Gaussian Process Regression to develop a surrogate model that predicts crash parameters with high accuracy while providing uncertainty estimates. The model was trained using data generated from thermoforming and crash simulations over a range of material and process parameters. Validation against new simulation data demonstrated the model's predictive accuracy with mean absolute percentage errors within 8.08% for all output variables. Additionally, a Monte Carlo uncertainty propagation study revealed the impact of input variability on outputs. The results highlight the efficacy of the Gaussian Process Regression model in capturing complex relationships within the dataset, offering a robust and efficient tool for the design optimization of composite battery enclosures.
Advancing EEG-Based Gaze Prediction Using Depthwise Separable Convolution and Enhanced Pre-Processing
Venue: arXiv cs.LG 2024
Year: 2024

In the field of EEG-based gaze prediction, the application of deep learning to interpret complex neural data poses significant challenges. This study evaluates the effectiveness of pre-processing techniques and the effect of additional depthwise separable convolution on EEG vision transformers (ViTs) in a pretrained model architecture. We introduce a novel method, the EEG Deeper Clustered Vision Transformer (EEG-DCViT), which combines depthwise separable convolutional neural networks (CNNs) with vision transformers, enriched by a pre-processing strategy involving data clustering. The new approach demonstrates superior performance, establishing a new benchmark with a Root Mean Square Error (RMSE) of 51.6 mm. This achievement underscores the impact of pre-processing and model refinement in enhancing EEG-based applications.
Harnessing the Power of LLMs in Source Code Vulnerability Detection
Venue: arXiv cs.SE 2024
Year: 2024

Software vulnerabilities, caused by unintentional flaws in source code, are a primary root cause of cyberattacks. Static analysis of source code has been widely used to detect these unintentional defects introduced by software developers. Large Language Models (LLMs) have demonstrated human-like conversational abilities due to their capacity to capture complex patterns in sequential data, such as natural languages. In this paper, we harness LLMs' capabilities to analyze source code and detect known vulnerabilities. To ensure the proposed vulnerability detection method is universal across multiple programming languages, we convert source code to LLVM IR and train LLMs on these intermediate representations. We conduct extensive experiments on various LLM architectures and compare their accuracy. Our comprehensive experiments on real-world and synthetic codes from NVD and SARD demonstrate high accuracy in identifying source code vulnerabilities.
FacialPulse: An Efficient RNN-based Depression Detection via Temporal Facial Landmarks
Venue: arXiv cs.CV 2024
Year: 2024

Depression is a prevalent mental health disorder that significantly impacts individuals' lives and well-being. Early detection and intervention are crucial for effective treatment and management of depression. Recently, there are many end-to-end deep learning methods leveraging the facial expression features for automatic depression detection. However, most current methods overlook the temporal dynamics of facial expressions. Although very recent 3DCNN methods remedy this gap, they introduce more computational cost due to the selection of CNN-based backbones and redundant facial features. To address the above limitations, by considering the timing correlation of facial expressions, we propose a novel framework called FacialPulse, which recognizes depression with high accuracy and speed. By harnessing the bidirectional nature and proficiently addressing long-term dependencies, the Facial Motion Modeling Module (FMMM) is designed in FacialPulse to fully capture temporal features. Since the proposed FMMM has parallel processing capabilities and has the gate mechanism to mitigate gradient vanishing, this module can also significantly boost the training speed. Besides, to effectively use facial landmarks to replace original images to decrease information redundancy, a Facial Landmark Calibration Module (FLCM) is designed to eliminate facial landmark errors to further improve recognition accuracy. Extensive experiments on the AVEC2014 dataset and MMDA dataset (a depression dataset) demonstrate the superiority of FacialPulse on recognition accuracy and speed, with the average MAE (Mean Absolute Error) decreased by 21% compared to baselines, and the recognition speed increased by 100% compared to state-of-the-art methods. Codes are released at https://github.com/volatileee/FacialPulse.
CLIP-based Point Cloud Classification via Point Cloud to Image Translation
Venue: arXiv cs.CV 2024
Year: 2024

Point cloud understanding is an inherently challenging problem because of the sparse and unordered structure of the point cloud in the 3D space. Recently, Contrastive Vision-Language Pre-training (CLIP) based point cloud classification model i.e. PointCLIP has added a new direction in the point cloud classification research domain. In this method, at first multi-view depth maps are extracted from the point cloud and passed through the CLIP visual encoder. To transfer the 3D knowledge to the network, a small network called an adapter is fine-tuned on top of the CLIP visual encoder. PointCLIP has two limitations. Firstly, the point cloud depth maps lack image information which is essential for tasks like classification and recognition. Secondly, the adapter only relies on the global representation of the multi-view features. Motivated by this observation, we propose a Pretrained Point Cloud to Image Translation Network (PPCITNet) that produces generalized colored images along with additional salient visual cues to the point cloud depth maps so that it can achieve promising performance on point cloud classification and understanding. In addition, we propose a novel viewpoint adapter that combines the view feature processed by each viewpoint as well as the global intertwined knowledge that exists across the multi-view features. The experimental results demonstrate the superior performance of the proposed model over existing state-of-the-art CLIP-based models on ModelNet10, ModelNet40, and ScanobjectNN datasets.
Unlocking Exocentric Video-Language Data for Egocentric Video Representation Learning
Venue: arXiv cs.CV 2024
Year: 2024

We present EMBED (Egocentric Models Built with Exocentric Data), a method designed to transform exocentric video-language data for egocentric video representation learning. Large-scale exocentric data covers diverse activities with significant potential for egocentric learning, but inherent disparities between egocentric and exocentric data pose challenges in utilizing one view for the other seamlessly. Egocentric videos predominantly feature close-up hand-object interactions, whereas exocentric videos offer a broader perspective on human activities. Additionally, narratives in egocentric datasets are typically more action-centric and closely linked with the visual content, in contrast to the narrative styles found in exocentric datasets. To address these challenges, we employ a data transformation framework to adapt exocentric data for egocentric training, focusing on identifying specific video clips that emphasize hand-object interactions and transforming narration styles to align with egocentric perspectives. By applying both vision and language style transfer, our framework creates a new egocentric dataset derived from exocentric video-language data. Through extensive evaluations, we demonstrate the effectiveness of EMBED, achieving state-of-the-art results across various egocentric downstream tasks, including an absolute improvement of 4.7% on the Epic-Kitchens-100 multi-instance retrieval and 6.2% on the EGTEA classification benchmarks in zero-shot settings. Furthermore, EMBED enables egocentric video-language models to perform competitively in exocentric tasks. Finally, we showcase EMBED's application across various exocentric datasets, exhibiting strong generalization capabilities when applied to different exocentric datasets.
mucAI at WojoodNER 2024: Arabic Named Entity Recognition with Nearest Neighbor Search
Venue: arXiv cs.CL 2024
Year: 2024

Named Entity Recognition (NER) is a task in Natural Language Processing (NLP) that aims to identify and classify entities in text into predefined categories. However, when applied to Arabic data, NER encounters unique challenges stemming from the language's rich morphological inflections, absence of capitalization cues, and spelling variants, where a single word can comprise multiple morphemes. In this paper, we introduce Arabic KNN-NER, our submission to the Wojood NER Shared Task 2024 (ArabicNLP 2024). We have participated in the shared sub-task 1 Flat NER. In this shared sub-task, we tackle fine-grained flat-entity recognition for Arabic text, where we identify a single main entity and possibly zero or multiple sub-entities for each word. Arabic KNN-NER augments the probability distribution of a fine-tuned model with another label probability distribution derived from performing a KNN search over the cached training data. Our submission achieved 91% on the test set on the WojoodFine dataset, placing Arabic KNN-NER on top of the leaderboard for the shared task.
Prioritising Response-able IP Practices in Digitization of Electoral Processes in Africa
Venue: arXiv cs.CY 2024
Year: 2024

Globally, people widely regard technology as a solution to global social problems. In a democratic society, its citizens view technology as a way to ensure commitment and sustaining the nation's democracy by allowing them to participate actively in the democratic process. However, despite the hype surrounding technology and development, many developing countries still experience democratic challenges. The democratic challenges have further led to barriers that shape the political landscape, resulting in delusion, disappointment, and failures in the democratic and public good processes, such as the electoral process. This paper explores the relationship between intellectual property (IP) practices and the adoption of digital technologies used in democratic electoral processes. Specifically, it examines how the prioritisation of IP by technology service providers can disrupt socio-material relationships in democratic electoral processes and outcomes. Because of the hard boundaries associated with IP it creates an environment where the systems are controlled solely by technology IP owners, while the consequences of electoral processes are borne by citizens. This questions the response-ability and trust-ability of digital technologies in running democratic processes. Drawing from the parallels in Kenya's general elections of 2017 and 2022, this paper illustrates how IP practices form a hard boundary that impels technology owners to micromanage electoral processes, leading to tensions that potentially create conflict. This finding can be used by decision-makers to adopt digital technologies and protect IP without compromising electoral processes and disrupting relationships in the wider society.
Centralized Defense: Logging and Mitigation of Kubernetes Misconfigurations with Open Source Tools
Venue: arXiv cs.CR 2024
Year: 2024

Kubernetes, an open-source platform for automating the deployment, scaling, and management of containerized applications, is widely used for its efficiency and scalability. However, its complexity and extensive configuration options often lead to security vulnerabilities if not managed properly. This paper presents a detailed analysis of misconfigurations in Kubernetes environments and their significant impact on system reliability and security. A centralized logging solution was developed to detect such misconfigurations, detailing the integration process with a Kubernetes cluster and the implementation of role-based access control. Utilizing a combination of open-source tools, the solution systematically identifies misconfigurations and aggregates diagnostic data into a central repository. The effectiveness of the solution was evaluated using specific metrics, such as the total cycle time for running the central logging solution against the individual open source tools.
MTDSense: AI-Based Fingerprinting of Moving Target Defense Techniques in Software-Defined Networking
Venue: arXiv cs.CR 2024
Year: 2024

Moving target defenses (MTD) are proactive security techniques that enhance network security by confusing the attacker and limiting their attack window. MTDs have been shown to have significant benefits when evaluated against traditional network attacks, most of which are automated and untargeted. However, little has been done to address an attacker who is aware the network uses an MTD. In this work, we propose a novel approach named MTDSense, which can determine when the MTD has been triggered using the footprints the MTD operation leaves in the network traffic. MTDSense uses unsupervised clustering to identify traffic following an MTD trigger and extract the MTD interval. An attacker can use this information to maximize their attack window and tailor their attacks, which has been shown to significantly reduce the effectiveness of MTD. Through analyzing the attacker's approach, we propose and evaluate two new MTD update algorithms that aim to reduce the information leaked into the network by the MTD. We present an extensive experimental evaluation by creating, to our knowledge, the first dataset of the operation of an IP-shuffling MTD in a software-defined network. Our work reveals that despite previous results showing the effectiveness of MTD as a defense, traditional implementations of MTD are highly susceptible to a targeted attacker.
Building and Eroding: Exogenous and Endogenous Factors that Influence Subjective Trust in Visualization
Venue: arXiv cs.HC 2024
Year: 2024

Trust is a subjective yet fundamental component of human-computer interaction, and is a determining factor in shaping the efficacy of data visualizations. Prior research has identified five dimensions of trust assessment in visualizations (credibility, clarity, reliability, familiarity, and confidence), and observed that these dimensions tend to vary predictably along with certain features of the visualization being evaluated. This raises a further question: how do the design features driving viewers trust assessment vary with the characteristics of the viewers themselves? By reanalyzing data from these studies through the lens of individual differences, we build a more detailed map of the relationships between design features, individual characteristics, and trust behaviors. In particular, we model the distinct contributions of endogenous design features (such as visualization type, or the use of color) and exogenous user characteristics (such as visualization literacy), as well as the interactions between them. We then use these findings to make recommendations for individualized and adaptive visualization design.
Bi-Level Spatial and Channel-aware Transformer for Learned Image Compression
Venue: arXiv cs.CV 2024
Year: 2024

Recent advancements in learned image compression (LIC) methods have demonstrated superior performance over traditional hand-crafted codecs. These learning-based methods often employ convolutional neural networks (CNNs) or Transformer-based architectures. However, these nonlinear approaches frequently overlook the frequency characteristics of images, which limits their compression efficiency. To address this issue, we propose a novel Transformer-based image compression method that enhances the transformation stage by considering frequency components within the feature map. Our method integrates a novel Hybrid Spatial-Channel Attention Transformer Block (HSCATB), where a spatial-based branch independently handles high and low frequencies at the attention layer, and a Channel-aware Self-Attention (CaSA) module captures information across channels, significantly improving compression performance. Additionally, we introduce a Mixed Local-Global Feed Forward Network (MLGFFN) within the Transformer block to enhance the extraction of diverse and rich information, which is crucial for effective compression. These innovations collectively improve the transformation's ability to project data into a more decorrelated latent space, thereby boosting overall compression efficiency. Experimental results demonstrate that our framework surpasses state-of-the-art LIC methods in rate-distortion performance.
Hate Speech Detection and Classification in Amharic Text with Deep Learning
Venue: arXiv cs.CL 2024
Year: 2024

Hate speech is a growing problem on social media. It can seriously impact society, especially in countries like Ethiopia, where it can trigger conflicts among diverse ethnic and religious groups. While hate speech detection in resource rich languages are progressing, for low resource languages such as Amharic are lacking. To address this gap, we develop Amharic hate speech data and SBi-LSTM deep learning model that can detect and classify text into four categories of hate speech: racial, religious, gender, and non-hate speech. We have annotated 5k Amharic social media post and comment data into four categories. The data is annotated using a custom annotation tool by a total of 100 native Amharic speakers. The model achieves a 94.8 F1-score performance. Future improvements will include expanding the dataset and develop state-of-the art models. Keywords: Amharic hate speech detection, classification, Amharic dataset, Deep Learning, SBi-LSTM
Large Language Models for Biomedical Text Simplification: Promising But Not There Yet
Venue: arXiv cs.CL 2024
Year: 2024

In this system report, we describe the models and methods we used for our participation in the PLABA2023 task on biomedical abstract simplification, part of the TAC 2023 tracks. The system outputs we submitted come from the following three categories: 1) domain fine-tuned T5-like models including Biomedical-T5 and Lay-SciFive; 2) fine-tuned BARTLarge model with controllable attributes (via tokens) BART-w-CTs; 3) ChatGPTprompting. We also present the work we carried out for this task on BioGPT finetuning. In the official automatic evaluation using SARI scores, BeeManc ranks 2nd among all teams and our model LaySciFive ranks 3rd among all 13 evaluated systems. In the official human evaluation, our model BART-w-CTs ranks 2nd on Sentence-Simplicity (score 92.84), 3rd on Term-Simplicity (score 82.33) among all 7 evaluated systems; It also produced a high score 91.57 on Fluency in comparison to the highest score 93.53. In the second round of submissions, our team using ChatGPT-prompting ranks the 2nd in several categories including simplified term accuracy score 92.26 and completeness score 96.58, and a very similar score on faithfulness score 95.3 to re-evaluated PLABA-base-1 (95.73) via human evaluations. Our codes, fine-tuned models, prompts, and data splits from the system development stage will be available at https://github.com/ HECTA-UoM/PLABA-MU
Lower Bounds for Approximate (& Exact) k-Disjoint-Shortest-Paths
Venue: arXiv cs.DS 2024
Year: 2024

Given a graph $G=(V,E)$ and a set $T=\{ (s_i, t_i) : 1\leq i\leq k \}\subseteq V\times V$ of $k$ pairs, the $k$-vertex-disjoint-paths (resp. $k$-edge-disjoint-paths) problem asks to determine whether there exist~$k$ pairwise vertex-disjoint (resp. edge-disjoint) paths $P_1, P_2, ..., P_k$ in $G$ such that, for each $1\leq i\leq k$, $P_i$ connects $s_i$ to $t_i$. Both the edge-disjoint and vertex-disjoint versions in undirected graphs are famously known to be FPT (parameterized by $k$) due to the Graph Minor Theory of Robertson and Seymour. Eilam-Tzoreff [DAM `98] introduced a variant, known as the $k$-disjoint-shortest-paths problem, where each individual path is further required to be a shortest path connecting its pair. They showed that the $k$-disjoint-shortest-paths problem is NP-complete on both directed and undirected graphs; this holds even if the graphs are planar and have unit edge lengths. We focus on four versions of the problem, corresponding to considering edge/vertex disjointness, and to considering directed/undirected graphs. Building on the reduction of Chitnis [SIDMA `23] for $k$-edge-disjoint-paths on planar DAGs, we obtain the following inapproximability lower bound for each of the four versions of $k$-disjoint-shortest-paths on $n$-vertex graphs: - Under Gap-ETH, there exists a constant $δ>0$ such that for any constant $0<ε\leq \frac{1}{2}$ and any computable function $f$, there is no $(\frac{1}{2}+ε)$-approx in $f(k)\cdot n^{δ\cdot k}$ time. We further strengthen our results as follows: Directed: Inapprox lower bound for edge-disjoint (resp. vertex-disjoint) paths holds even if the input graph is a planar (resp. 1-planar) DAG with max in-degree and max out-degree at most $2$. Undirected: Inapprox lower bound for edge-disjoint (resp. vertex-disjoint) paths hold even if the input graph is planar (resp. 1-planar) and has max degree $4$.