OpenRaiser/Intern-Atlas huggingface dataset:
Federated PCA on Grassmann Manifold for IoT Anomaly Detection
Venue: arXiv cs.LG 2024
Year: 2024

With the proliferation of the Internet of Things (IoT) and the rising interconnectedness of devices, network security faces significant challenges, especially from anomalous activities. While traditional machine learning-based intrusion detection systems (ML-IDS) effectively employ supervised learning methods, they possess limitations such as the requirement for labeled data and challenges with high dimensionality. Recent unsupervised ML-IDS approaches such as AutoEncoders and Generative Adversarial Networks (GAN) offer alternative solutions but pose challenges in deployment onto resource-constrained IoT devices and in interpretability. To address these concerns, this paper proposes a novel federated unsupervised anomaly detection framework, FedPCA, that leverages Principal Component Analysis (PCA) and the Alternating Directions Method Multipliers (ADMM) to learn common representations of distributed non-i.i.d. datasets. Building on the FedPCA framework, we propose two algorithms, FEDPE in Euclidean space and FEDPG on Grassmann manifolds. Our approach enables real-time threat detection and mitigation at the device level, enhancing network resilience while ensuring privacy. Moreover, the proposed algorithms are accompanied by theoretical convergence rates even under a subsampling scheme, a novel result. Experimental results on the UNSW-NB15 and TON-IoT datasets show that our proposed methods offer performance in anomaly detection comparable to nonlinear baselines, while providing significant improvements in communication and memory efficiency, underscoring their potential for securing IoT networks.
Aging-Resistant Wideband Precoding in 5G and Beyond Using 3D Convolutional Neural Networks
Venue: arXiv cs.IT 2024
Year: 2024

To meet the ever-increasing demand for higher data rates, 5G and 6G technologies are shifting transceivers to higher carrier frequencies, to support wider bandwidths and more antenna elements. Nevertheless, this solution poses several key challenges: i) increasing the carrier frequency and bandwidth leads to greater channel frequency selectivity in time and frequency domains, and ii) the greater the number of antennas the greater the the pilot overhead for channel estimation and the more prohibitively complex it becomes to determine the optimal precoding matrix. This paper presents two deep-learning frameworks to solve these issues. Firstly, we propose a 3D convolutional neural network (CNN) that is based on image super-resolution and captures the correlations between the transmitting and receiving antennas and the frequency domains to combat frequency selectivity. Secondly, we devise a deep learning-based framework to combat the time selectivity of the channel that treats channel aging as a distortion that can be mitigated through deep learning-based image restoration techniques. Simulation results show that combining both frameworks leads to a significant improvement in performance compared to existing techniques with little increase in complexity.
EDHOC is a New Security Handshake Standard: An Overview of Security Analysis
Venue: arXiv cs.CR 2024
Year: 2024

The paper wraps up the call for formal analysis of the new security handshake protocol EDHOC by providing an overview of the protocol as it was standardized, a summary of the formal security analyses conducted by the community, and a discussion on open venues for future work.
CM-DQN: A Value-Based Deep Reinforcement Learning Model to Simulate Confirmation Bias
Venue: arXiv cs.LG 2024
Year: 2024

In human decision-making tasks, individuals learn through trials and prediction errors. When individuals learn the task, some are more influenced by good outcomes, while others weigh bad outcomes more heavily. Such confirmation bias can lead to different learning effects. In this study, we propose a new algorithm in Deep Reinforcement Learning, CM-DQN, which applies the idea of different update strategies for positive or negative prediction errors, to simulate the human decision-making process when the task's states are continuous while the actions are discrete. We test in Lunar Lander environment with confirmatory, disconfirmatory bias and non-biased to observe the learning effects. Moreover, we apply the confirmation model in a multi-armed bandit problem (environment in discrete states and discrete actions), which utilizes the same idea as our proposed algorithm, as a contrast experiment to algorithmically simulate the impact of different confirmation bias in decision-making process. In both experiments, confirmatory bias indicates a better learning effect.
Use of social networks to motivate computer-engineering students to participate in self-assessment activities
Venue: arXiv cs.CY 2024
Year: 2024

Motivation is essential in the learning process of university students, and teachers should have a wide range of strategies to address this issue. The emergence of social technologies has had a considerable influence in e-learning systems, and a number of experts state that their use is a good method to motivate students and to increase their participation in activities. This study attempts to determine whether social networks and social applications should be viewed as many other tools or whether they can actually provide extra motivation for students to participate. The study compared the percentage of student participation in tasks of self-assessment. The experiments covered three traditional strategies of student motivation and another one in which social networks were used to introduce, explain and deliver the self-assessment tasks. The case with a higher participation was the one in which students obtained a reward from the completion of the activity. Despite this result, the statistical analysis indicated that the use of social networks obtained similar results as a strategy of continuous and regular motivational speeches.
Fine-Grained Classification for Poisonous Fungi Identification with Transfer Learning
Venue: arXiv cs.CV 2024
Year: 2024

FungiCLEF 2024 addresses the fine-grained visual categorization (FGVC) of fungi species, with a focus on identifying poisonous species. This task is challenging due to the size and class imbalance of the dataset, subtle inter-class variations, and significant intra-class variability amongst samples. In this paper, we document our approach in tackling this challenge through the use of ensemble classifier heads on pre-computed image embeddings. Our team (DS@GT) demonstrate that state-of-the-art self-supervised vision models can be utilized as robust feature extractors for downstream application of computer vision tasks without the need for task-specific fine-tuning on the vision backbone. Our approach achieved the best Track 3 score (0.345), accuracy (78.4%) and macro-F1 (0.577) on the private test set in post competition evaluation. Our code is available at https://github.com/dsgt-kaggle-clef/fungiclef-2024.
Pan-cancer Histopathology WSI Pre-training with Position-aware Masked Autoencoder
Venue: arXiv cs.CV 2024
Year: 2024

Large-scale pre-training models have promoted the development of histopathology image analysis. However, existing self-supervised methods for histopathology images primarily focus on learning patch features, while there is a notable gap in the availability of pre-training models specifically designed for WSI-level feature learning. In this paper, we propose a novel self-supervised learning framework for pan-cancer WSI-level representation pre-training with the designed position-aware masked autoencoder (PAMA). Meanwhile, we propose the position-aware cross-attention (PACA) module with a kernel reorientation (KRO) strategy and an anchor dropout (AD) mechanism. The KRO strategy can capture the complete semantic structure and eliminate ambiguity in WSIs, and the AD contributes to enhancing the robustness and generalization of the model. We evaluated our method on 7 large-scale datasets from multiple organs for pan-cancer classification tasks. The results have demonstrated the effectiveness and generalization of PAMA in discriminative WSI representation learning and pan-cancer WSI pre-training. The proposed method was also compared with 8 WSI analysis methods. The experimental results have indicated that our proposed PAMA is superior to the state-of-the-art methods. The code and checkpoints are available at https://github.com/WkEEn/PAMA.
A Systematic Mapping Study on Teaching of Security Concepts in Programming Courses
Venue: arXiv cs.PL 2024
Year: 2024

Context: To effectively defend against ever-evolving cybersecurity threats, software systems should be made as secure as possible. To achieve this, software developers should understand potential vulnerabilities and apply secure coding practices. To prepare these skilled professionals, it is important that cybersecurity concepts are included in programming courses taught at universities. Objective: To present a comprehensive and unbiased literature review on teaching of cybersecurity concepts in programming courses taught at universities. Method: We perform a Systematic Mapping Study. We present six research questions, define our selection criteria, and develop a classification scheme. Results and Conclusions: We select 24 publications. Our results show a wide range of research contributions. We also outline guidelines and identify opportunities for future studies. The guidelines include coverage of security knowledge categories and evaluation of contributions. We suggest that future studies should cover security issues, negative impacts, and countermeasures, as well as apply evaluation techniques that examine students' knowledge. The opportunities for future studies are related to advanced courses, security knowledge frameworks, and programming environments. Furthermore, there is a need of a holistic security framework that covers the security concepts identified in this study and is suitable for education.
MLRS-PDS: A Meta-learning recommendation of dynamic ensemble selection pipelines
Venue: arXiv cs.LG 2024
Year: 2024

Dynamic Selection (DS), where base classifiers are chosen from a classifier's pool for each new instance at test time, has shown to be highly effective in pattern recognition. However, instability and redundancy in the classifier pools can impede computational efficiency and accuracy in dynamic ensemble selection. This paper introduces a meta-learning recommendation system (MLRS) to recommend the optimal pool generation scheme for DES methods tailored to individual datasets. The system employs a meta-model built from dataset meta-features to predict the most suitable pool generation scheme and DES method for a given dataset. Through an extensive experimental study encompassing 288 datasets, we demonstrate that this meta-learning recommendation system outperforms traditional fixed pool or DES method selection strategies, highlighting the efficacy of a meta-learning approach in refining DES method selection. The source code, datasets, and supplementary results can be found in this project's GitHub repository: https://github.com/Menelau/MLRS-PDS.
KaiRacters: Character-level-based Writer Retrieval for Greek Papyri
Venue: arXiv cs.CV 2024
Year: 2024

This paper presents a character-based approach for enhancing writer retrieval performance in the context of Greek papyri. Our contribution lies in introducing character-level annotations for frequently used characters, in our case the trigram kai and four additional letters (epsilon, kappa, mu, omega), in Greek texts. We use a state-of-the-art writer retrieval approach based on NetVLAD and compare a character-level-based feature aggregation method against the current default baseline of using small patches located at SIFT keypoint locations for building the page descriptors. We demonstrate that by using only about 15 characters per page, we are able to boost the performance up to 4% mAP (a relative improvement of 11%) on the GRK-120 dataset. Additionally, our qualitative analysis offers insights into the similarity scores of SIFT patches and specific characters. We publish the dataset with character-level annotations, including a quality label and our binarized images for further research.
Machine Unlearning for Medical Imaging
Venue: arXiv cs.LG 2024
Year: 2024

Machine unlearning is the process of removing the impact of a particular set of training samples from a pretrained model. It aims to fulfill the "right to be forgotten", which grants the individuals such as patients the right to reconsider their contribution in models including medical imaging models. In this study, we evaluate the effectiveness (performance) and computational efficiency of different unlearning algorithms in medical imaging domain. Our evaluations demonstrate that the considered unlearning algorithms perform well on the retain set (samples whose influence on the model is allowed to be retained) and forget set (samples whose contribution to the model should be eliminated), and show no bias against male or female samples. They, however, adversely impact the generalization of the model, especially for larger forget set sizes. Moreover, they might be biased against easy or hard samples, and need additional computational overhead for hyper-parameter tuning. In conclusion, machine unlearning seems promising for medical imaging, but the existing unlearning algorithms still needs further improvements to become more practical for medical applications.
Evaluating the method reproducibility of deep learning models in the biodiversity domain
Venue: arXiv cs.IR 2024
Year: 2024

Artificial Intelligence (AI) is revolutionizing biodiversity research by enabling advanced data analysis, species identification, and habitats monitoring, thereby enhancing conservation efforts. Ensuring reproducibility in AI-driven biodiversity research is crucial for fostering transparency, verifying results, and promoting the credibility of ecological findings.This study investigates the reproducibility of deep learning (DL) methods within the biodiversity domain. We design a methodology for evaluating the reproducibility of biodiversity-related publications that employ DL techniques across three stages. We define ten variables essential for method reproducibility, divided into four categories: resource requirements, methodological information, uncontrolled randomness, and statistical considerations. These categories subsequently serve as the basis for defining different levels of reproducibility. We manually extract the availability of these variables from a curated dataset comprising 61 publications identified using the keywords provided by biodiversity experts. Our study shows that the dataset is shared in 47% of the publications; however, a significant number of the publications lack comprehensive information on deep learning methods, including details regarding randomness.
Arabic Automatic Story Generation with Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs) have recently emerged as a powerful tool for a wide range of language generation tasks. Nevertheless, this progress has been slower in Arabic. In this work, we focus on the task of generating stories from LLMs. For our training, we use stories acquired through machine translation (MT) as well as GPT-4. For the MT data, we develop a careful pipeline that ensures we acquire high-quality stories. For our GPT-41 data, we introduce crafted prompts that allow us to generate data well-suited to the Arabic context in both Modern Standard Arabic (MSA) and two Arabic dialects (Egyptian and Moroccan). For example, we generate stories tailored to various Arab countries on a wide host of topics. Our manual evaluation shows that our model fine-tuned on these training datasets can generate coherent stories that adhere to our instructions. We also conduct an extensive automatic and human evaluation comparing our models against state-of-the-art proprietary and open-source models. Our datasets and models will be made publicly available at https: //github.com/UBC-NLP/arastories.
Completeness of Finitely Weighted Kleene Algebra With Tests
Venue: arXiv cs.LO 2024
Year: 2024

Building on Ésik and Kuich's completeness result for finitely weighted Kleene algebra, we establish relational and language completeness results for finitely weighted Kleene algebra with tests. Similarly as Ésik and Kuich, we assume that the finite semiring of weights is commutative, partially ordered and zero-bounded, but we also assume that it is integral. We argue that finitely weighted Kleene algebra with tests is a natural framework for equational reasoning about weighted programs in cases where an upper bound on admissible weights is assumed.
Explaining Graph Neural Networks for Node Similarity on Graphs
Venue: arXiv cs.LG 2024
Year: 2024

Similarity search is a fundamental task for exploiting information in various applications dealing with graph data, such as citation networks or knowledge graphs. While this task has been intensively approached from heuristics to graph embeddings and graph neural networks (GNNs), providing explanations for similarity has received less attention. In this work we are concerned with explainable similarity search over graphs, by investigating how GNN-based methods for computing node similarities can be augmented with explanations. Specifically, we evaluate the performance of two prominent approaches towards explanations in GNNs, based on the concepts of mutual information (MI), and gradient-based explanations (GB). We discuss their suitability and empirically validate the properties of their explanations over different popular graph benchmarks. We find that unlike MI explanations, gradient-based explanations have three desirable properties. First, they are actionable: selecting inputs depending on them results in predictable changes in similarity scores. Second, they are consistent: the effect of selecting certain inputs overlaps very little with the effect of discarding them. Third, they can be pruned significantly to obtain sparse explanations that retain the effect on similarity scores.
Feasibility Study on Active Learning of Smart Surrogates for Scientific Simulations
Venue: arXiv cs.LG 2024
Year: 2024

High-performance scientific simulations, important for comprehension of complex systems, encounter computational challenges especially when exploring extensive parameter spaces. There has been an increasing interest in developing deep neural networks (DNNs) as surrogate models capable of accelerating the simulations. However, existing approaches for training these DNN surrogates rely on extensive simulation data which are heuristically selected and generated with expensive computation -- a challenge under-explored in the literature. In this paper, we investigate the potential of incorporating active learning into DNN surrogate training. This allows intelligent and objective selection of training simulations, reducing the need to generate extensive simulation data as well as the dependency of the performance of DNN surrogates on pre-defined training simulations. In the problem context of constructing DNN surrogates for diffusion equations with sources, we examine the efficacy of diversity- and uncertainty-based strategies for selecting training simulations, considering two different DNN architecture. The results set the groundwork for developing the high-performance computing infrastructure for Smart Surrogates that supports on-the-fly generation of simulation data steered by active learning strategies to potentially improve the efficiency of scientific simulations.
The Language of Weather: Social Media Reactions to Weather Accounting for Climatic and Linguistic Baselines
Venue: arXiv cs.HC 2024
Year: 2024

This study explores how different weather conditions influence public sentiment on social media, focusing on Twitter data from the UK. By considering climate and linguistic baselines, we improve the accuracy of weather-related sentiment analysis. Our findings show that emotional responses to weather are complex, influenced by combinations of weather variables and regional language differences. The results highlight the importance of context-sensitive methods for better understanding public mood in response to weather, which can enhance impact-based forecasting and risk communication in the context of climate change.
Leveraging Self-Supervised Learning for MIMO-OFDM Channel Representation and Generation
Venue: arXiv cs.IT 2024
Year: 2024

In communications theory, the capacity of multiple input multiple output-orthogonal frequency division multiplexing (MIMO-OFDM) systems is fundamentally determined by wireless channels, which exhibit both diversity and correlation in spatial, frequency and temporal domains. It is further envisioned to exploit the inherent nature of channels, namely representation, to achieve geolocation-based MIMO transmission for 6G, exemplified by the fully-decoupled radio access network (FD-RAN). Accordingly, this paper first employs self-supervised learning to obtain channel representation from unlabeled channel, then proposes a channel generation assisted approach for determining MIMO precoding matrix solely based on geolocation. Specifically, we exploit the small-scale temporal domain variations of channels at a fixed geolocation, and design an ingenious pretext task tailored for contrastive learning. Then, a Transformer-based encoder is trained to output channel representations. We further develop a conditional diffusion generator to generate channel representations from geolocation. Finally, a Transformer-encoder-based decoder is utilized to reconstruct channels from generated representations, where the optimal channel is selected for calculating the precoding matrix for both single and dual BS transmission. We conduct experiments on a public ray-tracing channel dataset, and the extensive simulation results demonstrate the effectiveness of our channel representation method, and also showcase the performance improvement in geolocation-based MIMO transmission.
High-Performance Sorting-Based k-mer Counting in Distributed Memory with Flexible Hybrid Parallelism
Venue: arXiv cs.DC 2024
Year: 2024

In generating large quantities of DNA data, high-throughput sequencing technologies require advanced bioinformatics infrastructures for efficient data analysis. k-mer counting, the process of quantifying the frequency of fixed-length k DNA subsequences, is a fundamental step in various bioinformatics pipelines, including genome assembly and protein prediction. Due to the growing volume of data, the scaling of the counting process is critical. In the literature, distributed memory software uses hash tables, which exhibit poor cache friendliness and consume excessive memory. They often also lack support for flexible parallelism, which makes integration into existing bioinformatics pipelines difficult. In this work, we propose HySortK, a highly efficient sorting-based distributed memory k-mer counter. HySortK reduces the communication volume through a carefully designed communication scheme and domain-specific optimization strategies. Furthermore, we introduce an abstract task layer for flexible hybrid parallelism to address load imbalances in different scenarios. HySortK achieves a 2-10x speedup compared to the GPU baseline on 4 and 8 nodes. Compared to state-of-the-art CPU software, HySortK achieves up to 2x speedup while reducing peak memory usage by 30% on 16 nodes. Finally, we integrated HySortK into an existing genome assembly pipeline and achieved up to 1.8x speedup, proving its flexibility and practicality in real-world scenarios.