OpenRaiser/Intern-Atlas huggingface dataset:
Dual-Channel Latent Factor Analysis Enhanced Graph Contrastive Learning for Recommendation
Venue: arXiv cs.LG 2024
Year: 2024

Graph Neural Networks (GNNs) are powerful learning methods for recommender systems owing to their robustness in handling complicated user-item interactions. Recently, the integration of contrastive learning with GNNs has demonstrated remarkable performance in recommender systems to handle the issue of highly sparse user-item interaction data. Yet, some available graph contrastive learning (GCL) techniques employ stochastic augmentation, i.e., nodes or edges are randomly perturbed on the user-item bipartite graph to construct contrastive views. Such a stochastic augmentation strategy not only brings noise perturbation but also cannot utilize global collaborative signals effectively. To address it, this study proposes a latent factor analysis (LFA) enhanced GCL approach, named LFA-GCL. Our model exclusively incorporates LFA to implement the unconstrained structural refinement, thereby obtaining an augmented global collaborative graph accurately without introducing noise signals. Experiments on four public datasets show that the proposed LFA-GCL outperforms the state-of-the-art models.
Kolmogorov-Arnold Network for Online Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

Kolmogorov-Arnold Networks (KANs) have shown potential as an alternative to Multi-Layer Perceptrons (MLPs) in neural networks, providing universal function approximation with fewer parameters and reduced memory usage. In this paper, we explore the use of KANs as function approximators within the Proximal Policy Optimization (PPO) algorithm. We evaluate this approach by comparing its performance to the original MLP-based PPO using the DeepMind Control Proprio Robotics benchmark. Our results indicate that the KAN-based reinforcement learning algorithm can achieve comparable performance to its MLP-based counterpart, often with fewer parameters. These findings suggest that KANs may offer a more efficient option for reinforcement learning models.
Automated PMC-based Power Modeling Methodology for Modern Mobile GPUs
Venue: arXiv cs.PF 2024
Year: 2024

The rise of machine learning workload on smartphones has propelled GPUs into one of the most power-hungry components of modern smartphones and elevates the need for optimizing the GPU power draw by mobile apps. Optimizing the power consumption of mobile GPUs in turn requires accurate estimation of their power draw during app execution. In this paper, we observe that the prior-art, utilization-frequency based GPU models cannot capture the diverse micro-architectural usage of modern mobile GPUs.We show that these models suffer poor modeling accuracy under diverse GPU workload, and study whether performance monitoring counter (PMC)-based models recently proposed for desktop/server GPUs can be applied to accurately model mobile GPU power. Our study shows that the PMCs that come with dominating mobile GPUs used in modern smartphones are sufficient to model mobile GPU power, but exhibit multicollinearity if used altogether. We present APGPM, the mobile GPU power modeling methodology that automatically selects an optimal set of PMCs that maximizes the GPU power model accuracy. Evaluation on two representative mobile GPUs shows that APGPM-generated GPU power models reduce the MAPE modeling error of prior-art by 1.95x to 2.66x (i.e., by 11.3% to 15.4%) while using only 4.66% to 20.41% of the total number of available PMCs.
Axiomatic Characterisations of Sample-based Explainers
Venue: arXiv cs.AI 2024
Year: 2024

Explaining decisions of black-box classifiers is both important and computationally challenging. In this paper, we scrutinize explainers that generate feature-based explanations from samples or datasets. We start by presenting a set of desirable properties that explainers would ideally satisfy, delve into their relationships, and highlight incompatibilities of some of them. We identify the entire family of explainers that satisfy two key properties which are compatible with all the others. Its instances provide sufficient reasons, called weak abductive explanations.We then unravel its various subfamilies that satisfy subsets of compatible properties. Indeed, we fully characterize all the explainers that satisfy any subset of compatible properties. In particular, we introduce the first (broad family of) explainers that guarantee the existence of explanations and their global consistency.We discuss some of its instances including the irrefutable explainer and the surrogate explainer whose explanations can be found in polynomial time.
GlitchProber: Advancing Effective Detection and Mitigation of Glitch Tokens in Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs) have achieved unprecedented success in the field of natural language processing. However, the black-box nature of their internal mechanisms has brought many concerns about their trustworthiness and interpretability. Recent research has discovered a class of abnormal tokens in the model's vocabulary space and named them "glitch tokens". Those tokens, once included in the input, may induce the model to produce incorrect, irrelevant, or even harmful results, drastically undermining the reliability and practicality of LLMs. In this work, we aim to enhance the understanding of glitch tokens and propose techniques for their detection and mitigation. We first reveal the characteristic features induced by glitch tokens on LLMs, which are evidenced by significant deviations in the distributions of attention patterns and dynamic information from intermediate model layers. Based on the insights, we develop GlitchProber, a tool for efficient glitch token detection and mitigation. GlitchProber utilizes small-scale sampling, principal component analysis for accelerated feature extraction, and a simple classifier for efficient vocabulary screening. Taking one step further, GlitchProber rectifies abnormal model intermediate layer values to mitigate the destructive effects of glitch tokens. Evaluated on five mainstream open-source LLMs, GlitchProber demonstrates higher efficiency, precision, and recall compared to existing approaches, with an average F1 score of 0.86 and an average repair rate of 50.06%. GlitchProber unveils a novel path to address the challenges posed by glitch tokens and inspires future research toward more robust and interpretable LLMs.
Towards a Generative Approach for Emotion Detection and Reasoning
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs) have demonstrated impressive performance in mathematical and commonsense reasoning tasks using chain-of-thought (CoT) prompting techniques. But can they perform emotional reasoning by concatenating `Let's think step-by-step' to the input prompt? In this paper we investigate this question along with introducing a novel approach to zero-shot emotion detection and emotional reasoning using LLMs. Existing state of the art zero-shot approaches rely on textual entailment models to choose the most appropriate emotion label for an input text. We argue that this strongly restricts the model to a fixed set of labels which may not be suitable or sufficient for many applications where emotion analysis is required. Instead, we propose framing the problem of emotion analysis as a generative question-answering (QA) task. Our approach uses a two step methodology of generating relevant context or background knowledge to answer the emotion detection question step-by-step. Our paper is the first work on using a generative approach to jointly address the tasks of emotion detection and emotional reasoning for texts. We evaluate our approach on two popular emotion detection datasets and also release the fine-grained emotion labels and explanations for further training and fine-tuning of emotional reasoning systems.
Energy performance of LR-FHSS: analysis and evaluation
Venue: arXiv cs.NI 2024
Year: 2024

Long-range frequency hopping spread spectrum (LR-FHSS) is a pivotal advancement in the LoRaWAN protocol that is designed to enhance the network's capacity and robustness, particularly in densely populated environments. Although energy consumption is paramount in LoRaWAN-based end devices, {this is the first study} in the literature, to our knowledge, that models the impact of this novel mechanism on energy consumption. In this article, we provide a comprehensive energy consumption analytical model of LR-FHSS, focusing on three critical metrics: average current consumption, battery lifetime, and energy efficiency of data transmission. The model is based on measurements performed on real hardware in a fully operational LR-FHSS network. While in our evaluation, LR-FHSS can show worse consumption figures than LoRa, we find that with optimal configuration, the battery lifetime of LR-FHSS end devices can reach 2.5 years for a 50 min notification period. For the most energy-efficient payload size, this lifespan can be extended to a theoretical maximum of up to 16 years with a one-day notification interval using a cell-coin battery.
GuidedNet: Semi-Supervised Multi-Organ Segmentation via Labeled Data Guide Unlabeled Data
Venue: arXiv cs.CV 2024
Year: 2024

Semi-supervised multi-organ medical image segmentation aids physicians in improving disease diagnosis and treatment planning and reduces the time and effort required for organ annotation.Existing state-of-the-art methods train the labeled data with ground truths and train the unlabeled data with pseudo-labels. However, the two training flows are separate, which does not reflect the interrelationship between labeled and unlabeled data.To address this issue, we propose a semi-supervised multi-organ segmentation method called GuidedNet, which leverages the knowledge from labeled data to guide the training of unlabeled data. The primary goals of this study are to improve the quality of pseudo-labels for unlabeled data and to enhance the network's learning capability for both small and complex organs.A key concept is that voxel features from labeled and unlabeled data that are close to each other in the feature space are more likely to belong to the same class.On this basis, a 3D Consistent Gaussian Mixture Model (3D-CGMM) is designed to leverage the feature distributions from labeled data to rectify the generated pseudo-labels.Furthermore, we introduce a Knowledge Transfer Cross Pseudo Supervision (KT-CPS) strategy, which leverages the prior knowledge obtained from the labeled data to guide the training of the unlabeled data, thereby improving the segmentation accuracy for both small and complex organs. Extensive experiments on two public datasets, FLARE22 and AMOS, demonstrated that GuidedNet is capable of achieving state-of-the-art performance. The source code with our proposed model are available at https://github.com/kimjisoo12/GuidedNet.
DAFT-GAN: Dual Affine Transformation Generative Adversarial Network for Text-Guided Image Inpainting
Venue: arXiv cs.CV 2024
Year: 2024

In recent years, there has been a significant focus on research related to text-guided image inpainting. However, the task remains challenging due to several constraints, such as ensuring alignment between the image and the text, and maintaining consistency in distribution between corrupted and uncorrupted regions. In this paper, thus, we propose a dual affine transformation generative adversarial network (DAFT-GAN) to maintain the semantic consistency for text-guided inpainting. DAFT-GAN integrates two affine transformation networks to combine text and image features gradually for each decoding block. Moreover, we minimize information leakage of uncorrupted features for fine-grained image generation by encoding corrupted and uncorrupted regions of the masked image separately. Our proposed model outperforms the existing GAN-based models in both qualitative and quantitative assessments with three benchmark datasets (MS-COCO, CUB, and Oxford) for text-guided image inpainting.
Generalisation First, Memorisation Second? Memorisation Localisation for Natural Language Classification Tasks
Venue: arXiv cs.CL 2024
Year: 2024

Memorisation is a natural part of learning from real-world data: neural models pick up on atypical input-output combinations and store those training examples in their parameter space. That this happens is well-known, but how and where are questions that remain largely unanswered. Given a multi-layered neural model, where does memorisation occur in the millions of parameters? Related work reports conflicting findings: a dominant hypothesis based on image classification is that lower layers learn generalisable features and that deeper layers specialise and memorise. Work from NLP suggests this does not apply to language models, but has been mainly focused on memorisation of facts. We expand the scope of the localisation question to 12 natural language classification tasks and apply 4 memorisation localisation techniques. Our results indicate that memorisation is a gradual process rather than a localised one, establish that memorisation is task-dependent, and give nuance to the generalisation first, memorisation second hypothesis.
Exploiting the Lock: Leveraging MiG-V's Logic Locking for Secret-Data Extraction
Venue: arXiv cs.CR 2024
Year: 2024

The MiG-V was designed for high-security applications and is the first commercially available logic-locked RISC-V processor on the market. In this context logic locking was used to protect the RISC-V processor design during the untrusted manufacturing process by using key-driven logic gates to obfuscate the original design. Although this method defends against malicious modifications, such as hardware Trojans, logic locking's impact on the RISC-V processor's data confidentiality during runtime has not been thoroughly examined. In this study, we evaluate the impact of logic locking on data confidentiality. By altering the logic locking key of the MiG-V while running SSL cryptographic algorithms, we identify data leakages resulting from the exploitation of the logic locking hardware. We show that changing a single bit of the logic locking key can expose 100% of the cryptographic encryption key. This research reveals a critical security flaw in logic locking, highlighting the need for comprehensive security assessments beyond logic locking key-recovery attacks.
Conceptual Design and Implementation of FIDO2 compatible Smart Card for Decentralized Financial Transaction System
Venue: arXiv cs.CR 2024
Year: 2024

With challenges and limitations associated with security in the fintech industry, the rise to the need for data protection increases. However, the current existing passwordless and password-based peer to peer transactions in online banking systems are vulnerable to advanced forms of digital attacks. The influx of modern data protection methods keeps better records of the transactions, but it still does not address the issue of authentication and account takeovers during transactions. To the address the mentioned issue, this paper proposes a novel and robust peer to peer transaction system which employs best cloud security practices, proper use of cryptography and trusted computing to mitigate common vulnerabilities. We will be implementing FIDO2 compatible Smart Card to securely authenticate the user using physical smart cards and store the records in the cloud which enables access control by allowing access only when an access is requested. The standard incorporates multiple layers of security on cloud computing models to ensure secrecy of the said data. Services of the standard adhere to regulations provides by the government and assures privacy to the information of the payee or the end-user. The whole system has been implemented in the Internet of Things scenario.
Looking Back, Moving Forward: A First-Person Perspective Of How Past Artificial Intelligence Encounters Shape Today's Creative Practice
Venue: arXiv cs.HC 2024
Year: 2024

This visual narrative is a first-person reflection of the previous pictorial at the 1st International Workshop on Explainable AI for the Arts (XAIxArts) at ACM Creativity and Cognition 2023. The initial workshop pictorial explored a relationship between researcher and artificial intelligence, navigating creative challenges throughout the 2023 teaching block. It concluded by raising crucial questions regarding attribution transparency, the ethical dimensions of the creative process, and the delicate balance between inspiration and plagiarism. Subsequent discussions at the workshop yielded valuable insights, particularly concerning interpreting the creative journey. This follow-up visual narrative reflects the enduring impact of Makayla Lewis's interaction with AI. A self-portrait that delves into the interplay of creativity and introspection.
Get Confused Cautiously: Textual Sequence Memorization Erasure with Selective Entropy Maximization
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) have been found to memorize and recite some of the textual sequences from their training set verbatim, raising broad concerns about privacy and copyright issues when using LLMs. This Textual Sequence Memorization (TSM) phenomenon leads to a high demand to regulate LLM output to prevent it from generating certain memorized text to meet user requirements. However, our empirical study reveals that existing methods for TSM erasure fail to forget massive memorized samples without substantially jeopardizing the model utility. To achieve a better trade-off between the effectiveness of TSM erasure and model utility in LLMs, our paper proposes a new framework based on Entropy Maximization with Selective Optimization (EMSO), where the updated weights are chosen with a novel contrastive gradient metric without any participation of additional model or data. Our analysis shows that training with the entropy maximization loss has a more stable optimization process and better keeps model utility than existing methods. The contrastive gradient metric localizes the most influential weight for TSM erasure by taking both the gradient magnitude and direction into consideration. Extensive experiments across three model scales demonstrate that our method excels in handling large-scale forgetting requests while preserving model ability in language generation and reasoning.
MathPartner: An Artificial Intelligence Cloud Service
Venue: arXiv cs.SC 2024
Year: 2024

In a broad sense, artificial intelligence is a service to find a solution to complex intellectual problems. In this sense, the MathPartner service provides artificial intelligence that allows us to formulate questions and receive answers to questions formulated in a mathematical language. For mathematicians and physicists today, such a language is \LaTeX. The MathPartner service uses a dialect of \LaTeX, which is called Mathpar. The service is a cloud-based computer algebra system and provides users with the opportunity to solve many mathematical problems. In this publication, we focus only on a small class of extremum problems, which are widely applied in economics, management, logistics, and in many engineering fields. In particular, we consider the shortest path problem and discuss an algorithm that is based on the tropical mathematics. The ability to work with many types of classical and tropical algebras, which are freely available to users, is an important distinguishing feature of this intelligent tool for symbolic-numerical calculations. We also consider the use of the simplex algorithm for solving optimization problems.
Approximate Distributed Monitoring under Partial Synchrony: Balancing Speed and Accuracy
Venue: arXiv cs.LO 2024
Year: 2024

In distributed systems with processes that do not share a global clock, \emph{partial synchrony} is achieved by clock synchronization that guarantees bounded clock skew among all applications. Existing solutions for distributed runtime verification under partial synchrony against temporal logic specifications are exact but suffer from significant computational overhead. In this paper, we propose an \emph{approximate} distributed monitoring algorithm for Signal Temporal Logic (STL) that mitigates this issue by abstracting away potential interleaving behaviors. This conservative abstraction enables a significant speedup of the distributed monitors, albeit with a tradeoff in accuracy. We address this tradeoff with a methodology that combines our approximate monitor with its exact counterpart, resulting in enhanced efficiency without sacrificing precision. We evaluate our approach with multiple experiments, showcasing its efficacy in both real-world applications and synthetic examples.
A GNN Model with Adaptive Weights for Session-Based Recommendation Systems
Venue: arXiv cs.IR 2024
Year: 2024

Session-based recommendation systems aim to model users' interests based on their sequential interactions to predict the next item in an ongoing session. In this work, we present a novel approach that can be used in session-based recommendations (SBRs). Our goal is to enhance the prediction accuracy of an existing session-based recommendation model, the SR-GNN model, by introducing an adaptive weighting mechanism applied to the graph neural network (GNN) vectors. This mechanism is designed to incorporate various types of side information obtained through different methods during the study. Items are assigned varying degrees of importance within each session as a result of the weighting mechanism. We hypothesize that this adaptive weighting strategy will contribute to more accurate predictions and thus improve the overall performance of SBRs in different scenarios. The adaptive weighting strategy can be utilized to address the cold start problem in SBRs by dynamically adjusting the importance of items in each session, thus providing better recommendations in cold start situations, such as for new users or newly added items. Our experimental evaluations on the Dressipi dataset demonstrate the effectiveness of the proposed approach compared to traditional models in enhancing the user experience and highlighting its potential to optimize the recommendation results in real-world applications.
Graph Neural Networks as Ordering Heuristics for Parallel Graph Coloring
Venue: arXiv cs.LG 2024
Year: 2024

The graph coloring problem asks for an assignment of the minimum number of distinct colors to vertices in an undirected graph with the constraint that no pair of adjacent vertices share the same color. The problem is a thoroughly studied NP-hard combinatorial problem with several real-world applications. As such, a number of greedy heuristics have been suggested that strike a good balance between coloring quality, execution time, and also parallel scalability. In this work, we introduce a graph neural network (GNN) based ordering heuristic and demonstrate that it outperforms existing greedy ordering heuristics both on quality and performance. Previous results have demonstrated that GNNs can produce high-quality colorings but at the expense of excessive running time. The current paper is the first that brings the execution time down to compete with existing greedy heuristics. Our GNN model is trained using both supervised and unsupervised techniques. The experimental results show that a 2-layer GNN model can achieve execution times between the largest degree first (LF) and smallest degree last (SL) ordering heuristics while outperforming both on coloring quality. Increasing the number of layers improves the coloring quality further, and it is only at four layers that SL becomes faster than the GNN. Finally, our GNN-based coloring heuristic achieves superior scaling in the parallel setting compared to both SL and LF.