OpenRaiser/Intern-Atlas huggingface dataset:
All-domain Moveline Evolution Network for Click-Through Rate Prediction
Venue: arXiv cs.IR 2024
Year: 2024

E-commerce app users exhibit behaviors that are inherently logically consistent. A series of multi-scenario user behaviors interconnect to form the scene-level all-domain user moveline, which ultimately reveals the user's true intention. Traditional CTR prediction methods typically focus on the item-level interaction between the target item and the historically interacted items. However, the scene-level interaction between the target item and the user moveline remains underexplored. There are two challenges when modeling the interaction with preceding all-domain user moveline: (i) Heterogeneity between items and scenes: Unlike traditional user behavior sequences that utilize items as carriers, the user moveline utilizes scenes as carriers. The heterogeneity between items and scenes complicates the process of aligning interactions within a unified representation space. (ii) Temporal misalignment of linked scene-level and item-level behaviors: In the preceding user moveline with a fixed sampling length, certain critical scene-level behaviors are closely linked to subsequent item-level behaviors. However, it is impossible to establish a complete temporal alignment that clearly identifies which specific scene-level behaviors correspond to which item-level behaviors. To address these challenges and pioneer modeling user intent from the perspective of the all-domain moveline, we propose All-domain Moveline Evolution Network (AMEN). AMEN not only transfers interactions between items and scenes to homogeneous representation spaces, but also introduces a Temporal Sequential Pairwise (TSP) mechanism to understand the nuanced associations between scene-level and item-level behaviors, ensuring that the all-domain user moveline differentially influences CTR predictions for user's favored and unfavored items. Online A/B testing demonstrates that our method achieves a +11.6% increase in CTCVR.
Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering
Venue: arXiv cs.AI 2024
Year: 2024

The evolution of machine learning has increasingly prioritized the development of powerful models and more scalable supervision signals. However, the emergence of foundation models presents significant challenges in providing effective supervision signals necessary for further enhancing their capabilities. Consequently, there is an urgent need to explore novel supervision signals and technical approaches. In this paper, we propose verifier engineering, a novel post-training paradigm specifically designed for the era of foundation models. The core of verifier engineering involves leveraging a suite of automated verifiers to perform verification tasks and deliver meaningful feedback to foundation models. We systematically categorize the verifier engineering process into three essential stages: search, verify, and feedback, and provide a comprehensive review of state-of-the-art research developments within each stage. We believe that verifier engineering constitutes a fundamental pathway toward achieving Artificial General Intelligence.
Reliable Poisoned Sample Detection against Backdoor Attacks Enhanced by Sharpness Aware Minimization
Venue: arXiv cs.CV 2024
Year: 2024

Backdoor attack has been considered as a serious security threat to deep neural networks (DNNs). Poisoned sample detection (PSD) that aims at filtering out poisoned samples from an untrustworthy training dataset has shown very promising performance for defending against data poisoning based backdoor attacks. However, we observe that the detection performance of many advanced methods is likely to be unstable when facing weak backdoor attacks, such as low poisoning ratio or weak trigger strength. To further verify this observation, we make a statistical investigation among various backdoor attacks and poisoned sample detections, showing a positive correlation between backdoor effect and detection performance. It inspires us to strengthen the backdoor effect to enhance detection performance. Since we cannot achieve that goal via directly manipulating poisoning ratio or trigger strength, we propose to train one model using the Sharpness-Aware Minimization (SAM) algorithm, rather than the vanilla training algorithm. We also provide both empirical and theoretical analysis about how SAM training strengthens the backdoor effect. Then, this SAM trained model can be seamlessly integrated with any off-the-shelf PSD method that extracts discriminative features from the trained model for detection, called SAM-enhanced PSD. Extensive experiments on several benchmark datasets show the reliable detection performance of the proposed method against both weak and strong backdoor attacks, with significant improvements against various attacks ($+34.38\%$ TPR on average), over the conventional PSD methods (i.e., without SAM enhancement). Overall, this work provides new insights about PSD and proposes a novel approach that can complement existing detection methods, which may inspire more in-depth explorations in this field.
Design and Development of a Localized E-Commerce Solution for Students focussing on Economical Sharing
Venue: arXiv cs.CY 2024
Year: 2024

The rapid adoption of e-commerce has transformed how students access goods and resources. However, existing platforms often fail to address the specific needs of campus communities, where students face challenges such as financial constraints, lack of access to affordable goods, and inefficient resource circulation. This research proposes ShareSpace, a localized web application designed specifically for college students to facilitate the buying, and selling of mainly second-hand goods. By addressing imbalances like surplus items left behind by seniors and shortages experienced by juniors, ShareSpace promotes sustainability and affordability within the campus ecosystem. Leveraging modern technologies such as Node.js, React.js, and MongoDB, the project demonstrates the feasibility of creating a student-centric e-commerce solution. The study highlights how ShareSpace solves the challenges of economical pricing and content moderation using proposed solutions. This study also explores the limitations of existing solutions and evaluates the potential of ShareSpace to encourage sustainable consumption and resourcefulness among students.
gpuPairHMM: High-speed Pair-HMM Forward Algorithm for DNA Variant Calling on GPUs
Venue: arXiv cs.DC 2024
Year: 2024

The continually increasing volume of DNA sequence data has resulted in a growing demand for fast implementations of core algorithms. Computation of pairwise alignments between candidate haplotypes and sequencing reads using Pair-HMMs is a key component in DNA variant calling tools such as the GATK HaplotypeCaller but can be highly time consuming due to its quadratic time complexity and the large number of pairs to be aligned. Unfortunately, previous approaches to accelerate this task using the massively parallel processing capabilities of modern GPUs are limited by inefficient memory access schemes. This established the need for significantly faster solutions. We address this need by presenting gpuPairHMM -- a novel GPU-based parallelization scheme for the dynamic-programming based Pair-HMM forward algorithm based on wavefronts and warp-shuffles. It gains efficiency by minimizing both memory accesses and instructions. We show that our approach achieves close-to-peak performance on several generations of modern CUDA-enabled GPUs (Volta, Ampere, Ada, Hopper). It also outperforms prior implementations on GPUs, CPUs, and FPGAs by a factor of at least 8.6, 10.4, and 14.5, respectively. gpuPairHMM is publicly available at https://github.com/asbschmidt/gpuPairHMM.
Topology-aware Preemptive Scheduling for Co-located LLM Workloads
Venue: arXiv cs.DC 2024
Year: 2024

Hosting diverse large language model workloads in a unified resource pool through co-location is cost-effective. For example, long-running chat services generally follow diurnal traffic patterns, which inspire co-location of batch jobs to fulfill resource valleys between successive peaks, and thus to saturate resource allocation in cluster-wide scope. These heterogeneous workloads often have different business priorities, and therefore preemption can be leveraged for resource elasticity. However, workloads often have distinct topology preferences as well. The resources released by lower-priority instances may fail to meet the requirements of high-priority online services which are usually latency-sensitive. The root cause behind such mis-match is a lack of topology awareness of resource scheduler, especially during preemption. To bridge this gap, we develop a fine-grained topology-aware method for preemptive scheduling of hybrid workloads. The method ensures that the resources freed by preempted tasks adhere to the topological affinity needs of high-priority preemptors in a guaranteed or best-effort manner. This dynamic alignment significantly increases the efficiency of preemption and improves overall scheduled performance for LLM workloads by $55\%$.
Analysis of Generalized Hebbian Learning Algorithm for Neuromorphic Hardware Using Spinnaker
Venue: arXiv cs.NE 2024
Year: 2024

Neuromorphic computing, inspired by biological neural networks, has emerged as a promising approach for solving complex machine learning tasks with greater efficiency and lower power consumption. The integration of biologically plausible learning algorithms, such as the Generalized Hebbian Algorithm (GHA), is key to enhancing the performance of neuromorphic systems. In this paper, we explore the application of GHA in large-scale neuromorphic platforms, specifically SpiNNaker, a hardware designed to simulate large neural networks. Our results demonstrate significant improvements in classification accuracy, showcasing the potential of biologically inspired learning algorithms in advancing the field of neuromorphic computing.
Signaling and Social Learning in Swarms of Robots
Venue: arXiv cs.RO 2024
Year: 2024

This paper investigates the role of communication in improving coordination within robot swarms, focusing on a paradigm where learning and execution occur simultaneously in a decentralized manner. We highlight the role communication can play in addressing the credit assignment problem (individual contribution to the overall performance), and how it can be influenced by it. We propose a taxonomy of existing and future works on communication, focusing on information selection and physical abstraction as principal axes for classification: from low-level lossless compression with raw signal extraction and processing to high-level lossy compression with structured communication models. The paper reviews current research from evolutionary robotics, multi-agent (deep) reinforcement learning, language models, and biophysics models to outline the challenges and opportunities of communication in a collective of robots that continuously learn from one another through local message exchanges, illustrating a form of social learning.
The ethical landscape of robot-assisted surgery. A systematic review
Venue: arXiv cs.CY 2024
Year: 2024

Background: Robot-assisted surgery has been widely adopted in recent years. However, compared to other health technologies operating in close proximity to patients in a vulnerable state, ethical issues of robot-assisted surgery have received less attention. Against the background of increasing automation that are expected to raise new ethical issues, this systematic review aims to map the state of the ethical debate in this field. Methods: A protocol was registered in the international prospective register of systematic reviews (PROSPERO CRD42023397951). Medline via PubMed, EMBASE, CINHAL, Philosophers' Index, IEEE Xplorer, Web of Science (Core Collection), Scopus and Google Scholar were searched in January 2023. Screening, extraction, and analysis were conducted independently by two authors. A qualitative narrative synthesis was performed. Results: Out of 1,723 records, 66 records were included in the final dataset. Seven major strands of the ethical debate emerged during analysis. These include questions of harms and benefits, responsibility and control, professional-patient relationship, ethical issues in surgical training and learning, justice, translational questions, and economic considerations. Discussion: The identified themes testify to a broad range of different and differing ethical issues requiring careful deliberation and integration into the surgical ethos. Looking forward, we argue that a different perspective in addressing robotic surgical devices might be helpful to consider upcoming challenges of automation.
Can Highlighting Help GitHub Maintainers Track Security Fixes?
Venue: arXiv cs.CR 2024
Year: 2024

In recent years, the rapid growth of security vulnerabilities poses great challenges to tracing and managing them. For example, it was reported that the NVD database experienced significant delays due to the shortage of maintainers. Such delay creates challenges for third-party security personnel (e.g., administrators) to trace the information related to the CVE. To help security personnel trace a vulnerability patch, we build a retrieval system that automatically retrieves the patch in the repository. Inspired by existing work on explainable machine learning, we ask the following research question: can explanations help security maintainers make decisions in patch tracing? First, we investigate using LIME (a widely used explainable machine learning method) to highlight the rationale tokens in the commit message and code. In addition, we propose an explanation method called TfIdf-Highlight, which leverages the Tf-Idf statistics to select the most informative words in the repository and the dataset. We evaluate the effectiveness of highlighting using two experiments. First, we compare LIME and TfIdf-Highlight using a faithfulness score (i.e., sufficiency and comprehensiveness) defined for ranking. We find that TfIdf-Highlight significantly outperforms LIME's sufficiency scores by 15\% and slightly outperforms the comprehensiveness scores. Second, we conduct a blind human labeling experiment by asking the annotators to guess the patch under 3 settings (TfIdf-Highlight, LIME, and no highlight). We find that the helpfulness score for TfIdf-Highlight is higher than LIME while the labeling accuracies of LIME and TfIdf-Highlight are similar. Nevertheless, highlighting does not improve the accuracy over non-highlighting.
Do Captioning Metrics Reflect Music Semantic Alignment?
Venue: arXiv cs.SD 2024
Year: 2024

Music captioning has emerged as a promising task, fueled by the advent of advanced language generation models. However, the evaluation of music captioning relies heavily on traditional metrics such as BLEU, METEOR, and ROUGE which were developed for other domains, without proper justification for their use in this new field. We present cases where traditional metrics are vulnerable to syntactic changes, and show they do not correlate well with human judgments. By addressing these issues, we aim to emphasize the need for a critical reevaluation of how music captions are assessed.
Wideband Ultrasonic Acoustic Underwater Channels: Measurements and Characterization
Venue: arXiv cs.IT 2024
Year: 2024

In this work we present the results of a measurement campaign carried out in the Mediterranean sea aimed at characterizing the underwater acoustic channel in a wideband at ultrasonic frequencies centered at 80 kHz with a width of 96 kHz, covering two octaves from 32 to 128 kHz. So far, these type of wideband measurements are not found in the literature. Periodic orthogonal frequency division multiplexing (OFMD) sounding signals using Zadoff-Chu sequences have been specially designed for this purpose. The collected data has been post-processed to estimate the time-variant impulse and frequency responses and relevant parameters for system design like the time coherence, bandwidth coherence, delay spread and Doppler bandwidth. The statistical behavior of the channel gain random fluctuation has also been analyzed. This information has been extracted for both the global channel and each path separately. The wide bandwidth of the measurements have allowed the characterization of the channel in a scarcely explored ultrasonic band with an accuracy that is far beyond what is reported in previous works.
Advacheck at GenAI Detection Task 1: AI Detection Powered by Domain-Aware Multi-Tasking
Venue: arXiv cs.CL 2024
Year: 2024

The paper describes a system designed by Advacheck team to recognise machine-generated and human-written texts in the monolingual subtask of GenAI Detection Task 1 competition. Our developed system is a multi-task architecture with shared Transformer Encoder between several classification heads. One head is responsible for binary classification between human-written and machine-generated texts, while the other heads are auxiliary multiclass classifiers for texts of different domains from particular datasets. As multiclass heads were trained to distinguish the domains presented in the data, they provide a better understanding of the samples. This approach led us to achieve the first place in the official ranking with 83.07% macro F1-score on the test set and bypass the baseline by 10%. We further study obtained system through ablation, error and representation analyses, finding that multi-task learning outperforms single-task mode and simultaneous tasks form a cluster structure in embeddings space.
WoodYOLO: A Novel Object Detector for Wood Species Detection in Microscopic Images
Venue: arXiv cs.CV 2024
Year: 2024

Wood species identification plays a crucial role in various industries, from ensuring the legality of timber products to advancing ecological conservation efforts. This paper introduces WoodYOLO, a novel object detection algorithm specifically designed for microscopic wood fiber analysis. Our approach adapts the YOLO architecture to address the challenges posed by large, high-resolution microscopy images and the need for high recall in localization of the cell type of interest (vessel elements). Our results show that WoodYOLO significantly outperforms state-of-the-art models, achieving performance gains of 12.9% and 6.5% in F2 score over YOLOv10 and YOLOv7, respectively. This improvement in automated wood cell type localization capabilities contributes to enhancing regulatory compliance, supporting sustainable forestry practices, and promoting biodiversity conservation efforts globally.
An Internet Voting System Fatally Flawed in Creative New Ways
Venue: arXiv cs.CR 2024
Year: 2024

The recently published "MERGE" protocol is designed to be used in the prototype CAC-vote system. The voting kiosk and protocol transmit votes over the internet and then transmit voter-verifiable paper ballots through the mail. In the MERGE protocol, the votes transmitted over the internet are used to tabulate the results and determine the winners, but audits and recounts use the paper ballots that arrive in time. The enunciated motivation for the protocol is to allow (electronic) votes from overseas military voters to be included in preliminary results before a (paper) ballot is received from the voter. MERGE contains interesting ideas that are not inherently unsound; but to make the system trustworthy--to apply the MERGE protocol--would require major changes to the laws, practices, and technical and logistical abilities of U.S. election jurisdictions. The gap between theory and practice is large and unbridgeable for the foreseeable future. Promoters of this research project at DARPA, the agency that sponsored the research, should acknowledge that MERGE is internet voting (election results rely on votes transmitted over the internet except in the event of a full hand count) and refrain from claiming that it could be a component of trustworthy elections without sweeping changes to election law and election administration throughout the U.S.
Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrieval
Venue: arXiv cs.IR 2024
Year: 2024

The field of bioinformatics has seen significant progress, making the cross-modal text-molecule retrieval task increasingly vital. This task focuses on accurately retrieving molecule structures based on textual descriptions, by effectively aligning textual descriptions and molecules to assist researchers in identifying suitable molecular candidates. However, many existing approaches overlook the details inherent in molecule sub-structures. In this work, we introduce the Optimal TRansport-based Multi-grained Alignments model (ORMA), a novel approach that facilitates multi-grained alignments between textual descriptions and molecules. Our model features a text encoder and a molecule encoder. The text encoder processes textual descriptions to generate both token-level and sentence-level representations, while molecules are modeled as hierarchical heterogeneous graphs, encompassing atom, motif, and molecule nodes to extract representations at these three levels. A key innovation in ORMA is the application of Optimal Transport (OT) to align tokens with motifs, creating multi-token representations that integrate multiple token alignments with their corresponding motifs. Additionally, we employ contrastive learning to refine cross-modal alignments at three distinct scales: token-atom, multitoken-motif, and sentence-molecule, ensuring that the similarities between correctly matched text-molecule pairs are maximized while those of unmatched pairs are minimized. To our knowledge, this is the first attempt to explore alignments at both the motif and multi-token levels. Experimental results on the ChEBI-20 and PCdes datasets demonstrate that ORMA significantly outperforms existing state-of-the-art (SOTA) models.
ResLearn: Transformer-based Residual Learning for Metaverse Network Traffic Prediction
Venue: arXiv cs.AI 2024
Year: 2024

Our work proposes a comprehensive solution for predicting Metaverse network traffic, addressing the growing demand for intelligent resource management in eXtended Reality (XR) services. We first introduce a state-of-the-art testbed capturing a real-world dataset of virtual reality (VR), augmented reality (AR), and mixed reality (MR) traffic, made openly available for further research. To enhance prediction accuracy, we then propose a novel view-frame (VF) algorithm that accurately identifies video frames from traffic while ensuring privacy compliance, and we develop a Transformer-based progressive error-learning algorithm, referred to as ResLearn for Metaverse traffic prediction. ResLearn significantly improves time-series predictions by using fully connected neural networks to reduce errors, particularly during peak traffic, outperforming prior work by 99%. Our contributions offer Internet service providers (ISPs) robust tools for real-time network management to satisfy Quality of Service (QoS) and enhance user experience in the Metaverse.