OpenRaiser/Intern-Atlas huggingface dataset:
A New Class of Linear Codes
Venue: arXiv cs.IT 2024
Year: 2024

Let $n$ be a prime power, $r$ be a prime with $r\mid n-1$, and $\varepsilon\in (0,1/2)$. Using the theory of multiplicative character sums and superelliptic curves, we construct new codes over $\mathbb F_r$ having length $n$, relative distance $(r-1)/r+O(n^{-\varepsilon})$ and rate $n^{-1/2-\varepsilon}$. When $r=2$, our binary codes have exponential size when compared to all previously known families of linear and non-linear codes with relative distance asymptotic to $1/2$, such as Delsarte--Goethals codes. Moreover, concatenating with a Reed--Solomon code gives a family of codes of length $n$, asymptotic distance $1/2$ and rate $Ω(n^{-\varepsilon})$ for any fixed small $\varepsilon>0$, improving our initial construction. Such rate is also asymptotically better than the one by Kschischang and Tasbihi obtained by concatenating a Reed--Solomon with Reed--Muller, improving by a factor in $Ω(n^{1/2}/\log(n))$.
Playing the MEV Game on a First-Come-First-Served Blockchain
Venue: arXiv cs.CR 2024
Year: 2024

Maximal Extractable Value (MEV) searching has gained prominence on the Ethereum blockchain since the surge in Decentralized Finance activities. In Ethereum, MEV extraction primarily hinges on fee payments to block proposers. However, in First-Come-First-Served (FCFS) blockchain networks, the focus shifts to latency optimizations, akin to High-Frequency Trading in Traditional Finance. This paper illustrates the dynamics of the MEV extraction game in an FCFS network, specifically Algorand. We introduce an arbitrage detection algorithm tailored to the unique time constraints of FCFS networks and assess its effectiveness. Additionally, our experiments investigate potential optimizations in Algorand's network layer to secure optimal execution positions. Our analysis reveals that while the states of relevant trading pools are updated approximately every six blocks on median, pursuing MEV at the block state level is not viable on Algorand, as arbitrage opportunities are typically executed within the blocks they appear. Our algorithm's performance under varying time constraints underscores the importance of timing in arbitrage discovery. Furthermore, our network-level experiments identify critical transaction prioritization strategies for Algorand's FCFS network. Key among these is reducing latency in connections with relays that are well-connected to high-staked proposers.
Carrying over algorithm in transformers
Venue: arXiv cs.LG 2024
Year: 2024

Addition is perhaps one of the simplest arithmetic tasks one can think of and is usually performed using the carrying over algorithm. This algorithm consists of two tasks: adding digits in the same position and carrying over a one whenever necessary. We study how transformer models implement this algorithm and how the two aforementioned tasks are allocated to different parts of the network. We first focus on two-layer encoder-only models and show that the carrying over algorithm is implemented in a modular fashion. The first layer is mostly responsible for adding digits in the same position. The second layer first decides, in the attention, which positions need a carried one or not, and then performs the carrying of the one in the final MLP. We provide a simple way of precisely identifying which neurons are responsible for that task. This implementation of the carrying over algorithm occurs across a range of hyperparameters for two as well as three-layer models. For small decoder-only models, we observe the same implementation and provide suggestive evidence for its existence in three 7B large language models.
Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities
Venue: arXiv cs.CV 2024
Year: 2024

The rise of large foundation models, trained on extensive datasets, is revolutionizing the field of AI. Models such as SAM, DALL-E2, and GPT-4 showcase their adaptability by extracting intricate patterns and performing effectively across diverse tasks, thereby serving as potent building blocks for a wide range of AI applications. Autonomous driving, a vibrant front in AI applications, remains challenged by the lack of dedicated vision foundation models (VFMs). The scarcity of comprehensive training data, the need for multi-sensor integration, and the diverse task-specific architectures pose significant obstacles to the development of VFMs in this field. This paper delves into the critical challenge of forging VFMs tailored specifically for autonomous driving, while also outlining future directions. Through a systematic analysis of over 250 papers, we dissect essential techniques for VFM development, including data preparation, pre-training strategies, and downstream task adaptation. Moreover, we explore key advancements such as NeRF, diffusion models, 3D Gaussian Splatting, and world models, presenting a comprehensive roadmap for future research. To empower researchers, we have built and maintained https://github.com/zhanghm1995/Forge_VFM4AD, an open-access repository constantly updated with the latest advancements in forging VFMs for autonomous driving.
TrajVis: a visual clinical decision support system to translate artificial intelligence trajectory models in the precision management of chronic kidney disease
Venue: arXiv cs.HC 2024
Year: 2024

Objective: Our objective is to develop and validate TrajVis, an interactive tool that assists clinicians in using artificial intelligence (AI) models to leverage patients' longitudinal electronic medical records (EMR) for personalized precision management of chronic disease progression. Methods: We first perform requirement analysis with clinicians and data scientists to determine the visual analytics tasks of the TrajVis system as well as its design and functionalities. A graph AI model for chronic kidney disease (CKD) trajectory inference named DEPOT is used for system development and demonstration. TrajVis is implemented as a full-stack web application with synthetic EMR data derived from the Atrium Health Wake Forest Baptist Translational Data Warehouse and the Indiana Network for Patient Care research database. A case study with a nephrologist and a user experience survey of clinicians and data scientists are conducted to evaluate the TrajVis system. Results: The TrajVis clinical information system is composed of four panels: the Patient View for demographic and clinical information, the Trajectory View to visualize the DEPOT-derived CKD trajectories in latent space, the Clinical Indicator View to elucidate longitudinal patterns of clinical features and interpret DEPOT predictions, and the Analysis View to demonstrate personal CKD progression trajectories. System evaluations suggest that TrajVis supports clinicians in summarizing clinical data, identifying individualized risk predictors, and visualizing patient disease progression trajectories, overcoming the barriers of AI implementation in healthcare. Conclusion: TrajVis bridges the gap between the fast-growing AI/ML modeling and the clinical use of such models for personalized and precision management of chronic diseases.
Predicting Next Useful Location With Context-Awareness: The State-Of-The-Art
Venue: arXiv cs.LG 2024
Year: 2024

Predicting the future location of mobile objects reinforces location-aware services with proactive intelligence and helps businesses and decision-makers with better planning and near real-time scheduling in different applications such as traffic congestion control, location-aware advertisements, and monitoring public health and well-being. The recent developments in the smartphone and location sensors technology and the prevalence of using location-based social networks alongside the improvements in artificial intelligence and machine learning techniques provide an excellent opportunity to exploit massive amounts of historical and real-time contextual information to recognise mobility patterns and achieve more accurate and intelligent predictions. This survey provides a comprehensive overview of the next useful location prediction problem with context-awareness. First, we explain the concepts of context and context-awareness and define the next location prediction problem. Then we analyse nearly thirty studies in this field concerning the prediction method, the challenges addressed, the datasets and metrics used for training and evaluating the model, and the types of context incorporated. Finally, we discuss the advantages and disadvantages of different approaches, focusing on the usefulness of the predicted location and identifying the open challenges and future work on this subject by introducing two potential use cases of next location prediction in the automotive industry.
A Study on Training and Developing Large Language Models for Behavior Tree Generation
Venue: arXiv cs.CL 2024
Year: 2024

This paper presents an innovative exploration of the application potential of large language models (LLM) in addressing the challenging task of automatically generating behavior trees (BTs) for complex tasks. The conventional manual BT generation method is inefficient and heavily reliant on domain expertise. On the other hand, existing automatic BT generation technologies encounter bottlenecks related to task complexity, model adaptability, and reliability. In order to overcome these challenges, we propose a novel methodology that leverages the robust representation and reasoning abilities of LLMs. The core contribution of this paper lies in the design of a BT generation framework based on LLM, which encompasses the entire process, from data synthesis and model training to application developing and data verification. Synthetic data is introduced to train the BT generation model (BTGen model), enhancing its understanding and adaptability to various complex tasks, thereby significantly improving its overall performance. In order to ensure the effectiveness and executability of the generated BTs, we emphasize the importance of data verification and introduce a multilevel verification strategy. Additionally, we explore a range of agent design and development schemes with LLM as the central element. We hope that the work in this paper may provide a reference for the researchers who are interested in BT generation based on LLMs.
Octopus: A Fair Packet Delivery Service
Venue: arXiv cs.NI 2024
Year: 2024

The packet delivery fairness is critical in many applications in the cloud, such as exchange systems, consensus protocols, and online gaming applications. However, due to nonidentical and dynamic packet forwarding paths, as well as many in-network queuing delays, supporting packet delivery fairness is challenging in a shared compute environment. In this paper, we present Octopus, the first general fair packet delivery service to achieve packet arrival time variations smaller than tens of nanoseconds, with the existence of latency variations in the network. The key ideas of Octopus to support such good fairness come from repurposing hardware traffic shaping capabilities in modern NICs, and deploying agents at local SmartNICs to minimize latency variations from packet forwarding. Evaluation results show that Octopus has less than 40 ns unfairness for up to 99.97\% multicast packets.
Game Rewards Vulnerabilities: Software Vulnerability Detection with Zero-Sum Game and Prototype Learning
Venue: arXiv cs.SE 2024
Year: 2024

Recent years have witnessed a growing focus on automated software vulnerability detection. Notably, deep learning (DL)-based methods, which employ source code for the implicit acquisition of vulnerability patterns, have demonstrated superior performance compared to other approaches. However, the DL-based approaches are still hard to capture the vulnerability-related information from the whole code snippet, since the vulnerable parts usually account for only a small proportion. As evidenced by our experiments, the approaches tend to excessively emphasize semantic information, potentially leading to limited vulnerability detection performance in practical scenarios. First, they cannot well distinguish between the code snippets before (i.e., vulnerable code) and after (i.e., non-vulnerable code) developers' fixes due to the minimal code changes. Besides, substituting user-defined identifiers with placeholders (e.g., "VAR1" and "FUN1") in obvious performance degradation at up to 14.53% with respect to the F1 score. To mitigate these issues, we propose to leverage the vulnerable and corresponding fixed code snippets, in which the minimal changes can provide hints about semantic-agnostic features for vulnerability detection. In this paper, we propose a software vulneRability dEteCtion framework with zerO-sum game and prototype learNing, named RECON. In RECON, we propose a zero-sum game construction module. Distinguishing the vulnerable code from the corresponding fixed code is regarded as one player (i.e. Calibrator), while the conventional vulnerability detection is another player (i.e. Detector) in the zero-sum game. The goal is to capture the semantic-agnostic features of the first player for enhancing the second player's performance for vulnerability detection. Experiments on the public benchmark dataset show that RECON outperforms the state-of-the-art baseline by 6.29% in F1 score.
Three classes of propagation rules for generalized Reed-Solomon codes and their applications to EAQECCs
Venue: arXiv cs.IT 2024
Year: 2024

In this paper, we study the Hermitian hulls of generalized Reed-Solomon (GRS) codes over finite fields. For a given class of GRS codes, by extending the length, increasing the dimension, and extending the length and increasing the dimension at the same time, we obtain three classes of GRS codes with Hermitian hulls of arbitrary dimensions. Furthermore, based on some known $q^2$-ary Hermitian self-orthogonal GRS codes with dimension $q-1$, we obtain several classes of $q^2$-ary maximum distance separable (MDS) codes with Hermitian hulls of arbitrary dimensions. It is worth noting that the dimension of these MDS codes can be taken from $q$ to $\frac{n}{2}$, and the parameters of these MDS codes can be more flexible by propagation rules. As an application, we derive three new propagation rules for MDS entanglement-assisted quantum error correction codes (EAQECCs) constructed from GRS codes. Then, from the presently known GRS codes with Hermitian hulls, we can directly obtain many MDS EAQECCs with more flexible parameters. Finally, we present several new classes of (MDS) EAQECCs with flexible parameters, and the distance of these codes can be taken from $q+1$ to $\frac{n+2}{2}$.
Experimental Analysis of Type II Singularities and Assembly Change Points in a 3UPS+RPU Parallel Robot
Venue: arXiv cs.RO 2024
Year: 2024

Parallel robots (PRs) have singular configurations where the robot gains at least one degree of freedom and loses control. Theoretically, such singularity occurs when the Forward Jacobian-matrix determinant becomes zero (Type II). However, actual PRs could lose control owing to Type II singularities for determinant values near zero, but not zero, because manufacturing tolerances introduce errors that are complex to model due to their low repeatability. Thus, using an actual 3UPS+RPU PR, this paper presents three contributions: i) a proximity detection index for Type II singularities based on the angle between two Output Twist Screws. The index can identify which kinematic chains contribute to the singularity. ii) an experimental benchmark to study Type II singularities. iii) PR configurations where the proposed index is zero and the Forward Jacobian determinant is not. In this last configuration, the findings show that the actual robot is unable to handle external actions applied to the PR.
Enhancing Wind Speed and Wind Power Forecasting Using Shape-Wise Feature Engineering: A Novel Approach for Improved Accuracy and Robustness
Venue: arXiv cs.LG 2024
Year: 2024

Accurate prediction of wind speed and power is vital for enhancing the efficiency of wind energy systems. Numerous solutions have been implemented to date, demonstrating their potential to improve forecasting. Among these, deep learning is perceived as a revolutionary approach in the field. However, despite their effectiveness, the noise present in the collected data remains a significant challenge. This noise has the potential to diminish the performance of these algorithms, leading to inaccurate predictions. In response to this, this study explores a novel feature engineering approach. This approach involves altering the data input shape in both Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) and Autoregressive models for various forecasting horizons. The results reveal substantial enhancements in model resilience against noise resulting from step increases in data. The approach could achieve an impressive 83% accuracy in predicting unseen data up to the 24th steps. Furthermore, this method consistently provides high accuracy for short, mid, and long-term forecasts, outperforming the performance of individual models. These findings pave the way for further research on noise reduction strategies at different forecasting horizons through shape-wise feature engineering.
Ranking Heterogeneous Search Result Pages using the Interactive Probability Ranking Principle
Venue: arXiv cs.IR 2024
Year: 2024

The Probability Ranking Principle (PRP) ranks search results based on their expected utility derived solely from document contents, often overlooking the nuances of presentation and user interaction. However, with the evolution of Search Engine Result Pages (SERPs), now comprising a variety of result cards, the manner in which these results are presented is pivotal in influencing user engagement and satisfaction. This shift prompts the question: How does the PRP and its user-centric counterpart, the Interactive Probability Ranking Principle (iPRP), compare in the context of these heterogeneous SERPs? Our study draws a comparison between the PRP and the iPRP, revealing significant differences in their output. The iPRP, accounting for item-specific costs and interaction probabilities to determine the ``Expected Perceived Utility" (EPU), yields different result orderings compared to the PRP. We evaluate the effect of the EPU on the ordering of results by observing changes in the ranking within a heterogeneous SERP compared to the traditional ``ten blue links''. We find that changing the presentation affects the ranking of items according to the (iPRP) by up to 48\% (with respect to DCG, TBG and RBO) in ad-hoc search tasks on the TREC WaPo Collection. This work suggests that the iPRP should be employed when ranking heterogeneous SERPs to provide a user-centric ranking that adapts the ordering based on the presentation and user engagement.
ULT-model: Towards a one-legged unified locomotion template model for forward hopping with an upright trunk
Venue: arXiv cs.RO 2024
Year: 2024

While many advancements have been made in the development of template models for describing upright-trunk locomotion, the majority of the effort has been focused on the stance phase. In this paper, we develop a new compact dynamic model as a first step toward a fully unified locomotion template model (ULT-model) of an upright-trunk forward hopping system, which will also require a unified control law in the next step. We demonstrate that all locomotion subfunctions are enabled by adding just a point foot mass and a parallel leg actuator to the well-known trunk SLIP model and that a stable limit cycle can be achieved. This brings us closer toward the ultimate goal of enabling closed-loop dynamics for anchor matching and thus achieving simple, efficient, robust and stable upright-trunk gait control, as observed in biological systems.
Application of LLM Agents in Recruitment: A Novel Framework for Resume Screening
Venue: arXiv cs.CL 2024
Year: 2024

The automation of resume screening is a crucial aspect of the recruitment process in organizations. Automated resume screening systems often encompass a range of natural language processing (NLP) tasks. This paper introduces a novel Large Language Models (LLMs) based agent framework for resume screening, aimed at enhancing efficiency and time management in recruitment processes. Our framework is distinct in its ability to efficiently summarize and grade each resume from a large dataset. Moreover, it utilizes LLM agents for decision-making. To evaluate our framework, we constructed a dataset from actual resumes and simulated a resume screening process. Subsequently, the outcomes of the simulation experiment were compared and subjected to detailed analysis. The results demonstrate that our automated resume screening framework is 11 times faster than traditional manual methods. Furthermore, by fine-tuning the LLMs, we observed a significant improvement in the F1 score, reaching 87.73\%, during the resume sentence classification phase. In the resume summarization and grading phase, our fine-tuned model surpassed the baseline performance of the GPT-3.5 model. Analysis of the decision-making efficacy of the LLM agents in the final offer stage further underscores the potential of LLM agents in transforming resume screening processes.
OpenDPD: An Open-Source End-to-End Learning & Benchmarking Framework for Wideband Power Amplifier Modeling and Digital Pre-Distortion
Venue: arXiv cs.LG 2024
Year: 2024

With the rise in communication capacity, deep neural networks (DNN) for digital pre-distortion (DPD) to correct non-linearity in wideband power amplifiers (PAs) have become prominent. Yet, there is a void in open-source and measurement-setup-independent platforms for fast DPD exploration and objective DPD model comparison. This paper presents an open-source framework, OpenDPD, crafted in PyTorch, with an associated dataset for PA modeling and DPD learning. We introduce a Dense Gated Recurrent Unit (DGRU)-DPD, trained via a novel end-to-end learning architecture, outperforming previous DPD models on a digital PA (DPA) in the new digital transmitter (DTX) architecture with unconventional transfer characteristics compared to analog PAs. Measurements show our DGRU-DPD achieves an ACPR of -44.69/-44.47 dBc and an EVM of -35.22 dB for 200 MHz OFDM signals. OpenDPD code, datasets, and documentation are publicly available at https://github.com/lab-emi/OpenDPD.
Hallucination Detection and Hallucination Mitigation: An Investigation
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs), including ChatGPT, Bard, and Llama, have achieved remarkable successes over the last two years in a range of different applications. In spite of these successes, there exist concerns that limit the wide application of LLMs. A key problem is the problem of hallucination. Hallucination refers to the fact that in addition to correct responses, LLMs can also generate seemingly correct but factually incorrect responses. This report aims to present a comprehensive review of the current literature on both hallucination detection and hallucination mitigation. We hope that this report can serve as a good reference for both engineers and researchers who are interested in LLMs and applying them to real world tasks.