OpenRaiser/Intern-Atlas huggingface dataset:
One Arrow, Many Targets: Probing LLMs for Multi-Attribute Controllable Text Summarization
Venue: arXiv cs.CL 2024
Year: 2024

Text summarization is a well-established task within the natural language processing (NLP) community. However, the focus on controllable summarization tailored to user requirements is gaining traction only recently. While several efforts explore controllability in text summarization, the investigation of Multi-Attribute Controllable Summarization (MACS) remains limited. This work addresses this gap by examining the MACS task through the lens of large language models (LLMs), using various learning paradigms, particularly low-rank adapters. We experiment with different popular adapter fine-tuning strategies to assess the effectiveness of the resulting models in retaining cues and patterns associated with multiple controllable attributes. Additionally, we propose and evaluate a novel hierarchical adapter fusion technique to integrate learnings from two distinct controllable attributes. Subsquently, we present our findings, discuss the challenges encountered, and suggest potential avenues for advancing the MACS task.
ProGen: Revisiting Probabilistic Spatial-Temporal Time Series Forecasting from a Continuous Generative Perspective Using Stochastic Differential Equations
Venue: arXiv cs.LG 2024
Year: 2024

Accurate forecasting of spatiotemporal data remains challenging due to complex spatial dependencies and temporal dynamics. The inherent uncertainty and variability in such data often render deterministic models insufficient, prompting a shift towards probabilistic approaches, where diffusion-based generative models have emerged as effective solutions. In this paper, we present ProGen, a novel framework for probabilistic spatiotemporal time series forecasting that leverages Stochastic Differential Equations (SDEs) and diffusion-based generative modeling techniques in the continuous domain. By integrating a novel denoising score model, graph neural networks, and a tailored SDE, ProGen provides a robust solution that effectively captures spatiotemporal dependencies while managing uncertainty. Our extensive experiments on four benchmark traffic datasets demonstrate that ProGen outperforms state-of-the-art deterministic and probabilistic models. This work contributes a continuous, diffusion-based generative approach to spatiotemporal forecasting, paving the way for future research in probabilistic modeling and stochastic processes.
Disaggregated Database Management Systems
Venue: arXiv cs.DB 2024
Year: 2024

Modern applications demand high performance and cost efficient database management systems (DBMSs). Their workloads may be diverse, ranging from online transaction processing to analytics and decision support. The cloud infrastructure enables disaggregation of monolithic DBMSs into components that facilitate software-hardware co-design. This is realized using pools of hardware resources, i.e., CPUs, GPUs, memory, FPGA, NVM, etc., connected using high-speed networks. This disaggregation trend is being adopted by cloud DBMSs because hardware re-provisioning can be achieved by simply invoking software APIs. Disaggregated DBMSs separate processing from storage, enabling each to scale elastically and independently. They may disaggregate compute usage based on functionality, e.g., compute needed for writes from compute needed for queries and compute needed for compaction. They may also use disaggregated memory, e.g., for intermediate results in a shuffle or for remote caching. The DBMS monitors the characteristics of a workload and dynamically assembles its components that are most efficient and cost effective for the workload. This paper is a summary of a panel session that discussed the capability, challenges, and opportunities of these emerging DBMSs and disaggregated hardware systems.
Improving Energy Efficiency in Manufacturing: A Novel Expert System Shell
Venue: arXiv cs.AI 2024
Year: 2024

Expert systems are effective tools for automatically identifying energy efficiency potentials in manufacturing, thereby contributing significantly to global climate targets. These systems analyze energy data, pinpoint inefficiencies, and recommend optimizations to reduce energy consumption. Beyond systematic approaches for developing expert systems, there is a pressing need for simple and rapid software implementation solutions. Expert system shells, which facilitate the swift development and deployment of expert systems, are crucial tools in this process. They provide a template that simplifies the creation and integration of expert systems into existing manufacturing processes. This paper provides a comprehensive comparison of existing expert system shells regarding their suitability for improving energy efficiency, highlighting significant gaps and limitations. To address these deficiencies, we introduce a novel expert system shell, implemented in Jupyter Notebook, that provides a flexible and easily integrable solution for expert system development.
PARIS: A Practical, Adaptive Trace-Fetching and Real-Time Malicious Behavior Detection System
Venue: arXiv cs.CR 2024
Year: 2024

The escalating sophistication of cyber-attacks and the widespread utilization of stealth tactics have led to significant security threats globally. Nevertheless, the existing static detection methods exhibit limited coverage, and traditional dynamic monitoring approaches encounter challenges in bypassing evasion techniques. Thus, it has become imperative to implement nuanced and dynamic analysis to achieve precise behavior detection in real time. There are two pressing concerns associated with current dynamic malware behavior detection solutions. Firstly, the collection and processing of data entail a significant amount of overhead, making it challenging to be employed for real-time detection on the end host. Secondly, these approaches tend to treat malware as a singular entity, thereby overlooking varied behaviors within one instance. To fill these gaps, we propose PARIS, an adaptive trace fetching, lightweight, real-time malicious behavior detection system. Specifically, we monitor malicious behavior with Event Tracing for Windows (ETW) and learn to selectively collect maliciousness-related APIs or call stacks, significantly reducing the data collection overhead. As a result, we can monitor a wider range of APIs and detect more intricate attack behavior. We implemented a prototype of PARIS and evaluated the system overhead, the accuracy of comparative behavior recognition, and the impact of different models and parameters. The result demonstrates that PARIS can reduce over 98.8% of data compared to the raw ETW trace and hence decreases the overhead on the host in terms of memory, bandwidth, and CPU usage with a similar detection accuracy to the baselines that suffer from the high overhead. Furthermore, a breakdown evaluation shows that 80% of the memory and bandwidth savings and a complete reduction in CPU usage can be attributed to our adaptive trace-fetching collector.
Towards a Knowledge Graph for Teaching Knowledge Graphs
Venue: arXiv cs.IR 2024
Year: 2024

This poster paper describes the ongoing research project for the creation of a use-case-driven Knowledge Graph resource tailored to the needs of teaching education in Knowledge Graphs (KGs). We gather resources related to KG courses from lectures offered by the Semantic Web community, with the help of the COST Action Distributed Knowledge Graphs and the interest group on KGs at The Alan Turing Institute. Our goal is to create a resource-focused KG with multiple interconnected semantic layers that interlink topics, courses, and materials with each lecturer. Our approach formulates a domain KG in teaching and relates it with multiple Personal KGs created for the lecturers.
RA-WEBs: Remote Attestation for WEB services
Venue: arXiv cs.CR 2024
Year: 2024

Data theft and leakage, caused by external adversaries and insiders, demonstrate the need for protecting user data. Trusted Execution Environments (TEEs) offer a promising solution by creating secure environments that protect data and code from such threats. The rise of confidential computing on cloud platforms facilitates the deployment of TEE-enabled server applications, which are expected to be widely adopted in web services such as privacy-preserving LLM inference and secure data logging. One key feature is Remote Attestation (RA), which enables integrity verification of a TEE. However, $\textit{compatibility}$ issues with RA verification arise as no browsers natively support this feature, making prior solutions cumbersome and risky. To address these challenges, we propose $\texttt{RA-WEBs}$ ($\textbf{R}$emote $\textbf{A}$ttestation for $\textbf{Web}$ $\textbf{s}$ervices), a novel RA protocol designed for high compatibility with the current web ecosystem. $\texttt{RA-WEBs}$ leverages established web mechanisms for immediate deployability, enabling RA verification on existing browsers. We conduct a comprehensive security analysis, demonstrating $\texttt{RA-WEBs}$'s resilience against various threats. Our contributions include the $\texttt{RA-WEBs}$ proposal, a proof-of-concept implementation, an in-depth security analysis, and publicly available code for reproducible research.
The Role of Domain Randomization in Training Diffusion Policies for Whole-Body Humanoid Control
Venue: arXiv cs.RO 2024
Year: 2024

Humanoids have the potential to be the ideal embodiment in environments designed for humans. Thanks to the structural similarity to the human body, they benefit from rich sources of demonstration data, e.g., collected via teleoperation, motion capture, or even using videos of humans performing tasks. However, distilling a policy from demonstrations is still a challenging problem. While Diffusion Policies (DPs) have shown impressive results in robotic manipulation, their applicability to locomotion and humanoid control remains underexplored. In this paper, we investigate how dataset diversity and size affect the performance of DPs for humanoid whole-body control. In a simulated IsaacGym environment, we generate synthetic demonstrations by training Adversarial Motion Prior (AMP) agents under various Domain Randomization (DR) conditions, and we compare DPs fitted to datasets of different size and diversity. Our findings show that, although DPs can achieve stable walking behavior, successful training of locomotion policies requires significantly larger and more diverse datasets compared to manipulation tasks, even in simple scenarios.
Can Large Language Model Predict Employee Attrition?
Venue: arXiv cs.LG 2024
Year: 2024

Employee attrition poses significant costs for organizations, with traditional statistical prediction methods often struggling to capture modern workforce complexities. Machine learning (ML) advancements offer more scalable and accurate solutions, but large language models (LLMs) introduce new potential in human resource management by interpreting nuanced employee communication and detecting subtle turnover cues. This study leverages the IBM HR Analytics Attrition dataset to compare the predictive accuracy and interpretability of a fine-tuned GPT-3.5 model against traditional ML classifiers, including Logistic Regression, k-Nearest Neighbors (KNN), Support Vector Machine (SVM), Decision Tree, Random Forest, AdaBoost, and XGBoost. While traditional models are easier to use and interpret, LLMs can reveal deeper patterns in employee behavior. Our findings show that the fine-tuned GPT-3.5 model outperforms traditional methods with a precision of 0.91, recall of 0.94, and an F1-score of 0.92, while the best traditional model, SVM, achieved an F1-score of 0.82, with Random Forest and XGBoost reaching 0.80. These results highlight GPT-3.5's ability to capture complex patterns in attrition risk, offering organizations improved insights for retention strategies and underscoring the value of LLMs in HR applications.
Signer-Optimal Multiple-Time Post-Quantum Hash-Based Signature for Heterogeneous IoT Systems
Venue: arXiv cs.CR 2024
Year: 2024

Heterogeneous Internet of Things (IoTs) harboring resource-limited devices like wearable sensors are essential for next-generation networks. Ensuring the authentication and integrity of security-sensitive telemetry in these applications is vital. Digital signatures provide scalable authentication with non-repudiation and public verifiability, making them essential tools for IoTs. However, emerging quantum computers necessitate post-quantum (PQ) secure solutions, yet existing NIST-PQC standards are costlier than their conventional counterparts and unsuitable for resource-limited IoTs. There is a significant need for lightweight PQ-secure digital signatures that respect the resource constraints of low-end IoTs. We propose a new multiple-time hash-based signature called Maximum Utilization Multiple HORS (MUM-HORS) that offers PQ security, short signatures, fast signing, and high key utilization for an extended lifespan. MUM-HORS addresses the inefficiency and key loss issues of HORS in offline/online settings by introducing compact key management data structures and optimized resistance to weak-message attacks. We tested MUM-HORS on two embedded platforms (ARM Cortex A-72 and 8-bit AVR ATmega2560) and commodity hardware. Our experiments confirm up to 40x better utilization with the same signing capacity (2^20 messages, 128-bit security) compared to multiple-time HORS while achieving 2x and 156-2463x faster signing than conventional-secure and NIST PQ-secure schemes, respectively, on an ARM Cortex. These features make MUM-HORS ideal multiple-time PQ-secure signature for heterogeneous IoTs.
Hyperbox Mixture Regression for Process Performance Prediction in Antibody Production
Venue: arXiv cs.LG 2024
Year: 2024

This paper addresses the challenges of predicting bioprocess performance, particularly in monoclonal antibody (mAb) production, where conventional statistical methods often fall short due to time-series data's complexity and high dimensionality. We propose a novel Hyperbox Mixture Regression (HMR) model which employs hyperbox-based input space partitioning to enhance predictive accuracy while managing uncertainty inherent in bioprocess data. The HMR model is designed to dynamically generate hyperboxes for input samples in a single-pass process, thereby improving learning speed and reducing computational complexity. Our experimental study utilizes a dataset that contains 106 bioreactors. This study evaluates the model's performance in predicting critical quality attributes in monoclonal antibody manufacturing over a 15-day cultivation period. The results demonstrate that the HMR model outperforms comparable approximators in accuracy and learning speed and maintains interpretability and robustness under uncertain conditions. These findings underscore the potential of HMR as a powerful tool for enhancing predictive analytics in bioprocessing applications.
HiMemFormer: Hierarchical Memory-Aware Transformer for Multi-Agent Action Anticipation
Venue: arXiv cs.CV 2024
Year: 2024

Understanding and predicting human actions has been a long-standing challenge and is a crucial measure of perception in robotics AI. While significant progress has been made in anticipating the future actions of individual agents, prior work has largely overlooked a key aspect of real-world human activity -- interactions. To address this gap in human-like forecasting within multi-agent environments, we present the Hierarchical Memory-Aware Transformer (HiMemFormer), a transformer-based model for online multi-agent action anticipation. HiMemFormer integrates and distributes global memory that captures joint historical information across all agents through a transformer framework, with a hierarchical local memory decoder that interprets agent-specific features based on these global representations using a coarse-to-fine strategy. In contrast to previous approaches, HiMemFormer uniquely hierarchically applies the global context with agent-specific preferences to avoid noisy or redundant information in multi-agent action anticipation. Extensive experiments on various multi-agent scenarios demonstrate the significant performance of HiMemFormer, compared with other state-of-the-art methods.
Two-Timescale Model Caching and Resource Allocation for Edge-Enabled AI-Generated Content Services
Venue: arXiv cs.LG 2024
Year: 2024

Generative AI (GenAI) has emerged as a transformative technology, enabling customized and personalized AI-generated content (AIGC) services. In this paper, we address challenges of edge-enabled AIGC service provisioning, which remain underexplored in the literature. These services require executing GenAI models with billions of parameters, posing significant obstacles to resource-limited wireless edge. We subsequently introduce the formulation of joint model caching and resource allocation for AIGC services to balance a trade-off between AIGC quality and latency metrics. We obtain mathematical relationships of these metrics with the computational resources required by GenAI models via experimentation. Afterward, we decompose the formulation into a model caching subproblem on a long-timescale and a resource allocation subproblem on a short-timescale. Since the variables to be solved are discrete and continuous, respectively, we leverage a double deep Q-network (DDQN) algorithm to solve the former subproblem and propose a diffusion-based deep deterministic policy gradient (D3PG) algorithm to solve the latter. The proposed D3PG algorithm makes an innovative use of diffusion models as the actor network to determine optimal resource allocation decisions. Consequently, we integrate these two learning methods within the overarching two-timescale deep reinforcement learning (T2DRL) algorithm, the performance of which is studied through comparative numerical simulations.
Mao: Machine learning approach for NUMA optimization in Warehouse Scale Computers
Venue: arXiv cs.DC 2024
Year: 2024

Non-Uniform Memory Access (NUMA) architecture imposes numerous performance challenges to today's cloud workloads. Due to the complexity and the massive scale of modern warehouse-scale computers (WSCs), a lot of efforts need to be done to improve the memory access locality on the NUMA architecture. In Baidu, we have found that NUMA optimization has significant performance benefit to the major workloads like Search and Feed (Baidu's recommendation system). But how to conduct NUMA optimization within the large scale cluster brings a lot of subtle complexities and workload-specific scenario optimizations. In this paper, we will present a production environment deployed solution in Baidu called MAP (Memory Access Optimizer) that helps improve the memory access locality for Baidu's various workloads. MAO includes an online module and an offline module. The online module is responsible for the online monitoring, dynamic NUMA node binding and runtime optimization. Meanwhile the offline workload characterization module will proceed with the data analysis and resource-sensitivity module training. We also proposed a new performance model called "NUMA Sensitivity model" to address the impact of remote memory access to workload performance and projection of the potential performance improvements via NUMA optimization for a specific workload. Based on continuous data collected from online monitoring, this model is proved to be working properly in MAO. As of today, we have successfully deployed MAO to more than one hundred thousand servers. In our Feed product, we have achieved 12.1% average latency improvements and 9.8% CPU resource saving.
The Fairness of Maximum Nash Social Welfare Under Matroid Constraints and Beyond
Venue: arXiv cs.GT 2024
Year: 2024

We study the problem of fair allocation of a set of indivisible items among agents with additive valuations, under matroid constraints and two generalizations: $p$-extendible system and independence system constraints. The objective is to find fair and efficient allocations in which the subset of items assigned to every agent satisfies the given constraint. We focus on a common fairness notion of envy-freeness up to one item (EF1) and a well-known efficient (and fair) notion of the maximum Nash social welfare (Max-NSW). By using properties of matroids, we demonstrate that the Max-NSW allocation, implying Pareto optimality (PO), achieves a tight $1/2$-EF1 under matroid constraints. This result resolves an open question proposed in prior literature [26]. In particular, if agents have 2-valued ($\{1, a\}$) valuations, we prove that the Max-NSW allocation admits $\max\{1/a^2, 1/2\}$-EF1 and PO. Under strongly $p$-extendible system constraints, we show that the Max-NSW allocation guarantees $\max\{1/p, 1/4\}$-EF1 and PO for identical binary valuations. Indeed, the approximation of $1/4$ is the ratio for independence system constraints and additive valuations. Additionally, for lexicographic preferences, we study possibly feasible allocations other than Max-NSW admitting exactly EF1 and PO under the above constraints.
Polar R-CNN: End-to-End Lane Detection with Fewer Anchors
Venue: arXiv cs.CV 2024
Year: 2024

Lane detection is a critical and challenging task in autonomous driving, particularly in real-world scenarios where traffic lanes can be slender, lengthy, and often obscured by other vehicles, complicating detection efforts. Existing anchor-based methods typically rely on prior lane anchors to extract features and subsequently refine the location and shape of lanes. While these methods achieve high performance, manually setting prior anchors is cumbersome, and ensuring sufficient coverage across diverse datasets often requires a large amount of dense anchors. Furthermore, the use of Non-Maximum Suppression (NMS) to eliminate redundant predictions complicates real-world deployment and may underperform in complex scenarios. In this paper, we propose Polar R-CNN, an end-to-end anchor-based method for lane detection. By incorporating both local and global polar coordinate systems, Polar R-CNN facilitates flexible anchor proposals and significantly reduces the number of anchors required without compromising performance.Additionally, we introduce a triplet head with heuristic structure that supports NMS-free paradigm, enhancing deployment efficiency and performance in scenarios with dense lanes.Our method achieves competitive results on five popular lane detection benchmarks--Tusimple, CULane,LLAMAS, CurveLanes, and DL-Rai--while maintaining a lightweight design and straightforward structure. Our source code is available at https://github.com/ShqWW/PolarRCNN.