OpenRaiser/Intern-Atlas huggingface dataset:
BalanceDN: Load-Balancing Allocation of Interest for Fast Discovery in Content Centric Networks
Venue: arXiv cs.NI 2024
Year: 2024

In Named Data Networking (NDN), data is identified by unique names instead of IP addresses, and routers use the names of the content to forward Interest packets towards the producers of the requested content. However, the current content search mechanism in NDN is complex and slow. This mechanism not only creates congestion but also hinders practical deployment due to its slowness and cumbersome nature. To address this issue, we propose a methodology, called BalanceDN, that distributes content through the network such that sought content can be found quickly. BalanceDN uses a distributed allocation of resolvers as those used by the domain name system but differs in how content is distributed. Our approach avoids flooding the network with pending interest requests and also eliminates the need for blind search when the location of content is unknown. We tested our approach on ndnSIM; a simulation platform for NDN. The results show that the proposed routing scheme utilizes far fewer network resources compared to the NDN network when retrieving content. The proposed scheme accomplishes this performance gain by leveraging a load-balanced hashing mechanism to distribute and locate the name of the content on the distributed nameserver lookup service nodes.
MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations
Venue: arXiv cs.CL 2024
Year: 2024

Math word problems are critical K-8 educational tools, but writing them is time consuming and requires extensive expertise. To be educational, problems must be solvable, have accurate answers, and, most importantly, be educationally appropriate. We propose that language models have potential to support K-8 math education by automatically generating word problems. However, evaluating educational appropriateness is hard to quantify. We fill this gap by having teachers evaluate problems generated by LLMs, who find existing models and data often fail to be educationally appropriate. We then explore automatically generating educational word problems, ultimately using our expert annotations to finetune a 70B language model. Our model, MATHWELL, is the first K-8 word problem generator targeted at educational appropriateness. Further expert studies find MATHWELL generates problems far more solvable, accurate, and appropriate than public models. MATHWELL also matches GPT-4's problem quality while attaining more appropriate reading levels for K-8 students and avoiding generating harmful questions.
Differentially Private Bayesian Persuasion
Venue: arXiv cs.GT 2024
Year: 2024

The tension between persuasion and privacy preservation is common in real-world settings. Online platforms should protect the privacy of web users whose data they collect, even as they seek to disclose information about these data to selling advertising spaces. Similarly, hospitals may share patient data to attract research investments with the obligation to preserve patients' privacy. To deal with these issues, we develop a framework to study Bayesian persuasion under differential privacy constraints, where the sender must design an optimal signaling scheme for persuasion while guaranteeing the privacy of each agent's private information in the database. To understand how privacy constraints affect information disclosure, we explore two perspectives within Bayesian persuasion: one views the mechanism as releasing a posterior about the private data, while the other views it as sending an action recommendation. The posterior-based formulation helps consider privacy-utility tradeoffs, quantifying how the tightness of privacy constraints impacts the sender's optimal utility. For any instance in a common utility function family and a wide range of privacy levels, a significant constant utility gap can be found between any two of the three conditions: $ε$-differential privacy constraint, relaxation $(ε,δ)$-differential privacy constraint, and no privacy constraint. We further geometrically characterize optimal signaling schemes under different types of constraints ($ε$-differential privacy, $(ε,δ)$-differential privacy and Renyi differential privacy), all of which can be seen as finding concave hulls in constrained posterior regions. Meanwhile, by taking the action-based view of persuasion, we provide polynomial-time algorithms for computing optimal differentially private signaling schemes, as long as a mild homogeneous condition is met.
Symbolic Listings as Computation
Venue: arXiv cs.DM 2024
Year: 2024

We propose an algebraic model of computation which formally relates symbolic listings, complexity of Boolean functions, and low depth arithmetic circuit complexity. In this model algorithms are arithmetic formula expressing symbolic listings of YES instances of Boolean functions, and computation is executed via partial differential operators. We consider the Chow rank of an arithmetic formula as a measure of complexity and establish the Chow rank of multilinear polynomials with totally non-overlapping monomial support. We also provide Chow rank non-decreasing transformations from sets of graphs to sets of functional graphs.
ESFL: Efficient Split Federated Learning over Resource-Constrained Heterogeneous Wireless Devices
Venue: arXiv cs.LG 2024
Year: 2024

Federated learning (FL) allows multiple parties (distributed devices) to train a machine learning model without sharing raw data. How to effectively and efficiently utilize the resources on devices and the central server is a highly interesting yet challenging problem. In this paper, we propose an efficient split federated learning algorithm (ESFL) to take full advantage of the powerful computing capabilities at a central server under a split federated learning framework with heterogeneous end devices (EDs). By splitting the model into different submodels between the server and EDs, our approach jointly optimizes user-side workload and server-side computing resource allocation by considering users' heterogeneity. We formulate the whole optimization problem as a mixed-integer non-linear program, which is an NP-hard problem, and develop an iterative approach to obtain an approximate solution efficiently. Extensive simulations have been conducted to validate the significantly increased efficiency of our ESFL approach compared with standard federated learning, split learning, and splitfed learning.
Explainable Contrastive and Cost-Sensitive Learning for Cervical Cancer Classification
Venue: arXiv cs.CV 2024
Year: 2024

This paper proposes an efficient system for classifying cervical cancer cells using pre-trained convolutional neural networks (CNNs). We first fine-tune five pre-trained CNNs and minimize the overall cost of misclassification by prioritizing accuracy for certain classes that have higher associated costs or importance. To further enhance the performance of the models, supervised contrastive learning is included to make the models more adept at capturing important features and patterns. Extensive experimentation are conducted to evaluate the proposed system on the SIPaKMeD dataset. The experimental results demonstrate the effectiveness of the developed system, achieving an accuracy of 97.29%. To make our system more trustworthy, we have employed several explainable AI techniques to interpret how the models reached a specific decision. The implementation of the system can be found at - https://github.com/isha-67/CervicalCancerStudy.
High-performance finite elements with MFEM
Venue: arXiv cs.MS 2024
Year: 2024

The MFEM (Modular Finite Element Methods) library is a high-performance C++ library for finite element discretizations. MFEM supports numerous types of finite element methods and is the discretization engine powering many computational physics and engineering applications across a number of domains. This paper describes some of the recent research and development in MFEM, focusing on performance portability across leadership-class supercomputing facilities, including exascale supercomputers, as well as new capabilities and functionality, enabling a wider range of applications. Much of this work was undertaken as part of the Department of Energy's Exascale Computing Project (ECP) in collaboration with the Center for Efficient Exascale Discretizations (CEED).
Capsule Endoscopy Image Enhancement for Small Intestinal Villi Clarity
Venue: arXiv cs.CV 2024
Year: 2024

This paper presents, for the first time, an image enhancement methodology designed to enhance the clarity of small intestinal villi in Wireless Capsule Endoscopy (WCE) images. This method first separates the low-frequency and high-frequency components of small intestinal villi images using guided filtering. Subsequently, an adaptive light gain factor is generated based on the low-frequency component, and an adaptive gradient gain factor is derived from the convolution results of the Laplacian operator in different regions of small intestinal villi images. The obtained light gain factor and gradient gain factor are then combined to enhance the high-frequency components. Finally, the enhanced high-frequency component is fused with the original image to achieve adaptive sharpening of the edges of WCE small intestinal villi images. The experiments affirm that, compared to established WCE image enhancement methods, our approach not only accentuates the edge details of WCE small intestine villi images but also skillfully suppresses noise amplification, thereby preventing the occurrence of edge overshooting.
An Empirical Study of Challenges in Machine Learning Asset Management
Venue: arXiv cs.SE 2024
Year: 2024

In machine learning (ML), efficient asset management, including ML models, datasets, algorithms, and tools, is vital for resource optimization, consistent performance, and a streamlined development lifecycle. This enables quicker iterations, adaptability, reduced development-to-deployment time, and reliable outputs. Despite existing research, a significant knowledge gap remains in operational challenges like model versioning, data traceability, and collaboration, which are crucial for the success of ML projects. Our study aims to address this gap by analyzing 15,065 posts from developer forums and platforms, employing a mixed-method approach to classify inquiries, extract challenges using BERTopic, and identify solutions through open card sorting and BERTopic clustering. We uncover 133 topics related to asset management challenges, grouped into 16 macro-topics, with software dependency, model deployment, and model training being the most discussed. We also find 79 solution topics, categorized under 18 macro-topics, highlighting software dependency, feature development, and file management as key solutions. This research underscores the need for further exploration of identified pain points and the importance of collaborative efforts across academia, industry, and the research community.
Complexity of Manipulation and Bribery in Premise-Based Judgment Aggregation with Simple Formulas
Venue: arXiv cs.GT 2024
Year: 2024

Judgment aggregation is a framework to aggregate individual opinions on multiple, logically connected issues into a collective outcome. These opinions are cast by judges, which can be for example referees, experts, advisors or jurors, depending on the application and context. It is open to manipulative attacks such as \textsc{Manipulation} where judges cast their judgments strategically. Previous works have shown that most computational problems corresponding to these manipulative attacks are \NP-hard. This desired computational barrier, however, often relies on formulas that are either of unbounded size or of complex structure. We revisit the computational complexity for various \textsc{Manipulation} and \textsc{Bribery} problems in premise-based judgment aggregation, now focusing on simple and realistic formulas. We restrict all formulas to be clauses that are (positive) monotone, Horn-clauses, or have bounded length. For basic variants of \textsc{Manipulation}, we show that these restrictions make several variants, which were in general known to be \NP-hard, polynomial-time solvable. Moreover, we provide a P vs.\ NP dichotomy for a large class of clause restrictions (generalizing monotone and Horn clauses) by showing a close relationship between variants of \textsc{Manipulation} and variants of \textsc{Satisfiability}. For Hamming distance based \textsc{Manipulation}, we show that \NP-hardness even holds for positive monotone clauses of length three, but the problem becomes polynomial-time solvable for positive monotone clauses of length two. For \textsc{Bribery}, we show that \NP-hardness even holds for positive monotone clauses of length two, but it becomes polynomial-time solvable for the same clause set if there is a constant budget.
Understanding Public Perceptions of AI Conversational Agents: A Cross-Cultural Analysis
Venue: arXiv cs.HC 2024
Year: 2024

Conversational Agents (CAs) have increasingly been integrated into everyday life, sparking significant discussions on social media. While previous research has examined public perceptions of AI in general, there is a notable lack in research focused on CAs, with fewer investigations into cultural variations in CA perceptions. To address this gap, this study used computational methods to analyze about one million social media discussions surrounding CAs and compared people's discourses and perceptions of CAs in the US and China. We find Chinese participants tended to view CAs hedonically, perceived voice-based and physically embodied CAs as warmer and more competent, and generally expressed positive emotions. In contrast, US participants saw CAs more functionally, with an ambivalent attitude. Warm perception was a key driver of positive emotions toward CAs in both countries. We discussed practical implications for designing contextually sensitive and user-centric CAs to resonate with various users' preferences and needs.
LuaTaint: A Static Analysis System for Web Configuration Interface Vulnerability of Internet of Things Devices
Venue: arXiv cs.CR 2024
Year: 2024

The diversity of web configuration interfaces for IoT devices has exacerbated issues such as inadequate permission controls and insecure interfaces, resulting in various vulnerabilities. Owing to the varying interface configurations across various devices, the existing methods are inadequate for identifying these vulnerabilities precisely and comprehensively. This study addresses these issues by introducing an automated vulnerability detection system, called LuaTaint. It is designed for the commonly used web configuration interface of IoT devices. LuaTaint combines static taint analysis with a large language model (LLM) to achieve widespread and high-precision detection. The extensive traversal of the static analysis ensures the comprehensiveness of the detection. The system also incorporates rules related to page handler control logic within the taint detection process to enhance its precision and extensibility. Moreover, we leverage the prodigious abilities of LLM for code analysis tasks. By utilizing LLM in the process of pruning false alarms, the precision of LuaTaint is enhanced while significantly reducing its dependence on manual analysis. We develop a prototype of LuaTaint and evaluate it using 2,447 IoT firmware samples from 11 renowned vendors. LuaTaint has discovered 111 vulnerabilities. Moreover, LuaTaint exhibits a vulnerability detection precision rate of up to 89.29%.
Online Drone Scheduling for Last-mile Delivery
Venue: arXiv cs.DS 2024
Year: 2024

Delivering a parcel from the distribution hub to the customer's doorstep is called the \textit{last-mile delivery} step in delivery logistics. In this paper, we study a hybrid {\it truck-drones} model for the last-mile delivery step, in which a truck moves on a predefined path carrying parcels and drones deliver the parcels. We define the \textsc{online drone scheduling} problem, where the truck moves in a predefined path, and the customer's requests appear online during the truck's movement. The objective is to schedule a drone associated with every request to minimize the number of drones used subject to the battery budget of the drones and compatibility of the schedules. We propose a 3-competitive deterministic algorithm using the next-fit strategy and 2.7-competitive algorithms using the first-fit strategy for the problem with $O(\log n)$ worst-case time complexity per request, where $n$ is the maximum number of active requests at any time. We also introduce \textsc{online variable-size drone scheduling} problem (OVDS). Here, we know all the customer's requests in advance; however, the drones with different battery capacities appear online. The objective is to schedule customers' requests for drones to minimize the number of drones used. We propose a $(2α+ 1)$-competitive algorithm for the OVDS problem with total running time $O(n \log n)$ for $n$ customer requests, where $α$ is the ratio of the maximum battery capacity to the minimum battery capacity of the drones. Finally, we address how to generate intervals corresponding to each customer request when there are discrete stopping points on the truck's route, from where the drone can fly and meet with the truck.
StochCA: A Novel Approach for Exploiting Pretrained Models with Cross-Attention
Venue: arXiv cs.CV 2024
Year: 2024

Utilizing large-scale pretrained models is a well-known strategy to enhance performance on various target tasks. It is typically achieved through fine-tuning pretrained models on target tasks. However, na\"ıve fine-tuning may not fully leverage knowledge embedded in pretrained models. In this study, we introduce a novel fine-tuning method, called stochastic cross-attention (StochCA), specific to Transformer architectures. This method modifies the Transformer's self-attention mechanism to selectively utilize knowledge from pretrained models during fine-tuning. Specifically, in each block, instead of self-attention, cross-attention is performed stochastically according to the predefined probability, where keys and values are extracted from the corresponding block of a pretrained model. By doing so, queries and channel-mixing multi-layer perceptron layers of a target model are fine-tuned to target tasks to learn how to effectively exploit rich representations of pretrained models. To verify the effectiveness of StochCA, extensive experiments are conducted on benchmarks in the areas of transfer learning and domain generalization, where the exploitation of pretrained models is critical. Our experimental results show the superiority of StochCA over state-of-the-art approaches in both areas. Furthermore, we demonstrate that StochCA is complementary to existing approaches, i.e., it can be combined with them to further improve performance. Our code is available at https://github.com/daintlab/stochastic_cross_attention
DeepForge: Leveraging AI for Microstructural Control in Metal Forming via Model Predictive Control
Venue: arXiv cs.LG 2024
Year: 2024

This study presents a novel method for microstructure control in closed die hot forging that combines Model Predictive Control (MPC) with a developed machine learning model called DeepForge. DeepForge uses an architecture that combines 1D convolutional neural networks and gated recurrent units. It uses surface temperature measurements of a workpiece as input to predict microstructure changes during forging. The paper also details DeepForge's architecture and the finite element simulation model used to generate the data set, using a three-stroke forging process. The results demonstrate DeepForge's ability to predict microstructure with a mean absolute error of 0.4$\pm$0.3%. In addition, the study explores the use of MPC to adjust inter-stroke wait times, effectively counteracting temperature disturbances to achieve a target grain size of less than 35 microns within a specific 2D region of the workpiece. These results are then verified experimentally, demonstrating a significant step towards improved control and quality in forging processes where temperature can be used as an additional degree of freedom in the process.
LSTPrompt: Large Language Models as Zero-Shot Time Series Forecasters by Long-Short-Term Prompting
Venue: arXiv cs.CL 2024
Year: 2024

Time-series forecasting (TSF) finds broad applications in real-world scenarios. Prompting off-the-shelf Large Language Models (LLMs) demonstrates strong zero-shot TSF capabilities while preserving computational efficiency. However, existing prompting methods oversimplify TSF as language next-token predictions, overlooking its dynamic nature and lack of integration with state-of-the-art prompt strategies such as Chain-of-Thought. Thus, we propose LSTPrompt, a novel approach for prompting LLMs in zero-shot TSF tasks. LSTPrompt decomposes TSF into short-term and long-term forecasting sub-tasks, tailoring prompts to each. LSTPrompt guides LLMs to regularly reassess forecasting mechanisms to enhance adaptability. Extensive evaluations demonstrate consistently better performance of LSTPrompt than existing prompting methods, and competitive results compared to foundation TSF models.
From Text to Transformation: A Comprehensive Review of Large Language Models' Versatility
Venue: arXiv cs.CL 2024
Year: 2024

This groundbreaking study explores the expanse of Large Language Models (LLMs), such as Generative Pre-Trained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT) across varied domains ranging from technology, finance, healthcare to education. Despite their established prowess in Natural Language Processing (NLP), these LLMs have not been systematically examined for their impact on domains such as fitness, and holistic well-being, urban planning, climate modelling as well as disaster management. This review paper, in addition to furnishing a comprehensive analysis of the vast expanse and extent of LLMs' utility in diverse domains, recognizes the research gaps and realms where the potential of LLMs is yet to be harnessed. This study uncovers innovative ways in which LLMs can leave a mark in the fields like fitness and wellbeing, urban planning, climate modelling and disaster response which could inspire future researches and applications in the said avenues.
How Can LLM Guide RL? A Value-Based Approach
Venue: arXiv cs.LG 2024
Year: 2024

Reinforcement learning (RL) has become the de facto standard practice for sequential decision-making problems by improving future acting policies with feedback. However, RL algorithms may require extensive trial-and-error interactions to collect useful feedback for improvement. On the other hand, recent developments in large language models (LLMs) have showcased impressive capabilities in language understanding and generation, yet they fall short in exploration and self-improvement capabilities for planning tasks, lacking the ability to autonomously refine their responses based on feedback. Therefore, in this paper, we study how the policy prior provided by the LLM can enhance the sample efficiency of RL algorithms. Specifically, we develop an algorithm named LINVIT that incorporates LLM guidance as a regularization factor in value-based RL, leading to significant reductions in the amount of data needed for learning, particularly when the difference between the ideal policy and the LLM-informed policy is small, which suggests that the initial policy is close to optimal, reducing the need for further exploration. Additionally, we present a practical algorithm SLINVIT that simplifies the construction of the value function and employs subgoals to reduce the search complexity. Our experiments across three interactive environments ALFWorld, InterCode, and BlocksWorld demonstrate that our method achieves state-of-the-art success rates and also surpasses previous RL and LLM approaches in terms of sample efficiency. Our code is available at https://github.com/agentification/Language-Integrated-VI.