OpenRaiser/Intern-Atlas huggingface dataset:
Boosting Static Resource Leak Detection via LLM-based Resource-Oriented Intention Inference
Venue: arXiv cs.SE 2024
Year: 2024

Resource leaks, caused by resources not being released after acquisition, often lead to performance issues and system crashes. Existing static detection techniques rely on mechanical matching of predefined resource acquisition/release APIs and null-checking conditions to find unreleased resources, suffering from both (1) false negatives caused by the incompleteness of predefined resource acquisition/release APIs and (2) false positives caused by the incompleteness of resource reachability validation identification. To overcome these challenges, we propose InferROI, a novel approach that leverages the exceptional code comprehension capability of large language models (LLMs) to directly infer resource-oriented intentions (acquisition, release, and reachability validation) in code. InferROI first prompts the LLM to infer involved intentions for a given code snippet, and then incorporates a two-stage static analysis approach to check control-flow paths for resource leak detection based on the inferred intentions. We evaluate the effectiveness of InferROI in both resource-oriented intention inference and resource leak detection. Experimental results on the DroidLeaks and JLeaks datasets demonstrate InferROI achieves promising bug detection rate (59.3% and 62.5%) and false alarm rate (18.6% and 19.5%). Compared to three industrial static detectors, InferROI detects 14~45 and 149~485 more bugs in DroidLeaks and JLeaks, respectively. When applied to real-world open-source projects, InferROI identifies 29 unknown resource leak bugs (verified by authors), with 7 of them being confirmed by developers. In addition, the results of an ablation study underscores the importance of combining LLM-based inference with static analysis.
Hybrid Focal and Full-Range Attention Based Graph Transformers
Venue: arXiv cs.LG 2024
Year: 2024

The paradigm of Transformers using the self-attention mechanism has manifested its advantage in learning graph-structured data. Yet, Graph Transformers are capable of modeling full range dependencies but are often deficient in extracting information from locality. A common practice is to utilize Message Passing Neural Networks (MPNNs) as an auxiliary to capture local information, which however are still inadequate for comprehending substructures. In this paper, we present a purely attention-based architecture, namely Focal and Full-Range Graph Transformer (FFGT), which can mitigate the loss of local information in learning global correlations. The core component of FFGT is a new mechanism of compound attention, which combines the conventional full-range attention with K-hop focal attention on ego-nets to aggregate both global and local information. Beyond the scope of canonical Transformers, the FFGT has the merit of being more substructure-aware. Our approach enhances the performance of existing Graph Transformers on various open datasets, while achieves compatible SOTA performance on several Long-Range Graph Benchmark (LRGB) datasets even with a vanilla transformer. We further examine influential factors on the optimal focal length of attention via introducing a novel synthetic dataset based on SBM-PATTERN.
Examining Common Paradigms in Multi-Task Learning
Venue: arXiv cs.LG 2024
Year: 2024

While multi-task learning (MTL) has gained significant attention in recent years, its underlying mechanisms remain poorly understood. Recent methods did not yield consistent performance improvements over single task learning (STL) baselines, underscoring the importance of gaining more profound insights about challenges specific to MTL. In our study, we investigate paradigms in MTL in the context of STL: First, the impact of the choice of optimizer has only been mildly investigated in MTL. We show the pivotal role of common STL tools such as the Adam optimizer in MTL empirically in various experiments. To further investigate Adam's effectiveness, we theoretical derive a partial loss-scale invariance under mild assumptions. Second, the notion of gradient conflicts has often been phrased as a specific problem in MTL. We delve into the role of gradient conflicts in MTL and compare it to STL. For angular gradient alignment we find no evidence that this is a unique problem in MTL. We emphasize differences in gradient magnitude as the main distinguishing factor. Overall, we find surprising similarities between STL and MTL suggesting to consider methods from both fields in a broader context.
TinyAirNet: TinyML Model Transmission for Energy-efficient Image Retrieval from IoT Devices
Venue: arXiv cs.NI 2024
Year: 2024

This letter introduces an energy-efficient pull-based data collection framework for Internet of Things (IoT) devices that use Tiny Machine Learning (TinyML) to interpret data queries. A TinyML model is transmitted from the edge server to the IoT devices. The devices employ the model to facilitate the subsequent semantic queries. This reduces the transmission of irrelevant data, but receiving the ML model and its processing at the IoT devices consume additional energy. We consider the specific instance of image retrieval in a single device scenario and investigate the gain brought by the proposed scheme in terms of energy efficiency and retrieval accuracy, while considering the cost of computation and communication, as well as memory constraints. Numerical evaluation shows that, compared to a baseline scheme, the proposed scheme reaches up to 67% energy reduction under the accuracy constraint when many images are stored. Although focused on image retrieval, our analysis is indicative of a broader set of communication scenarios in which the preemptive transmission of an ML model can increase communication efficiency.
CloverLeaf on Intel Multi-Core CPUs: A Case Study in Write-Allocate Evasion
Venue: arXiv cs.DC 2024
Year: 2024

In this paper we analyze the MPI-only version of the CloverLeaf code from the SPEChpc 2021 benchmark suite on recent Intel Xeon "Ice Lake" and "Sapphire Rapids" server CPUs. We observe peculiar breakdowns in performance when the number of processes is prime. Investigating this effect, we create first-principles data traffic models for each of the stencil-like hotspot loops. With application measurements and microbenchmarks to study memory data traffic behavior, we can connect the breakdowns to SpecI2M, a new write-allocate evasion feature in current Intel CPUs. For serial and full-node cases we are able to predict the memory data volume analytically with an error of a few percent. We find that if the number of processes is prime, SpecI2M fails to work properly, which we can attribute to short inner loops emerging from the one-dimensional domain decomposition in this case. We can also rule out other possible causes of the prime number effect, such as breaking layer conditions, MPI communication overhead, and load imbalance.
Cross-Silo Federated Learning Across Divergent Domains with Iterative Parameter Alignment
Venue: arXiv cs.LG 2024
Year: 2024

Learning from the collective knowledge of data dispersed across private sources can provide neural networks with enhanced generalization capabilities. Federated learning, a method for collaboratively training a machine learning model across remote clients, achieves this by combining client models via the orchestration of a central server. However, current approaches face two critical limitations: i) they struggle to converge when client domains are sufficiently different, and ii) current aggregation techniques produce an identical global model for each client. In this work, we address these issues by reformulating the typical federated learning setup: rather than learning a single global model, we learn N models each optimized for a common objective. To achieve this, we apply a weighted distance minimization to model parameters shared in a peer-to-peer topology. The resulting framework, Iterative Parameter Alignment, applies naturally to the cross-silo setting, and has the following properties: (i) a unique solution for each participant, with the option to globally converge each model in the federation, and (ii) an optional early-stopping mechanism to elicit fairness among peers in collaborative learning settings. These characteristics jointly provide a flexible new framework for iteratively learning from peer models trained on disparate datasets. We find that the technique achieves competitive results on a variety of data partitions compared to state-of-the-art approaches. Further, we show that the method is robust to divergent domains (i.e. disjoint classes across peers) where existing approaches struggle.
Fusionize++: Improving Serverless Application Performance Using Dynamic Task Inlining and Infrastructure Optimization
Venue: arXiv cs.DC 2024
Year: 2024

The Function-as-a-Service (FaaS) execution model increases developer productivity by removing operational concerns such as managing hardware or software runtimes. Developers, however, still need to partition their applications into FaaS functions, which is error-prone and complex: Encapsulating only the smallest logical unit of an application as a FaaS function maximizes flexibility and reusability. Yet, it also leads to invocation overheads, additional cold starts, and may increase cost due to double billing during synchronous invocations. Conversely, deploying an entire application as a single FaaS function avoids these overheads but decreases flexibility. In this paper we present Fusionize, a framework that automates optimizing for this trade-off by automatically fusing application code into an optimized multi-function composition. Developers only need to write fine-grained application code following the serverless model, while Fusionize automatically fuses different parts of the application into FaaS functions, manages their interactions, and configures the underlying infrastructure. At runtime, it monitors application performance and adapts it to minimize request-response latency and costs. Real-world use cases show that Fusionize can improve the deployment artifacts of the application, reducing both median request-response latency and cost of an example IoT application by more than 35%.
Machine-learning optimized measurements of chaotic dynamical systems via the information bottleneck
Venue: arXiv cs.LG 2024
Year: 2024

Deterministic chaos permits a precise notion of a "perfect measurement" as one that, when obtained repeatedly, captures all of the information created by the system's evolution with minimal redundancy. Finding an optimal measurement is challenging, and has generally required intimate knowledge of the dynamics in the few cases where it has been done. We establish an equivalence between a perfect measurement and a variant of the information bottleneck. As a consequence, we can employ machine learning to optimize measurement processes that efficiently extract information from trajectory data. We obtain approximately optimal measurements for multiple chaotic maps and lay the necessary groundwork for efficient information extraction from general time series.
Tuning-less Object Naming with a Foundation Model
Venue: arXiv cs.CL 2024
Year: 2024

We implement a real-time object naming system that enables learning a set of named entities never seen. Our approach employs an existing foundation model that we consider ready to see anything before starting. It turns seen images into relatively small feature vectors that we associate with index to a gradually built vocabulary without any training of fine-tuning of the model. Our contribution is using the association mechanism known from transformers as attention. It has features that support generalization from irrelevant information for distinguishing the entities and potentially enable associating with much more than indices to vocabulary. As a result, the system can work in a one-shot manner and correctly name objects named in different contents. We also outline implementation details of the system modules integrated by a blackboard architecture. Finally, we investigate the system's quality, mainly how many objects it can handle in this way.
The Impact of Changes to Daylight Illumination level on Architectural experience in Offices Based on VR and EEG
Venue: arXiv cs.HC 2024
Year: 2024

This study investigates the influence of varying illumination levels on architectural experiences by employing a comprehensive approach that combines self-reported assessments and neurophysiological measurements. Thirty participants were exposed to nine distinct illumination conditions in a controlled virtual reality environment. Subjective assessments, collected through questionnaires in which participants were asked to rate how pleasant, interesting, exciting, calming, complex, bright and spacious they found the space. Objective measurements of brain activity were collected by electroencephalogram (EEG). Data analysis demonstrated that illumination levels significantly influenced cognitive engagement and different architectural experience indicators. This alignment between subjective assessment and EEG data underscores the relationship between illuminance and architectural experiences. The study bridges the gap between quantitative and qualitative assessments, providing a deeper understanding of the intricate connection between lighting conditions and human responses. These findings contribute to the enhancement of environmental design based on neuroscientific insights, emphasizing the critical role of well-considered daylighting design in positively influencing occupants' cognitive and emotional states within built environments.
Efficient Compression of Overparameterized Deep Models through Low-Dimensional Learning Dynamics
Venue: arXiv cs.LG 2024
Year: 2024

Overparameterized models have proven to be powerful tools for solving various machine learning tasks. However, overparameterization often leads to a substantial increase in computational and memory costs, which in turn requires extensive resources to train. In this work, we present a novel approach for compressing overparameterized models, developed through studying their learning dynamics. We observe that for many deep models, updates to the weight matrices occur within a low-dimensional invariant subspace. For deep linear models, we demonstrate that their principal components are fitted incrementally within a small subspace, and use these insights to propose a compression algorithm for deep linear networks that involve decreasing the width of their intermediate layers. We empirically evaluate the effectiveness of our compression technique on matrix recovery problems. Remarkably, by using an initialization that exploits the structure of the problem, we observe that our compressed network converges faster than the original network, consistently yielding smaller recovery errors. We substantiate this observation by developing a theory focused on deep matrix factorization. Finally, we empirically demonstrate how our compressed model has the potential to improve the utility of deep nonlinear models. Overall, our algorithm improves the training efficiency by more than 2x, without compromising generalization.
Characterizing Large Language Models as Rationalizers of Knowledge-intensive Tasks
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs) are proficient at generating fluent text with minimal task-specific supervision. Yet, their ability to provide well-grounded rationalizations for knowledge-intensive tasks remains under-explored. Such tasks, like commonsense multiple-choice questions, require rationales based on world knowledge to support predictions and refute alternate options. We consider the task of generating knowledge-guided rationalization in natural language by using expert-written examples in a few-shot manner. Surprisingly, crowd-workers preferred knowledge-grounded rationales over crowdsourced rationalizations, citing their factuality, sufficiency, and comprehensive refutations. Although LLMs-generated rationales were preferable, further improvements in conciseness and novelty are required. In another study, we show how rationalization of incorrect model predictions erodes humans' trust in LLM-generated rationales. Motivated by these observations, we create a two-stage pipeline to review task predictions and eliminate potential incorrect decisions before rationalization, enabling trustworthy rationale generation.
Vector Approximate Survey Propagation
Venue: arXiv cs.IT 2024
Year: 2024

Approximate Message Passing (AMP), originally designed to solve high-dimensional linear inverse problems, has found broad applications in signal processing and statistical inference. Among its key variants, Vector Approximate Message Passing (VAMP) and Generalized Approximate Survey Propagation (GASP) have demonstrated effectiveness even in scenarios where the assumed generative models differ from the true models. However, the maximum a posteriori (MAP) versions of VAMP and GASP have limitations: VAMP is restricted to differentiable priors and likelihoods, while GASP requires the measurement matrix to have independent identically distributed (i.i.d.) elements. To overcome these limitations, this paper introduces a new algorithm, Vector Approximate Survey Propagation (VASP). VASP utilizes survey propagation to handle non-differentiable priors and likelihoods, along with employs vector-form messages to account for correlations in the measurement matrix. Simulations reveal that VASP significantly surpasses VAMP and GASP in estimation accuracy, particularly when the assumed prior is discrete-supported and the measurement matrix is non-i.i.d.. Additionally, the state evolution (SE) of VASP, derived heuristically, accurately reflects the per-iteration mean squared error (MSE). A comparison between the SE and the free energy computed by Takahashi and Kabashima under the one-step replica symmetry breaking (1RSB) ansatz shows that the SE's fixed-point equations align with the free energy's saddle point equations. This suggests that VASP efficiently implements the postulated MAP estimator (which is NP-hard in the worst case) with cubic computational complexity, assuming the 1RSB ansatz is valid.
u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model
Venue: arXiv cs.CV 2024
Year: 2024

Recent advancements in multi-modal large language models (MLLMs) have led to substantial improvements in visual understanding, primarily driven by sophisticated modality alignment strategies. However, predominant approaches prioritize global or regional comprehension, with less focus on fine-grained, pixel-level tasks. To address this gap, we introduce u-LLaVA, an innovative unifying multi-task framework that integrates pixel, regional, and global features to refine the perceptual faculties of MLLMs. We commence by leveraging an efficient modality alignment approach, harnessing both image and video datasets to bolster the model's foundational understanding across diverse visual contexts. Subsequently, a joint instruction tuning method with task-specific projectors and decoders for end-to-end downstream training is presented. Furthermore, this work contributes a novel mask-based multi-task dataset comprising 277K samples, crafted to challenge and assess the fine-grained perception capabilities of MLLMs. The overall framework is simple, effective, and achieves state-of-the-art performance across multiple benchmarks. We also make our model, data, and code publicly accessible at https://github.com/OPPOMKLab/u-LLaVA.
FogROS2-Config: Optimizing Latency and Cost for Multi-Cloud Robot Applications
Venue: arXiv cs.RO 2024
Year: 2024

Cloud service providers provide over 50,000 distinct and dynamically changing set of cloud server options. To help roboticists make cost-effective decisions, we present FogROS2-Config, an open toolkit that takes ROS2 nodes as input and automatically runs relevant benchmarks to quickly return a menu of cloud compute services that tradeoff latency and cost. Because it is infeasible to try every hardware configuration, FogROS2-Config quickly samples tests a small set of edge case servers. We evaluate FogROS2-Config on three robotics application tasks: visual SLAM, grasp planning. and motion planning. FogROS2-Config can reduce the cost by up to 20x. By comparing with a Pareto frontier for cost and latency by running the application task on feasible server configurations, we evaluate cost and latency models and confirm that FogROS2-Config selects efficient hardware configurations to balance cost and latency.
Active Admission Control in a P2P Distributed Environment for Capacity Efficient Livestreaming in Mobile Wireless Networks
Venue: arXiv cs.NI 2024
Year: 2024

In this study, the Active Control in an Intelligent and Distributed Environment (ACIDE) media distribution model solution and algorithms are proposed for livestreaming in capacity efficient mobile wireless networks. The elements of the ACIDE model are a base station and a cluster formed by a number of peers able to establish peer to peer communications. The cluster peers are selected from a group of users interested in livestreaming the same media. The ACIDE model solution minimizes the bandwidth allocated to a cluster of n peers such that an uninterrupted media play for all peers is guaranteed. The livestream media is sent to the peers in packages and every media package is divided into n blocks. The blocks are distributed to the n peers of a cluster in two phases, such that the base station bandwidth is utilized during first phase only. The allocated bandwidth, the amount of bandwidth the base station has to allocate to a cluster, is minimized and its lower bound is equal to the bandwidth required for multicasting. In this study, the ACIDE model is used to address the problem of how to find the maximum number of peers n, chosen from a group of N users, that can be admitted to a cluster knowing the given allocated bandwidth, the amount of bandwidth that a base station allocates to a cluster in advance, prior to admitting users. When users become peers of an ACIDE cluster, the network capacity, the total number of users who are able to access live media, increases meaning that network resources are used more efficiently. The problem of finding the maximum number of peers n is addressed as an optimization problem, with the objective of having the entire given allocated bandwidth used by the peers admitted to the cluster. This problem is NP-complete and a non-optimal solution is proposed for peers selection such that all admitted peers play media continuously.
Establishing Performance Baselines in Fine-Tuning, Retrieval-Augmented Generation and Soft-Prompting for Non-Specialist LLM Users
Venue: arXiv cs.IR 2024
Year: 2024

Research into methods for improving the performance of large language models (LLMs) through fine-tuning, retrieval-augmented generation (RAG) and soft-prompting has tended to focus on the use of highly technical or high-cost techniques, making many of the newly discovered approaches comparatively inaccessible to non-technical users. In this paper we tested an unmodified version of GPT 3.5, a fine-tuned version, and the same unmodified model when given access to a vectorised RAG database, both in isolation and in combination with a basic, non-algorithmic soft prompt. In each case we tested the model's ability to answer a set of 100 questions relating primarily to events that occurred after September 2021 (the point at which GPT 3.5's training data set ends). We found that if commercial platforms are used and default settings are applied with no iteration in order to establish a baseline set of outputs, a fine-tuned model outperforms GPT 3.5 Turbo, while the RAG approach out-performed both. The application of a soft prompt significantly improved the performance of each approach.