OpenRaiser/Intern-Atlas huggingface dataset:
Continual Learning with Hebbian Plasticity in Sparse and Predictive Coding Networks: A Survey and Perspective
Venue: arXiv cs.NE 2024
Year: 2024

Recently, the use of bio-inspired learning techniques such as Hebbian learning and its closely-related Spike-Timing-Dependent Plasticity (STDP) variant have drawn significant attention for the design of compute-efficient AI systems that can continuously learn on-line at the edge. A key differentiating factor regarding this emerging class of neuromorphic continual learning system lies in the fact that learning must be carried using a data stream received in its natural order, as opposed to conventional gradient-based offline training, where a static training dataset is assumed available a priori and randomly shuffled to make the training set independent and identically distributed (i.i.d). In contrast, the emerging class of neuromorphic continual learning systems covered in this survey must learn to integrate new information on the fly in a non-i.i.d manner, which makes these systems subject to catastrophic forgetting. In order to build the next generation of neuromorphic AI systems that can continuously learn at the edge, a growing number of research groups are studying the use of Sparse and Predictive Coding-based Hebbian neural network architectures and the related Spiking Neural Networks (SNNs) equipped with STDP learning. However, since this research field is still emerging, there is a need for providing a holistic view of the different approaches proposed in the literature so far. To this end, this survey covers a number of recent works in the field of neuromorphic continual learning based on state-of-the-art Sparse and Predictive Coding technology; provides background theory to help interested researchers quickly learn the key concepts; and discusses important future research questions in light of the different works covered in this paper. It is hoped that this survey will contribute towards future research in the field of neuromorphic continual learning.
LangOcc: Self-Supervised Open Vocabulary Occupancy Estimation via Volume Rendering
Venue: arXiv cs.CV 2024
Year: 2024

The 3D occupancy estimation task has become an important challenge in the area of vision-based autonomous driving recently. However, most existing camera-based methods rely on costly 3D voxel labels or LiDAR scans for training, limiting their practicality and scalability. Moreover, most methods are tied to a predefined set of classes which they can detect. In this work we present a novel approach for open vocabulary occupancy estimation called LangOcc, that is trained only via camera images, and can detect arbitrary semantics via vision-language alignment. In particular, we distill the knowledge of the strong vision-language aligned encoder CLIP into a 3D occupancy model via differentiable volume rendering. Our model estimates vision-language aligned features in a 3D voxel grid using only images. It is trained in a self-supervised manner by rendering our estimations back to 2D space, where ground-truth features can be computed. This training mechanism automatically supervises the scene geometry, allowing for a straight-forward and powerful training method without any explicit geometry supervision. LangOcc outperforms LiDAR-supervised competitors in open vocabulary occupancy by a large margin, solely relying on vision-based training. We also achieve state-of-the-art results in self-supervised semantic occupancy estimation on the Occ3D-nuScenes dataset, despite not being limited to a specific set of categories, thus demonstrating the effectiveness of our proposed vision-language training.
Exploring Commercial Vehicle Detouring Patterns through the Application of Probe Trajectory Data
Venue: arXiv cs.CE 2024
Year: 2024

Understanding motorist detouring behavior is critical for both traffic operations and planning applications. However, measuring real-world detouring behavior is challenging due to the need to track the movement of individual vehicles. Recent developments in high-resolution vehicle trajectory data have enabled transportation professionals to observe real-world detouring behaviors without the need to install and maintain hardware such as license plate reading cameras. This paper investigates the feasibility of vehicle probe trajectory data to capture commercial motor vehicle (CMV) detouring behavior under three unique case studies. Before doing so, a validation analysis was conducted to investigate the ability of CMV probe trajectory data to represent overall CMV volumes at well-calibrated count stations near virtual weigh stations (VWS) in Maryland. The validation analysis showed strong positive correlations (above 0.75) at all VWS stations. Upon validating the data, a methodology was applied to assess CMV detour behaviors associated with CMV enforcement activities, congestion avoidance, and incident induced temporary road closures.
Energy Efficiency Optimization in Integrated Satellite-Terrestrial UAV-Enabled Cell-Free Massive MIMO
Venue: arXiv cs.IT 2024
Year: 2024

Integrating cell-free massive MIMO (CF-mMIMO) into satellite-unmanned aerial vehicle (UAV) networks offers an effective solution for enhancing connectivity. In this setup, UAVs serve as access points (APs) of a terrestrial CF-mMIMO network extending the satellite network capabilities, thereby ensuring robust, high-quality communication links. In this work, we propose a successive convex approximation algorithm for maximizing the downlink energy efficiency (EE) at the UAVs under per-UAV power budget and user quality-of-service constraints. We derive a closed-form expression for the EE that accounts for maximum-ratio transmission and statistical channel knowledge at the users. Simulation results show the effectiveness of the proposed algorithm in maximizing the EE at the UAV layer. Moreover, we observe that a few tens of UAVs transmitting with a fine-tuned power are sufficient to empower the service of satellite networks and significantly increase the spectral efficiency.
DexGANGrasp: Dexterous Generative Adversarial Grasping Synthesis for Task-Oriented Manipulation
Venue: arXiv cs.RO 2024
Year: 2024

We introduce DexGanGrasp, a dexterous grasping synthesis method that generates and evaluates grasps with single view in real time. DexGanGrasp comprises a Conditional Generative Adversarial Networks (cGANs)-based DexGenerator to generate dexterous grasps and a discriminator-like DexEvalautor to assess the stability of these grasps. Extensive simulation and real-world expriments showcases the effectiveness of our proposed method, outperforming the baseline FFHNet with an 18.57% higher success rate in real-world evaluation. We further extend DexGanGrasp to DexAfford-Prompt, an open-vocabulary affordance grounding pipeline for dexterous grasping leveraging Multimodal Large Language Models (MLLMs) and Vision Language Models (VLMs), to achieve task-oriented grasping with successful real-world deployments.
Gradient-based inference of abstract task representations for generalization in neural networks
Venue: arXiv cs.LG 2024
Year: 2024

Humans and many animals show remarkably adaptive behavior and can respond differently to the same input depending on their internal goals. The brain not only represents the intermediate abstractions needed to perform a computation but also actively maintains a representation of the computation itself (task abstraction). Such separation of the computation and its abstraction is associated with faster learning, flexible decision-making, and broad generalization capacity. We investigate if such benefits might extend to neural networks trained with task abstractions. For such benefits to emerge, one needs a task inference mechanism that possesses two crucial abilities: First, the ability to infer abstract task representations when no longer explicitly provided (task inference), and second, manipulate task representations to adapt to novel problems (task recomposition). To tackle this, we cast task inference as an optimization problem from a variational inference perspective and ground our approach in an expectation-maximization framework. We show that gradients backpropagated through a neural network to a task representation layer are an efficient heuristic to infer current task demands, a process we refer to as gradient-based inference (GBI). Further iterative optimization of the task representation layer allows for recomposing abstractions to adapt to novel situations. Using a toy example, a novel image classifier, and a language model, we demonstrate that GBI provides higher learning efficiency and generalization to novel tasks and limits forgetting. Moreover, we show that GBI has unique advantages such as preserving information for uncertainty estimation and detecting out-of-distribution samples.
Tutorial: Object as a Service (OaaS) Serverless Cloud Computing Paradigm
Venue: arXiv cs.DC 2024
Year: 2024

While the first generation of cloud computing systems mitigated the job of system administrators, the next generation of cloud computing systems is emerging to mitigate the burden for cloud developers -- facilitating the development of cloud-native applications. This paradigm shift is primarily happening by offering higher-level serverless abstractions, such as Function as a Service (FaaS). Although FaaS has successfully abstracted developers from the cloud resource management details, it falls short in abstracting the management of both data (i.e., state) and the non-functional aspects, such as Quality of Service (QoS) requirements. The lack of such abstractions implies developer intervention and is counterproductive to the objective of mitigating the burden of cloud-native application development. To further streamline cloud-native application development, we present Object-as-a-Service (OaaS) -- a serverless paradigm that borrows the object-oriented programming concepts to encapsulate application logic and data in addition to non-functional requirements into a single deployment package, thereby streamlining provider-agnostic cloud-native application development. We realized the OaaS paradigm through the development of an open-source platform called Oparaca. In this tutorial, we will present the concept and design of the OaaS paradigm and its implementation -- the Oparaca platform. Then, we give a tutorial on developing and deploying the application on the Oparaca platform and discuss its benefits and its optimal configurations to avoid potential overheads.
Vision Language Model-Empowered Contract Theory for AIGC Task Allocation in Teleoperation
Venue: arXiv cs.CV 2024
Year: 2024

Integrating low-light image enhancement techniques, in which diffusion-based AI-generated content (AIGC) models are promising, is necessary to enhance nighttime teleoperation. Remarkably, the AIGC model is computation-intensive, thus necessitating the allocation of AIGC tasks to edge servers with ample computational resources. Given the distinct cost of the AIGC model trained with varying-sized datasets and AIGC tasks possessing disparate demand, it is imperative to formulate a differential pricing strategy to optimize the utility of teleoperators and edge servers concurrently. Nonetheless, the pricing strategy formulation is under information asymmetry, i.e., the demand (e.g., the difficulty level of AIGC tasks and their distribution) of AIGC tasks is hidden information to edge servers. Additionally, manually assessing the difficulty level of AIGC tasks is tedious and unnecessary for teleoperators. To this end, we devise a framework of AIGC task allocation assisted by the Vision Language Model (VLM)-empowered contract theory, which includes two components: VLM-empowered difficulty assessment and contract theory-assisted AIGC task allocation. The first component enables automatic and accurate AIGC task difficulty assessment. The second component is capable of formulating the pricing strategy for edge servers under information asymmetry, thereby optimizing the utility of both edge servers and teleoperators. The simulation results demonstrated that our proposed framework can improve the average utility of teleoperators and edge servers by 10.88~12.43% and 1.4~2.17%, respectively. Code and data are available at https://github.com/ZiJun0819/VLM-Contract-Theory.
CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) excel in diverse tasks but often underperform in specialized fields due to limited domain-specific or proprietary corpus. Continual pre-training (CPT) enhances LLM capabilities by imbuing new domain-specific or proprietary knowledge while replaying general corpus to prevent catastrophic forgetting. The data mixture ratio of general corpus and domain-specific corpus, however, has been chosen heuristically, leading to sub-optimal training efficiency in practice. In this context, we attempt to re-visit the scaling behavior of LLMs under the hood of CPT, and discover a power-law relationship between loss, mixture ratio, and training tokens scale. We formalize the trade-off between general and domain-specific capabilities, leading to a well-defined Critical Mixture Ratio (CMR) of general and domain data. By striking the balance, CMR maintains the model's general ability and achieves the desired domain transfer, ensuring the highest utilization of available resources. Considering the balance between efficiency and effectiveness, CMR can be regarded as the optimal mixture ratio. Through extensive experiments, we ascertain the predictability of CMR, propose CMR scaling law and have substantiated its generalization. These findings offer practical guidelines for optimizing LLM training in specialized domains, ensuring both general and domain-specific performance while efficiently managing training resources.
Real-Time Automated donning and doffing detection of PPE based on Yolov4-tiny
Venue: arXiv cs.CV 2024
Year: 2024

Maintaining patient safety and the safety of healthcare workers (HCWs) in hospitals and clinics highly depends on following the proper protocol for donning and taking off personal protective equipment (PPE). HCWs can benefit from a feedback system during the putting on and removal process because the process is cognitively demanding and errors are common. Centers for Disease Control and Prevention (CDC) provided guidelines for correct PPE use which should be followed. A real time object detection along with a unique sequencing algorithms are used to identify and determine the donning and doffing process in real time. The purpose of this technical research is two-fold: The user gets real time alert to the step they missed in the sequence if they don't follow the proper procedure during donning or doffing. Secondly, the use of tiny machine learning (yolov4-tiny) in embedded system architecture makes it feasible and cost-effective to deploy in different healthcare settings.
Explainable Natural Language Processing for Corporate Sustainability Analysis
Venue: arXiv cs.CY 2024
Year: 2024

Sustainability commonly refers to entities, such as individuals, companies, and institutions, having a non-detrimental (or even positive) impact on the environment, society, and the economy. With sustainability becoming a synonym of acceptable and legitimate behaviour, it is being increasingly demanded and regulated. Several frameworks and standards have been proposed to measure the sustainability impact of corporations, including United Nations' sustainable development goals and the recently introduced global sustainability reporting framework, amongst others. However, the concept of corporate sustainability is complex due to the diverse and intricate nature of firm operations (i.e. geography, size, business activities, interlinks with other stakeholders). As a result, corporate sustainability assessments are plagued by subjectivity both within data that reflect corporate sustainability efforts (i.e. corporate sustainability disclosures) and the analysts evaluating them. This subjectivity can be distilled into distinct challenges, such as incompleteness, ambiguity, unreliability and sophistication on the data dimension, as well as limited resources and potential bias on the analyst dimension. Put together, subjectivity hinders effective cost attribution to entities non-compliant with prevailing sustainability expectations, potentially rendering sustainability efforts and its associated regulations futile. To this end, we argue that Explainable Natural Language Processing (XNLP) can significantly enhance corporate sustainability analysis. Specifically, linguistic understanding algorithms (lexical, semantic, syntactic), integrated with XAI capabilities (interpretability, explainability, faithfulness), can bridge gaps in analyst resources and mitigate subjectivity problems within data.
AI in Remote Patient Monitoring
Venue: arXiv cs.HC 2024
Year: 2024

The rapid evolution of Artificial Intelligence (AI) has significantly transformed healthcare, particularly in the domain of Remote Patient Monitoring (RPM). This chapter explores the integration of AI in RPM, highlighting real-life applications, system architectures, and the benefits it brings to patient care and healthcare systems. Through a comprehensive analysis of current technologies, methodologies, and case studies, I present a detailed overview of how AI enhances monitoring accuracy, predictive analytics, and personalized treatment plans. The chapter also discusses the challenges and future directions in this field, providing a comprehensive view of AI's role in revolutionizing remote patient care.
Graph Linear Canonical Transform Based on CM-CC-CM Decomposition
Venue: arXiv cs.IT 2024
Year: 2024

The graph linear canonical transform (GLCT) is presented as an extension of the graph Fourier transform (GFT) and the graph fractional Fourier transform (GFrFT), offering more flexibility as an effective tool for graph signal processing. In this paper, we introduce a GLCT based on chirp multiplication-chirp convolution-chirp multiplication decomposition (CM-CC-CM-GLCT), which irrelevant to sampling periods and without oversampling operation. Various properties and special cases of the CM-CC-CM-GLCT are derived and discussed. In terms of computational complexity, additivity, and reversibility, we compare the CM-CC-CM-GLCT and the GLCT based on the central discrete dilated Hermite function (CDDHFs-GLCT). Theoretical analysis demonstrates that the computational complexity of the CM-CC-CM-GLCT is significantly reduced. Simulation results indicate that the CM-CC-CM-GLCT achieves similar additivity to the CDDHFs-GLCT. Notably, the CM-CC-CM-GLCT exhibits better reversibility.
Mapping the Technological Future: A Topic, Sentiment, and Emotion Analysis in Social Media Discourse
Venue: arXiv cs.AI 2024
Year: 2024

People worldwide are currently confronted with a number of technological challenges, which act as a potent source of uncertainty. The uncertainty arising from the volatility and unpredictability of technology (such as AI) and its potential consequences is widely discussed on social media. This study uses BERTopic modelling along with sentiment and emotion analysis on 1.5 million tweets from 2021 to 2023 to identify anticipated tech-driven futures and capture the emotions communicated by 400 key opinion leaders (KOLs). Findings indicate positive sentiment significantly outweighs negative, with a prevailing dominance of positive anticipatory emotions. Specifically, the 'Hope' score is approximately 10.33\% higher than the median 'Anxiety' score. KOLs emphasize 'Optimism' and benefits over 'Pessimism' and challenges. The study emphasizes the important role KOLs play in shaping future visions through anticipatory discourse and emotional tone during times of technological uncertainty.
Large Language Models for Anomaly Detection in Computational Workflows: from Supervised Fine-Tuning to In-Context Learning
Venue: arXiv cs.SE 2024
Year: 2024

Anomaly detection in computational workflows is critical for ensuring system reliability and security. However, traditional rule-based methods struggle to detect novel anomalies. This paper leverages large language models (LLMs) for workflow anomaly detection by exploiting their ability to learn complex data patterns. Two approaches are investigated: 1) supervised fine-tuning (SFT), where pre-trained LLMs are fine-tuned on labeled data for sentence classification to identify anomalies, and 2) in-context learning (ICL) where prompts containing task descriptions and examples guide LLMs in few-shot anomaly detection without fine-tuning. The paper evaluates the performance, efficiency, generalization of SFT models, and explores zero-shot and few-shot ICL prompts and interpretability enhancement via chain-of-thought prompting. Experiments across multiple workflow datasets demonstrate the promising potential of LLMs for effective anomaly detection in complex executions.
Time-Shifted Alternating Gelfand-Pinsker Coding for Broadcast Channels
Venue: arXiv cs.IT 2024
Year: 2024

A coding scheme for broadcast channels (BCs) is proposed that shifts the users' code blocks by different amounts of time and applies alternating Gelfand-Pinsker encoding. The scheme achieves all rate tuples in Marton's region for two receiver BCs without time-sharing or rate-splitting. Simulations with short polar codes show that the method reduces the gap to capacity as compared to time-sharing.
Towards Neural Network based Cognitive Models of Dynamic Decision-Making by Humans
Venue: arXiv cs.LG 2024
Year: 2024

Modeling human cognitive processes in dynamic decision-making tasks has been an endeavor in AI for a long time because such models can help make AI systems more intuitive, personalized, mitigate any human biases, and enhance training in simulation. Some initial work has attempted to utilize neural networks (and large language models) but often assumes one common model for all humans and aims to emulate human behavior in aggregate. However, the behavior of each human is distinct, heterogeneous, and relies on specific past experiences in certain tasks. For instance, consider two individuals responding to a phishing email: one who has previously encountered and identified similar threats may recognize it quickly, while another without such experience might fall for the scam. In this work, we build on Instance Based Learning (IBL) that posits that human decisions are based on similar situations encountered in the past. However, IBL relies on simple fixed form functions to capture the mapping from past situations to current decisions. To that end, we propose two new attention-based neural network models to have open form non-linear functions to model distinct and heterogeneous human decision-making in dynamic settings. We experiment with two distinct datasets gathered from human subject experiment data, one focusing on detection of phishing email by humans and another where humans act as attackers in a cybersecurity setting and decide on an attack option. We conducted extensive experiments with our two neural network models, IBL, and GPT3.5, and demonstrate that the neural network models outperform IBL significantly in representing human decision-making, while providing similar interpretability of human decisions as IBL. Overall, our work yields promising results for further use of neural networks in cognitive modeling of human decision making.