OpenRaiser/Intern-Atlas huggingface dataset:
CausalStock: Deep End-to-end Causal Discovery for News-driven Stock Movement Prediction
Venue: arXiv cs.LG 2024
Year: 2024

There are two issues in news-driven multi-stock movement prediction tasks that are not well solved in the existing works. On the one hand, "relation discovery" is a pivotal part when leveraging the price information of other stocks to achieve accurate stock movement prediction. Given that stock relations are often unidirectional, such as the "supplier-consumer" relationship, causal relations are more appropriate to capture the impact between stocks. On the other hand, there is substantial noise existing in the news data leading to extracting effective information with difficulty. With these two issues in mind, we propose a novel framework called CausalStock for news-driven multi-stock movement prediction, which discovers the temporal causal relations between stocks. We design a lag-dependent temporal causal discovery mechanism to model the temporal causal graph distribution. Then a Functional Causal Model is employed to encapsulate the discovered causal relations and predict the stock movements. Additionally, we propose a Denoised News Encoder by taking advantage of the excellent text evaluation ability of large language models (LLMs) to extract useful information from massive news data. The experiment results show that CausalStock outperforms the strong baselines for both news-driven multi-stock movement prediction and multi-stock movement prediction tasks on six real-world datasets collected from the US, China, Japan, and UK markets. Moreover, getting benefit from the causal relations, CausalStock could offer a clear prediction mechanism with good explainability.
Improved Video VAE for Latent Video Diffusion Model
Venue: arXiv cs.CV 2024
Year: 2024

Variational Autoencoder (VAE) aims to compress pixel data into low-dimensional latent space, playing an important role in OpenAI's Sora and other latent video diffusion generation models. While most of existing video VAEs inflate a pretrained image VAE into the 3D causal structure for temporal-spatial compression, this paper presents two astonishing findings: (1) The initialization from a well-trained image VAE with the same latent dimensions suppresses the improvement of subsequent temporal compression capabilities. (2) The adoption of causal reasoning leads to unequal information interactions and unbalanced performance between frames. To alleviate these problems, we propose a keyframe-based temporal compression (KTC) architecture and a group causal convolution (GCConv) module to further improve video VAE (IV-VAE). Specifically, the KTC architecture divides the latent space into two branches, in which one half completely inherits the compression prior of keyframes from a lower-dimension image VAE while the other half involves temporal compression through 3D group causal convolution, reducing temporal-spatial conflicts and accelerating the convergence speed of video VAE. The GCConv in above 3D half uses standard convolution within each frame group to ensure inter-frame equivalence, and employs causal logical padding between groups to maintain flexibility in processing variable frame video. Extensive experiments on five benchmarks demonstrate the SOTA video reconstruction and generation capabilities of the proposed IV-VAE (https://wpy1999.github.io/IV-VAE/).
RL-Pruner: Structured Pruning Using Reinforcement Learning for CNN Compression and Acceleration
Venue: arXiv cs.CV 2024
Year: 2024

Convolutional Neural Networks (CNNs) have demonstrated exceptional performance in recent years. Compressing these models not only reduces storage requirements, making deployment to edge devices feasible, but also accelerates inference, thereby reducing latency and computational costs. Structured pruning, which removes filters at the layer level, directly modifies the model architecture. This approach achieves a more compact architecture while maintaining target accuracy, ensuring that the compressed model retains good compatibility and hardware efficiency. Our method is based on a key observation: filters in different layers of a neural network have varying importance to the model's performance. When the number of filters to prune is fixed, the optimal pruning distribution across different layers is uneven to minimize performance loss. Layers that are more sensitive to pruning should account for a smaller proportion of the pruning distribution. To leverage this insight, we propose RL-Pruner, which uses reinforcement learning to learn the optimal pruning distribution. RL-Pruner can automatically extract dependencies between filters in the input model and perform pruning, without requiring model-specific pruning implementations. We conducted experiments on models such as GoogleNet, ResNet, and MobileNet, comparing our approach to other structured pruning methods to validate its effectiveness. Our code is available at https://github.com/Beryex/RLPruner-CNN.
From Complexity to Simplicity: Using Python Instead of PsychoPy for fNIRS Data Collection
Venue: arXiv cs.HC 2024
Year: 2024

Functional near-infrared spectroscopy (fNIRS) is a non-invasive optical technique that measures brain activity by estimating blood oxygenation using near-infrared light. Traditionally, PsychoPy is used in many studies to send task-specific markers, requiring a separate device to interface with the fNIRS data collection system. In this work, we present a Python-based implementation to send markers directly, eliminating the need for an additional device. This approach allows researchers to run both marker transmission and fNIRS data collection on the same computer, simplifying the setup and enhancing accessibility. This streamlined solution reduces hardware requirements and makes fNIRS studies more efficient.
Does This Summary Answer My Question? Modeling Query-Focused Summary Readers with Rational Speech Acts
Venue: arXiv cs.AI 2024
Year: 2024

Query-focused summarization (QFS) is the task of generating a summary in response to a user-written query. Despite its user-oriented nature, there has been limited work in QFS in explicitly considering a user's understanding of a generated summary, potentially causing QFS systems to underperform at inference time. In this paper, we adapt the Rational Speech Act (RSA) framework, a model of human communication, to explicitly model a reader's understanding of a query-focused summary and integrate it within the generation method of existing QFS systems. In particular, we introduce the answer reconstruction objective which approximates a reader's understanding of a summary by their ability to use it to reconstruct the answer to their initial query. Using this objective, we are able to re-rank candidate summaries generated by existing QFS systems and select summaries that better align with their corresponding query and reference summary. More generally, our study suggests that a simple and effective way of improving a language generation system designed for a user-centered task may be to explicitly incorporate its user requirements into the system's generation procedure.
A Next-Generation Approach to Airline Reservations: Integrating Cloud Microservices with AI and Blockchain for Enhanced Operational Performance
Venue: arXiv cs.AI 2024
Year: 2024

This research proposes the development of a next generation airline reservation system that incorporates the Cloud microservices, distributed artificial intelligence modules and the blockchain technology to improve on the efficiency, safety and customer satisfaction. The traditional reservation systems encounter issues related to the expansion of the systems, the integrity of the data provided and the level of service offered to the customers, which is the main focus of this architecture through the modular and data centric design approaches. This will allow different operations such as reservations, payments, and customer data management among others to be performed separately thereby facilitating high availability of the system by 30% and enhancing performance of the system by 40% on its scalability. Such systems contain AI driven modules that utilize the past booking patterns along with the profile of the customer to estimate the demand and make recommendations, which increases to 25 % of customer engagement. Moreover, blockchain is effective in engaging an incorruptible ledger system for the all transactions therefore mitigating fraud incidences and increasing the clarity by 20%. The system was subjected to analysis using a simulator and using machine learning evaluations that rated it against other conventional systems. The results show that there were clear enhancements in the speed of transactions where the rates of secure data processing rose by 35%, and the system response time by 15 %. The system can also be used for other high transaction industries like logistics and hospitality. This structural design is indicative of how the use of advanced technologies will revolutionize the airline reservation sector. The implications are growing effectiveness, improvement in security and greater customer contentment.
On inside-out Dissections of Polygons and Polyhedra
Venue: arXiv cs.CG 2024
Year: 2024

In this work we study inside-out dissections of polygons and polyhedra. We first show that an arbitrary polygon can be inside-out dissected with $2n+1$ pieces, thereby improving the best previous upper bound of $4(n-2)$ pieces. Additionally, we establish that a regular polygon can be inside-out dissected with at most $6$ pieces. Lastly, we prove that any polyhedron that can be decomposed into finitely many regular tetrahedra and octahedra can be inside-out dissected.
Field Insights for Portable Vine Robots in Urban Search and Rescue
Venue: arXiv cs.RO 2024
Year: 2024

Soft, growing vine robots are well-suited for exploring cluttered, unknown environments, and are theorized to be performant during structural collapse incidents caused by earthquakes, fires, explosions, and material flaws. These vine robots grow from the tip, enabling them to navigate rubble-filled passageways easily. State-of-the-art vine robots have been tested in archaeological and other field settings, but their translational capabilities to urban search and rescue (USAR) are not well understood. To this end, we present a set of experiments designed to test the limits of a vine robot system, the Soft Pathfinding Robotic Observation Unit (SPROUT), operating in an engineered collapsed structure. Our testing is driven by a taxonomy of difficulty derived from the challenges USAR crews face navigating void spaces and their associated hazards. Initial experiments explore the viability of the vine robot form factor, both ideal and implemented, as well as the control and sensorization of the system. A secondary set of experiments applies domain-specific design improvements to increase the portability and reliability of the system. SPROUT can grow through tight apertures, around corners, and into void spaces, but requires additional development in sensorization to improve control and situational awareness.
Few-shot Semantic Learning for Robust Multi-Biome 3D Semantic Mapping in Off-Road Environments
Venue: arXiv cs.CV 2024
Year: 2024

Off-road environments pose significant perception challenges for high-speed autonomous navigation due to unstructured terrain, degraded sensing conditions, and domain-shifts among biomes. Learning semantic information across these conditions and biomes can be challenging when a large amount of ground truth data is required. In this work, we propose an approach that leverages a pre-trained Vision Transformer (ViT) with fine-tuning on a small (<500 images), sparse and coarsely labeled (<30% pixels) multi-biome dataset to predict 2D semantic segmentation classes. These classes are fused over time via a novel range-based metric and aggregated into a 3D semantic voxel map. We demonstrate zero-shot out-of-biome 2D semantic segmentation on the Yamaha (52.9 mIoU) and Rellis (55.5 mIoU) datasets along with few-shot coarse sparse labeling with existing data for improved segmentation performance on Yamaha (66.6 mIoU) and Rellis (67.2 mIoU). We further illustrate the feasibility of using a voxel map with a range-based semantic fusion approach to handle common off-road hazards like pop-up hazards, overhangs, and water features.
LFSamba: Marry SAM with Mamba for Light Field Salient Object Detection
Venue: arXiv cs.CV 2024
Year: 2024

A light field camera can reconstruct 3D scenes using captured multi-focus images that contain rich spatial geometric information, enhancing applications in stereoscopic photography, virtual reality, and robotic vision. In this work, a state-of-the-art salient object detection model for multi-focus light field images, called LFSamba, is introduced to emphasize four main insights: (a) Efficient feature extraction, where SAM is used to extract modality-aware discriminative features; (b) Inter-slice relation modeling, leveraging Mamba to capture long-range dependencies across multiple focal slices, thus extracting implicit depth cues; (c) Inter-modal relation modeling, utilizing Mamba to integrate all-focus and multi-focus images, enabling mutual enhancement; (d) Weakly supervised learning capability, developing a scribble annotation dataset from an existing pixel-level mask dataset, establishing the first scribble-supervised baseline for light field salient object detection.https://github.com/liuzywen/LFScribble
Adversarial Detection with a Dynamically Stable System
Venue: arXiv cs.AI 2024
Year: 2024

Adversarial detection is designed to identify and reject maliciously crafted adversarial examples(AEs) which are generated to disrupt the classification of target models. Presently, various input transformation-based methods have been developed on adversarial example detection, which typically rely on empirical experience and lead to unreliability against new attacks. To address this issue, we propose and conduct a Dynamically Stable System (DSS), which can effectively detect the adversarial examples from normal examples according to the stability of input examples. Particularly, in our paper, the generation of adversarial examples is considered as the perturbation process of a Lyapunov dynamic system, and we propose an example stability mechanism, in which a novel control term is added in adversarial example generation to ensure that the normal examples can achieve dynamic stability while the adversarial examples cannot achieve the stability. Then, based on the proposed example stability mechanism, a Dynamically Stable System (DSS) is proposed, which can utilize the disruption and restoration actions to determine the stability of input examples and detect the adversarial examples through changes in the stability of the input examples. In comparison with existing methods in three benchmark datasets(MNIST, CIFAR10, and CIFAR100), our evaluation results show that our proposed DSS can achieve ROC-AUC values of 99.83%, 97.81% and 94.47%, surpassing the state-of-the-art(SOTA) values of 97.35%, 91.10% and 93.49% in the other 7 methods.
WDMoE: Wireless Distributed Mixture of Experts for Large Language Models
Venue: arXiv cs.LG 2024
Year: 2024

Large Language Models (LLMs) have achieved significant success in various natural language processing tasks, but the role of wireless networks in supporting LLMs has not been thoroughly explored. In this paper, we propose a wireless distributed Mixture of Experts (WDMoE) architecture to enable collaborative deployment of LLMs across edge servers at the base station (BS) and mobile devices in wireless networks. Specifically, we decompose the MoE layer in LLMs by placing the gating network and the preceding neural network layer at BS, while distributing the expert networks among the devices. This deployment leverages the parallel inference capabilities of expert networks on mobile devices, effectively utilizing the limited computing and caching resources of these devices. Accordingly, we develop a performance metric for WDMoE-based LLMs, which accounts for both model capability and latency. To minimize the latency while maintaining accuracy, we jointly optimize expert selection and bandwidth allocation based on the performance metric. Moreover, we build a hardware testbed using NVIDIA Jetson kits to validate the effectiveness of WDMoE. Both theoretical simulations and practical hardware experiments demonstrate that the proposed method can significantly reduce the latency without compromising LLM performance.
Flight Time Improvement Using Adaptive Model Predictive Control for Unmanned Aerial Vehicles
Venue: arXiv cs.RO 2024
Year: 2024

Intelligent aerial platforms such as Unmanned Aerial Vehicles (UAVs) are expected to revolutionize various fields, including transportation, traffic management, field monitoring, industrial production, and agricultural management. Among these, precise control is a critical task that determines the performance and capabilities of UAV systems. However, current research primarily focuses on trajectory tracking and minimizing flight errors, with limited attention to improving flight time. In this paper, we propose a Model Predictive Control (MPC) approach aimed at minimizing flight time while addressing the limitations of the commonly used classical MPC controllers. Furthermore, the MPC method and its application for UAV control are presented in detail. Finally, the results demonstrate that the proposed controller outperforms the standard MPC in terms of efficiency. Moreover, this approach shows potential to become a foundation for integrating intelligent algorithms into basic controllers.
Research on an intelligent fault diagnosis method for nuclear power plants based on ETCN-SSA combined algorithm
Venue: arXiv cs.LG 2024
Year: 2024

Utilizing fault diagnosis methods is crucial for nuclear power professionals to achieve efficient and accurate fault diagnosis for nuclear power plants (NPPs). The performance of traditional methods is limited by their dependence on complex feature extraction and skilled expert knowledge, which can be time-consuming and subjective. This paper proposes a novel intelligent fault diagnosis method for NPPs that combines enhanced temporal convolutional network (ETCN) with sparrow search algorithm (SSA). ETCN utilizes temporal convolutional network (TCN), self-attention (SA) mechanism and residual block for enhancing performance. ETCN excels at extracting local features and capturing time series information, while SSA adaptively optimizes its hyperparameters for superior performance. The proposed method's performance is experimentally verified on a CPR1000 simulation dataset. Compared to other advanced intelligent fault diagnosis methods, the proposed one demonstrates superior performance across all evaluation metrics. This makes it a promising tool for NPP intelligent fault diagnosis, ultimately enhancing operational reliability.
Model Partition and Resource Allocation for Split Learning in Vehicular Edge Networks
Venue: arXiv cs.LG 2024
Year: 2024

The integration of autonomous driving technologies with vehicular networks presents significant challenges in privacy preservation, communication efficiency, and resource allocation. This paper proposes a novel U-shaped split federated learning (U-SFL) framework to address these challenges on the way of realizing in vehicular edge networks. U-SFL is able to enhance privacy protection by keeping both raw data and labels on the vehicular user (VU) side while enabling parallel processing across multiple vehicles. To optimize communication efficiency, we introduce a semantic-aware auto-encoder (SAE) that significantly reduces the dimensionality of transmitted data while preserving essential semantic information. Furthermore, we develop a deep reinforcement learning (DRL) based algorithm to solve the NP-hard problem of dynamic resource allocation and split point selection. Our comprehensive evaluation demonstrates that U-SFL achieves comparable classification performance to traditional split learning (SL) while substantially reducing data transmission volume and communication latency. The proposed DRL-based optimization algorithm shows good convergence in balancing latency, energy consumption, and learning performance.
MP-PINN: A Multi-Phase Physics-Informed Neural Network for Epidemic Forecasting
Venue: arXiv cs.AI 2024
Year: 2024

Forecasting temporal processes such as virus spreading in epidemics often requires more than just observed time-series data, especially at the beginning of a wave when data is limited. Traditional methods employ mechanistic models like the SIR family, which make strong assumptions about the underlying spreading process, often represented as a small set of compact differential equations. Data-driven methods such as deep neural networks make no such assumptions and can capture the generative process in more detail, but fail in long-term forecasting due to data limitations. We propose a new hybrid method called MP-PINN (Multi-Phase Physics-Informed Neural Network) to overcome the limitations of these two major approaches. MP-PINN instils the spreading mechanism into a neural network, enabling the mechanism to update in phases over time, reflecting the dynamics of the epidemics due to policy interventions. Experiments on COVID-19 waves demonstrate that MP-PINN achieves superior performance over pure data-driven or model-driven approaches for both short-term and long-term forecasting.
Designing Local Distributed Mechanisms
Venue: arXiv cs.DC 2024
Year: 2024

In this work we introduce a new notion: local mechanisms. These are truthful mechanisms that have an implementation as fast distributed algorithms and non-trivial approximation guarantees. We show how monotone distributed optimisation algorithms can be turned into truthful mechanisms using Myerson's Lemma. We demonstrate mechanisms for four fundamental graph problems: maximum-weight independent set, minimum-weight vertex cover, minimum-weight dominating set, and a variant of weighted colouring. We show how these mechanisms can be implemented in the distributed setting. The key observation is that computing the so-called critical prices of a monotone algorithm can be done with the same time complexity as the original algorithm in the LOCAL model of distributed computing. Our work establishes a new connection between algorithmic mechanism design and distributed graph algorithms. We pose several open questions, such as can critical prices be computed with small messages. It also points to the importance of designing monotone distributed optimisation algorithms. Our work extends previous work in Distributed Algorithmic Mechanism Design (DAMD) in a new direction. Instead of studying global problems like routing or leader election, we study local resource allocation problems. Our algorithms are simple and thus potentially practical. Local algorithms are particularly interesting for highly dynamic large-scale systems, and there are many potential future application domains, e.g. demand-side load management in electric grids or resource allocation in IoT computing.