OpenRaiser/Intern-Atlas huggingface dataset:
Uplifting Lower-Income Data: Strategies for Socioeconomic Perspective Shifts in Large Multi-modal Models
Venue: arXiv cs.CY 2024
Year: 2024

Recent work has demonstrated that the unequal representation of cultures and socioeconomic groups in training data leads to biased Large Multi-modal (LMM) models. To improve LMM model performance on underrepresented data, we propose and evaluate several prompting strategies using non-English, geographic, and socioeconomic attributes. We show that these geographic and socioeconomic integrated prompts favor retrieving topic appearances commonly found in data from low-income households across different countries leading to improved LMM model performance on lower-income data. Our analyses identify and highlight contexts where these strategies yield the most improvements.
Large Scale Hierarchical Industrial Demand Time-Series Forecasting incorporating Sparsity
Venue: arXiv cs.LG 2024
Year: 2024

Hierarchical time-series forecasting (HTSF) is an important problem for many real-world business applications where the goal is to simultaneously forecast multiple time-series that are related to each other via a hierarchical relation. Recent works, however, do not address two important challenges that are typically observed in many demand forecasting applications at large companies. First, many time-series at lower levels of the hierarchy have high sparsity i.e., they have a significant number of zeros. Most HTSF methods do not address this varying sparsity across the hierarchy. Further, they do not scale well to the large size of the real-world hierarchy typically unseen in benchmarks used in literature. We resolve both these challenges by proposing HAILS, a novel probabilistic hierarchical model that enables accurate and calibrated probabilistic forecasts across the hierarchy by adaptively modeling sparse and dense time-series with different distributional assumptions and reconciling them to adhere to hierarchical constraints. We show the scalability and effectiveness of our methods by evaluating them against real-world demand forecasting datasets. We deploy HAILS at a large chemical manufacturing company for a product demand forecasting application with over ten thousand products and observe a significant 8.5\% improvement in forecast accuracy and 23% better improvement for sparse time-series. The enhanced accuracy and scalability make HAILS a valuable tool for improved business planning and customer experience.
Supporters and Skeptics: LLM-based Analysis of Engagement with Mental Health (Mis)Information Content on Video-sharing Platforms
Venue: arXiv cs.SI 2024
Year: 2024

Over one in five adults in the US lives with a mental illness. In the face of a shortage of mental health professionals and offline resources, online short-form video content has grown to serve as a crucial conduit for disseminating mental health help and resources. However, the ease of content creation and access also contributes to the spread of misinformation, posing risks to accurate diagnosis and treatment. Detecting and understanding engagement with such content is crucial to mitigating their harmful effects on public health. We perform the first quantitative study of the phenomenon using YouTube Shorts and Bitchute as the sites of study. We contribute MentalMisinfo, a novel labeled mental health misinformation (MHMisinfo) dataset of 739 videos (639 from Youtube and 100 from Bitchute) and 135372 comments in total, using an expert-driven annotation schema. We first found that few-shot in-context learning with large language models (LLMs) are effective in detecting MHMisinfo videos. Next, we discover distinct and potentially alarming linguistic patterns in how audiences engage with MHMisinfo videos through commentary on both video-sharing platforms. Across the two platforms, comments could exacerbate prevailing stigma with some groups showing heightened susceptibility to and alignment with MHMisinfo. We discuss technical and public health-driven adaptive solutions to tackling the "epidemic" of mental health misinformation online.
Impact of Network Deployment on the Performance of NCR-assisted Networks
Venue: arXiv cs.NI 2024
Year: 2024

To address the need of coverage enhancement in the fifth generation (5G) of wireless cellular telecommunications, while taking into account possible bottlenecks related to deploying fiber based backhaul (e.g., required cost and time), the 3rd generation partnership project (3GPP) proposed in Release 18 the concept of network-controlled repeaters (NCRs). NCRs enhance previous radio frequency (RF) repeaters by exploring beamforming transmissions controlled by the network through side control information. In this context, this paper introduces the concept of NCR. Furthermore, we present a system level model that allows the performance evaluation of an NCR-assisted network. Finally, we evaluate the network deployment impact on the performance of NCR-assisted networks. As we show, with proper network planning, NCRs can boost the signal to interference-plus-noise ratio (SINR) of the user equipments (UEs) in a poor coverage of a macro base station. Furthermore, celledge UEs and uplink (UL) communications are the ones that benefit the most from the presence of NCRs.
Artificial intelligence and machine learning generated conjectures with TxGraffiti
Venue: arXiv cs.AI 2024
Year: 2024

\emph{TxGraffiti} is a machine learning and heuristic based artificial intelligence designed to automate the task of conjecturing in mathematics. Since its inception, TxGraffiti has generated many surprising conjectures leading to publication in respectable mathematical journals. In this paper we outline the machine learning and heuristic techniques implemented by TxGraffiti. We also recall its contributions to the mathematical literature and announce a new online version of the program available for anyone curious to explore conjectures in graph theory.
PWTO: A Heuristic Approach for Trajectory Optimization in Complex Terrains
Venue: arXiv cs.RO 2024
Year: 2024

This paper considers a trajectory planning problem for a robot navigating complex terrains, which arises in applications ranging from autonomous mining vehicles to planetary rovers. The problem seeks to find a low-cost dynamically feasible trajectory for the robot. The problem is challenging as it requires solving a non-linear optimization problem that often has many local minima due to the complex terrain. To address the challenge, we propose a method called Pareto-optimal Warm-started Trajectory Optimization (PWTO) that attempts to combine the benefits of graph search and trajectory optimization, two very different approaches to planning. PWTO first creates a state lattice using simplified dynamics of the robot and leverages a multi-objective graph search method to obtain a set of paths. Each of the paths is then used to warm-start a local trajectory optimization process, so that different local minima are explored to find a globally low-cost solution. In our tests, the solution cost computed by PWTO is often less than half of the costs computed by the baselines. In addition, we verify the trajectories generated by PWTO in Gazebo simulation in complex terrains with both wheeled and quadruped robots. The code of this paper is open sourced and can be found at https://github.com/rap-lab-org/public_pwto.
ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers
Venue: arXiv cs.CV 2024
Year: 2024

Vision Transformers (ViTs) have exhibited exceptional performance across diverse computer vision tasks, while their substantial parameter size incurs significantly increased memory and computational demands, impeding effective inference on resource-constrained devices. Quantization has emerged as a promising solution to mitigate these challenges, yet existing methods still suffer from significant accuracy loss at low-bit. We attribute this issue to the distinctive distributions of post-LayerNorm and post-GELU activations within ViTs, rendering conventional hardware-friendly quantizers ineffective, particularly in low-bit scenarios. To address this issue, we propose a novel framework called Activation-Distribution-Friendly post-training Quantization for Vision Transformers, ADFQ-ViT. Concretely, we introduce the Per-Patch Outlier-aware Quantizer to tackle irregular outliers in post-LayerNorm activations. This quantizer refines the granularity of the uniform quantizer to a per-patch level while retaining a minimal subset of values exceeding a threshold at full-precision. To handle the non-uniform distributions of post-GELU activations between positive and negative regions, we design the Shift-Log2 Quantizer, which shifts all elements to the positive region and then applies log2 quantization. Moreover, we present the Attention-score enhanced Module-wise Optimization which adjusts the parameters of each quantizer by reconstructing errors to further mitigate quantization error. Extensive experiments demonstrate ADFQ-ViT provides significant improvements over various baselines in image classification, object detection, and instance segmentation tasks at 4-bit. Specifically, when quantizing the ViT-B model to 4-bit, we achieve a 10.23% improvement in Top-1 accuracy on the ImageNet dataset.
OpenVNA: A Framework for Analyzing the Behavior of Multimodal Language Understanding System under Noisy Scenarios
Venue: arXiv cs.MM 2024
Year: 2024

We present OpenVNA, an open-source framework designed for analyzing the behavior of multimodal language understanding systems under noisy conditions. OpenVNA serves as an intuitive toolkit tailored for researchers, facilitating convenience batch-level robustness evaluation and on-the-fly instance-level demonstration. It primarily features a benchmark Python library for assessing global model robustness, offering high flexibility and extensibility, thereby enabling customization with user-defined noise types and models. Additionally, a GUI-based interface has been developed to intuitively analyze local model behavior. In this paper, we delineate the design principles and utilization of the created library and GUI-based web platform. Currently, OpenVNA is publicly accessible at \url{https://github.com/thuiar/OpenVNA}, with a demonstration video available at \url{https://youtu.be/0Z9cW7RGct4}.
MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Knowledge distillation is an effective technique for pre-trained language model compression. Although existing knowledge distillation methods perform well for the most typical model BERT, they could be further improved in two aspects: the relation-level knowledge could be further explored to improve model performance; and the setting of student attention head number could be more flexible to decrease inference time. Therefore, we are motivated to propose a novel knowledge distillation method MLKD-BERT to distill multi-level knowledge in teacher-student framework. Extensive experiments on GLUE benchmark and extractive question answering tasks demonstrate that our method outperforms state-of-the-art knowledge distillation methods on BERT. In addition, MLKD-BERT can flexibly set student attention head number, allowing for substantial inference time decrease with little performance drop.
Efficient Training of Language Models with Compact and Consistent Next Token Distributions
Venue: arXiv cs.CL 2024
Year: 2024

Maximizing the likelihood of the next token is an established, statistically sound objective for pre-training language models. In this paper we show that we can train better models faster by pre-aggregating the corpus with a collapsed $n$-gram distribution. Previous studies have proposed corpus-level $n$-gram statistics as a regularizer; however, the construction and querying of such $n$-grams, if done naively, prove to be costly and significantly impede training speed, thereby limiting their application in modern large language model pre-training. We introduce an alternative compact representation of the next token distribution that, in expectation, aligns with the complete $n$-gram distribution while markedly reducing variance across mini-batches compared to the standard next-token loss. Empirically, we demonstrate that both the $n$-gram regularized model and our approximation yield substantial improvements in model quality and convergence rate compared to existing methods. Furthermore, our approximation facilitates scalability of gains to larger datasets and models compared to the straightforward $n$-gram regularization method.
FedPot: A Quality-Aware Collaborative and Incentivized Honeypot-Based Detector for Smart Grid Networks
Venue: arXiv cs.NI 2024
Year: 2024

Honeypot technologies provide an effective defense strategy for the Industrial Internet of Things (IIoT), particularly in enhancing the Advanced Metering Infrastructure's (AMI) security by bolstering the network intrusion detection system. For this security paradigm to be fully realized, it necessitates the active participation of small-scale power suppliers (SPSs) in implementing honeypots and engaging in collaborative data sharing with traditional power retailers (TPRs). To motivate this interaction, TPRs incentivize data sharing with tangible rewards. However, without access to an SPS's confidential data, it is daunting for TPRs to validate shared data, thereby risking SPSs' privacy and increasing sharing costs due to voluminous honeypot logs. These challenges can be resolved by utilizing Federated Learning (FL), a distributed machine learning (ML) technique that allows for model training without data relocation. However, the conventional FL algorithm lacks the requisite functionality for both the security defense model and the rewards system of the AMI network. This work presents two solutions: first, an enhanced and cost-efficient FedAvg algorithm incorporating a novel data quality measure, and second, FedPot, the development of an effective security model with a fair incentives mechanism under an FL architecture. Accordingly, SPSs are limited to sharing the ML model they learn after efficiently measuring their local data quality, whereas TPRs can verify the participants' uploaded models and fairly compensate each participant for their contributions through rewards. Simulation results, drawn from realistic mircorgrid network log datasets, demonstrate that the proposed solutions outperform state-of-the-art techniques by enhancing the security model and guaranteeing fair reward distributions.
A Spatio-Temporal Representation Learning as an Alternative to Traditional Glosses in Sign Language Translation and Production
Venue: arXiv cs.CL 2024
Year: 2024

This work addresses the challenges associated with the use of glosses in both Sign Language Translation (SLT) and Sign Language Production (SLP). While glosses have long been used as a bridge between sign language and spoken language, they come with two major limitations that impede the advancement of sign language systems. First, annotating the glosses is a labor-intensive and time-consuming process, which limits the scalability of datasets. Second, the glosses oversimplify sign language by stripping away its spatio-temporal dynamics, reducing complex signs to basic labels and missing the subtle movements essential for precise interpretation. To address these limitations, we introduce Universal Gloss-level Representation (UniGloR), a framework designed to capture the spatio-temporal features inherent in sign language, providing a more dynamic and detailed alternative to the use of the glosses. The core idea of UniGloR is simple yet effective: We derive dense spatio-temporal representations from sign keypoint sequences using self-supervised learning and seamlessly integrate them into SLT and SLP tasks. Our experiments in a keypoint-based setting demonstrate that UniGloR either outperforms or matches the performance of previous SLT and SLP methods on two widely-used datasets: PHOENIX14T and How2Sign.
A Wolf in Sheep's Clothing: Practical Black-box Adversarial Attacks for Evading Learning-based Windows Malware Detection in the Wild
Venue: arXiv cs.CR 2024
Year: 2024

Given the remarkable achievements of existing learning-based malware detection in both academia and industry, this paper presents MalGuise, a practical black-box adversarial attack framework that evaluates the security risks of existing learning-based Windows malware detection systems under the black-box setting. MalGuise first employs a novel semantics-preserving transformation of call-based redividing to concurrently manipulate both nodes and edges of malware's control-flow graph, making it less noticeable. By employing a Monte-Carlo-tree-search-based optimization, MalGuise then searches for an optimized sequence of call-based redividing transformations to apply to the input Windows malware for evasions. Finally, it reconstructs the adversarial malware file based on the optimized transformation sequence while adhering to Windows executable format constraints, thereby maintaining the same semantics as the original. MalGuise is systematically evaluated against three state-of-the-art learning-based Windows malware detection systems under the black-box setting. Evaluation results demonstrate that MalGuise achieves a remarkably high attack success rate, mostly exceeding 95%, with over 91% of the generated adversarial malware files maintaining the same semantics. Furthermore, MalGuise achieves up to a 74.97% attack success rate against five anti-virus products, highlighting potential tangible security concerns to real-world users.
Domain-independent detection of known anomalies
Venue: arXiv cs.CV 2024
Year: 2024

One persistent obstacle in industrial quality inspection is the detection of anomalies. In real-world use cases, two problems must be addressed: anomalous data is sparse and the same types of anomalies need to be detected on previously unseen objects. Current anomaly detection approaches can be trained with sparse nominal data, whereas domain generalization approaches enable detecting objects in previously unseen domains. Utilizing those two observations, we introduce the hybrid task of domain generalization on sparse classes. To introduce an accompanying dataset for this task, we present a modification of the well-established MVTec AD dataset by generating three new datasets. In addition to applying existing methods for benchmark, we design two embedding-based approaches, Spatial Embedding MLP (SEMLP) and Labeled PatchCore. Overall, SEMLP achieves the best performance with an average image-level AUROC of 87.2 % vs. 80.4 % by MIRO. The new and openly available datasets allow for further research to improve industrial anomaly detection.
Towards Negotiative Dialogue for the Talkamatic Dialogue Manager
Venue: arXiv cs.CL 2024
Year: 2024

The paper describes a number of dialogue phenomena associated with negotiative dialogue, as implemented in a development version of the Talkamatic Dialogue Manager (TDM). This implementation is an initial step towards full coverage of general features of negotiative dialogue in TDM.
Free-SurGS: SfM-Free 3D Gaussian Splatting for Surgical Scene Reconstruction
Venue: arXiv cs.CV 2024
Year: 2024

Real-time 3D reconstruction of surgical scenes plays a vital role in computer-assisted surgery, holding a promise to enhance surgeons' visibility. Recent advancements in 3D Gaussian Splatting (3DGS) have shown great potential for real-time novel view synthesis of general scenes, which relies on accurate poses and point clouds generated by Structure-from-Motion (SfM) for initialization. However, 3DGS with SfM fails to recover accurate camera poses and geometry in surgical scenes due to the challenges of minimal textures and photometric inconsistencies. To tackle this problem, in this paper, we propose the first SfM-free 3DGS-based method for surgical scene reconstruction by jointly optimizing the camera poses and scene representation. Based on the video continuity, the key of our method is to exploit the immediate optical flow priors to guide the projection flow derived from 3D Gaussians. Unlike most previous methods relying on photometric loss only, we formulate the pose estimation problem as minimizing the flow loss between the projection flow and optical flow. A consistency check is further introduced to filter the flow outliers by detecting the rigid and reliable points that satisfy the epipolar geometry. During 3D Gaussian optimization, we randomly sample frames to optimize the scene representations to grow the 3D Gaussian progressively. Experiments on the SCARED dataset demonstrate our superior performance over existing methods in novel view synthesis and pose estimation with high efficiency. Code is available at https://github.com/wrld/Free-SurGS.
Efficient IoT Devices Localization Through Wi-Fi CSI Feature Fusion and Anomaly Detection
Venue: arXiv cs.IT 2024
Year: 2024

Internet of Things (IoT) device localization is fundamental to smart home functionalities, including indoor navigation and tracking of individuals. Traditional localization relies on relative methods utilizing the positions of anchors within a home environment, yet struggles with precision due to inherent inaccuracies in these anchor positions. In response, we introduce a cutting-edge smartphone-based localization system for IoT devices, leveraging the precise positioning capabilities of smartphones equipped with motion sensors. Our system employs artificial intelligence (AI) to merge channel state information from proximal trajectory points of a single smartphone, significantly enhancing line of sight (LoS) angle of arrival (AoA) estimation accuracy, particularly under severe multipath conditions. Additionally, we have developed an AI-based anomaly detection algorithm to further increase the reliability of LoSAoA estimation. This algorithm improves measurement reliability by analyzing the correlation between the accuracy of reversed feature reconstruction and the LoS-AoA estimation. Utilizing a straightforward least squares algorithm in conjunction with accurate LoS-AoA estimation and smartphone positional data, our system efficiently identifies IoT device locations. Validated through extensive simulations and experimental tests with a receiving antenna array comprising just two patch antenna elements in the horizontal direction, our methodology has been shown to attain decimeter-level localization accuracy in nearly 90% of cases, demonstrating robust performance even in challenging real-world scenarios. Additionally, our proposed anomaly detection algorithm trained on Wi-Fi data can be directly applied to ultra-wideband, also outperforming the most advanced techniques.