OpenRaiser/Intern-Atlas huggingface dataset:
Evaluation of waterway lock service quality in Yangtze Delta: from the perspectives of customer and supplier
Venue: arXiv cs.CY 2024
Year: 2024

In recent decades, the waterway locks in the Yangtze Delta, China, have become major traffic bottlenecks. To gain a comprehensive understanding of the crew's perspectives and primary concerns regarding lock services during vessel lockage, and to enhance customer satisfaction and improve vessel lockage efficiency, it is necessary to assess the waterway lock service quality (WLSQ). This paper presents an evaluation system for WLSQ from various stakeholders' viewpoints. Firstly, by employing questionnaire surveys and the structural equation model method, in conjunction with factor analysis, the WLSQ and its influencing factors in the Yangtze River Delta region are analyzed from a customer perspective. Secondly, the Analytic Hierarchy Process method is utilized, along with a dedicated questionnaire for service suppliers, to examine their concerns regarding the performance of vessel lock services. The findings indicate that there exists a cognitive bias towards factors influencing the WLSQ. Crew members express the greatest concern over vessel lockage delays, whereas vessel lockage safety is the primary concern for management department administrators. Furthermore, enhancing the supporting facilities of waterway locks can significantly increase crew members' satisfaction during vessel lockage. Improving staff skills, and safety conditions can also greatly enhance customers' tolerance for lockage delays. The results of this study will provide valuable insights for the lock management department, operators, and the government in formulating relevant policies to improve WLSQ and implementing ongoing service quality evaluations.
Graph Network Surrogate Model for Optimizing the Placement of Horizontal Injection Wells for CO2 Storage
Venue: arXiv cs.CE 2024
Year: 2024

Optimizing the locations of multiple CO2 injection wells will be essential as we proceed from demonstration-scale to large-scale carbon storage operations. Well placement optimization is, however, a computationally intensive task because the flow responses associated with many potential configurations must be evaluated. There is thus a need for efficient surrogate models for this application. In this work we develop and apply a graph network surrogate model (GNSM) to predict the global pressure and CO2 saturation fields in 3D geological models for arbitrary configurations of four horizontal wells. The GNSM uses an encoding-processing-decoding framework where the problem is represented in terms of computational graphs. Separate networks are applied for pressure and saturation predictions, and a multilayer perceptron is used to provide bottom-hole pressure (BHP) for each well at each time step. The GNSM is shown to achieve median relative errors of 4\% for pressure and 6\% for saturation over a test set involving very different plume shapes and dynamics. Speedup is about a factor of $120\times$ relative to high-fidelity simulation. The GNSM is applied for optimization using a differential evolution algorithm, where the goal is to minimize the CO2 footprint subject to constraints on the well configuration, plume location and well BHPs. Optimization results using the GNSM are shown to be comparable to those achieved using (much more expensive) high-fidelity simulation.
Taking a turn for the better: Conversation redirection throughout the course of mental-health therapy
Venue: arXiv cs.CL 2024
Year: 2024

Mental-health therapy involves a complex conversation flow in which patients and therapists continuously negotiate what should be talked about next. For example, therapists might try to shift the conversation's direction to keep the therapeutic process on track and avoid stagnation, or patients might push the discussion towards issues they want to focus on. How do such patient and therapist redirections relate to the development and quality of their relationship? To answer this question, we introduce a probabilistic measure of the extent to which a certain utterance immediately redirects the flow of the conversation, accounting for both the intention and the actual realization of such a change. We apply this new measure to characterize the development of patient-therapist relationships over multiple sessions in a very large, widely-used online therapy platform. Our analysis reveals that (1) patient control of the conversation's direction generally increases relative to that of the therapist as their relationship progresses; and (2) patients who have less control in the first few sessions are significantly more likely to eventually express dissatisfaction with their therapist and terminate the relationship.
The Transparent Relations Ontology (TRO): a vocabulary to describe conflicts of interest
Venue: arXiv cs.DL 2024
Year: 2024

The Transparent Relations Ontology (TRO) offers a vocabulary to publish data about relations between powerful parties that should be more transparent, in order to detect possible conflicts of interest. TRO is based on minimal modelling, reusing common vocabularies to offer a simple yet useful resource to publish interoperable data about pointers to relations that might result in corruption cases. Additionally, best practices have been followed in order to sustain a technically rigorous ontology development process. A usage example with real data is mentioned, integrating information from Basque Government's Open Data services and a news outlet. Building upon its foundational design, future enhancements of TRO could significantly amplify its utility in uncovering and scrutinizing opaque relationships that may lead to corruption.
TextToon: Real-Time Text Toonify Head Avatar from Single Video
Venue: arXiv cs.CV 2024
Year: 2024

We propose TextToon, a method to generate a drivable toonified avatar. Given a short monocular video sequence and a written instruction about the avatar style, our model can generate a high-fidelity toonified avatar that can be driven in real-time by another video with arbitrary identities. Existing related works heavily rely on multi-view modeling to recover geometry via texture embeddings, presented in a static manner, leading to control limitations. The multi-view video input also makes it difficult to deploy these models in real-world applications. To address these issues, we adopt a conditional embedding Tri-plane to learn realistic and stylized facial representations in a Gaussian deformation field. Additionally, we expand the stylization capabilities of 3D Gaussian Splatting by introducing an adaptive pixel-translation neural network and leveraging patch-aware contrastive learning to achieve high-quality images. To push our work into consumer applications, we develop a real-time system that can operate at 48 FPS on a GPU machine and 15-18 FPS on a mobile machine. Extensive experiments demonstrate the efficacy of our approach in generating textual avatars over existing methods in terms of quality and real-time animation. Please refer to our project page for more details: https://songluchuan.github.io/TextToon/.
The trade-off between data minimization and fairness in collaborative filtering
Venue: arXiv cs.IR 2024
Year: 2024

General Data Protection Regulations (GDPR) aim to safeguard individuals' personal information from harm. While full compliance is mandatory in the European Union and the California Privacy Rights Act (CPRA), it is not in other places. GDPR requires simultaneous compliance with all the principles such as fairness, accuracy, and data minimization. However, it overlooks the potential contradictions within its principles. This matter gets even more complex when compliance is required from decision-making systems. Therefore, it is essential to investigate the feasibility of simultaneously achieving the goals of GDPR and machine learning, and the potential tradeoffs that might be forced upon us. This paper studies the relationship between the principles of data minimization and fairness in recommender systems. We operationalize data minimization via active learning (AL) because, unlike many other methods, it can preserve a high accuracy while allowing for strategic data collection, hence minimizing the amount of data collection. We have implemented several active learning strategies (personalized and non-personalized) and conducted a comparative analysis focusing on accuracy and fairness on two publicly available datasets. The results demonstrate that different AL strategies may have different impacts on the accuracy of recommender systems with nearly all strategies negatively impacting fairness. There has been no to very limited work on the trade-off between data minimization and fairness, the pros and cons of active learning methods as tools for implementing data minimization, and the potential impacts of AL on fairness. By exploring these critical aspects, we offer valuable insights for developing recommender systems that are GDPR compliant.
SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers
Venue: arXiv cs.CL 2024
Year: 2024

The performance of Transformer models has been enhanced by increasing the number of parameters and the length of the processed text. Consequently, fine-tuning the entire model becomes a memory-intensive process. High-performance methods for parameter-efficient fine-tuning (PEFT) typically work with Attention blocks and often overlook MLP blocks, which contain about half of the model parameters. We propose a new selective PEFT method, namely SparseGrad, that performs well on MLP blocks. We transfer layer gradients to a space where only about 1\% of the layer's elements remain significant. By converting gradients into a sparse structure, we reduce the number of updated parameters. We apply SparseGrad to fine-tune BERT and RoBERTa for the NLU task and LLaMa-2 for the Question-Answering task. In these experiments, with identical memory requirements, our method outperforms LoRA and MeProp, robust popular state-of-the-art PEFT approaches.
The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks
Venue: arXiv cs.AI 2024
Year: 2024

There is increasing interest in tracking the capabilities of general intelligence foundation models. This study benchmarks leading large language models and vision language models against human performance on the Wechsler Adult Intelligence Scale (WAIS-IV), a comprehensive, population-normed assessment of underlying human cognition and intellectual abilities, with a focus on the domains of VerbalComprehension (VCI), Working Memory (WMI), and Perceptual Reasoning (PRI). Most models demonstrated exceptional capabilities in the storage, retrieval, and manipulation of tokens such as arbitrary sequences of letters and numbers, with performance on the Working Memory Index (WMI) greater or equal to the 99.5th percentile when compared to human population normative ability. Performance on the Verbal Comprehension Index (VCI) which measures retrieval of acquired information, and linguistic understanding about the meaning of words and their relationships to each other, also demonstrated consistent performance at or above the 98th percentile. Despite these broad strengths, we observed consistently poor performance on the Perceptual Reasoning Index (PRI; range 0.1-10th percentile) from multimodal models indicating profound inability to interpret and reason on visual information. Smaller and older model versions consistently performed worse, indicating that training data, parameter count and advances in tuning are resulting in significant advances in cognitive ability.
Structured Spatial Reasoning with Open Vocabulary Object Detectors
Venue: arXiv cs.CV 2024
Year: 2024

Reasoning about spatial relationships between objects is essential for many real-world robotic tasks, such as fetch-and-delivery, object rearrangement, and object search. The ability to detect and disambiguate different objects and identify their location is key to successful completion of these tasks. Several recent works have used powerful Vision and Language Models (VLMs) to unlock this capability in robotic agents. In this paper we introduce a structured probabilistic approach that integrates rich 3D geometric features with state-of-the-art open-vocabulary object detectors to enhance spatial reasoning for robotic perception. The approach is evaluated and compared against zero-shot performance of the state-of-the-art Vision and Language Models (VLMs) on spatial reasoning tasks. To enable this comparison, we annotate spatial clauses in real-world RGB-D Active Vision Dataset [1] and conduct experiments on this and the synthetic Semantic Abstraction [2] dataset. Results demonstrate the effectiveness of the proposed method, showing superior performance of grounding spatial relations over state of the art open-source VLMs by more than 20%.
Automated Creation of Digital Cousins for Robust Policy Learning
Venue: arXiv cs.RO 2024
Year: 2024

Training robot policies in the real world can be unsafe, costly, and difficult to scale. Simulation serves as an inexpensive and potentially limitless source of training data, but suffers from the semantics and physics disparity between simulated and real-world environments. These discrepancies can be minimized by training in digital twins, which serve as virtual replicas of a real scene but are expensive to generate and cannot produce cross-domain generalization. To address these limitations, we propose the concept of digital cousins, a virtual asset or scene that, unlike a digital twin, does not explicitly model a real-world counterpart but still exhibits similar geometric and semantic affordances. As a result, digital cousins simultaneously reduce the cost of generating an analogous virtual environment while also facilitating better robustness during sim-to-real domain transfer by providing a distribution of similar training scenes. Leveraging digital cousins, we introduce a novel method for their automated creation, and propose a fully automated real-to-sim-to-real pipeline for generating fully interactive scenes and training robot policies that can be deployed zero-shot in the original scene. We find that digital cousin scenes that preserve geometric and semantic affordances can be produced automatically, and can be used to train policies that outperform policies trained on digital twins, achieving 90% vs. 25% success rates under zero-shot sim-to-real transfer. Additional details are available at https://digital-cousins.github.io/.
NeRF-Accelerated Ecological Monitoring in Mixed-Evergreen Redwood Forest
Venue: arXiv cs.CV 2024
Year: 2024

Forest mapping provides critical observational data needed to understand the dynamics of forest environments. Notably, tree diameter at breast height (DBH) is a metric used to estimate forest biomass and carbon dioxide sequestration. Manual methods of forest mapping are labor intensive and time consuming, a bottleneck for large-scale mapping efforts. Automated mapping relies on acquiring dense forest reconstructions, typically in the form of point clouds. Terrestrial laser scanning (TLS) and mobile laser scanning (MLS) generate point clouds using expensive LiDAR sensing, and have been used successfully to estimate tree diameter. Neural radiance fields (NeRFs) are an emergent technology enabling photorealistic, vision-based reconstruction by training a neural network on a sparse set of input views. In this paper, we present a comparison of MLS and NeRF forest reconstructions for the purpose of trunk diameter estimation in a mixed-evergreen Redwood forest. In addition, we propose an improved DBH-estimation method using convex-hull modeling. Using this approach, we achieved 1.68 cm RMSE, which consistently outperformed standard cylinder modeling approaches. Our code contributions and forest datasets are freely available at https://github.com/harelab-ucsc/RedwoodNeRF.
Goal-oriented vessel detection with distributed computing in a LEO satellite constellation
Venue: arXiv cs.NI 2024
Year: 2024

Earth Observation (EO) has traditionally involved the transmission of a large volume of raw data to map the Earth surface. This results in congestion to the satellite network and delays in the availability of the results, invalidating the approach for timing-sensitive applications. Instead, the computation resources at the satellites can be used as an edge layer for compressing the data and/or doing inferences. In this paper, we investigate satellite edge computing for vessel detection with a LEO satellite constellation. First, we distribute the computation and inference load among the neighbouring satellites of the one taking the images, based on the VHRShips data set and YOLOv8. This semantic and fragmented information is then routed to a remote ground monitor through the whole constellation. The average and peak Age of Information (AoI) are reformulated to measure the freshness of the aggregated information at the receiver in this image-capture scenario. We then dimension the network (number of orbital planes and satellites per orbital plane) for a given target age and covered area that quantify the level of achievement of the task. The results show that 20 orbital planes with 20 satellites are necessary to keep the peak AoI below 60 seconds with a compression ratio > 23000, i.e., a size reduction of 99.996%, and for a approximately 100% probability of coverage.
Robust infrared small target detection using self-supervised and a contrario paradigms
Venue: arXiv cs.CV 2024
Year: 2024

Detecting small targets in infrared images poses significant challenges in defense applications due to the presence of complex backgrounds and the small size of the targets. Traditional object detection methods often struggle to balance high detection rates with low false alarm rates, especially when dealing with small objects. In this paper, we introduce a novel approach that combines a contrario paradigm with Self-Supervised Learning (SSL) to improve Infrared Small Target Detection (IRSTD). On the one hand, the integration of an a contrario criterion into a YOLO detection head enhances feature map responses for small and unexpected objects while effectively controlling false alarms. On the other hand, we explore SSL techniques to overcome the challenges of limited annotated data, common in IRSTD tasks. Specifically, we benchmark several representative SSL strategies for their effectiveness in improving small object detection performance. Our findings show that instance discrimination methods outperform masked image modeling strategies when applied to YOLO-based small object detection. Moreover, the combination of the a contrario and SSL paradigms leads to significant performance improvements, narrowing the gap with state-of-the-art segmentation methods and even outperforming them in frugal settings. This two-pronged approach offers a robust solution for improving IRSTD performance, particularly under challenging conditions.
Systematic Feature Design for Cycle Life Prediction of Lithium-Ion Batteries During Formation
Venue: arXiv cs.LG 2024
Year: 2024

Optimization of the formation step in lithium-ion battery manufacturing is challenging due to limited physical understanding of solid electrolyte interphase formation and the long testing time (~100 days) for cells to reach the end of life. We propose a systematic feature design framework that requires minimal domain knowledge for accurate cycle life prediction during formation. Two simple Q(V) features designed from our framework, extracted from formation data without any additional diagnostic cycles, achieved a median of 9.20% error for cycle life prediction, outperforming thousands of autoML models using pre-defined features. We attribute the strong performance of our designed features to their physical origins - the voltage ranges identified by our framework capture the effects of formation temperature and microscopic particle resistance heterogeneity. By designing highly interpretable features, our approach can accelerate formation research, leveraging the interplay between data-driven feature design and mechanistic understanding.
Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning
Venue: arXiv cs.CL 2024
Year: 2024

Network pruning has emerged as a potential solution to make LLMs cheaper to deploy. However, existing LLM pruning approaches universally rely on the C4 dataset as the calibration data for calculating pruning scores, leaving its optimality unexplored. In this study, we evaluate the choice of calibration data on LLM pruning, across a wide range of datasets that are most commonly used in LLM training and evaluation, including four pertaining datasets as well as three categories of downstream tasks encompassing nine datasets. Each downstream dataset is prompted with In-Context Learning (ICL) and Chain-of-Thought (CoT), respectively. Besides the already intriguing observation that the choice of calibration data significantly impacts the performance of pruned LLMs, our results also uncover several subtle and often unexpected findings, summarized as follows: (1) C4 is not the optimal choice for LLM pruning, even among commonly used pre-training datasets; (2) arithmetic datasets, when used as calibration data, performs on par or even better than pre-training datasets; (3) pruning with downstream datasets does not necessarily help the corresponding downstream task, compared to pre-training data; (4) ICL is widely beneficial to all data categories, whereas CoT is only useful on certain tasks. Our findings shed light on the importance of carefully selecting calibration data for LLM pruning and pave the way for more efficient deployment of these powerful models in real-world applications. We release our code at: https://github.com/abx393/llm-pruning-calibration-data.
Transducer Consistency Regularization for Speech to Text Applications
Venue: arXiv cs.CL 2024
Year: 2024

Consistency regularization is a commonly used practice to encourage the model to generate consistent representation from distorted input features and improve model generalization. It shows significant improvement on various speech applications that are optimized with cross entropy criterion. However, it is not straightforward to apply consistency regularization for the transducer-based approaches, which are widely adopted for speech applications due to the competitive performance and streaming characteristic. The main challenge is from the vast alignment space of the transducer optimization criterion and not all the alignments within the space contribute to the model optimization equally. In this study, we present Transducer Consistency Regularization (TCR), a consistency regularization method for transducer models. We apply distortions such as spec augmentation and dropout to create different data views and minimize the distribution difference. We utilize occupational probabilities to give different weights on transducer output distributions, thus only alignments close to oracle alignments would contribute to the model learning. Our experiments show the proposed method is superior to other consistency regularization implementations and could effectively reduce word error rate (WER) by 4.3\% relatively comparing with a strong baseline on the \textsc{Librispeech} dataset.