OpenRaiser/Intern-Atlas huggingface dataset:
Awareness and Adoption of AI Technologies in the Libraries of Karnataka
Venue: arXiv cs.CY 2024
Year: 2024

This study aims to determine the awareness and adoption of Artificial Intelligence (AI) technologies in the respondent libraries of Karnataka based on demographic variables such as gender, age, academic status, and professional experience. This study employed a survey research method to evaluate the awareness and adoption of AI technologies among the respondent library professionals in Karnataka. The study employed a stratified random sampling method to select a sample of 120 respondents from a diverse population, encompassing library professionals across multiple institution types including engineering colleges, medical colleges, and degree colleges. The Chi-square test was used to analyze the data. The study revealed that there is a statistically significant difference in the awareness and adoption of AI technologies based on the factor of gender. Whereas there no significant relationship exists between the degree of awareness and adoption of AI technologies based on factors such as age, academic ranking, and professional experience. AI-powered plagiarism detection, grammar checking, and ChatGPT are the most popularly employed AI technologies among the respondents. The respondents are of the perception that AI will support Librarians and not replace them.
The Design of a 3D Character Animation System for Digital Twins in the Metaverse
Venue: arXiv cs.HC 2024
Year: 2024

In the context of Industry 4.0, digital twin technology has emerged with rapid advancements as a powerful tool for visualizing and analyzing industrial assets. This technology has attracted considerable interest from researchers across diverse domains such as manufacturing, security, transportation, and gaming. The metaverse has emerged as a significant enabler in these domains, facilitating the integration of various technologies to create virtual replicas of physical assets. The utilization of 3D character animation, often referred to as avatars, is crucial for implementing the metaverse. Traditionally, costly motion capture technologies are employed for creating a realistic avatar system. To meet the needs of this evolving landscape, we have developed a modular framework tailored for asset digital twins as a more affordable alternative. This framework offers flexibility for the independent customization of individual system components. To validate our approach, we employ the English peg solitaire game as a use case, generating a solution tree using the breadth-first search algorithm. The results encompass both qualitative and quantitative findings of a data-driven 3D animation system utilizing motion primitives. The presented methodologies and infrastructure are adaptable and modular, making them applicable to asset digital twins across diverse business contexts. This case study lays the groundwork for pilot applications and can be tailored for education, health, or Industry 4.0 material development.
Predicting Winning Captions for Weekly New Yorker Comics
Venue: arXiv cs.CV 2024
Year: 2024

Image captioning using Vision Transformers (ViTs) represents a pivotal convergence of computer vision and natural language processing, offering the potential to enhance user experiences, improve accessibility, and provide textual representations of visual data. This paper explores the application of image captioning techniques to New Yorker cartoons, aiming to generate captions that emulate the wit and humor of winning entries in the New Yorker Cartoon Caption Contest. This task necessitates sophisticated visual and linguistic processing, along with an understanding of cultural nuances and humor. We propose several new baselines for using vision transformer encoder-decoder models to generate captions for the New Yorker cartoon caption contest.
Prompt Injection Attacks on Large Language Models in Oncology
Venue: arXiv cs.CR 2024
Year: 2024

Vision-language artificial intelligence models (VLMs) possess medical knowledge and can be employed in healthcare in numerous ways, including as image interpreters, virtual scribes, and general decision support systems. However, here, we demonstrate that current VLMs applied to medical tasks exhibit a fundamental security flaw: they can be attacked by prompt injection attacks, which can be used to output harmful information just by interacting with the VLM, without any access to its parameters. We performed a quantitative study to evaluate the vulnerabilities to these attacks in four state of the art VLMs which have been proposed to be of utility in healthcare: Claude 3 Opus, Claude 3.5 Sonnet, Reka Core, and GPT-4o. Using a set of N=297 attacks, we show that all of these models are susceptible. Specifically, we show that embedding sub-visual prompts in medical imaging data can cause the model to provide harmful output, and that these prompts are non-obvious to human observers. Thus, our study demonstrates a key vulnerability in medical VLMs which should be mitigated before widespread clinical adoption.
Towards Automated Solution Recipe Generation for Industrial Asset Management with LLM
Venue: arXiv cs.AI 2024
Year: 2024

This study introduces a novel approach to Industrial Asset Management (IAM) by incorporating Conditional-Based Management (CBM) principles with the latest advancements in Large Language Models (LLMs). Our research introduces an automated model-building process, traditionally reliant on intensive collaboration between data scientists and domain experts. We present two primary innovations: a taxonomy-guided prompting generation that facilitates the automatic creation of AI solution recipes and a set of LLM pipelines designed to produce a solution recipe containing a set of artifacts composed of documents, sample data, and models for IAM. These pipelines, guided by standardized principles, enable the generation of initial solution templates for heterogeneous asset classes without direct human input, reducing reliance on extensive domain knowledge and enhancing automation. We evaluate our methodology by assessing asset health and sustainability across a spectrum of ten asset classes. Our findings illustrate the potential of LLMs and taxonomy-based LLM prompting pipelines in transforming asset management, offering a blueprint for subsequent research and development initiatives to be integrated into a rapid client solution.
A maturity framework for data driven maintenance
Venue: arXiv cs.AI 2024
Year: 2024

Maintenance decisions range from the simple detection of faults to ultimately predicting future failures and solving the problem. These traditionally human decisions are nowadays increasingly supported by data and the ultimate aim is to make them autonomous. This paper explores the challenges encountered in data driven maintenance, and proposes to consider four aspects in a maturity framework: data / decision maturity, the translation from the real world to data, the computability of decisions (using models) and the causality in the obtained relations. After a discussion of the theoretical concepts involved, the exploration continues by considering a practical fault detection and identification problem. Two approaches, i.e. experience based and model based, are compared and discussed in terms of the four aspects in the maturity framework. It is observed that both approaches yield the same decisions, but still differ in the assignment of causality. This confirms that a maturity assessment not only concerns the type of decision, but should also include the other proposed aspects.
GraphBPE: Molecular Graphs Meet Byte-Pair Encoding
Venue: arXiv cs.LG 2024
Year: 2024

With the increasing attention to molecular machine learning, various innovations have been made in designing better models or proposing more comprehensive benchmarks. However, less is studied on the data preprocessing schedule for molecular graphs, where a different view of the molecular graph could potentially boost the model's performance. Inspired by the Byte-Pair Encoding (BPE) algorithm, a subword tokenization method popularly adopted in Natural Language Processing, we propose GraphBPE, which tokenizes a molecular graph into different substructures and acts as a preprocessing schedule independent of the model architectures. Our experiments on 3 graph-level classification and 3 graph-level regression datasets show that data preprocessing could boost the performance of models for molecular graphs, and GraphBPE is effective for small classification datasets and it performs on par with other tokenization methods across different model architectures.
Addressing Behavior Model Inaccuracies for Safe Motion Control in Uncertain Dynamic Environments
Venue: arXiv cs.RO 2024
Year: 2024

Uncertainties in the environment and behavior model inaccuracies compromise the state estimation of a dynamic obstacle and its trajectory predictions, introducing biases in estimation and shifts in predictive distributions. Addressing these challenges is crucial to safely control an autonomous system. In this paper, we propose a novel algorithm SIED-MPC, which synergistically integrates Simultaneous State and Input Estimation (SSIE) and Distributionally Robust Model Predictive Control (DR-MPC) using model confidence evaluation. The SSIE process produces unbiased state estimates and optimal input gap estimates to assess the confidence of the behavior model, defining the ambiguity radius for DR-MPC to handle predictive distribution shifts. This systematic confidence evaluation leads to producing safe inputs with an adequate level of conservatism. Our algorithm demonstrated a reduced collision rate in autonomous driving simulations through improved state estimation, with a 54% shorter average computation time.
Parsimonious Universal Function Approximator for Elastic and Elasto-Plastic Cavity Expansion Problems
Venue: arXiv cs.CE 2024
Year: 2024

Cavity expansion is a canonical problem in geotechnics, which can be described by partial differential equations (PDEs) and ordinary differential equations (ODEs). This study explores the potential of using a new solver, a physics-informed neural network (PINN), to calculate the stress field in an expanded cavity in the elastic and elasto-plastic regimes. Whilst PINNs have emerged as an effective universal function approximator for deriving the solutions of a wide range of governing PDEs/ODEs, their ability to solve elasto-plastic problems remains uncertain. A novel parsimonious loss function is first proposed to balance the simplicity and accuracy of PINN. The proposed method is applied to diverse material behaviours in the cavity expansion problem including isotropic, anisotropic elastic media, and elastic-perfectly plastic media with Tresca and Mohr-Coulomb yield criteria. The results indicate that the use of a parsimonious prior information-based loss function is highly beneficial to deriving the approximate solutions of complex PDEs with high accuracy. The present method allows for accurate derivation of solutions for both elastic and plastic mechanical responses of an expanded cavity. It also provides insights into how PINNs can be further advanced to solve more complex problems in geotechnical practice.
Practical Marketplace Optimization at Uber Using Causally-Informed Machine Learning
Venue: arXiv cs.LG 2024
Year: 2024

Budget allocation of marketplace levers, such as incentives for drivers and promotions for riders, has long been a technical and business challenge at Uber; understanding lever budget changes' impact and estimating cost efficiency to achieve predefined budgets is crucial, with the goal of optimal allocations that maximize business value; we introduce an end-to-end machine learning and optimization procedure to automate budget decision-making for cities, relying on feature store, model training and serving, optimizers, and backtesting; proposing state-of-the-art deep learning (DL) estimator based on S-Learner and a novel tensor B-Spline regression model, we solve high-dimensional optimization with ADMM and primal-dual interior point convex optimization, substantially improving Uber's resource allocation efficiency.
Shaping Integrity: Why Generative Artificial Intelligence Does Not Have to Undermine Education
Venue: arXiv cs.CY 2024
Year: 2024

This paper examines the role of generative artificial intelligence (GAI) in promoting academic integrity within educational settings. It explores how AI can be ethically integrated into classrooms to enhance learning experiences, foster intrinsic motivation, and support voluntary behavior change among students. By analyzing established ethical frameworks and educational theories such as deontological ethics, consequentialism, constructivist learning, and Self-Determination Theory (SDT), the paper argues that GAI, when used responsibly, can enhance digital literacy, encourage genuine knowledge construction, and uphold ethical standards in education. This research highlights the potential of GAI to create enriching, personalized learning environments that prepare students to navigate the complexities of the modern world ethically and effectively.
GPT Deciphering Fedspeak: Quantifying Dissent Among Hawks and Doves
Venue: arXiv cs.AI 2024
Year: 2024

Markets and policymakers around the world hang on the consequential monetary policy decisions made by the Federal Open Market Committee (FOMC). Publicly available textual documentation of their meetings provides insight into members' attitudes about the economy. We use GPT-4 to quantify dissent among members on the topic of inflation. We find that transcripts and minutes reflect the diversity of member views about the macroeconomic outlook in a way that is lost or omitted from the public statements. In fact, diverging opinions that shed light upon the committee's "true" attitudes are almost entirely omitted from the final statements. Hence, we argue that forecasting FOMC sentiment based solely on statements will not sufficiently reflect dissent among the hawks and doves.
Task Offloading in Fog Computing with Deep Reinforcement Learning: Future Research Directions Based on Security and Efficiency Enhancements
Venue: arXiv cs.CR 2024
Year: 2024

The surge in Internet of Things (IoT) devices and data generation highlights the limitations of traditional cloud computing in meeting demands for immediacy, Quality of Service, and location-aware services. Fog computing emerges as a solution, bringing computation, storage, and networking closer to data sources. This study explores the role of Deep Reinforcement Learning in enhancing fog computing's task offloading, aiming for operational efficiency and robust security. By reviewing current strategies and proposing future research directions, the paper shows the potential of Deep Reinforcement Learning in optimizing resource use, speeding up responses, and securing against vulnerabilities. It suggests advancing Deep Reinforcement Learning for fog computing, exploring blockchain for better security, and seeking energy-efficient models to improve the Internet of Things ecosystem. Incorporating artificial intelligence, our results indicate potential improvements in key metrics, such as task completion time, energy consumption, and security incident reduction. These findings provide a concrete foundation for future research and practical applications in optimizing fog computing architectures.
Robust Multimodal 3D Object Detection via Modality-Agnostic Decoding and Proximity-based Modality Ensemble
Venue: arXiv cs.CV 2024
Year: 2024

Recent advancements in 3D object detection have benefited from multi-modal information from the multi-view cameras and LiDAR sensors. However, the inherent disparities between the modalities pose substantial challenges. We observe that existing multi-modal 3D object detection methods heavily rely on the LiDAR sensor, treating the camera as an auxiliary modality for augmenting semantic details. This often leads to not only underutilization of camera data but also significant performance degradation in scenarios where LiDAR data is unavailable. Additionally, existing fusion methods overlook the detrimental impact of sensor noise induced by environmental changes, on detection performance. In this paper, we propose MEFormer to address the LiDAR over-reliance problem by harnessing critical information for 3D object detection from every available modality while concurrently safeguarding against corrupted signals during the fusion process. Specifically, we introduce Modality Agnostic Decoding (MOAD) that extracts geometric and semantic features with a shared transformer decoder regardless of input modalities and provides promising improvement with a single modality as well as multi-modality. Additionally, our Proximity-based Modality Ensemble (PME) module adaptively utilizes the strengths of each modality depending on the environment while mitigating the effects of a noisy sensor. Our MEFormer achieves state-of-the-art performance of 73.9% NDS and 71.5% mAP in the nuScenes validation set. Extensive analyses validate that our MEFormer improves robustness against challenging conditions such as sensor malfunctions or environmental changes. The source code is available at https://github.com/hanchaa/MEFormer
Genetic Algorithm-based Routing and Scheduling for Wildfire Suppression using a Team of UAVs
Venue: arXiv cs.RO 2024
Year: 2024

This paper addresses early wildfire management using a team of UAVs for the mitigation of fires. The early detection and mitigation systems help in alleviating the destruction with reduced resource utilization. A Genetic Algorithm-based Routing and Scheduling with Time constraints (GARST) is proposed to find the shortest schedule route to mitigate the fires as Single UAV Tasks (SUT). The objective of GARST is to compute the route and schedule of the UAVs so that the UAVS reach the assigned fire locations before the fire becomes a Multi UAV Task (MUT) and completely quench the fire using the extinguisher. The fitness function used for the genetic algorithm is the total quench time for mitigation of total fires. The selection, crossover, mutation operators, and elitist strategies collectively ensure the exploration and exploitation of the solution space, maintaining genetic diversity, preventing premature convergence, and preserving high-performing individuals for the effective optimization of solutions. The GARST effectively addresses the challenges posed by the NP-complete problem of routing and scheduling for growing tasks with time constraints. The GARST is able to handle infeasible scenarios effectively, contributing to the overall optimization of the wildfire management system.
Power-LLaVA: Large Language and Vision Assistant for Power Transmission Line Inspection
Venue: arXiv cs.CV 2024
Year: 2024

The inspection of power transmission line has achieved notable achievements in the past few years, primarily due to the integration of deep learning technology. However, current inspection approaches continue to encounter difficulties in generalization and intelligence, which restricts their further applicability. In this paper, we introduce Power-LLaVA, the first large language and vision assistant designed to offer professional and reliable inspection services for power transmission line by engaging in dialogues with humans. Moreover, we also construct a large-scale and high-quality dataset specialized for the inspection task. By employing a two-stage training strategy on the constructed dataset, Power-LLaVA demonstrates exceptional performance at a comparatively low training cost. Extensive experiments further prove the great capabilities of Power-LLaVA within the realm of power transmission line inspection. Code shall be released.
Data Processing Techniques for Modern Multimodal Models
Venue: arXiv cs.CV 2024
Year: 2024

Data processing plays an significant role in current multimodal model training. In this paper. we provide an comprehensive review of common data processing techniques used in modern multimodal model training with a focus on diffusion models and multimodal large language models (MLLMs). We summarized all techniques into four categories: data quality, data quantity, data distribution and data safety. We further present our findings in the choice of data process methods in different type of models. This study aims to provide guidance to multimodal models developers with effective data processing techniques.
Collaborative CP-NIZKs: Modular, Composable Proofs for Distributed Secrets
Venue: arXiv cs.CR 2024
Year: 2024

Non-interactive zero-knowledge (NIZK) proofs of knowledge have proven to be highly relevant for securely realizing a wide array of applications that rely on both privacy and correctness. They enable a prover to convince any party of the correctness of a public statement for a secret witness. However, most NIZKs do not natively support proving knowledge of a secret witness that is distributed over multiple provers. Previously, collaborative proofs [51] have been proposed to overcome this limitation. We investigate the notion of composability in this setting, following the Commit-and-Prove design of LegoSNARK [17]. Composability allows users to combine different, specialized NIZKs (e.g., one arithmetic circuit, one boolean circuit, and one for range proofs) with the aim of reducing the prove generation time. Moreover, it opens the door to efficient realizations of many applications in the collaborative setting such as mutually exclusive prover groups, combining collaborative and single-party proofs and efficiently implementing publicly auditable MPC (PA-MPC). We present the first, general definition for collaborative commit-and-prove NIZK (CP-NIZK) proofs of knowledge and construct distributed protocols to enable their realization. We implement our protocols for two commonly used NIZKs, Groth16 and Bulletproofs, and evaluate their practicality in a variety of computational settings. Our findings indicate that composability adds only minor overhead, especially for large circuits. We experimented with our construction in an application setting, and when compared to prior works, our protocols reduce latency by 18-55x while requiring only a fraction (0.2%) of the communication.
Alleviating Over-Smoothing via Aggregation over Compact Manifolds
Venue: arXiv cs.LG 2024
Year: 2024

Graph neural networks (GNNs) have achieved significant success in various applications. Most GNNs learn the node features with information aggregation of its neighbors and feature transformation in each layer. However, the node features become indistinguishable after many layers, leading to performance deterioration: a significant limitation known as over-smoothing. Past work adopted various techniques for addressing this issue, such as normalization and skip-connection of layer-wise output. After the study, we found that the information aggregations in existing work are all contracted aggregations, with the intrinsic property that features will inevitably converge to the same single point after many layers. To this end, we propose the aggregation over compacted manifolds method (ACM) that replaces the existing information aggregation with aggregation over compact manifolds, a special type of manifold, which avoids contracted aggregations. In this work, we theoretically analyze contracted aggregation and its properties. We also provide an extensive empirical evaluation that shows ACM can effectively alleviate over-smoothing and outperforms the state-of-the-art. The code can be found in https://github.com/DongzhuoranZhou/ACM.git.