OpenRaiser/Intern-Atlas huggingface dataset:
MEDiC: Autonomous Surgical Robotic Assistance to Maximizing Exposure for Dissection and Cautery
Venue: arXiv cs.RO 2024
Year: 2024

Surgical automation has the capability to improve the consistency of patient outcomes and broaden access to advanced surgical care in underprivileged communities. Shared autonomy, where the robot automates routine subtasks while the surgeon retains partial teleoperative control, offers great potential to make an impact. In this paper we focus on one important skill within surgical shared autonomy: Automating robotic assistance to maximize visual exposure and apply tissue tension for dissection and cautery. Ensuring consistent exposure to visualize the surgical site is crucial for both efficiency and patient safety. However, achieving this is highly challenging due to the complexities of manipulating deformable volumetric tissues that are prevalent in surgery.To address these challenges we propose \methodname, a framework for autonomous surgical robotic assistance to \methodfullname. We integrate a differentiable physics model with perceptual feedback to achieve our two key objectives: 1) Maximizing tissue exposure and applying tension for a specified dissection site through visual-servoing conrol and 2) Selecting optimal control positions for a dissection target based on deformable Jacobian analysis. We quantitatively assess our method through repeated real robot experiments on a tissue phantom, and showcase its capabilities through dissection experiments using shared autonomy on real animal tissue.
Transforming Multidimensional Time Series into Interpretable Event Sequences for Advanced Data Mining
Venue: arXiv cs.LG 2024
Year: 2024

This paper introduces a novel spatiotemporal feature representation model designed to address the limitations of traditional methods in multidimensional time series (MTS) analysis. The proposed approach converts MTS into one-dimensional sequences of spatially evolving events, preserving the complex coupling relationships between dimensions. By employing a variable-length tuple mining method, key spatiotemporal features are extracted, enhancing the interpretability and accuracy of time series analysis. Unlike conventional models, this unsupervised method does not rely on large training datasets, making it adaptable across different domains. Experimental results from motion sequence classification validate the model's superior performance in capturing intricate patterns within the data. The proposed framework has significant potential for applications across various fields, including backend services for monitoring and optimizing IT infrastructure, medical diagnosis through continuous patient monitoring and health trend analysis, and internet businesses for tracking user behavior and forecasting sales. This work offers a new theoretical foundation and technical support for advancing time series data mining and its practical applications in human behavior recognition and other domains.
Increasing Information-Carrying Capacity by Exploiting Diverse Traffic Characteristics in Multi-Band Optical Networks
Venue: arXiv cs.NI 2024
Year: 2024

Efficient network management in optical backbone networks is crucial for handling continuous traffic growth. In this work, we address the challenges of managing dynamic traffic in C- and C+L-band optical backbone networks while exploring application flexibility, namely the compressibility and delayability metrics. We propose a strategy, named Delay-Aware and Compression-Aware (DACA) provisioning algorithm, which reduces blocking probability, thereby increasing information-carrying capacity of the network compared to baseline strategies.
VERCEL: Verification and Rectification of Configuration Errors with Least Squares
Venue: arXiv cs.NI 2024
Year: 2024

We present Vercel, a network verification and automatic fault rectification tool that is based on a computationally tractable, algorithmically expressive, and mathematically aesthetic domain of linear algebra. Vercel works on abstracting out packet headers into standard basis vectors that are used to create a port-specific forwarding matrix $\mathcal{A}$, representing a set of packet headers/prefixes that a router forwards along a port. By equating this matrix $\mathcal{A}$ and a vector $b$ (that represents the set of all headers under consideration), we are able to apply \textit{least squares} (which produces a column rank agnostic solution) to compute which headers are reachable at the destination. Reachability now simply means evaluating if vector $b$ is in the column space of $\mathcal{A}$, which can efficiently be computed using least squares. Further, the use of vector representation and least squares opens new possibilities for understanding network behavior. For example, we are able to map rules, routing policies, what-if scenarios to the fundamental linear algebraic form, $\mathcal{A}x=b$, as well as determine how to configure forwarding tables appropriately. We show Vercel is faster than the state-of-art such as NetPlumber, Veriflow, APKeep, AP Verifier, when measured over diverse datasets. Vercel is almost as fast as Deltanet, when rules are verified in batches and provides better scalability, expressiveness and memory efficiency. A key highlight of Vercel is that while evaluating for reachability, the tool can incorporate intents, and transform these into auto-configurable table entries, implying a recommendation/correction system.
MANTA -- Model Adapter Native generations that's Affordable
Venue: arXiv cs.AI 2024
Year: 2024

The presiding model generation algorithms rely on simple, inflexible adapter selection to provide personalized results. We propose the model-adapter composition problem as a generalized problem to past work factoring in practical hardware and affordability constraints, and introduce MANTA as a new approach to the problem. Experiments on COCO 2014 validation show MANTA to be superior in image task diversity and quality at the cost of a modest drop in alignment. Our system achieves a $94\%$ win rate in task diversity and a $80\%$ task quality win rate versus the best known system, and demonstrates strong potential for direct use in synthetic data generation and the creative art domains.
The Ability of Large Language Models to Evaluate Constraint-satisfaction in Agent Responses to Open-ended Requests
Venue: arXiv cs.CL 2024
Year: 2024

Generative AI agents are often expected to respond to complex user requests that have No One Right Answer (NORA), e.g., "design a vegetarian meal plan below 1800 calories". Such requests may entail a set of constraints that the agent should adhere to. To successfully develop agents for NORA scenarios, an accurate automatic evaluation framework is essential, and specifically - one capable of validating the satisfaction of constraints in the agent's response. Recently, large language models (LLMs) have been adopted as versatile evaluators for many NORA tasks, but their ability to evaluate constraint-satisfaction in generated text remains unclear. To study this, we develop and release a novel Arithmetic Constraint-Satisfaction (ACS) benchmarking dataset. The dataset consists of complex user requests with corresponding constraints, agent responses and human labels indicating each constraint's satisfaction level in the response. A unique property of this dataset is that validating many of its constraints requires reviewing the response as a whole (in contrast to many other benchmarks that require the validation of a single independent item). Moreover, it assesses LLMs in performing reasoning, in-context data extraction, arithmetic calculations, and counting. We then benchmark both open and proprietary LLMs on evaluating constraint-satisfaction, and show that most models still have a significant headroom for improvement, and that errors primarily stem from reasoning issues. In addition, most models exhibit a skewed constraint-satisfaction prediction pattern, with higher accuracy where the ground-truth label is "satisfied". Lastly, few-shot prompting for our task proved to be rather challenging, since many of the studied models showed a degradation in performance when it was introduced.
J2N -- Nominal Adjective Identification and its Application
Venue: arXiv cs.CL 2024
Year: 2024

This paper explores the challenges posed by nominal adjectives (NAs) in natural language processing (NLP) tasks, particularly in part-of-speech (POS) tagging. We propose treating NAs as a distinct POS tag, "JN," and investigate its impact on POS tagging, BIO chunking, and coreference resolution. Our study shows that reclassifying NAs can improve the accuracy of syntactic analysis and structural understanding in NLP. We present experimental results using Hidden Markov Models (HMMs), Maximum Entropy (MaxEnt) models, and Spacy, demonstrating the feasibility and potential benefits of this approach. Additionally we finetuned a bert model to identify the NA in untagged text.
A Visualized Malware Detection Framework with CNN and Conditional GAN
Venue: arXiv cs.CR 2024
Year: 2024

Malware visualization analysis incorporating with Machine Learning (ML) has been proven to be a promising solution for improving security defenses on different platforms. In this work, we propose an integrated framework for addressing common problems experienced by ML utilizers in developing malware detection systems. Namely, a pictorial presentation system with extensions is designed to preserve the identities of benign/malign samples by encoding each variable into binary digits and mapping them into black and white pixels. A conditional Generative Adversarial Network based model is adopted to produce synthetic images and mitigate issues of imbalance classes. Detection models architected by Convolutional Neural Networks are for validating performances while training on datasets with and without artifactual samples. Result demonstrates accuracy rates of 98.51% and 97.26% for these two training scenarios.
Enhancing LLM-based Autonomous Driving Agents to Mitigate Perception Attacks
Venue: arXiv cs.CR 2024
Year: 2024

There is a growing interest in integrating Large Language Models (LLMs) with autonomous driving (AD) systems. However, AD systems are vulnerable to attacks against their object detection and tracking (ODT) functions. Unfortunately, our evaluation of four recent LLM agents against ODT attacks shows that the attacks are 63.26% successful in causing them to crash or violate traffic rules due to (1) misleading memory modules that provide past experiences for decision making, (2) limitations of prompts in identifying inconsistencies, and (3) reliance on ground truth perception data. In this paper, we introduce Hudson, a driving reasoning agent that extends prior LLM-based driving systems to enable safer decision making during perception attacks while maintaining effectiveness under benign conditions. Hudson achieves this by first instrumenting the AD software to collect real-time perception results and contextual information from the driving scene. This data is then formalized into a domain-specific language (DSL). To guide the LLM in detecting and making safe control decisions during ODT attacks, Hudson translates the DSL into natural language, along with a list of custom attack detection instructions. Following query execution, Hudson analyzes the LLM's control decision to understand its causal reasoning process. We evaluate the effectiveness of Hudson using a proprietary LLM (GPT-4) and two open-source LLMs (Llama and Gemma) in various adversarial driving scenarios. GPT-4, Llama, and Gemma achieve, on average, an attack detection accuracy of 83. 3%, 63. 6%, and 73. 6%. Consequently, they make safe control decisions in 86.4%, 73.9%, and 80% of the attacks. Our results, following the growing interest in integrating LLMs into AD systems, highlight the strengths of LLMs and their potential to detect and mitigate ODT attacks.
Beyond Words: Evaluating Large Language Models in Transportation Planning
Venue: arXiv cs.AI 2024
Year: 2024

The resurgence and rapid advancement of Generative Artificial Intelligence (GenAI) in 2023 has catalyzed transformative shifts across numerous industry sectors, including urban transportation and logistics. This study investigates the evaluation of Large Language Models (LLMs), specifically GPT-4 and Phi-3-mini, to enhance transportation planning. The study assesses the performance and spatial comprehension of these models through a transportation-informed evaluation framework that includes general geospatial skills, general transportation domain skills, and real-world transportation problem-solving. Utilizing a mixed-methods approach, the research encompasses an evaluation of the LLMs' general Geographic Information System (GIS) skills, general transportation domain knowledge as well as abilities to support human decision-making in the real-world transportation planning scenarios of congestion pricing. Results indicate that GPT-4 demonstrates superior accuracy and reliability across various GIS and transportation-specific tasks compared to Phi-3-mini, highlighting its potential as a robust tool for transportation planners. Nonetheless, Phi-3-mini exhibits competence in specific analytical scenarios, suggesting its utility in resource-constrained environments. The findings underscore the transformative potential of GenAI technologies in urban transportation planning. Future work could explore the application of newer LLMs and the impact of Retrieval-Augmented Generation (RAG) techniques, on a broader set of real-world transportation planning and operations challenges, to deepen the integration of advanced AI models in transportation management practices.
Modeling Pedestrian Crossing Behavior: A Reinforcement Learning Approach with Sensory Motor Constraints
Venue: arXiv cs.HC 2024
Year: 2024

Understanding pedestrian behavior is crucial for the safe deployment of Autonomous Vehicles (AVs) in urban environments. Traditional pedestrian behavior models often fall into two categories: mechanistic models, which do not generalize well to complex environments, and machine-learned models, which generally overlook sensory-motor constraints influencing human behavior and thus prone to fail in untrained scenarios. We hypothesize that sensory-motor constraints, fundamental to how humans perceive and interact with their surroundings, are essential for realistic simulations. Thus, we introduce a constrained reinforcement learning (RL) model that simulates the crossing decision and locomotion of pedestrians. It was constrained to emulate human sensory mechanisms with noisy visual perception and looming aversion. Additionally, human motor constraint was incorporated through a bio-mechanical model of walking. We gathered data from a human-in-the-loop experiment to understand pedestrian behavior. The findings reveal several phenomena not addressed by existing pedestrian models, regarding how pedestrians adapt their walking speed to the kinematics and behavior of the approaching vehicle. Our model successfully captures these human-like walking speed patterns, enabling us to understand these patterns as a trade-off between time pressure and walking effort. Importantly, the model retains the ability to reproduce various phenomena previously captured by a simpler version of the model. Additionally, phenomena related to external human-machine interfaces and light conditions were also included. Overall, our results not only demonstrate the potential of constrained RL in modeling pedestrian behaviors but also highlight the importance of sensory-motor mechanisms in modeling pedestrian-vehicle interactions.
tabulapdf: An R Package to Extract Tables from PDF Documents
Venue: arXiv cs.IR 2024
Year: 2024

tabulapdf is an R package that utilizes the Tabula Java library to import tables from PDF files directly into R. This tool can reduce time and effort in data extraction processes in fields like investigative journalism. It allows for automatic and manual table extraction, the latter facilitated through a Shiny interface, enabling manual areas selection with a computer mouse for data retrieval.
Is 3D chip technology the next growth engine for performance improvement?
Venue: arXiv cs.ET 2024
Year: 2024

The semiconductor industry is reaching a fascinating confluence in several evolutionary trends that will likely lead to a number of revolutionary changes in how computer systems are designed, implemented, scaled, and used. Since Moores Law, which has driven the evolution in systems for the last several decades, is imminently approaching real and severe limitations, the ability to create three-dimensional,3D, device stacks appears promising as a way to continue to integrate more devices into a chip.While on the one hand, this nascent ability to make 3D technology can be interpreted as merely an extension of Moores Law, on the other hand, the fact that systems can now be integrated across multiple planes poses some novel opportunities, as well as serious challenges and questions. In this paper, we explore these various challenges and opportunities and discuss structures and systems that are likely to be facilitated by 3D technology. We also describe the ways in which these systems are likely to change. Since 3D technology offers some different value propositions, we expect that some of the most important ways in which 3D technology will likely impact our approach to future systems design, implementation, and usage are not yet obvious to most system designers, and we outline several of them.
Computing String Covers in Sublinear Time
Venue: arXiv cs.DS 2024
Year: 2024

Let $T$ be a string of length $n$ over an integer alphabet of size $σ$. In the word RAM model, $T$ can be represented in $O(n /\log_σn)$ space. We show that a representation of all covers of $T$ can be computed in the optimal $O(n/\log_σn)$ time; in particular, the shortest cover can be computed within this time. We also design an $O(n(\logσ+ \log \log n)/\log n)$-sized data structure that computes in $O(1)$ time any element of the so-called (shortest) cover array of $T$, that is, the length of the shortest cover of any given prefix of $T$. As a by-product, we describe the structure of cover arrays of Fibonacci strings. On the negative side, we show that the shortest cover of a length-$n$ string cannot be computed using $o(n/\log n)$ operations in the PILLAR model of Charalampopoulos, Kociumaka, and Wellnitz (FOCS 2020).
Generative artificial intelligence usage by researchers at work: Effects of gender, career stage, type of workplace, and perceived barriers
Venue: arXiv cs.CY 2024
Year: 2024

The integration of generative artificial intelligence technology into research environments has become increasingly common in recent years, representing a significant shift in the way researchers approach their work. This paper seeks to explore the factors underlying the frequency of use of generative AI amongst researchers in their professional environments. As survey data may be influenced by a bias towards scientists interested in AI, potentially skewing the results towards the perspectives of these researchers, this study uses a regression model to isolate the impact of specific factors such as gender, career stage, type of workplace, and perceived barriers to using AI technology on the frequency of use of generative AI. It also controls for other relevant variables such as direct involvement in AI research or development, collaboration with AI companies, geographic location, and scientific discipline. Our results show that researchers who face barriers to AI adoption experience an 11% increase in tool use, while those who cite insufficient training resources experience an 8% decrease. Female researchers experience a 7% decrease in AI tool usage compared to men, while advanced career researchers experience a significant 19% decrease. Researchers associated with government advisory groups are 45% more likely to use AI tools frequently than those in government roles. Researchers in for-profit companies show an increase of 19%, while those in medical research institutions and hospitals show an increase of 16% and 15%, respectively. This paper contributes to a deeper understanding of the mechanisms driving the use of generative AI tools amongst researchers, with valuable implications for both academia and industry.
Medical Concept Normalization in a Low-Resource Setting
Venue: arXiv cs.CL 2024
Year: 2024

In the field of biomedical natural language processing, medical concept normalization is a crucial task for accurately mapping mentions of concepts to a large knowledge base. However, this task becomes even more challenging in low-resource settings, where limited data and resources are available. In this thesis, I explore the challenges of medical concept normalization in a low-resource setting. Specifically, I investigate the shortcomings of current medical concept normalization methods applied to German lay texts. Since there is no suitable dataset available, a dataset consisting of posts from a German medical online forum is annotated with concepts from the Unified Medical Language System. The experiments demonstrate that multilingual Transformer-based models are able to outperform string similarity methods. The use of contextual information to improve the normalization of lay mentions is also examined, but led to inferior results. Based on the results of the best performing model, I present a systematic error analysis and lay out potential improvements to mitigate frequent errors.
The X Types -- Mapping the Semantics of the Twitter Sphere
Venue: arXiv cs.CL 2024
Year: 2024

Social networks form a valuable source of world knowledge, where influential entities correspond to popular accounts. Unlike factual knowledge bases (KBs), which maintain a semantic ontology, structured semantic information is not available on social media. In this work, we consider a social KB of roughly 200K popular Twitter accounts, which denotes entities of interest. We elicit semantic information about those entities. In particular, we associate them with a fine-grained set of 136 semantic types, e.g., determine whether a given entity account belongs to a politician, or a musical artist. In the lack of explicit type information in Twitter, we obtain semantic labels for a subset of the accounts via alignment with the KBs of DBpedia and Wikidata. Given the labeled dataset, we finetune a transformer-based text encoder to generate semantic embeddings of the entities based on the contents of their accounts. We then exploit this evidence alongside network-based embeddings to predict the entities semantic types. In our experiments, we show high type prediction performance on the labeled dataset. Consequently, we apply our type classification model to all of the entity accounts in the social KB. Our analysis of the results offers insights about the global semantics of the Twitter sphere. We discuss downstream applications that should benefit from semantic type information and the semantic embeddings of social entities generated in this work. In particular, we demonstrate enhanced performance on the key task of entity similarity assessment using this information.