OpenRaiser/Intern-Atlas huggingface dataset:
Towards Semi-Autonomous Robotic Arm Manipulation Operator Intention Detection from Force Data
Venue: arXiv cs.RO 2024
Year: 2024

In hazardous environments like nuclear facilities, robotic systems are essential for executing tasks that would otherwise expose humans to dangerous radiation levels, which pose severe health risks and can be fatal. However, many operations in the nuclear environment require teleoperating robots, resulting in a significant cognitive load on operators as well as physical strain over extended periods of time. To address this challenge, we propose enhancing the teleoperation system with an assistive model capable of predicting operator intentions and dynamically adapting to their needs. The machine learning model processes robotic arm force data, analyzing spatiotemporal patterns to accurately detect the ongoing task before its completion. To support this approach, we collected a diverse dataset from teleoperation experiments involving glovebox tasks in nuclear applications. This dataset encompasses heterogeneous spatiotemporal data captured from the teleoperation system. We employ a hybrid Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) model to learn and forecast operator intentions based on the spatiotemporal data. By accurately predicting these intentions, the robot can execute tasks more efficiently and effectively, requiring minimal input from the operator. Our experiments validated the model using the dataset, focusing on tasks such as radiation surveys and object grasping. The proposed approach demonstrated an F1-score of 89% for task classification and an F1-score of 86% classification forecasted operator intentions over a 5-second window. These results highlight the potential of our method to improve the safety, precision, and efficiency of robotic operations in hazardous environments, thereby significantly reducing human radiation exposure.
Human-Centric Goal Reasoning with Ripple-Down Rules
Venue: arXiv cs.RO 2024
Year: 2024

ActorSim is a goal reasoning framework developed at the Naval Research Laboratory. Originally, all goal reasoning rules were hand-crafted. This work extends ActorSim with the capability of learning by demonstration, that is, when a human trainer disagrees with a decision made by the system, the trainer can take over and show the system the correct decision. The learning component uses Ripple-Down Rules (RDR) to build new decision rules to correctly handle similar cases in the future. The system is demonstrated using the RoboCup Rescue Agent Simulation, which simulates a city-wide disaster, requiring emergency services, including fire, ambulance and police, to be dispatched to different sites to evacuate civilians from dangerous situations. The RDRs are implemented in a scripting language, FrameScript, which is used to mediate between ActorSim and the agent simulator. Using Ripple-Down Rules, ActorSim can scale to an order of magnitude more goals than the previous version.
Temporal Analysis of Drifting Hashtags in Textual Data Streams: A Graph-Based Application
Venue: arXiv cs.SI 2024
Year: 2024

Initially supported by Twitter, hashtags are now used on several social media platforms. Hashtags are helpful for tagging, tracking, and grouping posts on similar topics. In this paper, based on a hashtag stream regarding the hashtag #mybodymychoice, we analyze hashtag drifts over time using concepts from graph analysis and textual data streams using the Girvan-Newman method to uncover hashtag communities in annual snapshots between 2018 and 2022. In addition, we offer insights about some correlated hashtags found in the study. Our approach can be useful for monitoring changes over time in opinions and sentiment patterns about an entity on social media. Even though the hashtag #mybodymychoice was initially coupled with women's rights, abortion, and bodily autonomy, we observe that it suffered drifts during the studied period across topics such as drug legalization, vaccination, political protests, war, and civil rights. The year 2021 was the most significant drifting year, in which the communities detected and their respective sizes suggest that #mybodymychoice had a significant drift to vaccination and Covid-19-related topics.
Engraving Oriented Joint Estimation of Pitch Spelling and Local and Global Keys
Venue: arXiv cs.SD 2024
Year: 2024

We revisit the problems of pitch spelling and tonality guessing with a new algorithm for their joint estimation from a MIDI file including information about the measure boundaries. Our algorithm does not only identify a global key but also local ones all along the analyzed piece. It uses Dynamic Programming techniques to search for an optimal spelling in term, roughly, of the number of accidental symbols that would be displayed in the engraved score. The evaluation of this number is coupled with an estimation of the global key and some local keys, one for each measure. Each of the three informations is used for the estimation of the other, in a multi-steps procedure. An evaluation conducted on a monophonic and a piano dataset, comprising 216 464 notes in total, shows a high degree of accuracy, both for pitch spelling (99.5% on average on the Bach corpus and 98.2% on the whole dataset) and global key signature estimation (93.0% on average, 95.58% on the piano dataset). Designed originally as a backend tool in a music transcription framework, this method should also be useful in other tasks related to music notation processing.
Experiments with Encoding Structured Data for Neural Networks
Venue: arXiv cs.AI 2024
Year: 2024

The project's aim is to create an AI agent capable of selecting good actions in a game-playing domain called Battlespace. Sequential domains like Battlespace are important testbeds for planning problems, as such, the Department of Defense uses such domains for wargaming exercises. The agents we developed combine Monte Carlo Tree Search (MCTS) and Deep Q-Network (DQN) techniques in an effort to navigate the game environment, avoid obstacles, interact with adversaries, and capture the flag. This paper will focus on the encoding techniques we explored to present complex structured data stored in a Python class, a necessary precursor to an agent.
How to Discern Important Urgent News?
Venue: arXiv cs.CL 2024
Year: 2024

We found that a simple property of clusters in a clustered dataset of news correlate strongly with importance and urgency of news (IUN) as assessed by LLM. We verified our finding across different news datasets, dataset sizes, clustering algorithms and embeddings. The found correlation should allow using clustering (as an alternative to LLM) for identifying the most important urgent news, or for filtering out unimportant articles.
HI-GAN: Hierarchical Inpainting GAN with Auxiliary Inputs for Combined RGB and Depth Inpainting
Venue: arXiv cs.CV 2024
Year: 2024

Inpainting involves filling in missing pixels or areas in an image, a crucial technique employed in Mixed Reality environments for various applications, particularly in Diminished Reality (DR) where content is removed from a user's visual environment. Existing methods rely on digital replacement techniques which necessitate multiple cameras and incur high costs. AR devices and smartphones use ToF depth sensors to capture scene depth maps aligned with RGB images. Despite speed and affordability, ToF cameras create imperfect depth maps with missing pixels. To address the above challenges, we propose Hierarchical Inpainting GAN (HI-GAN), a novel approach comprising three GANs in a hierarchical fashion for RGBD inpainting. EdgeGAN and LabelGAN inpaint masked edge and segmentation label images respectively, while CombinedRGBD-GAN combines their latent representation outputs and performs RGB and Depth inpainting. Edge images and particularly segmentation label images as auxiliary inputs significantly enhance inpainting performance by complementary context and hierarchical optimization. We believe we make the first attempt to incorporate label images into inpainting process.Unlike previous approaches requiring multiple sequential models and separate outputs, our work operates in an end-to-end manner, training all three models simultaneously and hierarchically. Specifically, EdgeGAN and LabelGAN are first optimized separately and further optimized inside CombinedRGBD-GAN to enhance inpainting quality. Experiments demonstrate that HI-GAN works seamlessly and achieves overall superior performance compared with existing approaches.
Revisiting Experience Replayable Conditions
Venue: arXiv cs.LG 2024
Year: 2024

Experience replay (ER) used in (deep) reinforcement learning is considered to be applicable only to off-policy algorithms. However, there have been some cases in which ER has been applied for on-policy algorithms, suggesting that off-policyness might be a sufficient condition for applying ER. This paper reconsiders more strict "experience replayable conditions" (ERC) and proposes the way of modifying the existing algorithms to satisfy ERC. In light of this, it is postulated that the instability of policy improvements represents a pivotal factor in ERC. The instability factors are revealed from the viewpoint of metric learning as i) repulsive forces from negative samples and ii) replays of inappropriate experiences. Accordingly, the corresponding stabilization tricks are derived. As a result, it is confirmed through numerical simulations that the proposed stabilization tricks make ER applicable to an advantage actor-critic, an on-policy algorithm. Moreover, its learning performance is comparable to that of a soft actor-critic, a state-of-the-art off-policy algorithm.
LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs
Venue: arXiv cs.LG 2024
Year: 2024

System logs are some of the most important information for the maintenance of software systems, which have become larger and more complex in recent years. The goal of log-based anomaly detection is to automatically detect system anomalies by analyzing the large number of logs generated in a short period of time, which is a critical challenge in the real world. Previous studies have used a log parser to extract templates from unstructured log data and detect anomalies on the basis of patterns of the template occurrences. These methods have limitations for logs with unknown templates. Furthermore, since most log anomalies are known to be point anomalies rather than contextual anomalies, detection methods based on occurrence patterns can cause unnecessary delays in detection. In this paper, we propose LogELECTRA, a new log anomaly detection model that analyzes a single line of log messages more deeply on the basis of self-supervised anomaly detection. LogELECTRA specializes in detecting log anomalies as point anomalies by applying ELECTRA, a natural language processing model, to analyze the semantics of a single line of log messages. LogELECTRA outperformed existing state-of-the-art methods in experiments on the public benchmark log datasets BGL, Sprit, and Thunderbird.
Prompt Learning for Multi-Label Code Smell Detection: A Promising Approach
Venue: arXiv cs.SE 2024
Year: 2024

Code smells indicate the potential problems of software quality so that developers can identify refactoring opportunities by detecting code smells. State-of-the-art approaches leverage heuristics, machine learning, and deep learning to detect code smells. However, existing approaches have not fully explored the potential of large language models (LLMs). In this paper, we propose \textit{PromptSmell}, a novel approach based on prompt learning for detecting multi-label code smell. Firstly, code snippets are acquired by traversing abstract syntax trees. Combined code snippets with natural language prompts and mask tokens, \textit{PromptSmell} constructs the input of LLMs. Secondly, to detect multi-label code smell, we leverage a label combination approach by converting a multi-label problem into a multi-classification problem. A customized answer space is added to the word list of pre-trained language models, and the probability distribution of intermediate answers is obtained by predicting the words at the mask positions. Finally, the intermediate answers are mapped to the target class labels by a verbalizer as the final classification result. We evaluate the effectiveness of \textit{PromptSmell} by answering six research questions. The experimental results demonstrate that \textit{PromptSmell} obtains an improvement of 11.17\% in $precision_{w}$ and 7.4\% in $F1_{w}$ compared to existing approaches.
Chain of Logic: Rule-Based Reasoning with Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Rule-based reasoning, a fundamental type of legal reasoning, enables us to draw conclusions by accurately applying a rule to a set of facts. We explore causal language models as rule-based reasoners, specifically with respect to compositional rules - rules consisting of multiple elements which form a complex logical expression. Reasoning about compositional rules is challenging because it requires multiple reasoning steps, and attending to the logical relationships between elements. We introduce a new prompting method, Chain of Logic, which elicits rule-based reasoning through decomposition (solving elements as independent threads of logic), and recomposition (recombining these sub-answers to resolve the underlying logical expression). This method was inspired by the IRAC (Issue, Rule, Application, Conclusion) framework, a sequential reasoning approach used by lawyers. We evaluate chain of logic across eight rule-based reasoning tasks involving three distinct compositional rules from the LegalBench benchmark and demonstrate it consistently outperforms other prompting methods, including chain of thought and self-ask, using open-source and commercial language models.
Subfield codes of $C_D$-codes over $\mathbb{F}_2[x]/\langle x^3-x \rangle$ are really nice!
Venue: arXiv cs.IT 2024
Year: 2024

A non-zero $\mathbb{F}$-linear map from a finite-dimensional commutative $\mathbb{F}$-algebra to $\mathbb{F}$ is called an $\mathbb{F}$-valued trace if its kernel does not contain any non-zero ideals. In this article, we utilize an $\mathbb{F}_2$-valued trace of the $\mathbb{F}_2$-algebra $\mathcal{R}_2:=\mathbb{F}_2[x]/\langle x^3-x\rangle$ to study binary subfield code $\mathcal{C}_D^{(2)}$ of $\mathcal{C}_D:=\{\left(x\cdot d\right)_{d\in D}: x\in \mathcal{R}_2^m\}$ for each defining set $D$ derived from a certain simplicial complex. For $m\in \mathbb{N}$ and $X\subseteq \{1, 2, \dots, m\}$, define $Δ_X:=\{v\in \mathbb{F}_2^m: \Supp(v)\subseteq X\}$ and $D:=(1+u^2)D_1+u^2D_2+(u+u^2)D_3,$ a subset of $\mathcal{R}_2^m,$ where $u=x+\langle x^3-x\rangle, D_1\in \{Δ_L, Δ_L^c\},\, D_2\in \{Δ_M, Δ_M^c\}$ and $ D_3\in \{Δ_N, Δ_N^c\}$, for $L, M, N\subseteq \{1, 2, \dots, m\}.$ The parameters and the Hamming weight distribution of the binary subfield code $\mathcal{C}_D^{(2)}$ of $\mathcal{C}_D$ are determined for each $D.$ These binary subfield codes are minimal under certain mild conditions on the cardinalities of $L, M$ and $N$. Moreover, most of these codes are distance-optimal. Consequently, we obtain a few infinite families of minimal, self-orthogonal and distance-optimal binary linear codes that are either $2$-weight or $4$-weight. It is worth mentioning that we have obtained several new distance-optimal binary linear codes.
Adversarial Curriculum Graph Contrastive Learning with Pair-wise Augmentation
Venue: arXiv cs.LG 2024
Year: 2024

Graph contrastive learning (GCL) has emerged as a pivotal technique in the domain of graph representation learning. A crucial aspect of effective GCL is the caliber of generated positive and negative samples, which is intrinsically dictated by their resemblance to the original data. Nevertheless, precise control over similarity during sample generation presents a formidable challenge, often impeding the effective discovery of representative graph patterns. To address this challenge, we propose an innovative framework: Adversarial Curriculum Graph Contrastive Learning (ACGCL), which capitalizes on the merits of pair-wise augmentation to engender graph-level positive and negative samples with controllable similarity, alongside subgraph contrastive learning to discern effective graph patterns therein. Within the ACGCL framework, we have devised a novel adversarial curriculum training methodology that facilitates progressive learning by sequentially increasing the difficulty of distinguishing the generated samples. Notably, this approach transcends the prevalent sparsity issue inherent in conventional curriculum learning strategies by adaptively concentrating on more challenging training data. Finally, a comprehensive assessment of ACGCL is conducted through extensive experiments on six well-known benchmark datasets, wherein ACGCL conspicuously surpasses a set of state-of-the-art baselines.
GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians
Venue: arXiv cs.GR 2024
Year: 2024

Hairstyle reflects culture and ethnicity at first glance. In the digital era, various realistic human hairstyles are also critical to high-fidelity digital human assets for beauty and inclusivity. Yet, realistic hair modeling and real-time rendering for animation is a formidable challenge due to its sheer number of strands, complicated structures of geometry, and sophisticated interaction with light. This paper presents GaussianHair, a novel explicit hair representation. It enables comprehensive modeling of hair geometry and appearance from images, fostering innovative illumination effects and dynamic animation capabilities. At the heart of GaussianHair is the novel concept of representing each hair strand as a sequence of connected cylindrical 3D Gaussian primitives. This approach not only retains the hair's geometric structure and appearance but also allows for efficient rasterization onto a 2D image plane, facilitating differentiable volumetric rendering. We further enhance this model with the "GaussianHair Scattering Model", adept at recreating the slender structure of hair strands and accurately capturing their local diffuse color in uniform lighting. Through extensive experiments, we substantiate that GaussianHair achieves breakthroughs in both geometric and appearance fidelity, transcending the limitations encountered in state-of-the-art methods for hair reconstruction. Beyond representation, GaussianHair extends to support editing, relighting, and dynamic rendering of hair, offering seamless integration with conventional CG pipeline workflows. Complementing these advancements, we have compiled an extensive dataset of real human hair, each with meticulously detailed strand geometry, to propel further research in this field.
How People Prompt to Create Interactive VR Scenes
Venue: arXiv cs.HC 2024
Year: 2024

Generative AI tools can provide people with the ability to create virtual environments and scenes with natural language prompts. Yet, how people will formulate such prompts is unclear -- particularly when they inhabit the environment that they are designing. For instance, it is likely that a person might say, "Put a chair here", while pointing at a location. If such linguistic features are common to people's prompts, we need to tune models to accommodate them. In this work, we present a wizard-of-oz elicitation study with 22 participants, where we studied people's implicit expectations when verbally prompting such programming agents to create interactive VR scenes. Our findings show that people prompt with several implicit expectations: (1) that agents have an embodied knowledge of the environment; (2) that agents understand embodied prompts by users; (3) that the agents can recall previous states of the scene and the conversation, and that (4) agents have a commonsense understanding of objects in the scene. Further, we found that participants prompt differently when they are prompting in situ (i.e. within the VR environment) versus ex situ (i.e. viewing the VR environment from the outside). To explore how our could be applied, we designed and built Oastaad, a conversational programming agent that allows non-programmers to design interactive VR experiences that they inhabit. Based on these explorations, we outline new opportunities and challenges for conversational programming agents that create VR environments.
Neural paraphrasing by automatically crawled and aligned sentence pairs
Venue: arXiv cs.CL 2024
Year: 2024

Paraphrasing is the task of re-writing an input text using other words, without altering the meaning of the original content. Conversational systems can exploit automatic paraphrasing to make the conversation more natural, e.g., talking about a certain topic using different paraphrases in different time instants. Recently, the task of automatically generating paraphrases has been approached in the context of Natural Language Generation (NLG). While many existing systems simply consist in rule-based models, the recent success of the Deep Neural Networks in several NLG tasks naturally suggests the possibility of exploiting such networks for generating paraphrases. However, the main obstacle toward neural-network-based paraphrasing is the lack of large datasets with aligned pairs of sentences and paraphrases, that are needed to efficiently train the neural models. In this paper we present a method for the automatic generation of large aligned corpora, that is based on the assumption that news and blog websites talk about the same events using different narrative styles. We propose a similarity search procedure with linguistic constraints that, given a reference sentence, is able to locate the most similar candidate paraphrases out from millions of indexed sentences. The data generation process is evaluated in the case of the Italian language, performing experiments using pointer-based deep neural architectures.
InSaAF: Incorporating Safety through Accuracy and Fairness | Are LLMs ready for the Indian Legal Domain?
Venue: arXiv cs.CL 2024
Year: 2024

Recent advancements in language technology and Artificial Intelligence have resulted in numerous Language Models being proposed to perform various tasks in the legal domain ranging from predicting judgments to generating summaries. Despite their immense potential, these models have been proven to learn and exhibit societal biases and make unfair predictions. In this study, we explore the ability of Large Language Models (LLMs) to perform legal tasks in the Indian landscape when social factors are involved. We present a novel metric, $β$-weighted $\textit{Legal Safety Score ($LSS_β$)}$, which encapsulates both the fairness and accuracy aspects of the LLM. We assess LLMs' safety by considering its performance in the $\textit{Binary Statutory Reasoning}$ task and its fairness exhibition with respect to various axes of disparities in the Indian society. Task performance and fairness scores of LLaMA and LLaMA--2 models indicate that the proposed $LSS_β$ metric can effectively determine the readiness of a model for safe usage in the legal sector. We also propose finetuning pipelines, utilising specialised legal datasets, as a potential method to mitigate bias and improve model safety. The finetuning procedures on LLaMA and LLaMA--2 models increase the $LSS_β$, improving their usability in the Indian legal domain. Our code is publicly released.