OpenRaiser/Intern-Atlas huggingface dataset:
Two new algorithms for solving Müller games and their applications
Venue: arXiv cs.GT 2023
Year: 2023

Müller games form a well-established class of games for model checking and verification. These games are played on directed graphs $\mathcal G$ where Player 0 and Player 1 play by generating an infinite path through the graph. The winner is determined by the set $X$ consisting of all vertices in the path that occur infinitely often. If $X$ belongs to $Ω$, a specified collection of subsets of $\mathcal G$, then Player 0 wins. Otherwise, Player 1 claims the win. These games are determined, enabling the partitioning of $\mathcal G$ into two sets $W_0$ and $W_1$ of winning positions for Player 0 and Player 1, respectively. Numerous algorithms exist that decide Müller games $\mathcal G$ by computing the sets $W_0$ and $W_1$. In this paper, we introduce two novel algorithms that outperform all previously known methods for deciding explicitly given Müller games, especially in the worst-case scenarios. The previously known algorithms either reduce Müller games to other known games (e.g. safety games) or recursively change the underlying graph $\mathcal G$ and the collection of sets in $Ω$. In contrast, our approach does not employ these techniques but instead leverages subgames, the sets within $Ω$, and their interactions. This distinct methodology sets our algorithms apart from prior approaches for deciding Müller games. Additionally, our algorithms offer enhanced clarity and ease of comprehension. Importantly, our techniques are applicable not only to Müller games but also to improving the performance of existing algorithms that handle other game classes, including coloured Müller games, McNaughton games, Rabin games, and Streett games.
The Quest for Content: A Survey of Search-Based Procedural Content Generation for Video Games
Venue: arXiv cs.SE 2023
Year: 2023

Video games demand is constantly increasing, which requires the costly production of large amounts of content. Towards this challenge, researchers have developed Search-Based Procedural Content Generation (SBPCG), that is, the (semi-)automated creation of content through search algorithms. We survey the current state of SBPCG, reporting work appeared in the field between 2011-2022 and identifying open research challenges. The results lead to recommendations for practitioners and to the identification of several potential future research avenues for SBPCG.
Towards Understanding Emotions in Informal Developer Interactions: A Gitter Chat Study
Venue: arXiv cs.SE 2023
Year: 2023

Emotions play a significant role in teamwork and collaborative activities like software development. While researchers have analyzed developer emotions in various software artifacts (e.g., issues, pull requests), few studies have focused on understanding the broad spectrum of emotions expressed in chats. As one of the most widely used means of communication, chats contain valuable information in the form of informal conversations, such as negative perspectives about adopting a tool. In this paper, we present a dataset of developer chat messages manually annotated with a wide range of emotion labels (and sub-labels), and analyze the type of information present in those messages. We also investigate the unique signals of emotions specific to chats and distinguish them from other forms of software communication. Our findings suggest that chats have fewer expressions of Approval and Fear but more expressions of Curiosity compared to GitHub comments. We also notice that Confusion is frequently observed when discussing programming-related information such as unexpected software behavior. Overall, our study highlights the potential of mining emotions in developer chats for supporting software maintenance and evolution tools.
VioLA: Aligning Videos to 2D LiDAR Scans
Venue: arXiv cs.CV 2023
Year: 2023

We study the problem of aligning a video that captures a local portion of an environment to the 2D LiDAR scan of the entire environment. We introduce a method (VioLA) that starts with building a semantic map of the local scene from the image sequence, then extracts points at a fixed height for registering to the LiDAR map. Due to reconstruction errors or partial coverage of the camera scan, the reconstructed semantic map may not contain sufficient information for registration. To address this problem, VioLA makes use of a pre-trained text-to-image inpainting model paired with a depth completion model for filling in the missing scene content in a geometrically consistent fashion to support pose registration. We evaluate VioLA on two real-world RGB-D benchmarks, as well as a self-captured dataset of a large office scene. Notably, our proposed scene completion module improves the pose registration performance by up to 20%.
Be Careful When Evaluating Explanations Regarding Ground Truth
Venue: arXiv cs.CV 2023
Year: 2023

Evaluating explanations of image classifiers regarding ground truth, e.g. segmentation masks defined by human perception, primarily evaluates the quality of the models under consideration rather than the explanation methods themselves. Driven by this observation, we propose a framework for $\textit{jointly}$ evaluating the robustness of safety-critical systems that $\textit{combine}$ a deep neural network with an explanation method. These are increasingly used in real-world applications like medical image analysis or robotics. We introduce a fine-tuning procedure to (mis)align model$\unicode{x2013}$explanation pipelines with ground truth and use it to quantify the potential discrepancy between worst and best-case scenarios of human alignment. Experiments across various model architectures and post-hoc local interpretation methods provide insights into the robustness of vision transformers and the overall vulnerability of such AI systems to potential adversarial attacks.
Anonymizing medical case-based explanations through disentanglement
Venue: arXiv cs.CV 2023
Year: 2023

Case-based explanations are an intuitive method to gain insight into the decision-making process of deep learning models in clinical contexts. However, medical images cannot be shared as explanations due to privacy concerns. To address this problem, we propose a novel method for disentangling identity and medical characteristics of images and apply it to anonymize medical images. The disentanglement mechanism replaces some feature vectors in an image while ensuring that the remaining features are preserved, obtaining independent feature vectors that encode the images' identity and medical characteristics. We also propose a model to manufacture synthetic privacy-preserving identities to replace the original image's identity and achieve anonymization. The models are applied to medical and biometric datasets, demonstrating their capacity to generate realistic-looking anonymized images that preserve their original medical content. Additionally, the experiments show the network's inherent capacity to generate counterfactual images through the replacement of medical features.
Learning to Control under Uncertainty with Data-Based Iterative Linear Quadratic Regulator
Venue: arXiv cs.RO 2023
Year: 2023

This paper studies the learning-to-control problem under process and sensing uncertainties for dynamical systems. In our previous work, we developed a data-based generalization of the iterative linear quadratic regulator (iLQR) to design closed-loop feedback control for high-dimensional dynamical systems with partial state observation. This method required perfect simulation rollouts which are not realistic in real applications. In this work, we briefly introduce this method and explore its efficacy under process and sensing uncertainties. We prove that in the fully observed case where the system dynamics are corrupted with noise but the measurements are perfect, it still converges to the global minimum. However, in the partially observed case where both process and measurement noise exist in the system, this method converges to a biased "optimum". Thus multiple rollouts need to be averaged to retrieve the true optimum. The analysis is verified in two nonlinear robotic examples simulated in the above cases.
SciDataFlow: A Tool for Improving the Flow of Data through Science
Venue: arXiv cs.DL 2023
Year: 2023

Managing data and code in open scientific research is complicated by two key problems: large datasets often cannot be stored alongside code in repository platforms like GitHub, and iterative analysis can lead to unnoticed changes to data, increasing the risk that analyses are based on older versions of data. Here, I introduce SciDataFlow: a fast, concurrent command-line tool paired with a simple Data Manifest specification. SciDataFlow streamlines tracking data changes, uploading data to remote repositories, and pulling in all data necessary to reproduce a computational analysis.
An Improved Transformer-based Model for Detecting Phishing, Spam, and Ham: A Large Language Model Approach
Venue: arXiv cs.CL 2023
Year: 2023

Phishing and spam detection is long standing challenge that has been the subject of much academic research. Large Language Models (LLM) have vast potential to transform society and provide new and innovative approaches to solve well-established challenges. Phishing and spam have caused financial hardships and lost time and resources to email users all over the world and frequently serve as an entry point for ransomware threat actors. While detection approaches exist, especially heuristic-based approaches, LLMs offer the potential to venture into a new unexplored area for understanding and solving this challenge. LLMs have rapidly altered the landscape from business, consumers, and throughout academia and demonstrate transformational potential for the potential of society. Based on this, applying these new and innovative approaches to email detection is a rational next step in academic research. In this work, we present IPSDM, our model based on fine-tuning the BERT family of models to specifically detect phishing and spam email. We demonstrate our fine-tuned version, IPSDM, is able to better classify emails in both unbalanced and balanced datasets. This work serves as an important first step towards employing LLMs to improve the security of our information systems.
More Robots are Coming: Large Multimodal Models (ChatGPT) can Solve Visually Diverse Images of Parsons Problems
Venue: arXiv cs.CL 2023
Year: 2023

The advent of large language models is reshaping computing education. Recent research has demonstrated that these models can produce better explanations than students, answer multiple-choice questions at or above the class average, and generate code that can pass automated tests in introductory courses. These capabilities have prompted instructors to rapidly adapt their courses and assessment methods to accommodate changes in learning objectives and the potential for academic integrity violations. While some scholars have advocated for the integration of visual problems as a safeguard against the capabilities of language models, new multimodal language models now have vision and language capabilities that may allow them to analyze and solve visual problems. In this paper, we evaluate the performance of two large multimodal models on visual assignments, with a specific focus on Parsons problems presented across diverse visual representations. Our results show that GPT-4V solved 96.7\% of these visual problems, struggling minimally with a single Parsons problem. Conversely, Bard performed poorly by only solving 69.2\% of problems, struggling with common issues like hallucinations and refusals. These findings suggest that merely transitioning to visual programming problems might not be a panacea to issues of academic integrity in the generative AI era.
GPT4All: An Ecosystem of Open Source Compressed Language Models
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) have recently achieved human-level performance on a range of professional and academic benchmarks. The accessibility of these models has lagged behind their performance. State-of-the-art LLMs require costly infrastructure; are only accessible via rate-limited, geo-locked, and censored web interfaces; and lack publicly available code and technical reports. In this paper, we tell the story of GPT4All, a popular open source repository that aims to democratize access to LLMs. We outline the technical details of the original GPT4All model family, as well as the evolution of the GPT4All project from a single model into a fully fledged open source ecosystem. It is our hope that this paper acts as both a technical overview of the original GPT4All models as well as a case study on the subsequent growth of the GPT4All open source ecosystem.
Leveraging a realistic synthetic database to learn Shape-from-Shading for estimating the colon depth in colonoscopy images
Venue: arXiv cs.CV 2023
Year: 2023

Colonoscopy is the choice procedure to diagnose colon and rectum cancer, from early detection of small precancerous lesions (polyps), to confirmation of malign masses. However, the high variability of the organ appearance and the complex shape of both the colon wall and structures of interest make this exploration difficult. Learned visuospatial and perceptual abilities mitigate technical limitations in clinical practice by proper estimation of the intestinal depth. This work introduces a novel methodology to estimate colon depth maps in single frames from monocular colonoscopy videos. The generated depth map is inferred from the shading variation of the colon wall with respect to the light source, as learned from a realistic synthetic database. Briefly, a classic convolutional neural network architecture is trained from scratch to estimate the depth map, improving sharp depth estimations in haustral folds and polyps by a custom loss function that minimizes the estimation error in edges and curvatures. The network was trained by a custom synthetic colonoscopy database herein constructed and released, composed of 248,400 frames (47 videos), with depth annotations at the level of pixels. This collection comprehends 5 subsets of videos with progressively higher levels of visual complexity. Evaluation of the depth estimation with the synthetic database reached a threshold accuracy of 95.65%, and a mean-RMSE of 0.451 cm, while a qualitative assessment with a real database showed consistent depth estimations, visually evaluated by the expert gastroenterologist coauthoring this paper. Finally, the method achieved competitive performance with respect to another state-of-the-art method using a public synthetic database and comparable results in a set of images with other five state-of-the-art methods.
Automated Annotation of Scientific Texts for ML-based Keyphrase Extraction and Validation
Venue: arXiv cs.IR 2023
Year: 2023

Advanced omics technologies and facilities generate a wealth of valuable data daily; however, the data often lacks the essential metadata required for researchers to find and search them effectively. The lack of metadata poses a significant challenge in the utilization of these datasets. Machine learning-based metadata extraction techniques have emerged as a potentially viable approach to automatically annotating scientific datasets with the metadata necessary for enabling effective search. Text labeling, usually performed manually, plays a crucial role in validating machine-extracted metadata. However, manual labeling is time-consuming; thus, there is an need to develop automated text labeling techniques in order to accelerate the process of scientific innovation. This need is particularly urgent in fields such as environmental genomics and microbiome science, which have historically received less attention in terms of metadata curation and creation of gold-standard text mining datasets. In this paper, we present two novel automated text labeling approaches for the validation of ML-generated metadata for unlabeled texts, with specific applications in environmental genomics. Our techniques show the potential of two new ways to leverage existing information about the unlabeled texts and the scientific domain. The first technique exploits relationships between different types of data sources related to the same research study, such as publications and proposals. The second technique takes advantage of domain-specific controlled vocabularies or ontologies. In this paper, we detail applying these approaches for ML-generated metadata validation. Our results show that the proposed label assignment approaches can generate both generic and highly-specific text labels for the unlabeled texts, with up to 44% of the labels matching with those suggested by a ML keyword extraction algorithm.
Meta-learning of semi-supervised learning from tasks with heterogeneous attribute spaces
Venue: arXiv cs.LG 2023
Year: 2023

We propose a meta-learning method for semi-supervised learning that learns from multiple tasks with heterogeneous attribute spaces. The existing semi-supervised meta-learning methods assume that all tasks share the same attribute space, which prevents us from learning with a wide variety of tasks. With the proposed method, the expected test performance on tasks with a small amount of labeled data is improved with unlabeled data as well as data in various tasks, where the attribute spaces are different among tasks. The proposed method embeds labeled and unlabeled data simultaneously in a task-specific space using a neural network, and the unlabeled data's labels are estimated by adapting classification or regression models in the embedding space. For the neural network, we develop variable-feature self-attention layers, which enable us to find embeddings of data with different attribute spaces with a single neural network by considering interactions among examples, attributes, and labels. Our experiments on classification and regression datasets with heterogeneous attribute spaces demonstrate that our proposed method outperforms the existing meta-learning and semi-supervised learning methods.
ScribblePolyp: Scribble-Supervised Polyp Segmentation through Dual Consistency Alignment
Venue: arXiv cs.CV 2023
Year: 2023

Automatic polyp segmentation models play a pivotal role in the clinical diagnosis of gastrointestinal diseases. In previous studies, most methods relied on fully supervised approaches, necessitating pixel-level annotations for model training. However, the creation of pixel-level annotations is both expensive and time-consuming, impeding the development of model generalization. In response to this challenge, we introduce ScribblePolyp, a novel scribble-supervised polyp segmentation framework. Unlike fully-supervised models, ScribblePolyp only requires the annotation of two lines (scribble labels) for each image, significantly reducing the labeling cost. Despite the coarse nature of scribble labels, which leave a substantial portion of pixels unlabeled, we propose a two-branch consistency alignment approach to provide supervision for these unlabeled pixels. The first branch employs transformation consistency alignment to narrow the gap between predictions under different transformations of the same input image. The second branch leverages affinity propagation to refine predictions into a soft version, extending additional supervision to unlabeled pixels. In summary, ScribblePolyp is an efficient model that does not rely on teacher models or moving average pseudo labels during training. Extensive experiments on the SUN-SEG dataset underscore the effectiveness of ScribblePolyp, achieving a Dice score of 0.8155, with the potential for a 1.8% improvement in the Dice score through a straightforward self-training strategy.
Large Language Models and Prompt Engineering for Biomedical Query Focused Multi-Document Summarisation
Venue: arXiv cs.CL 2023
Year: 2023

This paper reports on the use of prompt engineering and GPT-3.5 for biomedical query-focused multi-document summarisation. Using GPT-3.5 and appropriate prompts, our system achieves top ROUGE-F1 results in the task of obtaining short-paragraph-sized answers to biomedical questions in the 2023 BioASQ Challenge (BioASQ 11b). This paper confirms what has been observed in other domains: 1) Prompts that incorporated few-shot samples generally improved on their counterpart zero-shot variants; 2) The largest improvement was achieved by retrieval augmented generation. The fact that these prompts allow our top runs to rank within the top two runs of BioASQ 11b demonstrate the power of using adequate prompts for Large Language Models in general, and GPT-3.5 in particular, for query-focused summarisation.
Probabilistic Model Checking of Temporal Interaction Dynamics in the Supreme Court
Venue: arXiv cs.LO 2023
Year: 2023

The Supreme Court plays an extremely critical role in ensuring adherence to the rule of law and in strengthening the democracy. Due to this reason, modeling and analysis of small group interactions in the courtroom setting is an important task as it can help in understanding court decision-making. We apply probabilistic model checking for the modeling and analysis of temporal interaction dynamics in the context of the Supreme Court of the United States. We have used the transcripts of the oral arguments of cases from the Supreme Court for constructing a discrete-time Markov reward model (DTMRM). Next, we formulate interesting queries over interaction by using probabilistic computation tree logic (PCTL) and PCTL with rewards and verify them using a probabilistic symbolic model checker (PRISM). Our experimental results show that probabilistic model checking is very effective in identifying trends, hidden patterns, and how justices behave during the trials. These results not only provide valuable feedback to the justices but may also be used by the advocates and law students for finding better ways to present their arguments in the court.
Forceps with direct torque control
Venue: arXiv cs.RO 2023
Year: 2023

This study presents a conceptual design of laparoscopic forceps whose grasping torque can be directly controlled by the user. By integrating an adjustable constant torque mechanism, the handle opening angle is converted to the grasping torque irrespective of the jaw opening angle. This feature overcomes the limitation regarding of the lack of direct haptic feedback in laparoscopic minimally invasive surgery, preventing damage of delicate tissue during forceps grasping.
From "What" to "When" -- a Spiking Neural Network Predicting Rare Events and Time to their Occurrence
Venue: arXiv cs.NE 2023
Year: 2023

In the reinforcement learning (RL) tasks, the ability to predict receiving reward in the near or more distant future means the ability to evaluate the current state as more or less close to the target state (labelled by the reward signal). In the present work, we utilize a spiking neural network (SNN) to predict time to the next target event (reward - in case of RL). In the context of SNNs, events are represented as spikes emitted by network neurons or input nodes. It is assumed that target events are indicated by spikes emitted by a special network input node. Using description of the current state encoded in the form of spikes from the other input nodes, the network should predict approximate time of the next target event. This research paper presents a novel approach to learning the corresponding predictive model by an SNN consisting of leaky integrate-and-fire (LIF) neurons. The proposed method leverages specially designed local synaptic plasticity rules and a novel columnar-layered SNN architecture. Similar to our previous works, this study places a strong emphasis on the hardware-friendliness of the proposed models, ensuring their efficient implementation on modern and future neuroprocessors. The approach proposed was tested on a simple reward prediction task in the context of one of the RL benchmark ATARI games, ping-pong. It was demonstrated that the SNN described in this paper gives superior prediction accuracy in comparison with precise machine learning techniques, such as decision tree algorithms and convolutional neural networks.