OpenRaiser/Intern-Atlas huggingface dataset:
Customizing General-Purpose Foundation Models for Medical Report Generation
Venue: arXiv cs.CV 2023
Year: 2023

Medical caption prediction which can be regarded as a task of medical report generation (MRG), requires the automatic generation of coherent and accurate captions for the given medical images. However, the scarcity of labelled medical image-report pairs presents great challenges in the development of deep and large-scale neural networks capable of harnessing the potential artificial general intelligence power like large language models (LLMs). In this work, we propose customizing off-the-shelf general-purpose large-scale pre-trained models, i.e., foundation models (FMs), in computer vision and natural language processing with a specific focus on medical report generation. Specifically, following BLIP-2, a state-of-the-art vision-language pre-training approach, we introduce our encoder-decoder-based MRG model. This model utilizes a lightweight query Transformer to connect two FMs: the giant vision Transformer EVA-ViT-g and a bilingual LLM trained to align with human intentions (referred to as ChatGLM-6B). Furthermore, we conduct ablative experiments on the trainable components of the model to identify the crucial factors for effective transfer learning. Our findings demonstrate that unfreezing EVA-ViT-g to learn medical image representations, followed by parameter-efficient training of ChatGLM-6B to capture the writing styles of medical reports, is essential for achieving optimal results. Our best attempt (PCLmed Team) achieved the 4th and the 2nd, respectively, out of 13 participating teams, based on the BERTScore and ROUGE-1 metrics, in the ImageCLEFmedical Caption 2023 Caption Prediction Task competition.
Illumination Controllable Dehazing Network based on Unsupervised Retinex Embedding
Venue: arXiv cs.CV 2023
Year: 2023

On the one hand, the dehazing task is an illposedness problem, which means that no unique solution exists. On the other hand, the dehazing task should take into account the subjective factor, which is to give the user selectable dehazed images rather than a single result. Therefore, this paper proposes a multi-output dehazing network by introducing illumination controllable ability, called IC-Dehazing. The proposed IC-Dehazing can change the illumination intensity by adjusting the factor of the illumination controllable module, which is realized based on the interpretable Retinex theory. Moreover, the backbone dehazing network of IC-Dehazing consists of a Transformer with double decoders for high-quality image restoration. Further, the prior-based loss function and unsupervised training strategy enable IC-Dehazing to complete the parameter learning process without the need for paired data. To demonstrate the effectiveness of the proposed IC-Dehazing, quantitative and qualitative experiments are conducted on image dehazing, semantic segmentation, and object detection tasks. Code is available at https://github.com/Xiaofeng-life/ICDehazing.
Embodied Executable Policy Learning with Language-based Scene Summarization
Venue: arXiv cs.RO 2023
Year: 2023

Large Language models (LLMs) have shown remarkable success in assisting robot learning tasks, i.e., complex household planning. However, the performance of pretrained LLMs heavily relies on domain-specific templated text data, which may be infeasible in real-world robot learning tasks with image-based observations. Moreover, existing LLMs with text inputs lack the capability to evolve with non-expert interactions with environments. In this work, we introduce a novel learning paradigm that generates robots' executable actions in the form of text, derived solely from visual observations, using language-based summarization of these observations as the connecting bridge between both domains. Our proposed paradigm stands apart from previous works, which utilized either language instructions or a combination of language and visual data as inputs. Moreover, our method does not require oracle text summarization of the scene, eliminating the need for human involvement in the learning loop, which makes it more practical for real-world robot learning tasks. Our proposed paradigm consists of two modules: the SUM module, which interprets the environment using visual observations and produces a text summary of the scene, and the APM module, which generates executable action policies based on the natural language descriptions provided by the SUM module. We demonstrate that our proposed method can employ two fine-tuning strategies, including imitation learning and reinforcement learning approaches, to adapt to the target test tasks effectively. We conduct extensive experiments involving various SUM/APM model selections, environments, and tasks across 7 house layouts in the VirtualHome environment. Our experimental results demonstrate that our method surpasses existing baselines, confirming the effectiveness of this novel learning paradigm.
CAWL: A Cache-aware Write Performance Model of Linux Systems
Venue: arXiv cs.PF 2023
Year: 2023

The performance of data intensive applications is often dominated by their input/output (I/O) operations but the I/O stack of systems is complex and severely depends on system specific settings and hardware components. This situation makes generic performance optimisation challenging and costly for developers as they would have to run their application on a large variety of systems to evaluate their improvements. Here, simulation frameworks can help reducing the experimental overhead but they typically handle the topic of I/O rather coarse-grained, which leads to significant inaccuracies in performance predictions. Here, we propose a more accurate model of the write performance of Linux-based systems that takes different I/O methods and levels (via system calls, library calls, direct or indirect, etc.), the page cache, background writing, and the I/O throttling capabilities of the Linux kernel into account. With our model, we reduce, for example, the relative prediction error compared to a standard I/O model included in SimGrid for a random I/O scenario from 67 % down to 10 % relative error against real measurements of the simulated workload. In other scenarios the differences are even more pronounced.
DP-HyPO: An Adaptive Private Hyperparameter Optimization Framework
Venue: arXiv cs.LG 2023
Year: 2023

Hyperparameter optimization, also known as hyperparameter tuning, is a widely recognized technique for improving model performance. Regrettably, when training private ML models, many practitioners often overlook the privacy risks associated with hyperparameter optimization, which could potentially expose sensitive information about the underlying dataset. Currently, the sole existing approach to allow privacy-preserving hyperparameter optimization is to uniformly and randomly select hyperparameters for a number of runs, subsequently reporting the best-performing hyperparameter. In contrast, in non-private settings, practitioners commonly utilize ``adaptive'' hyperparameter optimization methods such as Gaussian process-based optimization, which select the next candidate based on information gathered from previous outputs. This substantial contrast between private and non-private hyperparameter optimization underscores a critical concern. In our paper, we introduce DP-HyPO, a pioneering framework for ``adaptive'' private hyperparameter optimization, aiming to bridge the gap between private and non-private hyperparameter optimization. To accomplish this, we provide a comprehensive differential privacy analysis of our framework. Furthermore, we empirically demonstrate the effectiveness of DP-HyPO on a diverse set of real-world datasets.
Flowsim: A Modular Simulation Platform for Microscopic Behavior Analysis of City-Scale Connected Autonomous Vehicles
Venue: arXiv cs.NI 2023
Year: 2023

As connected autonomous vehicles (CAVs) become increasingly prevalent, there is a growing need for simulation platforms that can accurately evaluate CAV behavior in large-scale environments. In this paper, we propose Flowsim, a novel simulator specifically designed to meet these requirements. Flowsim offers a modular and extensible architecture that enables the analysis of CAV behaviors in large-scale scenarios. It provides researchers with a customizable platform for studying CAV interactions, evaluating communication and networking protocols, assessing cybersecurity vulnerabilities, optimizing traffic management strategies, and developing and evaluating policies for CAV deployment. Flowsim is implemented in pure Python in approximately 1,500 lines of code, making it highly readable, understandable, and easily modifiable. We verified the functionality and performance of Flowsim via a series of experiments based on realistic traffic scenarios. The results show the effectiveness of Flowsim in providing a flexible and powerful simulation environment for evaluating CAV behavior and data flow. Flowsim is a valuable tool for researchers, policymakers, and industry professionals who are involved in the development, evaluation, and deployment of CAVs. The code of Flowsim is publicly available on GitHub under the MIT license.
From psychological traits to safety warnings: three studies on recommendations in a smart home environment
Venue: arXiv cs.HC 2023
Year: 2023

In this paper, we report on three experiments we have carried out in the context of the EMPATHY project, with the aim of helping users make better configuration choices in a smart home environment, and discuss our results. We found that there are psychological traits, such as Need for Cognition, which influence the way individuals tend to use recommendations, that there are non obvious relationships between the perceived usefulness of recommendations in different domains and individuals' ability to exploit suggestions on configuration choices, and that detailed, easy-to-understand security explanations are more persuasive than simple security warnings, when it comes to make decisions on the applicability of rules which might cause privacy and security risks.
Preliminary Guideline for Creating Boundary Artefacts in Software Engineering
Venue: arXiv cs.SE 2023
Year: 2023

Context: Software development benefits from having Boundary Artefacts (BAs), as a single artefact can supply stakeholders with different boundaries, facilitating collaboration among social worlds. When those artefacts display inconsistencies, such as incorrect information, the practitioners have decreased trust in the BA. As trust is an essential factor guiding the utilisation of BAs in software projects, it is necessary to understand which principles should be observed when creating them. Objective: This study aimed at develop and validate a preliminary guideline support the creation of trustworthy BAs. Method: We followed a multi-step approach. We developed our guideline through a literature review and previous results from our case study. Second, we submitted the guideline for an expert evaluation via two workshops and a survey. At last, we adjusted our guideline by incorporating the feedback obtained during the workshops. Results: We grouped the principles collected from a literature review into three categories. The first category (Scope) focuses on the scope, displaying principles referring to defining each boundary's target audience, needs, and terminology. The second category (Structure) relates to how the artefact's content is structured to meet stakeholders' needs. The third (Management) refers to principles that can guide the establishment of practices to manage the artefact throughout time. The expert validation revealed that the principles contribute to creating trustworthy BAs at different levels. Also, the relevance of the guideline and its usefulness. Conclusions: The guideline strengthen BA traits such as shared understanding, plasticity and ability to transfer. Practitioners can utilise the guideline to guide the creation or even evaluate current practices for existing BAs.
Two-level histograms for dealing with outliers and heavy tail distributions
Venue: arXiv cs.LG 2023
Year: 2023

Histograms are among the most popular methods used in exploratory analysis to summarize univariate distributions. In particular, irregular histograms are good non-parametric density estimators that require very few parameters: the number of bins with their lengths and frequencies. Many approaches have been proposed in the literature to infer these parameters, either assuming hypotheses about the underlying data distributions or exploiting a model selection approach. In this paper, we focus on the G-Enum histogram method, which exploits the Minimum Description Length (MDL) principle to build histograms without any user parameter and achieves state-of-the art performance w.r.t accuracy; parsimony and computation time. We investigate on the limits of this method in the case of outliers or heavy-tailed distributions. We suggest a two-level heuristic to deal with such cases. The first level exploits a logarithmic transformation of the data to split the data set into a list of data subsets with a controlled range of values. The second level builds a sub-histogram for each data subset and aggregates them to obtain a complete histogram. Extensive experiments show the benefits of the approach.
Towards the Exploitation of LLM-based Chatbot for Providing Legal Support to Palestinian Cooperatives
Venue: arXiv cs.CL 2023
Year: 2023

With the ever-increasing utilization of natural language processing (NLP), we started to witness over the past few years a significant transformation in our interaction with legal texts. This technology has advanced the analysis and enhanced the understanding of complex legal terminology and contexts. The development of recent large language models (LLMs), particularly ChatGPT, has also introduced a revolutionary contribution to the way that legal texts can be processed and comprehended. In this paper, we present our work on a cooperative-legal question-answering LLM-based chatbot, where we developed a set of legal questions about Palestinian cooperatives, associated with their regulations and compared the auto-generated answers by the chatbot to their correspondences that are designed by a legal expert. To evaluate the proposed chatbot, we have used 50 queries generated by the legal expert and compared the answers produced by the chart to their relevance judgments. Finding demonstrated that an overall accuracy rate of 82% has been achieved when answering the queries, while exhibiting an F1 score equivalent to 79%.
Sketch Beautification: Learning Part Beautification and Structure Refinement for Sketches of Man-made Objects
Venue: arXiv cs.CV 2023
Year: 2023

We present a novel freehand sketch beautification method, which takes as input a freely drawn sketch of a man-made object and automatically beautifies it both geometrically and structurally. Beautifying a sketch is challenging because of its highly abstract and heavily diverse drawing manner. Existing methods are usually confined to the distribution of their limited training samples and thus cannot beautify freely drawn sketches with rich variations. To address this challenge, we adopt a divide-and-combine strategy. Specifically, we first parse an input sketch into semantic components, beautify individual components by a learned part beautification module based on part-level implicit manifolds, and then reassemble the beautified components through a structure beautification module. With this strategy, our method can go beyond the training samples and handle novel freehand sketches. We demonstrate the effectiveness of our system with extensive experiments and a perceptive study.
A Complete Proof Synthesis Method for the Cube of Type Systems
Venue: arXiv cs.LO 2023
Year: 2023

We present a complete proof synthesis method for the eight type systems of Barendregt's cube extended with $η$-conversion. Because these systems verify the proofs-as-objects paradigm, the proof synthesis method is a one level process merging unification and resolution. Then we present a variant of this method, which is incomplete but much more efficient. At last we show how to turn this algorithm into a unification algorithm.
The Undecidability of Pattern Matching in Calculi where Primitive Recursive Functions are Representable
Venue: arXiv cs.LO 2023
Year: 2023

We prove that the pattern matching problem is undecidable in polymorphic lambda-calculi (as Girard's system F) and calculi supporting inductive types (as G{ö}del's system T) by reducing Hilbert's tenth problem to it. More generally pattern matching is undecidable in all the calculi in which primitive recursive functions can be fairly represented in a precised sense.
Sketch2Stress: Sketching with Structural Stress Awareness
Venue: arXiv cs.CV 2023
Year: 2023

In the process of product design and digital fabrication, the structural analysis of a designed prototype is a fundamental and essential step. However, such a step is usually invisible or inaccessible to designers at the early sketching phase. This limits the user's ability to consider a shape's physical properties and structural soundness. To bridge this gap, we introduce a novel approach Sketch2Stress that allows users to perform structural analysis of desired objects at the sketching stage. This method takes as input a 2D freehand sketch and one or multiple locations of user-assigned external forces. With the specially-designed two-branch generative-adversarial framework, it automatically predicts a normal map and a corresponding structural stress map distributed over the user-sketched underlying object. In this way, our method empowers designers to easily examine the stress sustained everywhere and identify potential problematic regions of their sketched object. Furthermore, combined with the predicted normal map, users are able to conduct a region-wise structural analysis efficiently by aggregating the stress effects of multiple forces in the same direction. Finally, we demonstrate the effectiveness and practicality of our system with extensive experiments and user studies.
Distributed Consensus Algorithm for Decision-Making in Multi-agent Multi-armed Bandit
Venue: arXiv cs.LG 2023
Year: 2023

We study a structured multi-agent multi-armed bandit (MAMAB) problem in a dynamic environment. A graph reflects the information-sharing structure among agents, and the arms' reward distributions are piecewise-stationary with several unknown change points. The agents face the identical piecewise-stationary MAB problem. The goal is to develop a decision-making policy for the agents that minimizes the regret, which is the expected total loss of not playing the optimal arm at each time step. Our proposed solution, Restarted Bayesian Online Change Point Detection in Cooperative Upper Confidence Bound Algorithm (RBO-Coop-UCB), involves an efficient multi-agent UCB algorithm as its core enhanced with a Bayesian change point detector. We also develop a simple restart decision cooperation that improves decision-making. Theoretically, we establish that the expected group regret of RBO-Coop-UCB is upper bounded by $\mathcal{O}(KNM\log T + K\sqrt{MT\log T})$, where K is the number of agents, M is the number of arms, and T is the number of time steps. Numerical experiments on synthetic and real-world datasets demonstrate that our proposed method outperforms the state-of-the-art algorithms.
SoK: Analysis of User-Centered Studies Focusing on Healthcare Privacy & Security
Venue: arXiv cs.HC 2023
Year: 2023

Sensitive information is intrinsically tied to interactions in healthcare, and its protection is of paramount importance for achieving high-quality patient outcomes. Research in healthcare privacy and security is predominantly focused on understanding the factors that increase the susceptibility of users to privacy and security breaches. To understand further, we systematically review 26 research papers in this domain to explore the existing user studies in healthcare privacy and security. Following the review, we conducted a card-sorting exercise, allowing us to identify 12 themes integral to this subject such as "Data Sharing," "Risk Awareness," and "Privacy." Further to the identification of these themes, we performed an in-depth analysis of the 26 research papers report on the insights into the discourse within the research community about healthcare privacy and security, particularly from the user perspective.
Cheating off your neighbors: Improving activity recognition through corroboration
Venue: arXiv cs.CV 2023
Year: 2023

Understanding the complexity of human activities solely through an individual's data can be challenging. However, in many situations, surrounding individuals are likely performing similar activities, while existing human activity recognition approaches focus almost exclusively on individual measurements and largely ignore the context of the activity. Consider two activities: attending a small group meeting and working at an office desk. From solely an individual's perspective, it can be difficult to differentiate between these activities as they may appear very similar, even though they are markedly different. Yet, by observing others nearby, it can be possible to distinguish between these activities. In this paper, we propose an approach to enhance the prediction accuracy of an individual's activities by incorporating insights from surrounding individuals. We have collected a real-world dataset from 20 participants with over 58 hours of data including activities such as attending lectures, having meetings, working in the office, and eating together. Compared to observing a single person in isolation, our proposed approach significantly improves accuracy. We regard this work as a first step in collaborative activity recognition, opening new possibilities for understanding human activity in group settings.
Learning Not to Spoof
Venue: arXiv cs.LG 2023
Year: 2023

As intelligent trading agents based on reinforcement learning (RL) gain prevalence, it becomes more important to ensure that RL agents obey laws, regulations, and human behavioral expectations. There is substantial literature concerning the aversion of obvious catastrophes like crashing a helicopter or bankrupting a trading account, but little around the avoidance of subtle non-normative behavior for which there are examples, but no programmable definition. Such behavior may violate legal or regulatory, rather than physical or monetary, constraints. In this article, I consider a series of experiments in which an intelligent stock trading agent maximizes profit but may also inadvertently learn to spoof the market in which it participates. I first inject a hand-coded spoofing agent to a multi-agent market simulation and learn to recognize spoofing activity sequences. Then I replace the hand-coded spoofing trader with a simple profit-maximizing RL agent and observe that it independently discovers spoofing as the optimal strategy. Finally, I introduce a method to incorporate the recognizer as normative guide, shaping the agent's perceived rewards and altering its selected actions. The agent remains profitable while avoiding spoofing behaviors that would result in even higher profit. After presenting the empirical results, I conclude with some recommendations. The method should generalize to the reduction of any unwanted behavior for which a recognizer can be learned.