OpenRaiser/Intern-Atlas huggingface dataset:
RAD-PHI2: Instruction Tuning PHI-2 for Radiology
Venue: arXiv cs.CL 2024
Year: 2024

Small Language Models (SLMs) have shown remarkable performance in general domain language understanding, reasoning and coding tasks, but their capabilities in the medical domain, particularly concerning radiology text, is less explored. In this study, we investigate the application of SLMs for general radiology knowledge specifically question answering related to understanding of symptoms, radiological appearances of findings, differential diagnosis, assessing prognosis, and suggesting treatments w.r.t diseases pertaining to different organ systems. Additionally, we explore the utility of SLMs in handling text-related tasks with respect to radiology reports within AI-driven radiology workflows. We fine-tune Phi-2, a SLM with 2.7 billion parameters using high-quality educational content from Radiopaedia, a collaborative online radiology resource. The resulting language model, RadPhi-2-Base, exhibits the ability to address general radiology queries across various systems (e.g., chest, cardiac). Furthermore, we investigate Phi-2 for instruction tuning, enabling it to perform specific tasks. By fine-tuning Phi-2 on both general domain tasks and radiology-specific tasks related to chest X-ray reports, we create Rad-Phi2. Our empirical results reveal that Rad-Phi2 Base and Rad-Phi2 perform comparably or even outperform larger models such as Mistral-7B-Instruct-v0.2 and GPT-4 providing concise and precise answers. In summary, our work demonstrates the feasibility and effectiveness of utilizing SLMs in radiology workflows both for knowledge related queries as well as for performing specific tasks related to radiology reports thereby opening up new avenues for enhancing the quality and efficiency of radiology practice.
Evaluating the Application of Large Language Models to Generate Feedback in Programming Education
Venue: arXiv cs.CL 2024
Year: 2024

This study investigates the application of large language models, specifically GPT-4, to enhance programming education. The research outlines the design of a web application that uses GPT-4 to provide feedback on programming tasks, without giving away the solution. A web application for working on programming tasks was developed for the study and evaluated with 51 students over the course of one semester. The results show that most of the feedback generated by GPT-4 effectively addressed code errors. However, challenges with incorrect suggestions and hallucinated issues indicate the need for further improvements.
BOP Challenge 2023 on Detection, Segmentation and Pose Estimation of Seen and Unseen Rigid Objects
Venue: arXiv cs.CV 2024
Year: 2024

We present the evaluation methodology, datasets and results of the BOP Challenge 2023, the fifth in a series of public competitions organized to capture the state of the art in model-based 6D object pose estimation from an RGB/RGB-D image and related tasks. Besides the three tasks from 2022 (model-based 2D detection, 2D segmentation, and 6D localization of objects seen during training), the 2023 challenge introduced new variants of these tasks focused on objects unseen during training. In the new tasks, methods were required to learn new objects during a short onboarding stage (max 5 minutes, 1 GPU) from provided 3D object models. The best 2023 method for 6D localization of unseen objects (GenFlow) notably reached the accuracy of the best 2020 method for seen objects (CosyPose), although being noticeably slower. The best 2023 method for seen objects (GPose) achieved a moderate accuracy improvement but a significant 43% run-time improvement compared to the best 2022 counterpart (GDRNPP). Since 2017, the accuracy of 6D localization of seen objects has improved by more than 50% (from 56.9 to 85.6 AR_C). The online evaluation system stays open and is available at: http://bop.felk.cvut.cz/.
iBRF: Improved Balanced Random Forest Classifier
Venue: arXiv cs.LG 2024
Year: 2024

Class imbalance poses a major challenge in different classification tasks, which is a frequently occurring scenario in many real-world applications. Data resampling is considered to be the standard approach to address this issue. The goal of the technique is to balance the class distribution by generating new samples or eliminating samples from the data. A wide variety of sampling techniques have been proposed over the years to tackle this challenging problem. Sampling techniques can also be incorporated into the ensemble learning framework to obtain more generalized prediction performance. Balanced Random Forest (BRF) and SMOTE-Bagging are some of the popular ensemble approaches. In this study, we propose a modification to the BRF classifier to enhance the prediction performance. In the original algorithm, the Random Undersampling (RUS) technique was utilized to balance the bootstrap samples. However, randomly eliminating too many samples from the data leads to significant data loss, resulting in a major decline in performance. We propose to alleviate the scenario by incorporating a novel hybrid sampling approach to balance the uneven class distribution in each bootstrap sub-sample. Our proposed hybrid sampling technique, when incorporated into the framework of the Random Forest classifier, termed as iBRF: improved Balanced Random Forest classifier, achieves better prediction performance than other sampling techniques used in imbalanced classification tasks. Experiments were carried out on 44 imbalanced datasets on which the original BRF classifier produced an average MCC score of 47.03% and an F1 score of 49.09%. Our proposed algorithm outperformed the approach by producing a far better MCC score of 53.04% and an F1 score of 55%. The results obtained signify the superiority of the iBRF algorithm and its potential to be an effective sampling technique in imbalanced learning.
The Influence of Extended Reality and Virtual Characters' Embodiment Levels on User Experience in Well-Being Activities
Venue: arXiv cs.HC 2024
Year: 2024

Millions of people have seen their daily habits transform, reducing physical activity and leading to mental health issues. This study explores how virtual characters impact motivation for well-being. Three prototypes with cartoon, robotic, and human-like avatars were tested by 22 participants. Results show that animated virtual avatars, especially with extended reality, boost motivation, enhance comprehension of activities, and heighten presence. Multiple output modalities, like audio and text, with character animations, improve the user experience. Notably, the cartoon-like character evoked positive responses. This research highlights virtual characters' potential to engage individuals in daily well-being activities.
Attention-based Class-Conditioned Alignment for Multi-Source Domain Adaptation of Object Detectors
Venue: arXiv cs.CV 2024
Year: 2024

Domain adaptation methods for object detection (OD) strive to mitigate the impact of distribution shifts by promoting feature alignment across source and target domains. Multi-source domain adaptation (MSDA) allows leveraging multiple annotated source datasets and unlabeled target data to improve the accuracy and robustness of the detection model. Most state-of-the-art MSDA methods for OD perform feature alignment in a class-agnostic manner. This is challenging since the objects have unique modality information due to variations in object appearance across domains. A recent prototype-based approach proposed a class-wise alignment, yet it suffers from error accumulation caused by noisy pseudo-labels that can negatively affect adaptation with imbalanced data. To overcome these limitations, we propose an attention-based class-conditioned alignment method for MSDA, designed to align instances of each object category across domains. In particular, an attention module combined with an adversarial domain classifier allows learning domain-invariant and class-specific instance representations. Experimental results on multiple benchmarking MSDA datasets indicate that our method outperforms state-of-the-art methods and exhibits robustness to class imbalance, achieved through a conceptually simple class-conditioning strategy. Our code is available at: https://github.com/imatif17/ACIA.
Search-based Ordered Password Generation of Autoregressive Neural Networks
Venue: arXiv cs.CR 2024
Year: 2024

Passwords are the most widely used method of authentication and password guessing is the essential part of password cracking and password security research. The progress of deep learning technology provides a promising way to improve the efficiency of password guessing. However, current research on neural network password guessing methods mostly focuses on model structure and has overlooked the generation method. Due to the randomness of sampling, not only the generated passwords have a large number of duplicates, but also the order in which passwords generated is random, leading to inefficient password attacks. In this paper, we propose SOPG, a search-based ordered password generation method, which enables the password guessing model based on autoregressive neural network to generate passwords in approximately descending order of probability. Experiment on comparison of SOPG and Random sampling shows passwords generated by SOPG do not repeat, and when they reach the same cover rate, SOPG requires fewer inferences and far fewer generated passwords than Random sampling, which brings great efficiency improvement to subsequent password attacks. We build SOPGesGPT, a password guessing model based on GPT, using SOPG to generate passwords. Compared with the most influential models OMEN, FLA, PassGAN, VAEPass and the latest model PassGPT in one-site test, experiments show that SOPGesGPT is far ahead in terms of both effective rate and cover rate. As to cover rate that everyone recognizes, SOPGesGPT reaches 35.06%, which is 254%, 298%, 421%, 380%, 81% higher than OMEN, FLA, PassGAN, VAEPass, and PassGPT respectively.
Den-SOFT: Dense Space-Oriented Light Field DataseT for 6-DOF Immersive Experience
Venue: arXiv cs.CV 2024
Year: 2024

We have built a custom mobile multi-camera large-space dense light field capture system, which provides a series of high-quality and sufficiently dense light field images for various scenarios. Our aim is to contribute to the development of popular 3D scene reconstruction algorithms such as IBRnet, NeRF, and 3D Gaussian splitting. More importantly, the collected dataset, which is much denser than existing datasets, may also inspire space-oriented light field reconstruction, which is potentially different from object-centric 3D reconstruction, for immersive VR/AR experiences. We utilized a total of 40 GoPro 10 cameras, capturing images of 5k resolution. The number of photos captured for each scene is no less than 1000, and the average density (view number within a unit sphere) is 134.68. It is also worth noting that our system is capable of efficiently capturing large outdoor scenes. Addressing the current lack of large-space and dense light field datasets, we made efforts to include elements such as sky, reflections, lights and shadows that are of interest to researchers in the field of 3D reconstruction during the data capture process. Finally, we validated the effectiveness of our provided dataset on three popular algorithms and also integrated the reconstructed 3DGS results into the Unity engine, demonstrating the potential of utilizing our datasets to enhance the realism of virtual reality (VR) and create feasible interactive spaces. The dataset is available at our project website.
Joint Optimization of STAR-RIS Assisted SWIPT Communication Systems
Venue: arXiv cs.NI 2024
Year: 2024

Simultaneous wireless information and power transfer (SWIPT) is an effective energy-saving technology, but its efficiency is hindered by environmental factors. The introduction of reconfigurable intelligent surfaces (RIS) has alleviated this issue, although it still faces significant constraints due to geographical limitations. This paper proposes a scheme that employs a simultaneously transmitting and reflecting (STAR)-RIS to assist SWIPT. It can overcome this limitation, achieve higher degrees of freedom (DoF), and provide better quality of service (QoS) for users on both sides of the ground. Meanwhile, we have considered a hybrid device based on a power splitting, which is capable of both energy harvesting and information decoding. We have proposed an efficient alternating optimization (AO) method to optimize the phase and amplitude vectors for reflection and transmission, beamforming and the optimal power splitting ratio, achieving an optimal balance between data rate and energy efficiency. Finally, simulation results demonstrate that the sum rate of the proposed model is superior to traditional RIS and other benchmark schemes.
Interactive Distance Field Mapping and Planning to Enable Human-Robot Collaboration
Venue: arXiv cs.RO 2024
Year: 2024

Human-robot collaborative applications require scene representations that are kept up-to-date and facilitate safe motions in dynamic scenes. In this letter, we present an interactive distance field mapping and planning (IDMP) framework that handles dynamic objects and collision avoidance through an efficient representation. We define interactive mapping and planning as the process of creating and updating the representation of the scene online while simultaneously planning and adapting the robot's actions based on that representation. The key aspect of this work is an efficient Gaussian Process field that performs incremental updates and handles dynamic objects reliably by identifying moving points via a simple and elegant formulation based on queries from a temporary latent model. In terms of mapping, IDMP is able to fuse point cloud data from single and multiple sensors, query the free space at any spatial resolution, and deal with moving objects without semantics. In terms of planning, IDMP allows seamless integration with gradient-based reactive planners facilitating dynamic obstacle avoidance for safe human-robot interactions. Our mapping performance is evaluated on both real and synthetic datasets. A comparison with similar state-of-the-art frameworks shows superior performance when handling dynamic objects and comparable or better performance in the accuracy of the computed distance and gradient field. Finally, we show how the framework can be used for fast motion planning in the presence of moving objects both in simulated and real-world scenes. An accompanying video, code, and datasets are made publicly available https://uts-ri.github.io/IDMP.
Federated Learning with Anomaly Detection via Gradient and Reconstruction Analysis
Venue: arXiv cs.CR 2024
Year: 2024

In the evolving landscape of Federated Learning (FL), the challenge of ensuring data integrity against poisoning attacks is paramount, particularly for applications demanding stringent privacy preservation. Traditional anomaly detection strategies often struggle to adapt to the distributed nature of FL, leaving a gap our research aims to bridge. We introduce a novel framework that synergizes gradient-based analysis with autoencoder-driven data reconstruction to detect and mitigate poisoned data with unprecedented precision. Our approach uniquely combines detecting anomalous gradient patterns with identifying reconstruction errors, significantly enhancing FL model security. Validated through extensive experiments on MNIST and CIFAR-10 datasets, our method outperforms existing solutions by 15\% in anomaly detection accuracy while maintaining a minimal false positive rate. This robust performance, consistent across varied data types and network sizes, underscores our framework's potential in securing FL deployments in critical domains such as healthcare and finance. By setting new benchmarks for anomaly detection within FL, our work paves the way for future advancements in distributed learning security.
Visual Foundation Models Boost Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

Unsupervised domain adaptation (UDA) is vital for alleviating the workload of labeling 3D point cloud data and mitigating the absence of labels when facing a newly defined domain. Various methods of utilizing images to enhance the performance of cross-domain 3D segmentation have recently emerged. However, the pseudo labels, which are generated from models trained on the source domain and provide additional supervised signals for the unseen domain, are inadequate when utilized for 3D segmentation due to their inherent noisiness and consequently restrict the accuracy of neural networks. With the advent of 2D visual foundation models (VFMs) and their abundant knowledge prior, we propose a novel pipeline VFMSeg to further enhance the cross-modal unsupervised domain adaptation framework by leveraging these models. In this work, we study how to harness the knowledge priors learned by VFMs to produce more accurate labels for unlabeled target domains and improve overall performance. We first utilize a multi-modal VFM, which is pre-trained on large scale image-text pairs, to provide supervised labels (VFM-PL) for images and point clouds from the target domain. Then, another VFM trained on fine-grained 2D masks is adopted to guide the generation of semantically augmented images and point clouds to enhance the performance of neural networks, which mix the data from source and target domains like view frustums (FrustumMixing). Finally, we merge class-wise prediction across modalities to produce more accurate annotations for unlabeled target domains. Our method is evaluated on various autonomous driving datasets and the results demonstrate a significant improvement for 3D segmentation task.
MR-MT3: Memory Retaining Multi-Track Music Transcription to Mitigate Instrument Leakage
Venue: arXiv cs.SD 2024
Year: 2024

This paper presents enhancements to the MT3 model, a state-of-the-art (SOTA) token-based multi-instrument automatic music transcription (AMT) model. Despite SOTA performance, MT3 has the issue of instrument leakage, where transcriptions are fragmented across different instruments. To mitigate this, we propose MR-MT3, with enhancements including a memory retention mechanism, prior token sampling, and token shuffling are proposed. These methods are evaluated on the Slakh2100 dataset, demonstrating improved onset F1 scores and reduced instrument leakage. In addition to the conventional multi-instrument transcription F1 score, new metrics such as the instrument leakage ratio and the instrument detection F1 score are introduced for a more comprehensive assessment of transcription quality. The study also explores the issue of domain overfitting by evaluating MT3 on single-instrument monophonic datasets such as ComMU and NSynth. The findings, along with the source code, are shared to facilitate future work aimed at refining token-based multi-instrument AMT models.
Ipelets for the Convex Polygonal Geometry
Venue: arXiv cs.CG 2024
Year: 2024

There are many structures, both classical and modern, involving convex polygonal geometries whose deeper understanding would be facilitated through interactive visualizations. The Ipe extensible drawing editor, developed by Otfried Cheong, is a widely used software system for generating geometric figures. One of its features is the capability to extend its functionality through programs called Ipelets. In this media submission, we showcase a collection of new Ipelets that construct a variety of geometric objects based on polygonal geometries. These include Macbeath regions, metric balls in the forward and reverse Funk distance, metric balls in the Hilbert metric, polar bodies, the minimum enclosing ball of a point set, and minimum spanning trees in both the Funk and Hilbert metrics. We also include a number of utilities on convex polygons, including union, intersection, subtraction, and Minkowski sum (previously implemented as a CGAL Ipelet). All of our Ipelets are programmed in Lua and are freely available.
Group-Mix SAM: Lightweight Solution for Industrial Assembly Line Applications
Venue: arXiv cs.CV 2024
Year: 2024

Since the advent of the Segment Anything Model(SAM) approximately one year ago, it has engendered significant academic interest and has spawned a large number of investigations and publications from various perspectives. However, the deployment of SAM in practical assembly line scenarios has yet to materialize due to its large image encoder, which weighs in at an imposing 632M. In this study, we have replaced the heavyweight image encoder with a lightweight one, thereby enabling the deployment of SAM in practical assembly line scenarios. Specifically, we have employed decoupled distillation to train the encoder of MobileSAM in a resource-limited setting. The entire knowledge distillation experiment can be completed in a single day on a single RTX 4090. The resulting lightweight SAM, referred to as Group-Mix SAM, had 37.63% (2.16M) fewer parameters and 42.5% (15614.7M) fewer floating-point operations compared to MobileSAM. However, on our constructed industrial dataset, MALSD, its mIoU was only marginally lower than that of MobileSAM, at 0.615. Finally, we conducted a comprehensive comparative experiment to demonstrate the superiority of Group-Mix SAM in the industrial domain. With its exceptional performance, our Group-Mix SAM is more suitable for practical assembly line applications.
Learning Physical Dynamics for Object-centric Visual Prediction
Venue: arXiv cs.CV 2024
Year: 2024

The ability to model the underlying dynamics of visual scenes and reason about the future is central to human intelligence. Many attempts have been made to empower intelligent systems with such physical understanding and prediction abilities. However, most existing methods focus on pixel-to-pixel prediction, which suffers from heavy computational costs while lacking a deep understanding of the physical dynamics behind videos. Recently, object-centric prediction methods have emerged and attracted increasing interest. Inspired by it, this paper proposes an unsupervised object-centric prediction model that makes future predictions by learning visual dynamics between objects. Our model consists of two modules, perceptual, and dynamic module. The perceptual module is utilized to decompose images into several objects and synthesize images with a set of object-centric representations. The dynamic module fuses contextual information, takes environment-object and object-object interaction into account, and predicts the future trajectory of objects. Extensive experiments are conducted to validate the effectiveness of the proposed method. Both quantitative and qualitative experimental results demonstrate that our model generates higher visual quality and more physically reliable predictions compared to the state-of-the-art methods.