OpenRaiser/Intern-Atlas huggingface dataset:
Video Denoising in Fluorescence Guided Surgery
Venue: arXiv cs.CV 2024
Year: 2024

Fluorescence guided surgery (FGS) is a promising surgical technique that gives surgeons a unique view of tissue that is used to guide their practice by delineating tissue types and diseased areas. As new fluorescent contrast agents are developed that have low fluorescent photon yields, it becomes increasingly important to develop computational models to allow FGS systems to maintain good video quality in real time environments. To further complicate this task, FGS has a difficult bias noise term from laser leakage light (LLL) that represents unfiltered excitation light that can be on the order of the fluorescent signal. Most conventional video denoising methods focus on zero mean noise, and non-causal processing, both of which are violated in FGS. Luckily in FGS, often a co-located reference video is also captured which we use to simulate the LLL and assist in the denoising processes. In this work, we propose an accurate noise simulation pipeline that includes LLL and propose three baseline deep learning based algorithms for FGS video denoising.
To Optimize Human-in-the-loop Learning in Repeated Routing Games
Venue: arXiv cs.GT 2024
Year: 2024

Today navigation applications (e.g., Waze and Google Maps) enable human users to learn and share the latest traffic observations, yet such information sharing simply aids selfish users to predict and choose the shortest paths to jam each other. Prior routing game studies focus on myopic users in oversimplified one-shot scenarios to regulate selfish routing via information hiding or pricing mechanisms. For practical human-in-the-loop learning (HILL) in repeated routing games, we face non-myopic users of differential past observations and need new mechanisms (preferably non-monetary) to persuade users to adhere to the optimal path recommendations. We model the repeated routing game in a typical parallel transportation network, which generally contains one deterministic path and $N$ stochastic paths. We first prove that no matter under the information sharing mechanism in use or the latest routing literature's hiding mechanism, the resultant price of anarchy (PoA) for measuring the efficiency loss from social optimum can approach infinity, telling arbitrarily poor exploration-exploitation tradeoff over time. Then we propose a novel user-differential probabilistic recommendation (UPR) mechanism to differentiate and randomize path recommendations for users with differential learning histories. We prove that our UPR mechanism ensures interim individual rationality for all users and significantly reduces $\text{PoA}=\infty$ to close-to-optimal $\text{PoA}=1+\frac{1}{4N+3}$, which cannot be further reduced by any other non-monetary mechanism. In addition to theoretical analysis, we conduct extensive experiments using real-world datasets to generalize our routing graphs and validate the close-to-optimal performance of UPR mechanism.
Content-Aware Preserving Image Generation
Venue: arXiv cs.CV 2024
Year: 2024

Remarkable progress has been achieved in image generation with the introduction of generative models. However, precisely controlling the content in generated images remains a challenging task due to their fundamental training objective. This paper addresses this challenge by proposing a novel image generation framework explicitly designed to incorporate desired content in output images. The framework utilizes advanced encoding techniques, integrating subnetworks called content fusion and frequency encoding modules. The frequency encoding module first captures features and structures of reference images by exclusively focusing on selected frequency components. Subsequently, the content fusion module generates a content-guiding vector that encapsulates desired content features. During the image generation process, content-guiding vectors from real images are fused with projected noise vectors. This ensures the production of generated images that not only maintain consistent content from guiding images but also exhibit diverse stylistic variations. To validate the effectiveness of the proposed framework in preserving content attributes, extensive experiments are conducted on widely used benchmark datasets, including Flickr-Faces-High Quality, Animal Faces High Quality, and Large-scale Scene Understanding datasets.
EEG Spectral Analysis in Gray Zone Between Healthy and Insomnia
Venue: arXiv cs.HC 2024
Year: 2024

This study investigates the sleep characteristics and brain activity of individuals in the gray zone of insomnia, a population that experiences sleep disturbances yet does not fully meet the clinical criteria for chronic insomnia. Thirteen healthy participants and thirteen individuals from the gray zone were assessed using polysomnography and electroencephalogram to analyze both sleep architecture and neural activity. Although no significant differences in objective sleep quality or structure were found between the groups, gray zone individuals reported higher insomnia severity index scores, indicating subjective sleep difficulties. Electroencephalogram analysis revealed increased delta and alpha activity during the wake stage, suggesting lingering sleep inertia, while non-rapid eye movement stages 1 and 2 exhibited elevated beta and gamma activity, often associated with chronic insomnia. However, these high-frequency patterns were not observed in non-rapid eye movement stage 3 or rapid eye movement sleep, suggesting less severe disruptions compared to chronic insomnia. This study emphasizes that despite normal polysomnography findings, EEG patterns in gray zone individuals suggest a potential risk for chronic insomnia, highlighting the need for early identification and tailored intervention strategies to prevent progression.
Research on Domain-Specific Chinese Spelling Correction Method Based on Plugin Extension Modules
Venue: arXiv cs.CL 2024
Year: 2024

This paper proposes a Chinese spelling correction method based on plugin extension modules, aimed at addressing the limitations of existing models in handling domain-specific texts. Traditional Chinese spelling correction models are typically trained on general-domain datasets, resulting in poor performance when encountering specialized terminology in domain-specific texts. To address this issue, we design an extension module that learns the features of domain-specific terminology, thereby enhancing the model's correction capabilities within specific domains. This extension module can provide domain knowledge to the model without compromising its general spelling correction performance, thus improving its accuracy in specialized fields. Experimental results demonstrate that after integrating extension modules for medical, legal, and official document domains, the model's correction performance is significantly improved compared to the baseline model without any extension modules.
A Self-Supervised Robotic System for Autonomous Contact-Based Spatial Mapping of Semiconductor Properties
Venue: arXiv cs.RO 2024
Year: 2024

Integrating robotically driven contact-based material characterization techniques into self-driving laboratories can enhance measurement quality, reliability, and throughput. While deep learning models support robust autonomy, current methods lack reliable pixel-precision positioning and require extensive labeled data. To overcome these challenges, we propose an approach for building self-supervised autonomy into contact-based robotic systems that teach the robot to follow domain expert measurement principles at high-throughputs. Firstly, we design a vision-based, self-supervised convolutional neural network (CNN) architecture that uses differentiable image priors to optimize domain-specific objectives, refining the pixel precision of predicted robot contact poses by 20.0% relative to existing approaches. Secondly, we design a reliable graph-based planner for generating distance-minimizing paths to accelerate the robot measurement throughput and decrease planning variance by 6x. We demonstrate the performance of this approach by autonomously driving a 4-degree-of-freedom robotic probe for 24 hours to characterize semiconductor photoconductivity at 3,025 uniquely predicted poses across a gradient of drop-casted perovskite film compositions, achieving throughputs over 125 measurements per hour. Spatially mapping photoconductivity onto each drop-casted film reveals compositional trends and regions of inhomogeneity, valuable for identifying manufacturing process defects. With this self-supervised CNN-driven robotic system, we enable high-precision and reliable automation of contact-based characterization techniques at high throughputs, thereby allowing the measurement of previously inaccessible yet important semiconductor properties for self-driving laboratories.
Self-Supervised Learning of Grasping Arbitrary Objects On-the-Move
Venue: arXiv cs.RO 2024
Year: 2024

Mobile grasping enhances manipulation efficiency by utilizing robots' mobility. This study aims to enable a commercial off-the-shelf robot for mobile grasping, requiring precise timing and pose adjustments. Self-supervised learning can develop a generalizable policy to adjust the robot's velocity and determine grasp position and orientation based on the target object's shape and pose. Due to mobile grasping's complexity, action primitivization and step-by-step learning are crucial to avoid data sparsity in learning from trial and error. This study simplifies mobile grasping into two grasp action primitives and a moving action primitive, which can be operated with limited degrees of freedom for the manipulator. This study introduces three fully convolutional neural network (FCN) models to predict static grasp primitive, dynamic grasp primitive, and residual moving velocity error from visual inputs. A two-stage grasp learning approach facilitates seamless FCN model learning. The ablation study demonstrated that the proposed method achieved the highest grasping accuracy and pick-and-place efficiency. Furthermore, randomizing object shapes and environments in the simulation effectively achieved generalizable mobile grasping.
Physics-informed Machine Learning for Battery Pack Thermal Management
Venue: arXiv cs.LG 2024
Year: 2024

With the popularity of electric vehicles, the demand for lithium-ion batteries is increasing. Temperature significantly influences the performance and safety of batteries. Battery thermal management systems can effectively control the temperature of batteries; therefore, the performance and safety can be ensured. However, the development process of battery thermal management systems is time-consuming and costly due to the extensive training dataset needed by data-driven models requiring enormous computational costs for finite element analysis. Therefore, a new approach to constructing surrogate models is needed in the era of AI. Physics-informed machine learning enforces the physical laws in surrogate models, making it the perfect candidate for estimating battery pack temperature distribution. In this study, we first developed a 21700 battery pack indirect liquid cooling system with cold plates on the top and bottom with thermal paste surrounding the battery cells. Then, the simplified finite element model was built based on experiment results. Due to the high coolant flow rate, the cold plates can be considered as constant temperature boundaries, while battery cells are the heat sources. The physics-informed convolutional neural network served as a surrogate model to estimate the temperature distribution of the battery pack. The loss function was constructed considering the heat conduction equation based on the finite difference method. The physics-informed loss function helped the convergence of the training process with less data. As a result, the physics-informed convolutional neural network showed more than 15 percents improvement in accuracy compared to the data-driven method with the same training data.
Autonomous Robotic Pepper Harvesting: Imitation Learning in Unstructured Agricultural Environments
Venue: arXiv cs.RO 2024
Year: 2024

Automating tasks in outdoor agricultural fields poses significant challenges due to environmental variability, unstructured terrain, and diverse crop characteristics. We present a robotic system for autonomous pepper harvesting designed to operate in these unprotected, complex settings. Utilizing a custom handheld shear-gripper, we collected 300 demonstrations to train a visuomotor policy, enabling the system to adapt to varying field conditions and crop diversity. We achieved a success rate of 28.95% with a cycle time of 31.71 seconds, comparable to existing systems tested under more controlled conditions like greenhouses. Our system demonstrates the feasibility and effectiveness of leveraging imitation learning for automated harvesting in unstructured agricultural environments. This work aims to advance scalable, automated robotic solutions for agriculture in natural settings.
TEESlice: Protecting Sensitive Neural Network Models in Trusted Execution Environments When Attackers have Pre-Trained Models
Venue: arXiv cs.CR 2024
Year: 2024

Trusted Execution Environments (TEE) are used to safeguard on-device models. However, directly employing TEEs to secure the entire DNN model is challenging due to the limited computational speed. Utilizing GPU can accelerate DNN's computation speed but commercial widely-available GPUs usually lack security protection. To this end, scholars introduce TSDP, a method that protects privacy-sensitive weights within TEEs and offloads insensitive weights to GPUs. Nevertheless, current methods do not consider the presence of a knowledgeable adversary who can access abundant publicly available pre-trained models and datasets. This paper investigates the security of existing methods against such a knowledgeable adversary and reveals their inability to fulfill their security promises. Consequently, we introduce a novel partition before training strategy, which effectively separates privacy-sensitive weights from other components of the model. Our evaluation demonstrates that our approach can offer full model protection with a computational cost reduced by a factor of 10. In addition to traditional CNN models, we also demonstrate the scalability to large language models. Our approach can compress the private functionalities of the large language model to lightweight slices and achieve the same level of protection as the shielding-whole-model baseline.
Large Language Models as User-Agents for Evaluating Task-Oriented-Dialogue Systems
Venue: arXiv cs.CL 2024
Year: 2024

Traditionally, offline datasets have been used to evaluate task-oriented dialogue (TOD) models. These datasets lack context awareness, making them suboptimal benchmarks for conversational systems. In contrast, user-agents, which are context-aware, can simulate the variability and unpredictability of human conversations, making them better alternatives as evaluators. Prior research has utilized large language models (LLMs) to develop user-agents. Our work builds upon this by using LLMs to create user-agents for the evaluation of TOD systems. This involves prompting an LLM, using in-context examples as guidance, and tracking the user-goal state. Our evaluation of diversity and task completion metrics for the user-agents shows improved performance with the use of better prompts. Additionally, we propose methodologies for the automatic evaluation of TOD models within this dynamic framework.
Journal Quality Factors from ChatGPT: More meaningful than Impact Factors?
Venue: arXiv cs.DL 2024
Year: 2024

Purpose: Journal Impact Factors and other citation-based indicators are widely used and abused to help select journals to publish in or to estimate the value of a published article. Nevertheless, citation rates primarily reflect scholarly impact rather than other quality dimensions, including societal impact, originality, and rigour. In contrast, Journal Quality Factors (JQFs) are average quality score estimates given to a journal's articles by ChatGPT. Design: JQFs were compared with Polish, Norwegian and Finnish journal ranks and with journal citation rates for 1,300 journals with 130,000 articles from 2021 in large monodisciplinary journals in the 25 out of 27 Scopus broad fields of research for which it was possible. Outliers were also examined. Findings: JQFs correlated positively and mostly strongly (median correlation: 0.641) with journal ranks in 24 out of the 25 broad fields examined, indicating a nearly science-wide ability for ChatGPT to estimate journal quality. Journal citation rates had similarly high correlations with national journal ranks, however, so JQFs are not a universally better indicator. An examination of journals with JQFs not matching their journal ranks suggested that abstract styles may affect the result, such as whether the societal contexts of research are mentioned. Limitations: Different journal rankings may have given different findings because there is no agreed meaning for journal quality. Implications: The results suggest that JQFs are plausible as journal quality indicators in all fields and may be useful for the (few) research and evaluation contexts where journal quality is an acceptable proxy for article quality, and especially for fields like mathematics for which citations are not strong indicators of quality. Originality: This is the first attempt to estimate academic journal value with a Large Language Model.
Strategic Roadmap for Quantum- Resistant Security: A Framework for Preparing Industries for the Quantum Threat
Venue: arXiv cs.CR 2024
Year: 2024

As quantum computing continues to advance, its ability to compromise widely used cryptographic systems projects a significant challenge to modern cybersecurity. This paper outlines a strategic roadmap for industries to anticipate and mitigate the risks posed by quantum attacks. Our study explores the development of a quantum-resistant cryptographic solutioning framework for the industry, offering a practical and strategic approach to mitigating quantum attacks. We, here, propose a novel strategic framework, coined name STL-QCRYPTO, outlines tailored, industry-specific methodologies to implement quantum-safe security systems, ensuring long-term protection against the disruptive potential of quantum computing. The following fourteen high-risk sectors: Financial Services, Banking, Healthcare, Critical Infrastructure, Government & Defence, E-commerce, Energy & Utilities, Automotive & Transportation, Cloud Computing & Data Storage, Insurance, Internet & Telecommunications, Blockchain Applications, Metaverse Applications, and Multiagent AI Systems - are critically assessed for their vulnerability to quantum threats. The evaluation emphasizes practical approaches for the deployment of quantum-safe security systems to safeguard these industries against emerging quantum-enabled cyber risks. Additionally, the paper addresses the technical, operational, and regulatory hurdles associated with adopting quantum-resistant technologies. By presenting a structured timeline and actionable recommendations, this roadmap with proposed framework prepares industries with the essential strategy to safeguard their potential security threats in the quantum computing era.
Hollywood's misrepresentation of death: A comparison of overall and by-gender mortality causes in film and the real world
Venue: arXiv cs.CY 2024
Year: 2024

The common phrase 'representation matters' asserts that media has a measurable and important impact on civic society's perception of self and others. The representation of health in media, in particular, may reflect and perpetuate a society's disease burden. Here, for the top 10 major causes of death in the United States, we examine how cinematic representation of overall and by-gender mortality diverges from reality. Using crowd-sourced data on film deaths from Cinemorgue Wiki, we employ natural language processing (NLP) techniques to analyze shifts in representation of deaths in movies versus the 2021 National Vital Statistic Survey (NVSS) top ten mortality causes. Overall, movies strongly overrepresent suicide and, to a lesser degree, accidents. In terms of gender, movies overrepresent men and underrepresent women for nearly every major mortality cause, including heart disease and cerebrovascular disease. The two exceptions for which women are overrepresented are suicide and accidents. We discuss the implications of under- and over-representing causes of death overall and by gender, as well as areas of future research.
Physics-informed neural networks need a physicist to be accurate: the case of mass and heat transport in Fischer-Tropsch catalyst particles
Venue: arXiv cs.LG 2024
Year: 2024

Physics-Informed Neural Networks (PINNs) have emerged as an influential technology, merging the swift and automated capabilities of machine learning with the precision and dependability of simulations grounded in theoretical physics. PINNs are often employed to solve algebraic or differential equations to replace some or even all steps of multi-stage computational workflows, leading to their significant speed-up. However, wide adoption of PINNs is still hindered by reliability issues, particularly at extreme ends of the input parameter ranges. In this study, we demonstrate this in the context of a system of coupled non-linear differential reaction-diffusion and heat transfer equations related to Fischer-Tropsch synthesis, which are solved by a finite-difference method with a PINN used in evaluating their source terms. It is shown that the testing strategies traditionally used to assess the accuracy of neural networks as function approximators can overlook the peculiarities which ultimately cause instabilities of the finite-difference solver. We propose a domain knowledge-based modifications to the PINN architecture ensuring its correct asymptotic behavior. When combined with an improved numerical scheme employed as an initial guess generator, the proposed modifications are shown to recover the overall stability of the simulations, while preserving the speed-up brought by PINN as the workflow component. We discuss the possible applications of the proposed hybrid transport equation solver in context of chemical reactors simulations.
Influence of Depth Camera Noise Models on Respiration Estimation
Venue: arXiv cs.CV 2024
Year: 2024

Depth cameras are an interesting modality for capturing vital signs such as respiratory rate. Plenty approaches exist to extract vital signs in a controlled setting, but in order to apply them more flexibly for example in multi-camera settings, a simulated environment is needed to generate enough data for training and testing of new algorithms. We show first results of a 3D-rendering simulation pipeline that focuses on different noise models in order to generate realistic, depth-camera based respiratory signals using both synthetic and real respiratory signals as a baseline. While most noise can be accurately modelled as Gaussian in this context, we can show that as soon as the available image resolution is too low, the differences between different noise models surface.