OpenRaiser/Intern-Atlas huggingface dataset:
Interpretable Differential Diagnosis with Dual-Inference Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Automatic differential diagnosis (DDx) is an essential medical task that generates a list of potential diseases as differentials based on patient symptom descriptions. In practice, interpreting these differential diagnoses yields significant value but remains under-explored. Given the powerful capabilities of large language models (LLMs), we investigated using LLMs for interpretable DDx. Specifically, we curated the first DDx dataset with expert-derived interpretation on 570 clinical notes. Besides, we proposed Dual-Inf, a novel framework that enabled LLMs to conduct bidirectional inference (i.e., from symptoms to diagnoses and vice versa) for DDx interpretation. Both human and automated evaluation validated its efficacy in predicting and elucidating differentials across four base LLMs. In addition, Dual-Inf could reduce interpretation errors and hold promise for rare disease explanations. To the best of our knowledge, it is the first work that customizes LLMs for DDx explanation and comprehensively evaluates their interpretation performance. Overall, our study bridges a critical gap in DDx interpretation and enhances clinical decision-making.
FALFormer: Feature-aware Landmarks self-attention for Whole-slide Image Classification
Venue: arXiv cs.CV 2024
Year: 2024

Slide-level classification for whole-slide images (WSIs) has been widely recognized as a crucial problem in digital and computational pathology. Current approaches commonly consider WSIs as a bag of cropped patches and process them via multiple instance learning due to the large number of patches, which cannot fully explore the relationship among patches; in other words, the global information cannot be fully incorporated into decision making. Herein, we propose an efficient and effective slide-level classification model, named as FALFormer, that can process a WSI as a whole so as to fully exploit the relationship among the entire patches and to improve the classification performance. FALFormer is built based upon Transformers and self-attention mechanism. To lessen the computational burden of the original self-attention mechanism and to process the entire patches together in a WSI, FALFormer employs Nyström self-attention which approximates the computation by using a smaller number of tokens or landmarks. For effective learning, FALFormer introduces feature-aware landmarks to enhance the representation power of the landmarks and the quality of the approximation. We systematically evaluate the performance of FALFormer using two public datasets, including CAMELYON16 and TCGA-BRCA. The experimental results demonstrate that FALFormer achieves superior performance on both datasets, outperforming the state-of-the-art methods for the slide-level classification. This suggests that FALFormer can facilitate an accurate and precise analysis of WSIs, potentially leading to improved diagnosis and prognosis on WSIs.
DuInNet: Dual-Modality Feature Interaction for Point Cloud Completion
Venue: arXiv cs.CV 2024
Year: 2024

To further promote the development of multimodal point cloud completion, we contribute a large-scale multimodal point cloud completion benchmark ModelNet-MPC with richer shape categories and more diverse test data, which contains nearly 400,000 pairs of high-quality point clouds and rendered images of 40 categories. Besides the fully supervised point cloud completion task, two additional tasks including denoising completion and zero-shot learning completion are proposed in ModelNet-MPC, to simulate real-world scenarios and verify the robustness to noise and the transfer ability across categories of current methods. Meanwhile, considering that existing multimodal completion pipelines usually adopt a unidirectional fusion mechanism and ignore the shape prior contained in the image modality, we propose a Dual-Modality Feature Interaction Network (DuInNet) in this paper. DuInNet iteratively interacts features between point clouds and images to learn both geometric and texture characteristics of shapes with the dual feature interactor. To adapt to specific tasks such as fully supervised, denoising, and zero-shot learning point cloud completions, an adaptive point generator is proposed to generate complete point clouds in blocks with different weights for these two modalities. Extensive experiments on the ShapeNet-ViPC and ModelNet-MPC benchmarks demonstrate that DuInNet exhibits superiority, robustness and transfer ability in all completion tasks over state-of-the-art methods. The code and dataset will be available soon.
Stable Weight Updating: A Key to Reliable PDE Solutions Using Deep Learning
Venue: arXiv cs.AI 2024
Year: 2024

Background: Deep learning techniques, particularly neural networks, have revolutionized computational physics, offering powerful tools for solving complex partial differential equations (PDEs). However, ensuring stability and efficiency remains a challenge, especially in scenarios involving nonlinear and time-dependent equations. Methodology: This paper introduces novel residual-based architectures, namely the Simple Highway Network and the Squared Residual Network, designed to enhance stability and accuracy in physics-informed neural networks (PINNs). These architectures augment traditional neural networks by incorporating residual connections, which facilitate smoother weight updates and improve backpropagation efficiency. Results: Through extensive numerical experiments across various examples including linear and nonlinear, time-dependent and independent PDEs we demonstrate the efficacy of the proposed architectures. The Squared Residual Network, in particular, exhibits robust performance, achieving enhanced stability and accuracy compared to conventional neural networks. These findings underscore the potential of residual-based architectures in advancing deep learning for PDEs and computational physics applications.
Greit-HRNet: Grouped Lightweight High-Resolution Network for Human Pose Estimation
Venue: arXiv cs.CV 2024
Year: 2024

As multi-scale features are necessary for human pose estimation tasks, high-resolution networks are widely applied. To improve efficiency, lightweight modules are proposed to replace costly point-wise convolutions in high-resolution networks, including channel weighting and spatial weighting methods. However, they fail to maintain the consistency of weights and capture global spatial information. To address these problems, we present a Grouped lightweight High-Resolution Network (Greit-HRNet), in which we propose a Greit block including a group method Grouped Channel Weighting (GCW) and a spatial weighting method Global Spatial Weighting (GSW). GCW modules group conditional channel weighting to make weights stable and maintain the high-resolution features with the deepening of the network, while GSW modules effectively extract global spatial information and exchange information across channels. In addition, we apply the Large Kernel Attention (LKA) method to improve the whole efficiency of our Greit-HRNet. Our experiments on both MS-COCO and MPII human pose estimation datasets demonstrate the superior performance of our Greit-HRNet, outperforming other state-of-the-art lightweight networks.
Secondary Structure-Guided Novel Protein Sequence Generation with Latent Graph Diffusion
Venue: arXiv cs.AI 2024
Year: 2024

The advent of deep learning has introduced efficient approaches for de novo protein sequence design, significantly improving success rates and reducing development costs compared to computational or experimental methods. However, existing methods face challenges in generating proteins with diverse lengths and shapes while maintaining key structural features. To address these challenges, we introduce CPDiffusion-SS, a latent graph diffusion model that generates protein sequences based on coarse-grained secondary structural information. CPDiffusion-SS offers greater flexibility in producing a variety of novel amino acid sequences while preserving overall structural constraints, thus enhancing the reliability and diversity of generated proteins. Experimental analyses demonstrate the significant superiority of the proposed method in producing diverse and novel sequences, with CPDiffusion-SS surpassing popular baseline methods on open benchmarks across various quantitative measurements. Furthermore, we provide a series of case studies to highlight the biological significance of the generation performance by the proposed method. The source code is publicly available at https://github.com/riacd/CPDiffusion-SS
Drantal-NeRF: Diffusion-Based Restoration for Anti-aliasing Neural Radiance Field
Venue: arXiv cs.CV 2024
Year: 2024

Aliasing artifacts in renderings produced by Neural Radiance Field (NeRF) is a long-standing but complex issue in the field of 3D implicit representation, which arises from a multitude of intricate causes and was mitigated by designing more advanced but complex scene parameterization methods before. In this paper, we present a Diffusion-based restoration method for anti-aliasing Neural Radiance Field (Drantal-NeRF). We consider the anti-aliasing issue from a low-level restoration perspective by viewing aliasing artifacts as a kind of degradation model added to clean ground truths. By leveraging the powerful prior knowledge encapsulated in diffusion model, we could restore the high-realism anti-aliasing renderings conditioned on aliased low-quality counterparts. We further employ a feature-wrapping operation to ensure multi-view restoration consistency and finetune the VAE decoder to better adapt to the scene-specific data distribution. Our proposed method is easy to implement and agnostic to various NeRF backbones. We conduct extensive experiments on challenging large-scale urban scenes as well as unbounded 360-degree scenes and achieve substantial qualitative and quantitative improvements.
Invisible Optical Adversarial Stripes on Traffic Sign against Autonomous Vehicles
Venue: arXiv cs.CR 2024
Year: 2024

Camera-based computer vision is essential to autonomous vehicle's perception. This paper presents an attack that uses light-emitting diodes and exploits the camera's rolling shutter effect to create adversarial stripes in the captured images to mislead traffic sign recognition. The attack is stealthy because the stripes on the traffic sign are invisible to human. For the attack to be threatening, the recognition results need to be stable over consecutive image frames. To achieve this, we design and implement GhostStripe, an attack system that controls the timing of the modulated light emission to adapt to camera operations and victim vehicle movements. Evaluated on real testbeds, GhostStripe can stably spoof the traffic sign recognition results for up to 94\% of frames to a wrong class when the victim vehicle passes the road section. In reality, such attack effect may fool victim vehicles into life-threatening incidents. We discuss the countermeasures at the levels of camera sensor, perception model, and autonomous driving system.
IDA-VLM: Towards Movie Understanding via ID-Aware Large Vision-Language Model
Venue: arXiv cs.CV 2024
Year: 2024

The rapid advancement of Large Vision-Language models (LVLMs) has demonstrated a spectrum of emergent capabilities. Nevertheless, current models only focus on the visual content of a single scenario, while their ability to associate instances across different scenes has not yet been explored, which is essential for understanding complex visual content, such as movies with multiple characters and intricate plots. Towards movie understanding, a critical initial step for LVLMs is to unleash the potential of character identities memory and recognition across multiple visual scenarios. To achieve the goal, we propose visual instruction tuning with ID reference and develop an ID-Aware Large Vision-Language Model, IDA-VLM. Furthermore, our research introduces a novel benchmark MM-ID, to examine LVLMs on instance IDs memory and recognition across four dimensions: matching, location, question-answering, and captioning. Our findings highlight the limitations of existing LVLMs in recognizing and associating instance identities with ID reference. This paper paves the way for future artificial intelligence systems to possess multi-identity visual inputs, thereby facilitating the comprehension of complex visual narratives like movies.
Can social media shape the security of next-generation connected vehicles?
Venue: arXiv cs.SI 2024
Year: 2024

The increasing adoption of connectivity and electronic components in vehicles makes these systems valuable targets for attackers. While automotive vendors prioritize safety, there remains a critical need for comprehensive assessment and analysis of cyber risks. In this context, this paper proposes a Social Media Automotive Threat Intelligence (SOCMATI) framework, specifically designed for the emerging field of automotive cybersecurity. The framework leverages advanced intelligence techniques and machine learning models to extract valuable insights from social media. Four use cases illustrate the framework's potential by demonstrating how it can significantly enhance threat assessment procedures within the automotive industry.
The Computational Learning of Construction Grammars: State of the Art and Prospective Roadmap
Venue: arXiv cs.CL 2024
Year: 2024

This paper documents and reviews the state of the art concerning computational models of construction grammar learning. It brings together prior work on the computational learning of form-meaning pairings, which has so far been studied in several distinct areas of research. The goal of this paper is threefold. First of all, it aims to synthesise the variety of methodologies that have been proposed to date and the results that have been obtained. Second, it aims to identify those parts of the challenge that have been successfully tackled and reveal those that require further research. Finally, it aims to provide a roadmap which can help to boost and streamline future research efforts on the computational learning of large-scale, usage-based construction grammars.
On Sampling from Ising Models with Spectral Constraints
Venue: arXiv cs.DS 2024
Year: 2024

We consider the problem of sampling from the Ising model when the underlying interaction matrix has eigenvalues lying within an interval of length $γ$. Recent work in this setting has shown various algorithmic results that apply roughly when $γ< 1$, notably with nearly-linear running times based on the classical Glauber dynamics. However, the optimality of the range of $γ$ was not clear since previous inapproximability results developed for the antiferromagnetic case (where the matrix has entries $\leq 0$) apply only for $γ>2$. To this end, Kunisky (SODA'24) recently provided evidence that the problem becomes hard already when $γ>1$ based on the low-degree hardness for an inference problem on random matrices. Based on this, he conjectured that sampling from the Ising model in the same range of $γ$ is NP-hard. Here we confirm this conjecture, complementing in particular the known algorithmic results by showing NP-hardness results for approximately counting and sampling when $γ>1$, with strong inapproximability guarantees; we also obtain a more refined hardness result for matrices where only a constant number of entries per row are allowed to be non-zero. The main observation in our reductions is that, for $γ>1$, Glauber dynamics mixes slowly when the interactions are all positive (ferromagnetic) for the complete and random regular graphs, due to a bimodality in the underlying distribution. While ferromagnetic interactions typically preclude NP-hardness results, here we work around this by introducing in an appropriate way mild antiferromagnetism, keeping the spectrum roughly within the same range. This allows us to exploit the bimodality of the aforementioned graphs and show the target NP-hardness by adapting suitably previous inapproximability techniques developed for antiferromagnetic systems.
Boosting Medical Image Synthesis via Registration-guided Consistency and Disentanglement Learning
Venue: arXiv cs.CV 2024
Year: 2024

Medical image synthesis remains challenging due to misalignment noise during training. Existing methods have attempted to address this challenge by incorporating a registration-guided module. However, these methods tend to overlook the task-specific constraints on the synthetic and registration modules, which may cause the synthetic module to still generate spatially aligned images with misaligned target images during training, regardless of the registration module's function. Therefore, this paper proposes registration-guided consistency and incorporates disentanglement learning for medical image synthesis. The proposed registration-guided consistency architecture fosters task-specificity within the synthetic and registration modules by applying identical deformation fields before and after synthesis, while enforcing output consistency through an alignment loss. Moreover, the synthetic module is designed to possess the capability of disentangling anatomical structures and specific styles across various modalities. An anatomy consistency loss is introduced to further compel the synthetic module to preserve geometrical integrity within latent spaces. Experiments conducted on both an in-house abdominal CECT-CT dataset and a publicly available pelvic MR-CT dataset have demonstrated the superiority of the proposed method.
Joint UPF and Edge Applications Placement and Routing in 5G & Beyond
Venue: arXiv cs.NI 2024
Year: 2024

The development of 5G networks has enabled support for a vast number of applications with stringent traffic requirements, both in terms of communication and computation. Furthermore, the proximity of the entities, such as edge servers and User Plane Functions (UPFs) that provide these resources is of paramount importance. However, with the ever-increasing demand from these applications, operators often find their resources insufficient to accommodate all requests. Some of these demands can be forwarded to external entities, not owned by the operator. This introduces a cost, reducing the operator's profit. Hence, to maximize operator's profit, it is important to place the demands optimally in internal or external edge nodes. To this end, we formulate a constrained optimization problem that captures this objective and the inter-play between different parameters, which turns out to be NP-hard. Therefore, we resort to proposing a heuristic algorithm which ranks the demands according to their value to the operator and amount of resources they need. Results show that our approach outperforms the benchmark algorithms, deviating from the optimal solution by only ~3% on average.
APTAS for bin packing with general cost structures
Venue: arXiv cs.DS 2024
Year: 2024

We consider the following generalization of the bin packing problem. We are given a set of items each of which is associated with a rational size in the interval [0,1], and a monotone non-decreasing non-negative cost function f defined over the cardinalities of the subsets of items. A feasible solution is a partition of the set of items into bins subject to the constraint that the total size of items in every bin is at most 1. Unlike bin packing, the goal function is to minimize the total cost of the bins where the cost of a bin is the value of f applied on the cardinality of the subset of items packed into the bin. We present an APTAS for this strongly NP-hard problem. We also provide a complete complexity classification of the problem with respect to the choice of f.
The V-Lab VR Educational Application Framework
Venue: arXiv cs.HC 2024
Year: 2024

This paper presents the V-Lab, a VR application development framework for educational scenarios mainly involving scientific processes executed in laboratory environments such as chemistry and biology laboratories. This work is an extension of the Onlabs simulator which has been developed by the Hellenic Open University as a distance teaching enabler for similar subjects, helping to alleviate the need for access to the physical laboratory infrastructure; thus, shortening training periods of students in the laboratory and making their training during the periods of physical presence more productive and secure. The extensions of the Onlabs to deliver an enhanced and modular framework that can be extended to multiple educational scenarios is the work performed within the context of the European project XR2Learn (Leveraging the European XR industry technologies to empower immersive learning and training).
Multi-User Localization and Tracking with Spatiotemporal Correlation in Multi-RIS-Assisted Systems
Venue: arXiv cs.IT 2024
Year: 2024

As a promising technique, reconfigurable intelligent surfaces (RISs) exhibit its tremendous potential for high accuracy positioning. In this paper, we investigates multi-user localization and tracking problem in multi-RISs-assisted system. In particular, we incorporate statistical spatiotemporal correlation of multi-user locations and develop a general spatiotemporal Markov random field model (ST-+MRF) to capture multi-user dynamic motion states. To achieve superior performance, a novel multi-user tracking algorithm is proposed based on Bayesian inference to effectively utilize the correlation among users. Besides that, considering the necessity of RISs configuration for tracking performance, we further propose a predictive RISs beamforming optimization scheme via semidefinite relaxation (SDR). Compared to other pioneering work, finally, we confirm that the proposed strategy by alternating tracking algorithm and RISs optimization, can achieve significant performance gains over benchmark schemes.
Digital Twin sensors in cultural heritage applications
Venue: arXiv cs.CY 2024
Year: 2024

The paper concerns the extension of the Heritage Digital Twin Ontology introduced in previous work to describe the reactivity of digital twins used for cultural heritage documentation by including the semantic description of sensors and activators and all the process of interacting with the real world. After analysing previous work on the use of digital twins in cultural heritage, a summary description of the Heritage Digital Twin Ontology is provided, and the existing applications of digital twins to cultural heritage are overviewed, with references to reviews summarizing the large production of scientific contributions on the topic. Then a novel ontology, named Reactive Digital Twin Ontology is described, in which sensors, activators and the decision processes are also semantically described, turning the previous synchronic approach to cultural heritage documentation into a diachronic one. Some case studies exemplify this theory.