OpenRaiser/Intern-Atlas huggingface dataset:
Towards Case-based Interpretability for Medical Federated Learning
Venue: arXiv cs.LG 2024
Year: 2024

We explore deep generative models to generate case-based explanations in a medical federated learning setting. Explaining AI model decisions through case-based interpretability is paramount to increasing trust and allowing widespread adoption of AI in clinical practice. However, medical AI training paradigms are shifting towards federated learning settings in order to comply with data protection regulations. In a federated scenario, past data is inaccessible to the current user. Thus, we use a deep generative model to generate synthetic examples that protect privacy and explain decisions. Our proof-of-concept focuses on pleural effusion diagnosis and uses publicly available Chest X-ray data.
Temporally-consistent 3D Reconstruction of Birds
Venue: arXiv cs.CV 2024
Year: 2024

This paper deals with 3D reconstruction of seabirds which recently came into focus of environmental scientists as valuable bio-indicators for environmental change. Such 3D information is beneficial for analyzing the bird's behavior and physiological shape, for example by tracking motion, shape, and appearance changes. From a computer vision perspective birds are especially challenging due to their rapid and oftentimes non-rigid motions. We propose an approach to reconstruct the 3D pose and shape from monocular videos of a specific breed of seabird - the common murre. Our approach comprises a full pipeline of detection, tracking, segmentation, and temporally consistent 3D reconstruction. Additionally, we propose a temporal loss that extends current single-image 3D bird pose estimators to the temporal domain. Moreover, we provide a real-world dataset of 10000 frames of video observations on average capture nine birds simultaneously, comprising a large variety of motions and interactions, including a smaller test set with bird-specific keypoint labels. Using our temporal optimization, we achieve state-of-the-art performance for the challenging sequences in our dataset.
Transmitter Actions for Secure Integrated Sensing and Communication
Venue: arXiv cs.IT 2024
Year: 2024

This work models a secure integrated sensing and communication (ISAC) system as a wiretap channel with action-dependent channel states and channel output feedback, e.g., obtained through reflections. The transmitted message is split into a common and a secure message, both of which must be reliably recovered at the legitimate receiver, while the secure message needs to be kept secret from the eavesdropper. The transmitter actions, such as beamforming vector design, affect the corresponding state at each channel use. The action sequence is modeled to depend on both the transmitted message and channel output feedback. For perfect channel output feedback, the secrecy-distortion regions are provided for physically-degraded and reversely-physically-degraded secure ISAC channels with transmitter actions. The corresponding rate regions when the entire message should be kept secret are also provided. The results are illustrated through characterizing the secrecy-distortion region of a binary example.
Synthetic Networks That Preserve Edge Connectivity
Venue: arXiv cs.SI 2024
Year: 2024

Since true communities within real-world networks are rarely known, synthetic networks with planted ground truths are valuable for evaluating the performance of community detection methods. Of the synthetic network generation tools available, Stochastic Block Models (SBMs) produce networks with ground truth clusters that well approximate input parameters from real-world networks and clusterings. However, we show that SBMs can produce disconnected ground truth clusters, even when given parameters from clusterings where all clusters are connected. Here we describe the REalistic Cluster Connectivity Simulator (RECCS), a technique that modifies an SBM synthetic network to improve the fit to a given clustered real-world network with respect to edge connectivity within clusters, while maintaining the good fit with respect to other network and cluster statistics. Using real-world networks up to 13.9 million nodes in size, we show that RECCS, applied to stochastic block models, results in synthetic networks that have a better fit to cluster edge connectivity than unmodified SBMs, while providing roughly the same quality fit for other network and clustering parameters as unmodified SBMs.
Hierarchical Network Fusion for Multi-Modal Electron Micrograph Representation Learning with Foundational Large Language Models
Venue: arXiv cs.CV 2024
Year: 2024

Characterizing materials with electron micrographs is a crucial task in fields such as semiconductors and quantum materials. The complex hierarchical structure of micrographs often poses challenges for traditional classification methods. In this study, we propose an innovative backbone architecture for analyzing electron micrographs. We create multi-modal representations of the micrographs by tokenizing them into patch sequences and, additionally, representing them as vision graphs, commonly referred to as patch attributed graphs. We introduce the Hierarchical Network Fusion (HNF), a multi-layered network structure architecture that facilitates information exchange between the multi-modal representations and knowledge integration across different patch resolutions. Furthermore, we leverage large language models (LLMs) to generate detailed technical descriptions of nanomaterials as auxiliary information to assist in the downstream task. We utilize a cross-modal attention mechanism for knowledge fusion across cross-domain representations(both image-based and linguistic insights) to predict the nanomaterial category. This multi-faceted approach promises a more comprehensive and accurate representation and classification of micrographs for nanomaterial identification. Our framework outperforms traditional methods, overcoming challenges posed by distributional shifts, and facilitating high-throughput screening.
Submodular Maximization Approaches for Equitable Client Selection in Federated Learning
Venue: arXiv cs.LG 2024
Year: 2024

In a conventional Federated Learning framework, client selection for training typically involves the random sampling of a subset of clients in each iteration. However, this random selection often leads to disparate performance among clients, raising concerns regarding fairness, particularly in applications where equitable outcomes are crucial, such as in medical or financial machine learning tasks. This disparity typically becomes more pronounced with the advent of performance-centric client sampling techniques. This paper introduces two novel methods, namely SUBTRUNC and UNIONFL, designed to address the limitations of random client selection. Both approaches utilize submodular function maximization to achieve more balanced models. By modifying the facility location problem, they aim to mitigate the fairness concerns associated with random selection. SUBTRUNC leverages client loss information to diversify solutions, while UNIONFL relies on historical client selection data to ensure a more equitable performance of the final model. Moreover, these algorithms are accompanied by robust theoretical guarantees regarding convergence under reasonable assumptions. The efficacy of these methods is demonstrated through extensive evaluations across heterogeneous scenarios, revealing significant improvements in fairness as measured by a client dissimilarity metric.
Understanding Uncertainty-based Active Learning Under Model Mismatch
Venue: arXiv cs.LG 2024
Year: 2024

Instead of randomly acquiring training data points, Uncertainty-based Active Learning (UAL) operates by querying the label(s) of pivotal samples from an unlabeled pool selected based on the prediction uncertainty, thereby aiming at minimizing the labeling cost for model training. The efficacy of UAL critically depends on the model capacity as well as the adopted uncertainty-based acquisition function. Within the context of this study, our analytical focus is directed toward comprehending how the capacity of the machine learning model may affect UAL efficacy. Through theoretical analysis, comprehensive simulations, and empirical studies, we conclusively demonstrate that UAL can lead to worse performance in comparison with random sampling when the machine learning model class has low capacity and is unable to cover the underlying ground truth. In such situations, adopting acquisition functions that directly target estimating the prediction performance may be beneficial for improving the performance of UAL.
Revisit the Partial Coloring Method: Prefix Spencer and Sampling
Venue: arXiv cs.DS 2024
Year: 2024

As the most powerful tool in discrepancy theory, the partial coloring method has wide applications in many problems including the Beck-Fiala problem and Spencer's celebrated result. Currently, there are two major algorithmic methods for the partial coloring method: the first approach uses linear algebraic tools; and the second is called Gaussian measure algorithm. We explore the advantages of these two methods and show the following results for them separately. 1. Spencer conjectured that the prefix discrepancy of any $\mathbf{A} \in \{0,1\}^{m \times n}$ is $O(\sqrt{m})$. We show how to find a partial coloring with prefix discrepancy $O(\sqrt{m})$ and $Ω(n)$ entries in $\{ \pm 1\}$ efficiently. To the best of our knowledge, this provides the first partial coloring whose prefix discrepancy is almost optimal. However, unlike the classical discrepancy problem, there is no reduction on the number of variables $n$ for the prefix problem. By recursively applying partial coloring, we obtain a full coloring with prefix discrepancy $O(\sqrt{m} \cdot \log \frac{O(n)}{m})$. Prior to this work, the best bounds of the prefix Spencer conjecture for arbitrarily large $n$ were $2m$ and $O(\sqrt{m \log n})$. 2. Our second result extends the first linear algebraic approach to a sampling algorithm in Spencer's classical setting. On the first hand, Spencer proved that there are $1.99^m$ good colorings with discrepancy $O(\sqrt{m})$. Hence a natural question is to design efficient random sampling algorithms in Spencer's setting. On the other hand, some applications of discrepancy theory, prefer a random solution instead of a fixed one. Our second result is an efficient sampling algorithm whose random output has min-entropy $Ω(n)$ and discrepancy $O(\sqrt{m})$. Moreover, our technique extends the linear algebraic framework by incorporating leverage scores of randomized matrix algorithms.
Self-Parameterization Based Multi-Resolution Mesh Convolution Networks
Venue: arXiv cs.CV 2024
Year: 2024

This paper addresses the challenges of designing mesh convolution neural networks for 3D mesh dense prediction. While deep learning has achieved remarkable success in image dense prediction tasks, directly applying or extending these methods to irregular graph data, such as 3D surface meshes, is nontrivial due to the non-uniform element distribution and irregular connectivity in surface meshes which make it difficult to adapt downsampling, upsampling, and convolution operations. In addition, commonly used multiresolution networks require repeated high-to-low and then low-to-high processes to boost the performance of recovering rich, high-resolution representations. To address these challenges, this paper proposes a self-parameterization-based multi-resolution convolution network that extends existing image dense prediction architectures to 3D meshes. The novelty of our approach lies in two key aspects. First, we construct a multi-resolution mesh pyramid directly from the high-resolution input data and propose area-aware mesh downsampling/upsampling operations that use sequential bijective inter-surface mappings between different mesh resolutions. The inter-surface mapping redefines the mesh, rather than reshaping it, which thus avoids introducing unnecessary errors. Second, we maintain the high-resolution representation in the multi-resolution convolution network, enabling multi-scale fusions to exchange information across parallel multi-resolution subnetworks, rather than through connections of high-to-low resolution subnetworks in series. These features differentiate our approach from most existing mesh convolution networks and enable more accurate mesh dense predictions, which is confirmed in experiments.
ICFRNet: Image Complexity Prior Guided Feature Refinement for Real-time Semantic Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

In this paper, we leverage image complexity as a prior for refining segmentation features to achieve accurate real-time semantic segmentation. The design philosophy is based on the observation that different pixel regions within an image exhibit varying levels of complexity, with higher complexities posing a greater challenge for accurate segmentation. We thus introduce image complexity as prior guidance and propose the Image Complexity prior-guided Feature Refinement Network (ICFRNet). This network aggregates both complexity and segmentation features to produce an attention map for refining segmentation features within an Image Complexity Guided Attention (ICGA) module. We optimize the network in terms of both segmentation and image complexity prediction tasks with a combined loss function. Experimental results on the Cityscapes and CamViD datasets have shown that our ICFRNet achieves higher accuracy with a competitive efficiency for real-time segmentation.
GenOnet: Generative Open xG Network Simulation with Multi-Agent LLM and ns-3
Venue: arXiv cs.NI 2024
Year: 2024

The move toward Sixth-Generation (6G) networks relies on open interfaces and protocols for seamless interoperability across devices, vendors, and technologies. In this context, open 6G development involves multiple disciplines and requires advanced simulation approaches for testing. In this demo paper, we propose a generative simulation approach based on a multi-agent Large Language Model (LLM) and Network Simulator 3 (ns-3), called Generative Open xG Network Simulation (GenOnet), to effectively generate, debug, execute, and interpret simulated Open Fifth-Generation (5G) environments. The first version of GenOnet application represents a specialized adaptation of the OpenAI GPT models. It incorporates supplementary tools, agents, 5G standards, and seamless integration with ns-3 simulation capabilities, supporting both C++ variants and Python implementations. This release complies with the latest Open Radio Access Network (O-RAN) and 3GPP standards.
Analyzing the Impact of Splicing Artifacts in Partially Fake Speech Signals
Venue: arXiv cs.SD 2024
Year: 2024

Speech deepfake detection has recently gained significant attention within the multimedia forensics community. Related issues have also been explored, such as the identification of partially fake signals, i.e., tracks that include both real and fake speech segments. However, generating high-quality spliced audio is not as straightforward as it may appear. Spliced signals are typically created through basic signal concatenation. This process could introduce noticeable artifacts that can make the generated data easier to detect. We analyze spliced audio tracks resulting from signal concatenation, investigate their artifacts and assess whether such artifacts introduce any bias in existing datasets. Our findings reveal that by analyzing splicing artifacts, we can achieve a detection EER of 6.16% and 7.36% on PartialSpoof and HAD datasets, respectively, without needing to train any detector. These results underscore the complexities of generating reliable spliced audio data and lead to discussions that can help improve future research in this area.
Informativeness and Trust in Bayesian Persuasion
Venue: arXiv cs.GT 2024
Year: 2024

A persuasion policy successfully persuades an agent to pick a particular action only if the information is designed in a manner that convinces the agent that it is in their best interest to pick that action. Thus, it is natural to ask, what makes the agent trust the persuader's suggestion? We study a Bayesian persuasion interaction between a sender and a receiver where the sender has access to private information and the receiver attempts to recover this information from messages sent by the sender. The sender crafts these messages in an attempt to maximize its utility which depends on the source symbol and the symbol recovered by the receiver. Our goal is to characterize the \textit{Stackelberg game value}, and the amount of true information revealed by the sender during persuasion. We find that the SGV is given by the optimal value of a \textit{linear program} on probability distributions constrained by certain \textit{trust constraints}. These constraints encode that any signal in a persuasion strategy must contain more truth than untruth and thus impose a fundamental bound on the extent of obfuscation a sender can perform. We define \textit{informativeness} of the sender as the minimum expected number of symbols truthfully revealed by the sender in any accumulation point of a sequence of $\varepsilon$-equilibrium persuasion strategies, and show that it is given by another linear program. Informativeness is a fundamental bound on the amount of information the sender must reveal to persuade a receiver. Closed form expressions for the SGV and the informativeness are presented for structured utility functions. This work generalizes our previous work where the sender and the receiver were constrained to play only deterministic strategies and a similar notion of informativeness was characterized. Comparisons between the previous and current notions are discussed.
PAM: A Propagation-Based Model for Segmenting Any 3D Objects across Multi-Modal Medical Images
Venue: arXiv cs.CV 2024
Year: 2024

Volumetric segmentation is important in medical imaging, but current methods face challenges like requiring lots of manual annotations and being tailored to specific tasks, which limits their versatility. General segmentation models used for natural images don't perform well with the unique features of medical images. There's a strong need for an adaptable approach that can effectively handle different 3D medical structures and imaging modalities. In this study, we present PAM (Propagating Anything Model), a segmentation approach that uses a 2D prompt, like a bounding box or sketch, to create a complete 3D segmentation of medical image volumes. PAM works by modeling relationships between slices, maintaining information flow across the 3D structure. It combines a CNN-based UNet for processing within slices and a Transformer-based attention module for propagating information between slices, leading to better generalizability across various imaging modalities. PAM significantly outperformed existing models like MedSAM and SegVol, with an average improvement of over 18.1% in dice similarity coefficient (DSC) across 44 medical datasets and various object types. It also showed stable performance despite prompt deviations and different propagation setups, and faster inference speeds compared to other models. PAM's one-view prompt design made it more efficient, reducing interaction time by about 63.6% compared to two-view prompts. Thanks to its focus on structural relationships, PAM handled unseen and complex objects well, showing a unique ability to generalize to new situations. PAM represents an advancement in medical image segmentation, effectively reducing the need for extensive manual work and specialized training. Its adaptability makes it a promising tool for more automated and reliable analysis in clinical settings.
$\boldsymbol{Steiner}$-Hardness: A Query Hardness Measure for Graph-Based ANN Indexes
Venue: arXiv cs.DB 2024
Year: 2024

Graph-based indexes have been widely employed to accelerate approximate similarity search of high-dimensional vectors. However, the performance of graph indexes to answer different queries varies vastly, leading to an unstable quality of service for downstream applications. This necessitates an effective measure to test query hardness on graph indexes. Nonetheless, popular distance-based hardness measures like LID lose their effects due to the ignorance of the graph structure. In this paper, we propose $Steiner$-hardness, a novel connection-based graph-native query hardness measure. Specifically, we first propose a theoretical framework to analyze the minimum query effort on graph indexes and then define $Steiner$-hardness as the minimum effort on a representative graph. Moreover, we prove that our $Steiner$-hardness is highly relevant to the classical Directed $Steiner$ Tree (DST) problems. In this case, we design a novel algorithm to reduce our problem to DST problems and then leverage their solvers to help calculate $Steiner$-hardness efficiently. Compared with LID and other similar measures, $Steiner$-hardness shows a significantly better correlation with the actual query effort on various datasets. Additionally, an unbiased evaluation designed based on $Steiner$-hardness reveals new ranking results, indicating a meaningful direction for enhancing the robustness of graph indexes. This paper is accepted by PVLDB 2025.
Prediction of COPD Using Machine Learning, Clinical Summary Notes, and Vital Signs
Venue: arXiv cs.LG 2024
Year: 2024

Chronic obstructive pulmonary disease (COPD) is a chronic inflammatory lung disease that causes obstructed airflow from the lungs. In the United States, more than 15.7 million Americans have been diagnosed with COPD, with 96% of individuals living with at least one other chronic health condition. It is the 4th leading cause of death in the country. Over 2.2 million patients are admitted to hospitals annually due to COPD exacerbations. Monitoring and predicting patient exacerbations on-time could save their life. This paper presents two different predictive models to predict COPD exacerbation using AI and natural language processing (NLP) approaches. These models use respiration summary notes, symptoms, and vital signs. To train and test these models, data records containing physiologic signals and vital signs time series were used. These records were captured from patient monitors and comprehensive clinical data obtained from hospital medical information systems for tens of thousands of Intensive Care Unit (ICU) patients. We achieved an area under the Receiver operating characteristic (ROC) curve of 0.82 in detection and prediction of COPD exacerbation.
Optimizing Luxury Vehicle Dealership Networks: A Graph Neural Network Approach to Site Selection
Venue: arXiv cs.LG 2024
Year: 2024

This study presents a novel application of Graph Neural Networks (GNNs) to optimize dealership network planning for a luxury car manufacturer in the U.S. By conducting a comprehensive literature review on dealership location determinants, the study identifies 65 county-level explanatory variables, augmented by two additional measures of regional interconnectedness derived from social and mobility data. An ablation study involving 34 variable combinations and ten state-of-the-art GNN operators reveals key insights into the predictive power of various variables, particularly highlighting the significance of competition, demographic factors, and mobility patterns in influencing dealership location decisions. The analysis pinpoints seven specific counties as promising targets for network expansion. This research not only illustrates the effectiveness of GNNs in solving complex geospatial decision-making problems but also provides actionable recommendations and valuable methodological insights for industry practitioners.