OpenRaiser/Intern-Atlas huggingface dataset:
Harnessing Stacked Intelligent Metasurface for Enhanced Cell-Free Massive MIMO Systems: A Low-Power and Cost Approach
Venue: arXiv cs.IT 2024
Year: 2024

In this paper, we explore the integration of low-power, low-cost stacked intelligent metasurfaces (SIM) into cell-free (CF) massive multiple-input multiple-output (mMIMO) systems to enhance access point (AP) capabilities and address high power consumption and cost challenges. Specifically, we investigate the uplink performance of a SIM-enhanced CF mMIMO system and propose a novel system framework. First, the closed-form expressions of the spectral efficiency (SE) are obtained using the unique two-layer signal processing framework of CF mMIMO systems. Second, to mitigate inter-user interference, an interference-based greedy algorithm for pilot allocation is introduced. Third, a wave-based beamforming algorithm for SIM is proposed, based only on statistical channel state information, which effectively reduces the fronthaul costs. Finally, a max-min SE power control algorithm is proposed to improve the performance of UE with inferior channel conditions. The results indicate that increasing the number of SIM layers and meta-atoms leads to significant performance improvements and allows for a reduction in the number of APs and AP antennas, thus lowering the costs. In particular, the best SE performance is achieved with the deployment of 20 APs plus 1200 SIM meta-atoms. Finally, the proposed wave-based beamforming algorithm can enhance the SE performance of SIM-enhanced CF-mMIMO systems by 57\%, significantly outperforming traditional CF mMIMO systems.
Extended Reality System for Robotic Learning from Human Demonstration
Venue: arXiv cs.RO 2024
Year: 2024

Many real-world tasks are intuitive for a human to perform, but difficult to encode algorithmically when utilizing a robot to perform the tasks. In these scenarios, robotic systems can benefit from expert demonstrations to learn how to perform each task. In many settings, it may be difficult or unsafe to use a physical robot to provide these demonstrations, for example, considering cooking tasks such as slicing with a knife. Extended reality provides a natural setting for demonstrating robotic trajectories while bypassing safety concerns and providing a broader range of interaction modalities. We propose the Robot Action Demonstration in Extended Reality (RADER) system, a generic extended reality interface for learning from demonstration. We additionally present its application to an existing state-of-the-art learning from demonstration approach and show comparable results between demonstrations given on a physical robot and those given using our extended reality system.
LI-GS: Gaussian Splatting with LiDAR Incorporated for Accurate Large-Scale Reconstruction
Venue: arXiv cs.RO 2024
Year: 2024

Large-scale 3D reconstruction is critical in the field of robotics, and the potential of 3D Gaussian Splatting (3DGS) for achieving accurate object-level reconstruction has been demonstrated. However, ensuring geometric accuracy in outdoor and unbounded scenes remains a significant challenge. This study introduces LI-GS, a reconstruction system that incorporates LiDAR and Gaussian Splatting to enhance geometric accuracy in large-scale scenes. 2D Gaussain surfels are employed as the map representation to enhance surface alignment. Additionally, a novel modeling method is proposed to convert LiDAR point clouds to plane-constrained multimodal Gaussian Mixture Models (GMMs). The GMMs are utilized during both initialization and optimization stages to ensure sufficient and continuous supervision over the entire scene while mitigating the risk of over-fitting. Furthermore, GMMs are employed in mesh extraction to eliminate artifacts and improve the overall geometric quality. Experiments demonstrate that our method outperforms state-of-the-art methods in large-scale 3D reconstruction, achieving higher accuracy compared to both LiDAR-based methods and Gaussian-based methods with improvements of 52.6% and 68.7%, respectively.
The Relevance of Item-Co-Exposure For Exposure Bias Mitigation
Venue: arXiv cs.IR 2024
Year: 2024

Through exposing items to users, implicit feedback recommender systems influence the logged interactions, and, ultimately, their own recommendations. This effect is called exposure bias and it can lead to issues such as filter bubbles and echo chambers. Previous research employed the multinomial logit model (MNL) with exposure information to reduce exposure bias on synthetic data. This extended abstract summarizes our previous study in which we investigated whether (i) these findings hold for human-generated choices, (ii) other discrete choice models mitigate bias better, and (iii) an item's estimated relevance can depend on the relevances of the other items that were presented with it. We collected a data set of biased and unbiased choices in a controlled online user study and measured the effects of overexposure and competition. We found that (i) the discrete choice models effectively mitigated exposure bias on human-generated choice data, (ii) there were no significant differences in robustness among the different discrete choice models, and (iii) only multivariate discrete choice models were robust to competition between items. We conclude that discrete choice models mitigate exposure bias effectively because they consider item-co-exposure. Moreover, exposing items alongside more or less popular items can bias future recommendations significantly and item exposure must be tracked for overcoming exposure bias. We consider our work vital for understanding what exposure bias it, how it forms, and how it can be mitigated.
Data-driven surrogate model for etch rate profiles using sensor data from a reactive ion etcher
Venue: arXiv cs.CE 2024
Year: 2024

Reliable predictions of the etch rate profile are desirable in semiconductor manufacturing to prevent etch rate target misses and yield rate excursions. Conventional methods for analyzing etch rate require extensive metrology, which adds considerable costs to manufacturing. We demonstrate a data-driven method to predict the etch rate profiles of a capacitively-coupled plasma RIE etcher from the tool's sensor data. The model employs a hybrid autoencoder-multiquadric interpolation-based approach, with the autoencoder being used to encode the features of the wafers' etch rate profiles into a latent space representation. The tool's sensor data is then used to construct interpolation maps for the latent space variables using multiquadric radial basis functions, which are then used to generate synthetic wafer etch rate profiles using the decoder. The accuracy of the model is determined using experimental data, and the errors are analyzed in interpolation and extrapolation.
A New People-Object Interaction Dataset and NVS Benchmarks
Venue: arXiv cs.CV 2024
Year: 2024

Recently, NVS in human-object interaction scenes has received increasing attention. Existing human-object interaction datasets mainly consist of static data with limited views, offering only RGB images or videos, mostly containing interactions between a single person and objects. Moreover, these datasets exhibit complexities in lighting environments, poor synchronization, and low resolution, hindering high-quality human-object interaction studies. In this paper, we introduce a new people-object interaction dataset that comprises 38 series of 30-view multi-person or single-person RGB-D video sequences, accompanied by camera parameters, foreground masks, SMPL models, some point clouds, and mesh files. Video sequences are captured by 30 Kinect Azures, uniformly surrounding the scene, each in 4K resolution 25 FPS, and lasting for 1$\sim$19 seconds. Meanwhile, we evaluate some SOTA NVS models on our dataset to establish the NVS benchmarks. We hope our work can inspire further research in humanobject interaction.
Introducing the Large Medical Model: State of the art healthcare cost and risk prediction with transformers trained on patient event sequences
Venue: arXiv cs.LG 2024
Year: 2024

With U.S. healthcare spending approaching $5T (NHE Fact Sheet 2024), and 25% of it estimated to be wasteful (Waste in the US the health care system: estimated costs and potential for savings, n.d.), the need to better predict risk and optimal patient care is evermore important. This paper introduces the Large Medical Model (LMM), a generative pre-trained transformer (GPT) designed to guide and predict the broad facets of patient care and healthcare administration. The model is trained on medical event sequences from over 140M longitudinal patient claims records with a specialized vocabulary built from medical terminology systems and demonstrates a superior capability to forecast healthcare costs and identify potential risk factors. Through experimentation and validation, we showcase the LMM's proficiency in not only in cost and risk predictions, but also in discerning intricate patterns within complex medical conditions and an ability to identify novel relationships in patient care. The LMM is able to improve both cost prediction by 14.1% over the best commercial models and chronic conditions prediction by 1.9% over the best transformer models in research predicting a broad set of conditions. The LMM is a substantial advancement in healthcare analytics, offering the potential to significantly enhance risk assessment, cost management, and personalized medicine.
Data Poisoning and Leakage Analysis in Federated Learning
Venue: arXiv cs.LG 2024
Year: 2024

Data poisoning and leakage risks impede the massive deployment of federated learning in the real world. This chapter reveals the truths and pitfalls of understanding two dominating threats: {\em training data privacy intrusion} and {\em training data poisoning}. We first investigate training data privacy threat and present our observations on when and how training data may be leaked during the course of federated training. One promising defense strategy is to perturb the raw gradient update by adding some controlled randomized noise prior to sharing during each round of federated learning. We discuss the importance of determining the proper amount of randomized noise and the proper location to add such noise for effective mitigation of gradient leakage threats against training data privacy. Then we will review and compare different training data poisoning threats and analyze why and when such data poisoning induced model Trojan attacks may lead to detrimental damage on the performance of the global model. We will categorize and compare representative poisoning attacks and the effectiveness of their mitigation techniques, delivering an in-depth understanding of the negative impact of data poisoning. Finally, we demonstrate the potential of dynamic model perturbation in simultaneously ensuring privacy protection, poisoning resilience, and model performance. The chapter concludes with a discussion on additional risk factors in federated learning, including the negative impact of skewness, data and algorithmic biases, as well as misinformation in training data. Powered by empirical evidence, our analytical study offers some transformative insights into effective privacy protection and security assurance strategies in attack-resilient federated learning.
TACE: Tumor-Aware Counterfactual Explanations
Venue: arXiv cs.CV 2024
Year: 2024

The application of deep learning in medical imaging has significantly advanced diagnostic capabilities, enhancing both accuracy and efficiency. Despite these benefits, the lack of transparency in these AI models, often termed "black boxes," raises concerns about their reliability in clinical settings. Explainable AI (XAI) aims to mitigate these concerns by developing methods that make AI decisions understandable and trustworthy. In this study, we propose Tumor Aware Counterfactual Explanations (TACE), a framework designed to generate reliable counterfactual explanations for medical images. Unlike existing methods, TACE focuses on modifying tumor-specific features without altering the overall organ structure, ensuring the faithfulness of the counterfactuals. We achieve this by including an additional step in the generation process which allows to modify only the region of interest (ROI), thus yielding more reliable counterfactuals as the rest of the organ remains unchanged. We evaluate our method on mammography images and brain MRI. We find that our method far exceeds existing state-of-the-art techniques in quality, faithfulness, and generation speed of counterfactuals. Indeed, more faithful explanations lead to a significant improvement in classification success rates, with a 10.69% increase for breast cancer and a 98.02% increase for brain tumors. The code of our work is available at https://github.com/ispamm/TACE.
Towards Unbiased Evaluation of Time-series Anomaly Detector
Venue: arXiv cs.LG 2024
Year: 2024

Time series anomaly detection (TSAD) is an evolving area of research motivated by its critical applications, such as detecting seismic activity, sensor failures in industrial plants, predicting crashes in the stock market, and so on. Across domains, anomalies occur significantly less frequently than normal data, making the F1-score the most commonly adopted metric for anomaly detection. However, in the case of time series, it is not straightforward to use standard F1-score because of the dissociation between `time points' and `time events'. To accommodate this, anomaly predictions are adjusted, called as point adjustment (PA), before the $F_1$-score evaluation. However, these adjustments are heuristics-based, and biased towards true positive detection, resulting in over-estimated detector performance. In this work, we propose an alternative adjustment protocol called ``Balanced point adjustment'' (BA). It addresses the limitations of existing point adjustment methods and provides guarantees of fairness backed by axiomatic definitions of TSAD evaluation.
Cross-Chirality Palmprint Verification: Left is Right for the Right Palmprint
Venue: arXiv cs.CV 2024
Year: 2024

Palmprint recognition has emerged as a prominent biometric authentication method, owing to its high discriminative power and user-friendly nature. This paper introduces a novel Cross-Chirality Palmprint Verification (CCPV) framework that challenges the conventional wisdom in traditional palmprint verification systems. Unlike existing methods that typically require storing both left and right palmprints, our approach enables verification using either palm while storing only one palmprint template. The core of our CCPV framework lies in a carefully designed matching rule. This rule involves flipping both the gallery and query palmprints and calculating the average distance between each pair as the final matching distance. This approach effectively reduces matching variance and enhances overall system robustness. We introduce a novel cross-chirality loss function to construct a discriminative and robust cross-chirality feature space. This loss enforces representation consistency across four palmprint variants: left, right, flipped left, and flipped right. The resulting compact feature space, coupled with the model's enhanced discriminative representation capability, ensures robust performance across various scenarios. We conducted extensive experiments to validate the efficacy of our proposed method. The evaluation encompassed multiple public datasets and considered both closed-set and open-set settings. The results demonstrate the CCPV framework's effectiveness and highlight its potential for real-world applications in palmprint authentication systems.
Improved Image Classification with Manifold Neural Networks
Venue: arXiv cs.LG 2024
Year: 2024

Graph Neural Networks (GNNs) have gained popularity in various learning tasks, with successful applications in fields like molecular biology, transportation systems, and electrical grids. These fields naturally use graph data, benefiting from GNNs' message-passing framework. However, the potential of GNNs in more general data representations, especially in the image domain, remains underexplored. Leveraging the manifold hypothesis, which posits that high-dimensional data lies in a low-dimensional manifold, we explore GNNs' potential in this context. We construct an image manifold using variational autoencoders, then sample the manifold to generate graphs where each node is an image. This approach reduces data dimensionality while preserving geometric information. We then train a GNN to predict node labels corresponding to the image labels in the classification task, and leverage convergence of GNNs to manifold neural networks to analyze GNN generalization. Experiments on MNIST and CIFAR10 datasets demonstrate that GNNs generalize effectively to unseen graphs, achieving competitive accuracy in classification tasks.
A Cognitive Approach to Improving Binary Reverse Engineering with Immersive Virtual Reality
Venue: arXiv cs.HC 2024
Year: 2024

Through its affordances, immersive virtual reality (VR) offers a means to apply embodied and external cognition from the physical realm to solving analytical problems that are typically only conceptual. We present an example of executing a structured analysis following the tenets of cognitive systems engineering to derive immersive affordances applicable to a difficult analytical problem, in our case, reverse engineering (RE) binary programs. We conducted a basic cognitive task analysis of the problem to reveal features of its cognitive model and their associated fundamental cognitive phenomena, and then we mapped those concepts to immersive affordances associated with those concepts. We implemented a subset of those affordances in a VR system facilitating discovery of features of a binary program. Feedback from RE practitioners drove the initial development of the system and we are preparing for a formal effectiveness study to inform the direction of future research.
Interpret the Predictions of Deep Networks via Re-Label Distillation
Venue: arXiv cs.CV 2024
Year: 2024

Interpreting the predictions of a black-box deep network can facilitate the reliability of its deployment. In this work, we propose a re-label distillation approach to learn a direct map from the input to the prediction in a self-supervision manner. The image is projected into a VAE subspace to generate some synthetic images by randomly perturbing its latent vector. Then, these synthetic images can be annotated into one of two classes by identifying whether their labels shift. After that, using the labels annotated by the deep network as teacher, a linear student model is trained to approximate the annotations by mapping these synthetic images to the classes. In this manner, these re-labeled synthetic images can well describe the local classification mechanism of the deep network, and the learned student can provide a more intuitive explanation towards the predictions. Extensive experiments verify the effectiveness of our approach qualitatively and quantitatively.
Bilateral Sharpness-Aware Minimization for Flatter Minima
Venue: arXiv cs.CV 2024
Year: 2024

Sharpness-Aware Minimization (SAM) enhances generalization by reducing a Max-Sharpness (MaxS). Despite the practical success, we empirically found that the MAxS behind SAM's generalization enhancements face the "Flatness Indicator Problem" (FIP), where SAM only considers the flatness in the direction of gradient ascent, resulting in a next minimization region that is not sufficiently flat. A better Flatness Indicator (FI) would bring a better generalization of neural networks. Because SAM is a greedy search method in nature. In this paper, we propose to utilize the difference between the training loss and the minimum loss over the neighborhood surrounding the current weight, which we denote as Min-Sharpness (MinS). By merging MaxS and MinS, we created a better FI that indicates a flatter direction during the optimization. Specially, we combine this FI with SAM into the proposed Bilateral SAM (BSAM) which finds a more flatter minimum than that of SAM. The theoretical analysis proves that BSAM converges to local minima. Extensive experiments demonstrate that BSAM offers superior generalization performance and robustness compared to vanilla SAM across various tasks, i.e., classification, transfer learning, human pose estimation, and network quantization. Code is publicly available at: https://github.com/ajiaaa/BSAM.
Designing an Intervention Tool for End-User Algorithm Audits in Personalized Recommendation Systems
Venue: arXiv cs.HC 2024
Year: 2024

As algorithms increasingly shape user experiences on personalized recommendation platforms, there is a growing need for tools that empower end users to audit these algorithms for potential bias and harms. This paper introduces a novel intervention tool, MapMyFeed, designed to support everyday user audits. The tool addresses key challenges associated with user-driven algorithm audits, such as low algorithm literacy, unstructured audit paths, and the presence of noise. MapMyFeed assists users by offering guiding prompts, tracking audit paths via a browser extension, and visualizing audit results through a live dashboard. The tool will not only foster users' algorithmic literacy and awareness but also enhance more transparent and fair recommendation systems.
FreeAvatar: Robust 3D Facial Animation Transfer by Learning an Expression Foundation Model
Venue: arXiv cs.GR 2024
Year: 2024

Video-driven 3D facial animation transfer aims to drive avatars to reproduce the expressions of actors. Existing methods have achieved remarkable results by constraining both geometric and perceptual consistency. However, geometric constraints (like those designed on facial landmarks) are insufficient to capture subtle emotions, while expression features trained on classification tasks lack fine granularity for complex emotions. To address this, we propose \textbf{FreeAvatar}, a robust facial animation transfer method that relies solely on our learned expression representation. Specifically, FreeAvatar consists of two main components: the expression foundation model and the facial animation transfer model. In the first component, we initially construct a facial feature space through a face reconstruction task and then optimize the expression feature space by exploring the similarities among different expressions. Benefiting from training on the amounts of unlabeled facial images and re-collected expression comparison dataset, our model adapts freely and effectively to any in-the-wild input facial images. In the facial animation transfer component, we propose a novel Expression-driven Multi-avatar Animator, which first maps expressive semantics to the facial control parameters of 3D avatars and then imposes perceptual constraints between the input and output images to maintain expression consistency. To make the entire process differentiable, we employ a trained neural renderer to translate rig parameters into corresponding images. Furthermore, unlike previous methods that require separate decoders for each avatar, we propose a dynamic identity injection module that allows for the joint training of multiple avatars within a single network.