OpenRaiser/Intern-Atlas huggingface dataset:
Re-thinking Richardson-Lucy without Iteration Cutoffs: Physically Motivated Bayesian Deconvolution
Venue: arXiv cs.CV 2024
Year: 2024

Richardson-Lucy deconvolution is widely used to restore images from degradation caused by the broadening effects of a point spread function and corruption by photon shot noise, in order to recover an underlying object. In practice, this is achieved by iteratively maximizing a Poisson emission likelihood. However, the RL algorithm is known to prefer sparse solutions and overfit noise, leading to high-frequency artifacts. The structure of these artifacts is sensitive to the number of RL iterations, and this parameter is typically hand-tuned to achieve reasonable perceptual quality of the inferred object. Overfitting can be mitigated by introducing tunable regularizers or other ad hoc iteration cutoffs in the optimization as otherwise incorporating fully realistic models can introduce computational bottlenecks. To resolve these problems, we present Bayesian deconvolution, a rigorous deconvolution framework that combines a physically accurate image formation model avoiding the challenges inherent to the RL approach. Our approach achieves deconvolution while satisfying the following desiderata: I deconvolution is performed in the spatial domain (as opposed to the frequency domain) where all known noise sources are accurately modeled and integrated in the spirit of providing full probability distributions over the density of the putative object recovered; II the probability distribution is estimated without making assumptions on the sparsity or continuity of the underlying object; III unsupervised inference is performed and converges to a stable solution with no user-dependent parameter tuning or iteration cutoff; IV deconvolution produces strictly positive solutions; and V implementation is amenable to fast, parallelizable computation.
When Two Wrongs Don't Make a Right" -- Examining Confirmation Bias and the Role of Time Pressure During Human-AI Collaboration in Computational Pathology
Venue: arXiv cs.HC 2024
Year: 2024

Artificial intelligence (AI)-based decision support systems hold promise for enhancing diagnostic accuracy and efficiency in computational pathology. However, human-AI collaboration can introduce and amplify cognitive biases, such as confirmation bias caused by false confirmation when erroneous human opinions are reinforced by inaccurate AI output. This bias may worsen when time pressure, ubiquitously present in routine pathology, strains practitioners' cognitive resources. We quantified confirmation bias triggered by AI-induced false confirmation and examined the role of time constraints in a web-based experiment, where trained pathology experts (n=28) estimated tumor cell percentages. Our results suggest that AI integration may fuel confirmation bias, evidenced by a statistically significant positive linear-mixed-effects model coefficient linking AI recommendations mirroring flawed human judgment and alignment with system advice. Conversely, time pressure appeared to weaken this relationship. These findings highlight potential risks of AI use in healthcare and aim to support the safe integration of clinical decision support systems.
Mixed Reality Teleoperation Assistance for Direct Control of Humanoids
Venue: arXiv cs.RO 2024
Year: 2024

Teleoperation plays a crucial role in enabling robot operations in challenging environments, yet existing limitations in effectiveness and accuracy necessitate the development of innovative strategies for improving teleoperated tasks. This article introduces a novel approach that utilizes mixed reality and assistive autonomy to enhance the efficiency and precision of humanoid robot teleoperation. By leveraging Probabilistic Movement Primitives, object detection, and Affordance Templates, the assistance combines user motion with autonomous capabilities, achieving task efficiency while maintaining human-like robot motion. Experiments and feasibility studies on the Nadia robot confirm the effectiveness of the proposed framework.
MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
Venue: arXiv cs.LG 2024
Year: 2024

The emergence of Mixture of Experts (MoE) LLMs has significantly advanced the development of language models. Compared to traditional LLMs, MoE LLMs outperform traditional LLMs by achieving higher performance with considerably fewer activated parameters. Despite this efficiency, their enormous parameter size still leads to high deployment costs. In this paper, we introduce a two-stage compression method tailored for MoE to reduce the model size and decrease the computational cost. First, in the inter-expert pruning stage, we analyze the importance of each layer and propose the Layer-wise Genetic Search and Block-wise KT-Reception Field with the non-uniform pruning ratio to prune the individual expert. Second, in the intra-expert decomposition stage, we apply the low-rank decomposition to further compress the parameters within the remaining experts. Extensive experiments on Qwen1.5-MoE-A2.7B, DeepSeek-V2-Lite, and Mixtral-8$\times$7B demonstrate that our proposed methods can both reduce the model size and enhance inference efficiency while maintaining performance in various zero-shot tasks. The code will be available at \url{https://github.com/xiaochengsky/MoEI-2.git}
Online Moderation in Competitive Action Games: How Intervention Affects Player Behaviors
Venue: arXiv cs.CY 2024
Year: 2024

Online competitive action games have flourished as a space for entertainment and social connections, yet they face challenges from a small percentage of players engaging in disruptive behaviors. This study delves into the under-explored realm of understanding the effects of moderation on player behavior within online gaming on an example of a popular title - Call of Duty(R): Modern Warfare(R)II. We employ a quasi-experimental design and causal inference techniques to examine the impact of moderation in a real-world industry-scale moderation system. We further delve into novel aspects around the impact of delayed moderation, as well as the severity of applied punishment. We examine these effects on a set of four disruptive behaviors including cheating, offensive user name, chat, and voice. Our findings uncover the dual impact moderation has on reducing disruptive behavior and discouraging disruptive players from participating. We further uncover differences in the effectiveness of quick and delayed moderation and the varying severity of punishment. Our examination of real-world gaming interactions sets a precedent in understanding the effectiveness of moderation and its impact on player behavior. Our insights offer actionable suggestions for the most promising avenues for improving real-world moderation practices, as well as the heterogeneous impact moderation has on indifferent players.
BinEnhance: An Enhancement Framework Based on External Environment Semantics for Binary Code Search
Venue: arXiv cs.SE 2024
Year: 2024

Binary code search plays a crucial role in applications like software reuse detection. Currently, existing models are typically based on either internal code semantics or a combination of function call graphs (CG) and internal code semantics. However, these models have limitations. Internal code semantic models only consider the semantics within the function, ignoring the inter-function semantics, making it difficult to handle situations such as function inlining. The combination of CG and internal code semantics is insufficient for addressing complex real-world scenarios. To address these limitations, we propose BinEnhance, a novel framework designed to leverage the inter-function semantics to enhance the expression of internal code semantics for binary code search. Specifically, BinEnhance constructs an External Environment Semantic Graph (EESG), which establishes a stable and analogous external environment for homologous functions by using different inter-function semantic relations (e.g., call, location, data-co-use). After the construction of EESG, we utilize the embeddings generated by existing internal code semantic models to initialize nodes of EESG. Finally, we design a Semantic Enhancement Model (SEM) that uses Relational Graph Convolutional Networks (RGCNs) and a residual block to learn valuable external semantics on the EESG for generating the enhanced semantics embedding. In addition, BinEnhance utilizes data feature similarity to refine the cosine similarity of semantic embeddings. We conduct experiments under six different tasks (e.g., under function inlining scenario) and the results illustrate the performance and robustness of BinEnhance. The application of BinEnhance to HermesSim, Asm2vec, TREX, Gemini, and Asteria on two public datasets results in an improvement of Mean Average Precision (MAP) from 53.6% to 69.7%. Moreover, the efficiency increases fourfold.
Mewz: Lightweight Execution Environment for WebAssembly with High Isolation and Portability using Unikernels
Venue: arXiv cs.DC 2024
Year: 2024

Cloud computing requires isolation and portability for workloads. Cloud vendors must isolate each user's resources from others to prevent them from attacking other users or the whole system. Users may want to move their applications to different environments, for instance other cloud, on-premise servers, or edge devices. Virtual machines (VMs) and containers are widely used to achieve these requirements. However, there are two problems with combined use of VMs and containers. First, container images depend on host operating systems and CPU architectures. Users need to manage different container images for each platform to run the same codes on different OSes and ISAs. Second, performance is degraded by the overheads of both VMs and containers. Previous researches have solved each of these problems separately, but no solution solves both problems simultaneously. Therefore, execution environments of applications on cloud are required to be more lightweight and portable while ensuring isolation is required. We propose a new system that combines WebAssembly (Wasm) and unikernels. Wasm is a portable binary format, so it can be run on any host operating systems and architectures. Unikernels are kernels statically linked with applications, which reduces the overhead of guest kernel. In this approach, users deploy applications as a Wasm binary and it runs as a unikernel on cloud. To realize this system, we propose a mechanism to convert a Wasm binary into a unikernel image with the Wasm AoT-compiled to native code. We developed a unikernel with Wasm System Interface (WASI) API and an Ahead-of-Time (AoT) compiler that converts Wasm to native code. We evaluated the performance of the system by running a simple HTTP server compiled into Wasm and native code. The performance was improved by 30\% compared to running it with an existing Wasm runtime on Linux on a virtual machine.
Do LLMs Know to Respect Copyright Notice?
Venue: arXiv cs.CL 2024
Year: 2024

Prior study shows that LLMs sometimes generate content that violates copyright. In this paper, we study another important yet underexplored problem, i.e., will LLMs respect copyright information in user input, and behave accordingly? The research problem is critical, as a negative answer would imply that LLMs will become the primary facilitator and accelerator of copyright infringement behavior. We conducted a series of experiments using a diverse set of language models, user prompts, and copyrighted materials, including books, news articles, API documentation, and movie scripts. Our study offers a conservative evaluation of the extent to which language models may infringe upon copyrights when processing user input containing protected material. This research emphasizes the need for further investigation and the importance of ensuring LLMs respect copyright regulations when handling user input to prevent unauthorized use or reproduction of protected content. We also release a benchmark dataset serving as a test bed for evaluating infringement behaviors by LLMs and stress the need for future alignment.
HIP: Hierarchical Point Modeling and Pre-training for Visual Information Extraction
Venue: arXiv cs.CV 2024
Year: 2024

End-to-end visual information extraction (VIE) aims at integrating the hierarchical subtasks of VIE, including text spotting, word grouping, and entity labeling, into a unified framework. Dealing with the gaps among the three subtasks plays a pivotal role in designing an effective VIE model. OCR-dependent methods heavily rely on offline OCR engines and inevitably suffer from OCR errors, while OCR-free methods, particularly those employing a black-box model, might produce outputs that lack interpretability or contain hallucinated content. Inspired by CenterNet, DeepSolo, and ESP, we propose HIP, which models entities as HIerarchical Points to better conform to the hierarchical nature of the end-to-end VIE task. Specifically, such hierarchical points can be flexibly encoded and subsequently decoded into desired text transcripts, centers of various regions, and categories of entities. Furthermore, we devise corresponding hierarchical pre-training strategies, categorized as image reconstruction, layout learning, and language enhancement, to reinforce the cross-modality representation of the hierarchical encoders. Quantitative experiments on public benchmarks demonstrate that HIP outperforms previous state-of-the-art methods, while qualitative results show its excellent interpretability.
A Large-scale Time-aware Agents Simulation for Influencer Selection in Digital Advertising Campaigns
Venue: arXiv cs.SI 2024
Year: 2024

In the digital world, influencers are pivotal as opinion leaders, shaping the views and choices of their influencees. Modern advertising often follows this trend, where marketers choose appropriate influencers for product endorsements, based on thorough market analysis. Previous studies on influencer selection have typically relied on numerical representations of individual opinions and interactions, a method that simplifies the intricacies of social dynamics. In this work, we first introduce a Time-aware Influencer Simulator (TIS), helping promoters identify and select the right influencers to market their products, based on LLM simulation. To validate our approach, we conduct experiments on the public advertising campaign dataset SAGraph which encompasses social relationships, posts, and user interactions. The results show that our method outperforms traditional numerical feature-based approaches and methods using limited LLM agents. Our research shows that simulating user timelines and content lifecycles over time simplifies scaling, allowing for large-scale agent simulations in social networks. Additionally, LLM-based agents for social recommendations and advertising offer substantial benefits for decision-making in promotional campaigns.
Negative-Free Self-Supervised Gaussian Embedding of Graphs
Venue: arXiv cs.LG 2024
Year: 2024

Graph Contrastive Learning (GCL) has recently emerged as a promising graph self-supervised learning framework for learning discriminative node representations without labels. The widely adopted objective function of GCL benefits from two key properties: \emph{alignment} and \emph{uniformity}, which align representations of positive node pairs while uniformly distributing all representations on the hypersphere. The uniformity property plays a critical role in preventing representation collapse and is achieved by pushing apart augmented views of different nodes (negative pairs). As such, existing GCL methods inherently rely on increasing the quantity and quality of negative samples, resulting in heavy computational demands, memory overhead, and potential class collision issues. In this study, we propose a negative-free objective to achieve uniformity, inspired by the fact that points distributed according to a normalized isotropic Gaussian are uniformly spread across the unit hypersphere. Therefore, we can minimize the distance between the distribution of learned representations and the isotropic Gaussian distribution to promote the uniformity of node representations. Our method also distinguishes itself from other approaches by eliminating the need for a parameterized mutual information estimator, an additional projector, asymmetric structures, and, crucially, negative samples. Extensive experiments over seven graph benchmarks demonstrate that our proposal achieves competitive performance with fewer parameters, shorter training times, and lower memory consumption compared to existing GCL methods.
Graph Cross-Correlated Network for Recommendation
Venue: arXiv cs.IR 2024
Year: 2024

Collaborative filtering (CF) models have demonstrated remarkable performance in recommender systems, which represent users and items as embedding vectors. Recently, due to the powerful modeling capability of graph neural networks for user-item interaction graphs, graph-based CF models have gained increasing attention. They encode each user/item and its subgraph into a single super vector by combining graph embeddings after each graph convolution. However, each hop of the neighbor in the user-item subgraphs carries a specific semantic meaning. Encoding all subgraph information into single vectors and inferring user-item relations with dot products can weaken the semantic information between user and item subgraphs, thus leaving untapped potential. Exploiting this untapped potential provides insight into improving performance for existing recommendation models. To this end, we propose the Graph Cross-correlated Network for Recommendation (GCR), which serves as a general recommendation paradigm that explicitly considers correlations between user/item subgraphs. GCR first introduces the Plain Graph Representation (PGR) to extract information directly from each hop of neighbors into corresponding PGR vectors. Then, GCR develops Cross-Correlated Aggregation (CCA) to construct possible cross-correlated terms between PGR vectors of user/item subgraphs. Finally, GCR comprehensively incorporates the cross-correlated terms for recommendations. Experimental results show that GCR outperforms state-of-the-art models on both interaction prediction and click-through rate prediction tasks.
$B^4$: A Black-Box Scrubbing Attack on LLM Watermarks
Venue: arXiv cs.CL 2024
Year: 2024

Watermarking has emerged as a prominent technique for LLM-generated content detection by embedding imperceptible patterns. Despite supreme performance, its robustness against adversarial attacks remains underexplored. Previous work typically considers a grey-box attack setting, where the specific type of watermark is already known. Some even necessitates knowledge about hyperparameters of the watermarking method. Such prerequisites are unattainable in real-world scenarios. Targeting at a more realistic black-box threat model with fewer assumptions, we here propose $B^4$, a black-box scrubbing attack on watermarks. Specifically, we formulate the watermark scrubbing attack as a constrained optimization problem by capturing its objectives with two distributions, a Watermark Distribution and a Fidelity Distribution. This optimization problem can be approximately solved using two proxy distributions. Experimental results across 12 different settings demonstrate the superior performance of $B^4$ compared with other baselines.
Assessing the Impact of Sampling, Remixes, and Covers on Original Song Popularity
Venue: arXiv cs.SI 2024
Year: 2024

Music digitalization has introduced new forms of composition known as "musical borrowings", where composers use elements of existing songs -- such as melodies, lyrics, or beats -- to create new songs. Using Who Sampled data and Google Trends, we examine how the popularity of a borrowing song affects the original. Employing Regression Discontinuity Design (RDD) for short-term effects and Granger Causality for long-term impacts, we find evidence of causal popularity boosts in some cases. Borrowee songs can revive interest in older tracks, underscoring economic dynamics that may support fairer compensation in the music industry.
PMoL: Parameter Efficient MoE for Preference Mixing of LLM Alignment
Venue: arXiv cs.CL 2024
Year: 2024

Reinforcement Learning from Human Feedback (RLHF) has been proven to be an effective method for preference alignment of large language models (LLMs) and is widely used in the post-training process of LLMs. However, RLHF struggles with handling multiple competing preferences. This leads to a decrease in the alignment of LLMs with human preferences. To address this issue, we propose Preference Mixture of LoRAs (PMoL) from the perspective of model architecture, which can adapt to any number of preferences to mix. PMoL combines Mixture of Experts (MoE) and Low Rank Adaptor (LoRA). This architecture is innovatively applied to the research of preference alignment and has achieved significant performance improvement. The expert group soft loss is used to enable MoE with the ability to mix preferences. Through comprehensive evaluation by the reward model and GPT-4o, the experiment results show that PMoL has superior preference mixing capabilities compared to baseline methods. PMoL achieves better preference alignment with lower training costs.
ECG-PPS: Privacy Preserving Disease Diagnosis and Monitoring System for Real-Time ECG Signal
Venue: arXiv cs.CR 2024
Year: 2024

This study introduces the development of a state of the art, real time ECG monitoring and analysis system, incorporating cutting edge medical technology and innovative data security measures. Our system performs three distinct functions thaat real time ECG monitoring and disease detection, encrypted storage and synchronized visualization, and statistical analysis on encrypted data. At its core, the system uses a three lead ECG preamplifier connected through a serial port to capture, display, and record real time ECG data. These signals are securely stored in the cloud using robust encryption methods. Authorized medical personnel can access and decrypt this data on their computers, with AES encryption ensuring synchronized real time data tracking and visualization. Furthermore, the system performs statistical operations on the ECG data stored in the cloud without decrypting it, using Fully Homomorphic Encryption (FHE). This enables privacy preserving data analysis while ensuring the security and confidentiality of patient information. By integrating these independent functions, our system significantly enhances the security and efficiency of health monitoring. It supports critical tasks such as disease detection, patient monitoring, and preliminary intervention, all while upholding stringent data privacy standards. We provided detailed discussions on the system's architecture, hardware configuration, software implementation, and clinical performance. The results highlight the potential of this system to improve patient care through secure and efficient ECG monitoring and analysis. This work represents a significant leap forward in medical technology. By incorporating FHE into both data transmission and storage processes, we ensure continuous encryption of data throughout its lifecycle while enabling real time disease diagnosis.