OpenRaiser/Intern-Atlas huggingface dataset:
Mixture of Low-rank Experts for Transferable AI-Generated Image Detection
Venue: arXiv cs.CV 2024
Year: 2024

Generative models have shown a giant leap in synthesizing photo-realistic images with minimal expertise, sparking concerns about the authenticity of online information. This study aims to develop a universal AI-generated image detector capable of identifying images from diverse sources. Existing methods struggle to generalize across unseen generative models when provided with limited sample sources. Inspired by the zero-shot transferability of pre-trained vision-language models, we seek to harness the nontrivial visual-world knowledge and descriptive proficiency of CLIP-ViT to generalize over unknown domains. This paper presents a novel parameter-efficient fine-tuning approach, mixture of low-rank experts, to fully exploit CLIP-ViT's potential while preserving knowledge and expanding capacity for transferable detection. We adapt only the MLP layers of deeper ViT blocks via an integration of shared and separate LoRAs within an MoE-based structure. Extensive experiments on public benchmarks show that our method achieves superiority over state-of-the-art approaches in cross-generator generalization and robustness to perturbations. Remarkably, our best-performing ViT-L/14 variant requires training only 0.08% of its parameters to surpass the leading baseline by +3.64% mAP and +12.72% avg.Acc across unseen diffusion and autoregressive models. This even outperforms the baseline with just 0.28% of the training data. Our code and pre-trained models will be available at https://github.com/zhliuworks/CLIPMoLE.
Theoretical Analysis of the Three-Dimensional CAC Considering Connection and Communication Quality
Venue: arXiv cs.NI 2024
Year: 2024

In emergencies such as disasters, the number of voice calls (VoIP sessions) increases rapidly for a variety of purposes. Thus, a control server near a disaster area may not be able to connect to VoIP sessions due to congestion. To solve this problem, a Call Admission Control (CAC) is needed to determine whether a VoIP session requesting a connection can be accepted or rejected. A CAC has the purpose of guaranteeing the connection quality and communication quality of VoIP sessions. One conventional method classifies VoIP sessions into three classes (emergency VoIP sessions, VoIP sessions from the disaster area, and VoIP sessions from outside the disaster area) by focusing on the outgoing location and offers a CAC with a priority level for each. However, a conventional CAC method cannot be applied to VoIP networks because reception control is designed for Public Switched Telephone Networks (PSTN). When conventional methods are applied to VoIP networks, the connection quality is guaranteed, however the communication quality cannot be guaranteed because the packet dropping probability is not considered. In this paper, we propose a three-dimensional CAC that controls three classes of VoIP sessions and guarantees both communication and connection quality in VoIP networks during emergencies. A conventional CAC method and our proposed CAC method are evaluated in terms of the call blocking probability, which guarantees the connection quality, and packet dropping probability, which guarantees the communication quality, to show the effectiveness of the proposed method.
Tensorized Ant Colony Optimization for GPU Acceleration
Venue: arXiv cs.NE 2024
Year: 2024

Ant Colony Optimization (ACO) is renowned for its effectiveness in solving Traveling Salesman Problems, yet it faces computational challenges in CPU-based environments, particularly with large-scale instances. In response, we introduce a Tensorized Ant Colony Optimization (TensorACO) to utilize the advancements of GPU acceleration. As the core, TensorACO fully transforms ant system and ant path into tensor forms, a process we refer to as tensorization. For the tensorization of ant system, we propose a preprocessing method to reduce the computational overhead by calculating the probability transition matrix. In the tensorization of ant path, we propose an index mapping method to accelerate the update of pheromone matrix by replacing the mechanism of sequential path update with parallel matrix operations. Additionally, we introduce an Adaptive Independent Roulette (AdaIR) method to overcome the challenges of parallelizing ACO's selection mechanism on GPUs. Comprehensive experiments demonstrate the superior performance of TensorACO achieving up to 1921$\times$ speedup over standard ACO. Moreover, the AdaIR method further improves TensorACO's convergence speed by 80% and solution quality by 2%. Source codes are available at https://github.com/EMI-Group/tensoraco.
Online Learning under Haphazard Input Conditions: A Comprehensive Review and Analysis
Venue: arXiv cs.LG 2024
Year: 2024

The domain of online learning has experienced multifaceted expansion owing to its prevalence in real-life applications. Nonetheless, this progression operates under the assumption that the input feature space of the streaming data remains constant. In this survey paper, we address the topic of online learning in the context of haphazard inputs, explicitly foregoing such an assumption. We discuss, classify, evaluate, and compare the methodologies that are adept at modeling haphazard inputs, additionally providing the corresponding code implementations and their carbon footprint. Moreover, we classify the datasets related to the field of haphazard inputs and introduce evaluation metrics specifically designed for datasets exhibiting imbalance. The code of each methodology can be found at https://github.com/Rohit102497/HaphazardInputsReview
CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis
Venue: arXiv cs.CV 2024
Year: 2024

Neural Radiance Fields (NeRF) have achieved huge success in effectively capturing and representing 3D objects and scenes. However, to establish a ubiquitous presence in everyday media formats, such as images and videos, we need to fulfill three key objectives: 1. fast encoding and decoding time, 2. compact model sizes, and 3. high-quality renderings. Despite recent advancements, a comprehensive algorithm that adequately addresses all objectives has yet to be fully realized. In this work, we present CodecNeRF, a neural codec for NeRF representations, consisting of an encoder and decoder architecture that can generate a NeRF representation in a single forward pass. Furthermore, inspired by the recent parameter-efficient finetuning approaches, we propose a finetuning method to efficiently adapt the generated NeRF representations to a new test instance, leading to high-quality image renderings and compact code sizes. The proposed CodecNeRF, a newly suggested encoding-decoding-finetuning pipeline for NeRF, achieved unprecedented compression performance of more than 100x and remarkable reduction in encoding time while maintaining (or improving) the image quality on widely used 3D object datasets.
Towards Understanding the Influence of Reward Margin on Preference Model Performance
Venue: arXiv cs.CL 2024
Year: 2024

Reinforcement Learning from Human Feedback (RLHF) is a widely used framework for the training of language models. However, the process of using RLHF to develop a language model that is well-aligned presents challenges, especially when it comes to optimizing the reward model. Our research has found that existing reward models, when trained using the traditional ranking objective based on human preference data, often struggle to effectively distinguish between responses that are more or less favorable in real-world scenarios. To bridge this gap, our study introduces a novel method to estimate the preference differences without the need for detailed, exhaustive labels from human annotators. Our experimental results provide empirical evidence that incorporating margin values into the training process significantly improves the effectiveness of reward models. This comparative analysis not only demonstrates the superiority of our approach in terms of reward prediction accuracy but also highlights its effectiveness in practical applications.
Unleashing Large Language Models' Proficiency in Zero-shot Essay Scoring
Venue: arXiv cs.CL 2024
Year: 2024

Advances in automated essay scoring (AES) have traditionally relied on labeled essays, requiring tremendous cost and expertise for their acquisition. Recently, large language models (LLMs) have achieved great success in various tasks, but their potential is less explored in AES. In this paper, we show that our zero-shot prompting framework, Multi Trait Specialization (MTS), elicits LLMs' ample potential for essay scoring. In particular, we automatically decompose writing proficiency into distinct traits and generate scoring criteria for each trait. Then, an LLM is prompted to extract trait scores from several conversational rounds, each round scoring one of the traits based on the scoring criteria. Finally, we derive the overall score via trait averaging and min-max scaling. Experimental results on two benchmark datasets demonstrate that MTS consistently outperforms straightforward prompting (Vanilla) in average QWK across all LLMs and datasets, with maximum gains of 0.437 on TOEFL11 and 0.355 on ASAP. Additionally, with the help of MTS, the small-sized Llama2-13b-chat substantially outperforms ChatGPT, facilitating an effective deployment in real applications.
FPL+: Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

Adapting a medical image segmentation model to a new domain is important for improving its cross-domain transferability, and due to the expensive annotation process, Unsupervised Domain Adaptation (UDA) is appealing where only unlabeled images are needed for the adaptation. Existing UDA methods are mainly based on image or feature alignment with adversarial training for regularization, and they are limited by insufficient supervision in the target domain. In this paper, we propose an enhanced Filtered Pseudo Label (FPL+)-based UDA method for 3D medical image segmentation. It first uses cross-domain data augmentation to translate labeled images in the source domain to a dual-domain training set consisting of a pseudo source-domain set and a pseudo target-domain set. To leverage the dual-domain augmented images to train a pseudo label generator, domain-specific batch normalization layers are used to deal with the domain shift while learning the domain-invariant structure features, generating high-quality pseudo labels for target-domain images. We then combine labeled source-domain images and target-domain images with pseudo labels to train a final segmentor, where image-level weighting based on uncertainty estimation and pixel-level weighting based on dual-domain consensus are proposed to mitigate the adverse effect of noisy pseudo labels. Experiments on three public multi-modal datasets for Vestibular Schwannoma, brain tumor and whole heart segmentation show that our method surpassed ten state-of-the-art UDA methods, and it even achieved better results than fully supervised learning in the target domain in some cases.
On Linear Codes Whose Hermitian Hulls are MD
Venue: arXiv cs.IT 2024
Year: 2024

Hermitian hulls of linear codes are interesting for theoretical and practical reasons alike. In terms of recent application, linear codes whose hulls meet certain conditions have been utilized as ingredients to construct entanglement-assisted quantum error correcting codes. This family of quantum codes is often seen as a generalization of quantum stabilizer codes. Theoretically, compared with the Euclidean setup, the Hermitian case is much harder to deal with. Hermitian hulls of MDS linear codes with low dimensions have been explored, mostly from generalized Reed-Solomon codes. Characterizing Hermitian hulls which themselves are MDS appears to be more involved and has not been extensively studied. This paper introduces some tools to study linear codes whose Hermitian hulls are MDS. Using the tools, we then propose explicit constructions of such codes. We consider Hermitian hulls of both Reed-Solomon and non Reed-Solomon types of linear MDS codes. We demonstrate that, given the same Hermitian hull dimensions, the codes from our constructions have dimensions which are larger than those in the literature.
Weakly Supervised Deep Hyperspherical Quantization for Image Retrieval
Venue: arXiv cs.CV 2024
Year: 2024

Deep quantization methods have shown high efficiency on large-scale image retrieval. However, current models heavily rely on ground-truth information, hindering the application of quantization in label-hungry scenarios. A more realistic demand is to learn from inexhaustible uploaded images that are associated with informal tags provided by amateur users. Though such sketchy tags do not obviously reveal the labels, they actually contain useful semantic information for supervising deep quantization. To this end, we propose Weakly-Supervised Deep Hyperspherical Quantization (WSDHQ), which is the first work to learn deep quantization from weakly tagged images. Specifically, 1) we use word embeddings to represent the tags and enhance their semantic information based on a tag correlation graph. 2) To better preserve semantic information in quantization codes and reduce quantization error, we jointly learn semantics-preserving embeddings and supervised quantizer on hypersphere by employing a well-designed fusion layer and tailor-made loss functions. Extensive experiments show that WSDHQ can achieve state-of-art performance on weakly-supervised compact coding. Code is available at https://github.com/gimpong/AAAI21-WSDHQ.
Scalable and Efficient Hierarchical Visual Topological Mapping
Venue: arXiv cs.CV 2024
Year: 2024

Hierarchical topological representations can significantly reduce search times within mapping and localization algorithms. Although recent research has shown the potential for such approaches, limited consideration has been given to the suitability and comparative performance of different global feature representations within this context. In this work, we evaluate state-of-the-art hand-crafted and learned global descriptors using a hierarchical topological mapping technique on benchmark datasets and present results of a comprehensive evaluation of the impact of the global descriptor used. Although learned descriptors have been incorporated into place recognition methods to improve retrieval accuracy and enhance overall recall, the problem of scalability and efficiency when applied to longer trajectories has not been adequately addressed in a majority of research studies. Based on our empirical analysis of multiple runs, we identify that continuity and distinctiveness are crucial characteristics for an optimal global descriptor that enable efficient and scalable hierarchical mapping, and present a methodology for quantifying and contrasting these characteristics across different global descriptors. Our study demonstrates that the use of global descriptors based on an unsupervised learned Variational Autoencoder (VAE) excels in these characteristics and achieves significantly lower runtime. It runs on a consumer grade desktop, up to 2.3x faster than the second best global descriptor, NetVLAD, and up to 9.5x faster than the hand-crafted descriptor, PHOG, on the longest track evaluated (St Lucia, 17.6 km), without sacrificing overall recall performance.
The Hyperdrive Protocol: An Automated Market Maker for Fixed and Variable Rates
Venue: arXiv cs.GT 2024
Year: 2024

Hyperdrive is a protocol designed to facilitate the trading of fixed and variable rate assets. The protocol's unique pricing model consolidates liquidity into a single pool which addresses the challenges of fragmented liquidity across terms, eliminates the need for rollovers, and allows terms to be issued on demand. Its design meaningfully improves trading efficiency, liquidity provisioning, and user experience over existing fixed and variable rate protocol models.
Optimizing Privacy and Utility Tradeoffs for Group Interests Through Harmonization
Venue: arXiv cs.LG 2024
Year: 2024

We propose a novel problem formulation to address the privacy-utility tradeoff, specifically when dealing with two distinct user groups characterized by unique sets of private and utility attributes. Unlike previous studies that primarily focus on scenarios where all users share identical private and utility attributes and often rely on auxiliary datasets or manual annotations, we introduce a collaborative data-sharing mechanism between two user groups through a trusted third party. This third party uses adversarial privacy techniques with our proposed data-sharing mechanism to internally sanitize data for both groups and eliminates the need for manual annotation or auxiliary datasets. Our methodology ensures that private attributes cannot be accurately inferred while enabling highly accurate predictions of utility features. Importantly, even if analysts or adversaries possess auxiliary datasets containing raw data, they are unable to accurately deduce private features. Additionally, our data-sharing mechanism is compatible with various existing adversarially trained privacy techniques. We empirically demonstrate the effectiveness of our approach using synthetic and real-world datasets, showcasing its ability to balance the conflicting goals of privacy and utility.
Spatio-Temporal Attention and Gaussian Processes for Personalized Video Gaze Estimation
Venue: arXiv cs.CV 2024
Year: 2024

Gaze is an essential prompt for analyzing human behavior and attention. Recently, there has been an increasing interest in determining gaze direction from facial videos. However, video gaze estimation faces significant challenges, such as understanding the dynamic evolution of gaze in video sequences, dealing with static backgrounds, and adapting to variations in illumination. To address these challenges, we propose a simple and novel deep learning model designed to estimate gaze from videos, incorporating a specialized attention module. Our method employs a spatial attention mechanism that tracks spatial dynamics within videos. This technique enables accurate gaze direction prediction through a temporal sequence model, adeptly transforming spatial observations into temporal insights, thereby significantly improving gaze estimation accuracy. Additionally, our approach integrates Gaussian processes to include individual-specific traits, facilitating the personalization of our model with just a few labeled samples. Experimental results confirm the efficacy of the proposed approach, demonstrating its success in both within-dataset and cross-dataset settings. Specifically, our proposed approach achieves state-of-the-art performance on the Gaze360 dataset, improving by $2.5^\circ$ without personalization. Further, by personalizing the model with just three samples, we achieved an additional improvement of $0.8^\circ$. The code and pre-trained models are available at \url{https://github.com/jswati31/stage}.
Allowing humans to interactively guide machines where to look does not always improve human-AI team's classification accuracy
Venue: arXiv cs.CV 2024
Year: 2024

Via thousands of papers in Explainable AI (XAI), attention maps \cite{vaswani2017attention} and feature importance maps \cite{bansal2020sam} have been established as a common means for finding how important each input feature is to an AI's decisions. It is an interesting, unexplored question whether allowing users to edit the feature importance at test time would improve a human-AI team's accuracy on downstream tasks. In this paper, we address this question by leveraging CHM-Corr, a state-of-the-art, ante-hoc explainable classifier \cite{taesiri2022visual} that first predicts patch-wise correspondences between the input and training-set images, and then bases on them to make classification decisions. We build CHM-Corr++, an interactive interface for CHM-Corr, enabling users to edit the feature importance map provided by CHM-Corr and observe updated model decisions. Via CHM-Corr++, users can gain insights into if, when, and how the model changes its outputs, improving their understanding beyond static explanations. However, our study with 18 expert users who performed 1,400 decisions finds no statistical significance that our interactive approach improves user accuracy on CUB-200 bird image classification over static explanations. This challenges the hypothesis that interactivity can boost human-AI team accuracy and raises needs for future research. We open-source CHM-Corr++, an interactive tool for editing image classifier attention (see an interactive demo here: http://137.184.82.109:7080/). We release code and data on github: https://github.com/anguyen8/chm-corr-interactive.
Spatially Correlated RIS-Aided Secure Massive MIMO Under CSI and Hardware Imperfections
Venue: arXiv cs.IT 2024
Year: 2024

This paper investigates the integration of a reconfigurable intelligent surface (RIS) into a secure multiuser massive multiple-input multiple-output (MIMO) system in the presence of transceiver hardware impairments (HWI), imperfect channel state information (CSI), and spatially correlated channels. We first introduce a linear minimum-mean-square error estimation algorithm for the aggregate channel by considering the impact of transceiver HWI and RIS phase-shift errors. Then, we derive a lower bound for the achievable ergodic secrecy rate in the presence of a multi-antenna eavesdropper when artificial noise (AN) is employed at the base station (BS). In addition, the obtained expressions of the ergodic secrecy rate are further simplified in some noteworthy special cases to obtain valuable insights. To counteract the effects of HWI, we present a power allocation optimization strategy between the confidential signals and AN, which admits a fixed-point equation solution. Our analysis reveals that a non-zero ergodic secrecy rate is preserved if the total transmit power decreases no faster than $1/N$, where $N$ is the number of RIS elements. Moreover, the ergodic secrecy rate grows logarithmically with the number of BS antennas $M$ and approaches a certain limit in the asymptotic regime $N\rightarrow\infty$. Simulation results are provided to verify the derived analytical results. They reveal the impact of key design parameters on the secrecy rate. It is shown that, with the proposed power allocation strategy, the secrecy rate loss due to HWI can be counteracted by increasing the number of low-cost RIS elements.
Product Description and QA Assisted Self-Supervised Opinion Summarization
Venue: arXiv cs.CL 2024
Year: 2024

In e-commerce, opinion summarization is the process of summarizing the consensus opinions found in product reviews. However, the potential of additional sources such as product description and question-answers (QA) has been considered less often. Moreover, the absence of any supervised training data makes this task challenging. To address this, we propose a novel synthetic dataset creation (SDC) strategy that leverages information from reviews as well as additional sources for selecting one of the reviews as a pseudo-summary to enable supervised training. Our Multi-Encoder Decoder framework for Opinion Summarization (MEDOS) employs a separate encoder for each source, enabling effective selection of information while generating the summary. For evaluation, due to the unavailability of test sets with additional sources, we extend the Amazon, Oposum+, and Flipkart test sets and leverage ChatGPT to annotate summaries. Experiments across nine test sets demonstrate that the combination of our SDC approach and MEDOS model achieves on average a 14.5% improvement in ROUGE-1 F1 over the SOTA. Moreover, comparative analysis underlines the significance of incorporating additional sources for generating more informative summaries. Human evaluations further indicate that MEDOS scores relatively higher in coherence and fluency with 0.41 and 0.5 (-1 to 1) respectively, compared to existing models. To the best of our knowledge, we are the first to generate opinion summaries leveraging additional sources in a self-supervised setting.