OpenRaiser/Intern-Atlas huggingface dataset:
Unlocking the Potential of Large Language Models for Clinical Text Anonymization: A Comparative Study
Venue: arXiv cs.CL 2024
Year: 2024

Automated clinical text anonymization has the potential to unlock the widespread sharing of textual health data for secondary usage while assuring patient privacy and safety. Despite the proposal of many complex and theoretically successful anonymization solutions in literature, these techniques remain flawed. As such, clinical institutions are still reluctant to apply them for open access to their data. Recent advances in developing Large Language Models (LLMs) pose a promising opportunity to further the field, given their capability to perform various tasks. This paper proposes six new evaluation metrics tailored to the challenges of generative anonymization with LLMs. Moreover, we present a comparative study of LLM-based methods, testing them against two baseline techniques. Our results establish LLM-based models as a reliable alternative to common approaches, paving the way toward trustworthy anonymization of clinical text.
How Random is Random? Evaluating the Randomness and Humaness of LLMs' Coin Flips
Venue: arXiv cs.AI 2024
Year: 2024

One uniquely human trait is our inability to be random. We see and produce patterns where there should not be any and we do so in a predictable way. LLMs are supplied with human data and prone to human biases. In this work, we explore how LLMs approach randomness and where and how they fail through the lens of the well studied phenomena of generating binary random sequences. We find that GPT 4 and Llama 3 exhibit and exacerbate nearly every human bias we test in this context, but GPT 3.5 exhibits more random behavior. This dichotomy of randomness or humaness is proposed as a fundamental question of LLMs and that either behavior may be useful in different circumstances.
Submodular Maximization in Exactly $n$ Queries
Venue: arXiv cs.DS 2024
Year: 2024

In this work, we study the classical problem of maximizing a submodular function subject to a matroid constraint. We develop deterministic algorithms that are very parsimonious with respect to querying the submodular function, for both the case when the submodular function is monotone and the general submodular case. In particular, we present a 1/4 approximation algorithm for the monotone case that uses exactly one query per element, which gives the same total number of queries n as the number of queries required to compute the maximum singleton. For the general case, we present a constant factor approximation algorithm that requires 2 queries per element, which is the first algorithm for this problem with linear query complexity in the size of the ground set.
Non-Federated Multi-Task Split Learning for Heterogeneous Sources
Venue: arXiv cs.LG 2024
Year: 2024

With the development of edge networks and mobile computing, the need to serve heterogeneous data sources at the network edge requires the design of new distributed machine learning mechanisms. As a prevalent approach, Federated Learning (FL) employs parameter-sharing and gradient-averaging between clients and a server. Despite its many favorable qualities, such as convergence and data-privacy guarantees, it is well-known that classic FL fails to address the challenge of data heterogeneity and computation heterogeneity across clients. Most existing works that aim to accommodate such sources of heterogeneity stay within the FL operation paradigm, with modifications to overcome the negative effect of heterogeneous data. In this work, as an alternative paradigm, we propose a Multi-Task Split Learning (MTSL) framework, which combines the advantages of Split Learning (SL) with the flexibility of distributed network architectures. In contrast to the FL counterpart, in this paradigm, heterogeneity is not an obstacle to overcome, but a useful property to take advantage of. As such, this work aims to introduce a new architecture and methodology to perform multi-task learning for heterogeneous data sources efficiently, with the hope of encouraging the community to further explore the potential advantages we reveal. To support this promise, we first show through theoretical analysis that MTSL can achieve fast convergence by tuning the learning rate of the server and clients. Then, we compare the performance of MTSL with existing multi-task FL methods numerically on several image classification datasets to show that MTSL has advantages over FL in training speed, communication cost, and robustness to heterogeneous data.
Robustness of Online Proportional Response in Stochastic Online Fisher Markets: a Decentralized Approach
Venue: arXiv cs.GT 2024
Year: 2024

This study is focused on periodic Fisher markets where items with time-dependent and stochastic values are regularly replenished and buyers aim to maximize their utilities by spending budgets on these items. Traditional approaches of finding a market equilibrium in the single-period Fisher market rely on complete information about buyers' utility functions and budgets. However, it is impractical to consistently enforce buyers to disclose this private information in a periodic setting. We introduce a distributed auction algorithm, online proportional response, wherein buyers update bids solely based on the randomly fluctuating values of items in each period. The market then allocates items based on the bids provided by the buyers. Utilizing the known Shmyrev convex program that characterizes market equilibrium of a Fisher market, two performance metrics are proposed: the fairness regret is the cumulative difference in the objective value of a stochastic Shmyrev convex program between an online algorithm and an offline optimum, and the individual buyer's regret gauges the deviation in terms of utility for each buyer between the online algorithm and the offline optimum. Our algorithm attains a problem-dependent upper bound contingent on the number of items and buyers under stationary inputs in fairness regret. Additionally, we conduct analysis of regret under various non-stationary stochastic input models to demonstrate the algorithm's efficiency across diverse scenarios. The online proportional response algorithm addresses privacy concerns by allowing buyers to update bids without revealing sensitive information and ensures decentralized decision-making, fostering autonomy and potential improvements in buyer satisfaction. Furthermore, our algorithm is universally applicable to many worlds and shows the robust performance guarantees.
Contrastive Learning Via Equivariant Representation
Venue: arXiv cs.LG 2024
Year: 2024

Invariant Contrastive Learning (ICL) methods have achieved impressive performance across various domains. However, the absence of latent space representation for distortion (augmentation)-related information in the latent space makes ICL sub-optimal regarding training efficiency and robustness in downstream tasks. Recent studies suggest that introducing equivariance into Contrastive Learning (CL) can improve overall performance. In this paper, we revisit the roles of augmentation strategies and equivariance in improving CL's efficacy. We propose CLeVER (Contrastive Learning Via Equivariant Representation), a novel equivariant contrastive learning framework compatible with augmentation strategies of arbitrary complexity for various mainstream CL backbone models. Experimental results demonstrate that CLeVER effectively extracts and incorporates equivariant information from practical natural images, thereby improving the training efficiency and robustness of baseline models in downstream tasks and achieving state-of-the-art (SOTA) performance. Moreover, we find that leveraging equivariant information extracted by CLeVER simultaneously enhances rotational invariance and sensitivity across experimental tasks, and helps stabilize the framework when handling complex augmentations, particularly for models with small-scale backbones.
Impact Conflict Detection of IoT Services in Multi-resident Smart Homes
Venue: arXiv cs.DC 2024
Year: 2024

We propose a novel impact conflict detection framework for IoT services in multi-resident smart homes. The proposed impact assessment model is developed based on the integral of a signal deviation strategy. We mine the residents' previous service usage records to design a robust preference estimation model. We design an impact conflict detection approach using temporal proximity and preferential proximity techniques. Experimental results on real-world datasets demonstrate the effectiveness of the proposed approach.
From Seedling to Harvest: The GrowingSoy Dataset for Weed Detection in Soy Crops via Instance Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

Deep learning, particularly Convolutional Neural Networks (CNNs), has gained significant attention for its effectiveness in computer vision, especially in agricultural tasks. Recent advancements in instance segmentation have improved image classification accuracy. In this work, we introduce a comprehensive dataset for training neural networks to detect weeds and soy plants through instance segmentation. Our dataset covers various stages of soy growth, offering a chronological perspective on weed invasion's impact, with 1,000 meticulously annotated images. We also provide 6 state of the art models, trained in this dataset, that can understand and detect soy and weed in every stage of the plantation process. By using this dataset for weed and soy segmentation, we achieved a segmentation average precision of 79.1% and an average recall of 69.2% across all plant classes, with the YOLOv8X model. Moreover, the YOLOv8M model attained 78.7% mean average precision (mAp-50) in caruru weed segmentation, 69.7% in grassy weed segmentation, and 90.1% in soy plant segmentation.
E$^3$-Net: Efficient E(3)-Equivariant Normal Estimation Network
Venue: arXiv cs.CV 2024
Year: 2024

Point cloud normal estimation is a fundamental task in 3D geometry processing. While recent learning-based methods achieve notable advancements in normal prediction, they often overlook the critical aspect of equivariance. This results in inefficient learning of symmetric patterns. To address this issue, we propose E3-Net to achieve equivariance for normal estimation. We introduce an efficient random frame method, which significantly reduces the training resources required for this task to just 1/8 of previous work and improves the accuracy. Further, we design a Gaussian-weighted loss function and a receptive-aware inference strategy that effectively utilizes the local properties of point clouds. Our method achieves superior results on both synthetic and real-world datasets, and outperforms current state-of-the-art techniques by a substantial margin. We improve RMSE by 4% on the PCPNet dataset, 2.67% on the SceneNN dataset, and 2.44% on the FamousShape dataset.
Approaching 100% Confidence in Stream Summary through ReliableSketch
Venue: arXiv cs.DS 2024
Year: 2024

To approximate sums of values in key-value data streams, sketches are widely used in databases and networking systems. They offer high-confidence approximations for any given key while ensuring low time and space overhead. While existing sketches are proficient in estimating individual keys, they struggle to maintain this high confidence across all keys collectively, an objective that is critically important in both algorithm theory and its practical applications. We propose ReliableSketch, the first to control the error of all keys to less than $Λ$ with a small failure probability $Δ$, requiring only $O(1 + Δ\ln\ln(\frac{N}Λ))$ amortized time and $O(\frac{N}Λ + \ln(\frac{1}Δ))$ space. Furthermore, its simplicity makes it hardware-friendly, and we implement it on CPU servers, FPGAs, and programmable switches. Our experiments show that under the same small space, ReliableSketch not only keeps all keys' errors below $Λ$ but also achieves near-optimal throughput, outperforming competitors with thousands of uncontrolled estimations. We have made our source code publicly available.
CapeX: Category-Agnostic Pose Estimation from Textual Point Explanation
Venue: arXiv cs.CV 2024
Year: 2024

Conventional 2D pose estimation models are constrained by their design to specific object categories. This limits their applicability to predefined objects. To overcome these limitations, category-agnostic pose estimation (CAPE) emerged as a solution. CAPE aims to facilitate keypoint localization for diverse object categories using a unified model, which can generalize from minimal annotated support images. Recent CAPE works have produced object poses based on arbitrary keypoint definitions annotated on a user-provided support image. Our work departs from conventional CAPE methods, which require a support image, by adopting a text-based approach instead of the support image. Specifically, we use a pose-graph, where nodes represent keypoints that are described with text. This representation takes advantage of the abstraction of text descriptions and the structure imposed by the graph. Our approach effectively breaks symmetry, preserves structure, and improves occlusion handling. We validate our novel approach using the MP-100 benchmark, a comprehensive dataset spanning over 100 categories and 18,000 images. Under a 1-shot setting, our solution achieves a notable performance boost of 1.07\%, establishing a new state-of-the-art for CAPE. Additionally, we enrich the dataset by providing text description annotations, further enhancing its utility for future research.
MI 2 MI: Training Dyad with Collaborative Brain-Computer Interface and Cooperative Motor Imagery Tasks for Better BCI Performance
Venue: arXiv cs.HC 2024
Year: 2024

Collaborative brain-computer interface (cBCI) that conduct motor imagery (MI) among multiple users has the potential not only to improve overall BCI performance by integrating information from multiple users, but also to leverage individuals' performance in decision-making or control. However, existed research mostly focused on the brain signals changes through a single user, not noticing the possible interaction between users during the collaboration. In this work, we utilized cBCI and designed a cooperative four-classes MI task to train the dyad. A humanoid robot would stimulate the dyad to conduct both left/right hand and tongue/foot MI. Single user was asked to conduct single MI task before and after the cooperative MI task. The experiment results showed that our training could activate better performance (e.g., high quality of EEG /MI classification accuracy) for the single user than single MI task, and the single user also obtained better single MI performance after cooperative MI training.
Optimizing Age of Information in Random Access Networks: A Second-Order Approach for Active/Passive Users
Venue: arXiv cs.NI 2024
Year: 2024

In this paper, we study the moments of the Age of Information (AoI) for both active and passive users in a random access network. In this network, active users broadcast sensing data, while passive users detect in-band radio activities from out-of-network devices, such as jammers. Collisions occur when multiple active users transmit simultaneously. Passive users can detect radio activities only when no active user transmits. Each active user's transmission behavior follows a Markov process. We aim to minimize the weighted sum of any moments of AoI for both user types. To achieve this, we employ a second-order analysis of system behavior. Specifically, we characterize an active user's transmission Markov process using its mean and temporal variance. We show that any moment of the AoI can be approximated by a function of these two parameters. This insight enables us to analyze and optimize the transmission Markov process for active users. We apply this strategy to two different random access models. Simulation results show that policies derived from this strategy outperform other baseline policies.
On the use of first and second derivative approximations for biometric online signature recognition
Venue: arXiv cs.CV 2024
Year: 2024

This paper investigates the impact of different approximation methods in feature extraction for pattern recognition applications, specifically focused on delta and delta-delta parameters. Using MCYT330 online signature data-base, our experiments show that 11-point approximation outperforms 1-point approximation, resulting in a 1.4% improvement in identification rate, 36.8% reduction in random forgeries and 2.4% reduction in skilled forgeries
Cost-Effectiveness Analysis and Design of Cost-Efficient Cell-Free Massive MIMO Systems
Venue: arXiv cs.IT 2024
Year: 2024

Cell-free massive multi-input multi-output (MIMO) has recently attracted much attention, attributed to its potential to deliver uniform service quality. However, the adoption of a cell-free architecture raises concerns about the high implementation costs associated with deploying numerous distributed access points (APs) and the need for fronthaul network installation. To ensure the sustainability of next-generation wireless networks, it is crucial to improve cost-effectiveness, alongside achieving high performance. To address this, we conduct a cost analysis of cell-free massive MIMO and build a unified model with varying numbers of antennas per AP. Our objective is to explore whether employing multi-antenna APs could reduce system costs while maintaining performance. The analysis and evaluation result in the identification of a cost-effective design for cell-free massive MIMO, providing valuable insights for practical implementation.
Memory-guided Network with Uncertainty-based Feature Augmentation for Few-shot Semantic Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

The performance of supervised semantic segmentation methods highly relies on the availability of large-scale training data. To alleviate this dependence, few-shot semantic segmentation (FSS) is introduced to leverage the model trained on base classes with sufficient data into the segmentation of novel classes with few data. FSS methods face the challenge of model generalization on novel classes due to the distribution shift between base and novel classes. To overcome this issue, we propose a class-shared memory (CSM) module consisting of a set of learnable memory vectors. These memory vectors learn elemental object patterns from base classes during training whilst re-encoding query features during both training and inference, thereby improving the distribution alignment between base and novel classes. Furthermore, to cope with the performance degradation resulting from the intra-class variance across images, we introduce an uncertainty-based feature augmentation (UFA) module to produce diverse query features during training for improving the model's robustness. We integrate CSM and UFA into representative FSS works, with experimental results on the widely-used PASCAL-5$^i$ and COCO-20$^i$ datasets demonstrating the superior performance of ours over state of the art.
An Image Segmentation Model with Transformed Total Variation
Venue: arXiv cs.CV 2024
Year: 2024

Based on transformed $\ell_1$ regularization, transformed total variation (TTV) has robust image recovery that is competitive with other nonconvex total variation (TV) regularizers, such as TV$^p$, $0<p<1$. Inspired by its performance, we propose a TTV-regularized Mumford--Shah model with fuzzy membership function for image segmentation. To solve it, we design an alternating direction method of multipliers (ADMM) algorithm that utilizes the transformed $\ell_1$ proximal operator. Numerical experiments demonstrate that using TTV is more effective than classical TV and other nonconvex TV variants in image segmentation.
A Blueprint Architecture of Compound AI Systems for Enterprise
Venue: arXiv cs.DB 2024
Year: 2024

Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goal, there is a notable shift to building compound AI systems, wherein LLMs are integrated into an expansive software infrastructure with many components like models, retrievers, databases and tools. In this paper, we introduce a blueprint architecture for compound AI systems to operate in enterprise settings cost-effectively and feasibly. Our proposed architecture aims for seamless integration with existing compute and data infrastructure, with ``stream'' serving as the key orchestration concept to coordinate data and instructions among agents and other components. Task and data planners, respectively, break down, map, and optimize tasks and data to available agents and data sources defined in respective registries, given production constraints such as accuracy and latency.
Robust Visual Tracking via Iterative Gradient Descent and Threshold Selection
Venue: arXiv cs.CV 2024
Year: 2024

Visual tracking fundamentally involves regressing the state of the target in each frame of a video. Despite significant progress, existing regression-based trackers still tend to experience failures and inaccuracies. To enhance the precision of target estimation, this paper proposes a tracking technique based on robust regression. Firstly, we introduce a novel robust linear regression estimator, which achieves favorable performance when the error vector follows i.i.d Gaussian-Laplacian distribution. Secondly, we design an iterative process to quickly solve the problem of outliers. In fact, the coefficients are obtained by Iterative Gradient Descent and Threshold Selection algorithm (IGDTS). In addition, we expend IGDTS to a generative tracker, and apply IGDTS-distance to measure the deviation between the sample and the model. Finally, we propose an update scheme to capture the appearance changes of the tracked object and ensure that the model is updated correctly. Experimental results on several challenging image sequences show that the proposed tracker outperformance existing trackers.
Robust Fair Clustering with Group Membership Uncertainty Sets
Venue: arXiv cs.LG 2024
Year: 2024

We study the canonical fair clustering problem where each cluster is constrained to have close to population-level representation of each group. Despite significant attention, the salient issue of having incomplete knowledge about the group membership of each point has been superficially addressed. In this paper, we consider a setting where the assigned group memberships are noisy. We introduce a simple noise model that requires a small number of parameters to be given by the decision maker. We then present an algorithm for fair clustering with provable \emph{robustness} guarantees. Our framework enables the decision maker to trade off between the robustness and the clustering quality. Unlike previous work, our algorithms are backed by worst-case theoretical guarantees. Finally, we empirically verify the performance of our algorithm on real world datasets and show its superior performance over existing baselines.