OpenRaiser/Intern-Atlas huggingface dataset:
A Novel Kinesthetic Haptic Feedback Device Driven by Soft Electrohydraulic Actuators
Venue: arXiv cs.RO 2024
Year: 2024

Developing kinesthetic haptic devices with advanced haptic rendering capabilities is challenging due to the limitations on driving mechanisms. In this study, we introduce a novel soft electrohydraulic actuator and develop a kinesthetic haptic device utilizing it as the driving unit. We established a mathematical model and conducted testing experiments to demonstrate the device's ability to stably output controllable feedback force. Our experiments also demonstrates that this device exhibits fast response characteristics. By utilizing the easily controllable nature of the soft electrohydraulic actuator, we were able to achieve high-resolution controllable feedback force output. Furthermore, by modulating the waveform of the driving high voltage, the device acquired the capability to render variable frequency haptic vibration without adding any extra vibration actuator. Using this kinesthetic haptic device, we built a teleoperated robotic system, showcasing the device's potential application as a haptic force feedback system in the field of robotics.
GeneQuery: A General QA-based Framework for Spatial Gene Expression Predictions from Histology Images
Venue: arXiv cs.CV 2024
Year: 2024

Gene expression profiling provides profound insights into molecular mechanisms, but its time-consuming and costly nature often presents significant challenges. In contrast, whole-slide hematoxylin and eosin (H&E) stained histological images are readily accessible and allow for detailed examinations of tissue structure and composition at the microscopic level. Recent advancements have utilized these histological images to predict spatially resolved gene expression profiles. However, state-of-the-art works treat gene expression prediction as a multi-output regression problem, where each gene is learned independently with its own weights, failing to capture the shared dependencies and co-expression patterns between genes. Besides, existing works can only predict gene expression values for genes seen during training, limiting their ability to generalize to new, unseen genes. To address the above limitations, this paper presents GeneQuery, which aims to solve this gene expression prediction task in a question-answering (QA) manner for better generality and flexibility. Specifically, GeneQuery takes gene-related texts as queries and whole-slide images as contexts and then predicts the queried gene expression values. With such a transformation, GeneQuery can implicitly estimate the gene distribution by introducing the gene random variable. Besides, the proposed GeneQuery consists of two architecture implementations, i.e., spot-aware GeneQuery for capturing patterns between images and gene-aware GeneQuery for capturing patterns between genes. Comprehensive experiments on spatial transcriptomics datasets show that the proposed GeneQuery outperforms existing state-of-the-art methods on known and unseen genes. More results also demonstrate that GeneQuery can potentially analyze the tissue structure.
Exploring the Impact of Rewards on Developers' Proactive AI Accountability Behavior
Venue: arXiv cs.CY 2024
Year: 2024

The rapid integration of Artificial Intelligence (AI)-based systems offers benefits for various domains of the economy and society but simultaneously raises concerns due to emerging scandals. These scandals have led to the increasing importance of AI accountability to ensure that actors provide justification and victims receive compensation. However, AI accountability has a negative connotation due to its emphasis on penalizing sanctions, resulting in reactive approaches to emerging concerns. To counteract the prevalent negative view and offer a proactive approach to facilitate the AI accountability behavior of developers, we explore rewards as an alternative mechanism to sanctions. We develop a theoretical model grounded in Self-Determination Theory to uncover the potential impact of rewards and sanctions on AI developers. We further identify typical sanctions and bug bounties as potential reward mechanisms by surveying related research from various domains, including cybersecurity.
FastSwitch: Optimizing Context Switching Efficiency in Fairness-aware Large Language Model Serving
Venue: arXiv cs.LG 2024
Year: 2024

Serving numerous users and requests concurrently requires good fairness in Large Language Models (LLMs) serving system. This ensures that, at the same cost, the system can meet the Service Level Objectives (SLOs) of more users , such as time to first token (TTFT) and time between tokens (TBT), rather than allowing a few users to experience performance far exceeding the SLOs. To achieve better fairness, the preemption-based scheduling policy dynamically adjusts the priority of each request to maintain balance during runtime. However, existing systems tend to overly prioritize throughput, overlooking the overhead caused by preemption-induced context switching, which is crucial for maintaining fairness through priority adjustments. In this work, we identify three main challenges that result in this overhead. 1) Inadequate I/O utilization. 2) GPU idleness. 3) Unnecessary I/O transmission during multi-turn conversations. Our key insight is that the block-based KV cache memory policy in existing systems, while achieving near-zero memory waste, leads to discontinuity and insufficient granularity in the KV cache memory. To respond, we introduce FastSwitch, a fairness-aware serving system that not only aligns with existing KV cache memory allocation policy but also mitigates context switching overhead. Our evaluation shows that FastSwitch outperforms the state-of-the-art LLM serving system vLLM with speedups of 1.4-11.2x across different tail TTFT and TBT.
Continuous Autoregressive Models with Noise Augmentation Avoid Error Accumulation
Venue: arXiv cs.LG 2024
Year: 2024

Autoregressive models are typically applied to sequences of discrete tokens, but recent research indicates that generating sequences of continuous embeddings in an autoregressive manner is also feasible. However, such Continuous Autoregressive Models (CAMs) can suffer from a decline in generation quality over extended sequences due to error accumulation during inference. We introduce a novel method to address this issue by injecting random noise into the input embeddings during training. This procedure makes the model robust against varying error levels at inference. We further reduce error accumulation through an inference procedure that introduces low-level noise. Experiments on musical audio generation show that CAM substantially outperforms existing autoregressive and non-autoregressive approaches while preserving audio quality over extended sequences. This work paves the way for generating continuous embeddings in a purely autoregressive setting, opening new possibilities for real-time and interactive generative applications.
Multiple Choice Learning for Efficient Speech Separation with Many Speakers
Venue: arXiv cs.SD 2024
Year: 2024

Training speech separation models in the supervised setting raises a permutation problem: finding the best assignation between the model predictions and the ground truth separated signals. This inherently ambiguous task is customarily solved using Permutation Invariant Training (PIT). In this article, we instead consider using the Multiple Choice Learning (MCL) framework, which was originally introduced to tackle ambiguous tasks. We demonstrate experimentally on the popular WSJ0-mix and LibriMix benchmarks that MCL matches the performances of PIT, while being computationally advantageous. This opens the door to a promising research direction, as MCL can be naturally extended to handle a variable number of speakers, or to tackle speech separation in the unsupervised setting.
Personalised Serious Games and Gamification in Healthcare: Survey and Future Research Directions
Venue: arXiv cs.HC 2024
Year: 2024

Serious games and gamification (SGG) in eHealth have positive health impacts, but a personalized approach is needed due to diverse user contexts. This introduces challenges in achieving personalization in SGG. A literature search on Web of Science and PubMed identified 31 articles: 22 on serious games and 9 on gamification. These strategies are most applied in behavior change and rehabilitation, with machine learning and AI showing promise for personalization. Reusability of personalisation algorithms and domain knowledge is underemphasized, reported in only 10 articles. Future research should standardize personalized SGG development, focusing on component reuse to streamline design and enhance evaluation.
PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Image
Venue: arXiv cs.CV 2024
Year: 2024

We present PhyCAGE, the first approach for physically plausible compositional 3D asset generation from a single image. Given an input image, we first generate consistent multi-view images for components of the assets. These images are then fitted with 3D Gaussian Splatting representations. To ensure that the Gaussians representing objects are physically compatible with each other, we introduce a Physical Simulation-Enhanced Score Distillation Sampling (PSE-SDS) technique to further optimize the positions of the Gaussians. It is achieved by setting the gradient of the SDS loss as the initial velocity of the physical simulation, allowing the simulator to act as a physics-guided optimizer that progressively corrects the Gaussians' positions to a physically compatible state. Experimental results demonstrate that the proposed method can generate physically plausible compositional 3D assets given a single image.
FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion
Venue: arXiv cs.CV 2024
Year: 2024

Diffusion models are proficient at generating high-quality images. They are however effective only when operating at the resolution used during training. Inference at a scaled resolution leads to repetitive patterns and structural distortions. Retraining at higher resolutions quickly becomes prohibitive. Thus, methods enabling pre-existing diffusion models to operate at flexible test-time resolutions are highly desirable. Previous works suffer from frequent artifacts and often introduce large latency overheads. We propose two simple modules that combine to solve these issues. We introduce a Frequency Modulation (FM) module that leverages the Fourier domain to improve the global structure consistency, and an Attention Modulation (AM) module which improves the consistency of local texture patterns, a problem largely ignored in prior works. Our method, coined Fam diffusion, can seamlessly integrate into any latent diffusion model and requires no additional training. Extensive qualitative results highlight the effectiveness of our method in addressing structural and local artifacts, while quantitative results show state-of-the-art performance. Also, our method avoids redundant inference tricks for improved consistency such as patch-based or progressive generation, leading to negligible latency overheads.
Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuning
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) have demonstrated remarkable multilingual capabilities, yet challenges persist in adapting these models for low-resource languages. In this study, we investigate the effects of Low-Rank Adaptation (LoRA) Parameter-Efficient Fine-Tuning (PEFT) on multilingual Gemma models for Marathi, a language with limited resources. Using a translated Alpaca dataset with 52,000 instruction-response pairs, our findings reveal that while evaluation metrics often show a performance decline post-fine-tuning, manual assessments frequently suggest that the fine-tuned models outperform their original counterparts. The observations indicate improvements in target language generation capabilities but a reduction in reasoning abilities following language adaptation. These results underscore the need for improved evaluation methodologies and the creation of high-quality native datasets to accurately assess language-specific model performance in low-resource settings.
Exploring Depth Information for Detecting Manipulated Face Videos
Venue: arXiv cs.CV 2024
Year: 2024

Face manipulation detection has been receiving a lot of attention for the reliability and security of the face images/videos. Recent studies focus on using auxiliary information or prior knowledge to capture robust manipulation traces, which are shown to be promising. As one of the important face features, the face depth map, which has shown to be effective in other areas such as face recognition or face detection, is unfortunately paid little attention to in literature for face manipulation detection. In this paper, we explore the possibility of incorporating the face depth map as auxiliary information for robust face manipulation detection. To this end, we first propose a Face Depth Map Transformer (FDMT) to estimate the face depth map patch by patch from an RGB face image, which is able to capture the local depth anomaly created due to manipulation. The estimated face depth map is then considered as auxiliary information to be integrated with the backbone features using a Multi-head Depth Attention (MDA) mechanism that is newly designed. We also propose an RGB-Depth Inconsistency Attention (RDIA) module to effectively capture the inter-frame inconsistency for multi-frame input. Various experiments demonstrate the advantage of our proposed method for face manipulation detection.
Pruning Deep Convolutional Neural Network Using Conditional Mutual Information
Venue: arXiv cs.LG 2024
Year: 2024

Convolutional Neural Networks (CNNs) achieve high performance in image classification tasks but are challenging to deploy on resource-limited hardware due to their large model sizes. To address this issue, we leverage Mutual Information, a metric that provides valuable insights into how deep learning models retain and process information through measuring the shared information between input features or output labels and network layers. In this study, we propose a structured filter-pruning approach for CNNs that identifies and selectively retains the most informative features in each layer. Our approach successively evaluates each layer by ranking the importance of its feature maps based on Conditional Mutual Information (CMI) values, computed using a matrix-based Renyi α-order entropy numerical method. We propose several formulations of CMI to capture correlation among features across different layers. We then develop various strategies to determine the cutoff point for CMI values to prune unimportant features. This approach allows parallel pruning in both forward and backward directions and significantly reduces model size while preserving accuracy. Tested on the VGG16 architecture with the CIFAR-10 dataset, the proposed method reduces the number of filters by more than a third, with only a 0.32% drop in test accuracy.
CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
Venue: arXiv cs.CV 2024
Year: 2024

We present CAT4D, a method for creating 4D (dynamic 3D) scenes from monocular video. CAT4D leverages a multi-view video diffusion model trained on a diverse combination of datasets to enable novel view synthesis at any specified camera poses and timestamps. Combined with a novel sampling approach, this model can transform a single monocular video into a multi-view video, enabling robust 4D reconstruction via optimization of a deformable 3D Gaussian representation. We demonstrate competitive performance on novel view synthesis and dynamic scene reconstruction benchmarks, and highlight the creative capabilities for 4D scene generation from real or generated videos. See our project page for results and interactive demos: https://cat-4d.github.io/.
Counterfactual Learning-Driven Representation Disentanglement for Search-Enhanced Recommendation
Venue: arXiv cs.IR 2024
Year: 2024

For recommender systems in internet platforms, search activities provide additional insights into user interest through query-click interactions with items, and are thus widely used for enhancing personalized recommendation. However, these interacted items not only have transferable features matching users' interest helpful for the recommendation domain, but also have features related to users' unique intents in the search domain. Such domain gap of item features is neglected by most current search-enhanced recommendation methods. They directly incorporate these search behaviors into recommendation, and thus introduce partial negative transfer. To address this, we propose a Counterfactual learning-driven representation disentanglement framework for search-enhanced recommendation, based on the common belief that a user would click an item under a query not solely because of the item-query match but also due to the item's query-independent general features (e.g., color or style) that interest the user. These general features exclude the reflection of search-specific intents contained in queries, ensuring a pure match to users' underlying interest to complement recommendation. According to counterfactual thinking, how would user preferences and query match change for items if we removed their query-related features in search, we leverage search queries to construct counterfactual signals to disentangle item representations, isolating only query-independent general features. These representations subsequently enable feature augmentation and data augmentation for the recommendation scenario. Comprehensive experiments on real datasets demonstrate ClardRec is effective in both collaborative filtering and sequential recommendation scenarios.
Geospatial sustainability assessment of universal Fiber-To-The-Neighborhood (FTTnb) broadband infrastructure strategies for Sub-Saharan Africa
Venue: arXiv cs.NI 2024
Year: 2024

Broadband Internet access is an important way to help achieve the Sustainable Development Goals. Currently, fixed fiber infrastructure is essential for providing universal broadband, but has received relatively little research attention in low-income countries compared to other more cost-efficient wireless technologies. Yet, pushing out fiber broadband network to local areas is essential, even if the final access network is still wireless. Here, we design least-cost Fiber-To-The-Neighborhood (FTTnb) architectures using two spatial optimization Steiner Tree algorithms to jointly determine investment costs, environmental emissions, and Social Carbon Costs. We find that the average annualized per user emissions in low population density areas (<9 people per km2) range from 0.18-9.6 kg CO2 eq./user, compared to 0.015-0.12 kg CO2 eq./user for high population density areas (>958 people per km2). Moreover, Annualized Total Cost of Ownership per user is 12-90 times lower in high population density areas (>958 people per km2) compared to sparsely populated regions (<9 people per km2). Thus, 48% (about 550 million) of the total Sub-Saharan African population live in areas where FTTnb is viable within the next ten years.
On the Effectiveness of Incremental Training of Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Training large language models is a computationally intensive process that often requires substantial resources to achieve state-of-the-art results. Incremental layer-wise training has been proposed as a potential strategy to optimize the training process by progressively introducing layers, with the expectation that this approach would lead to faster convergence and more efficient use of computational resources. In this paper, we investigate the effectiveness of incremental training for LLMs, dividing the training process into multiple stages where layers are added progressively. Our experimental results indicate that while the incremental approach initially demonstrates some computational efficiency, it ultimately requires greater overall computational costs to reach comparable performance to traditional full-scale training. Although the incremental training process can eventually close the performance gap with the baseline, it does so only after significantly extended continual training. These findings suggest that incremental layer-wise training may not be a viable alternative for training large language models, highlighting its limitations and providing valuable insights into the inefficiencies of this approach.
Stratified Non-Negative Tensor Factorization
Venue: arXiv cs.LG 2024
Year: 2024

Non-negative matrix factorization (NMF) and non-negative tensor factorization (NTF) decompose non-negative high-dimensional data into non-negative low-rank components. NMF and NTF methods are popular for their intrinsic interpretability and effectiveness on large-scale data. Recent work developed Stratified-NMF, which applies NMF to regimes where data may come from different sources (strata) with different underlying distributions, and seeks to recover both strata-dependent information and global topics shared across strata. Applying Stratified-NMF to multi-modal data requires flattening across modes, and therefore loses geometric structure contained implicitly within the tensor. To address this problem, we extend Stratified-NMF to the tensor setting by developing a multiplicative update rule and demonstrating the method on text and image data. We find that Stratified-NTF can identify interpretable topics with lower memory requirements than Stratified-NMF. We also introduce a regularized version of the method and demonstrate its effects on image data.
NewsEdits 2.0: Learning the Intentions Behind Updating News
Venue: arXiv cs.CL 2024
Year: 2024

As events progress, news articles often update with new information: if we are not cautious, we risk propagating outdated facts. In this work, we hypothesize that linguistic features indicate factual fluidity, and that we can predict which facts in a news article will update using solely the text of a news article (i.e. not external resources like search engines). We test this hypothesis, first, by isolating fact-updates in large news revisions corpora. News articles may update for many reasons (e.g. factual, stylistic, narrative). We introduce the NewsEdits 2.0 taxonomy, an edit-intentions schema that separates fact updates from stylistic and narrative updates in news writing. We annotate over 9,200 pairs of sentence revisions and train high-scoring ensemble models to apply this schema. Then, taking a large dataset of silver-labeled pairs, we show that we can predict when facts will update in older article drafts with high precision. Finally, to demonstrate the usefulness of these findings, we construct a language model question asking (LLM-QA) abstention task. We wish the LLM to abstain from answering questions when information is likely to become outdated. Using our predictions, we show, LLM absention reaches near oracle levels of accuracy.
Perspectives on 6G Architectures
Venue: arXiv cs.NI 2024
Year: 2024

Mobile communications have been undergoing a generational change every ten years. While 5G network deployments are maturing, significant efforts are being made to standardize 6G, which is expected to be commercially introduced by 2030. This paper provides unique perspectives on the 6G network (radio and core) architecture(s) from the anticipated 6G use cases to meet the necessary performance requirements. To cater for the key 6G use cases, the 6G architecture must integrate different network-level functions in a multiplicity of virtual cloud environments, leveraging the advancements of distributed processing, artificial intelligence, and securely integrating different sub-networks e.g., terrestrial, and non-terrestrial networks into the overall 6G network. This paper characterizes the impact of 6G architectures from a deployment perspective with backwards compatibility in mind.