OpenRaiser/Intern-Atlas huggingface dataset:
Improving Knowledge Distillation in Transfer Learning with Layer-wise Learning Rates
Venue: arXiv cs.LG 2024
Year: 2024

Transfer learning methods start performing poorly when the complexity of the learning task is increased. Most of these methods calculate the cumulative differences of all the matched features and then use them to back-propagate that loss through all the layers. Contrary to these methods, in this work, we propose a novel layer-wise learning scheme that adjusts learning parameters per layer as a function of the differences in the Jacobian/Attention/Hessian of the output activations w.r.t. the network parameters. We applied this novel scheme for attention map-based and derivative-based (first and second order) transfer learning methods. We received improved learning performance and stability against a wide range of datasets. From extensive experimental evaluation, we observed that the performance boost achieved by our method becomes more significant with the increasing difficulty of the learning task.
Privacy or Transparency? Negotiated Smartphone Access as a Signifier of Trust in Romantic Relationships
Venue: arXiv cs.CR 2024
Year: 2024

In this work, we analyze two large-scale surveys to examine how individuals think about sharing smartphone access with romantic partners as a function of trust in relationships. We find that the majority of couples have access to each others' devices, but may have explicit or implicit boundaries on how this access is to be used. Investigating these boundaries and related social norms, we find that there is little consensus about the level of smartphone access (i.e., transparency), or lack thereof (i.e., privacy) that is desirable in romantic contexts. However, there is broad agreement that the level of access should be mutual and consensual. Most individuals understand trust to be the basis of their decisions about transparency and privacy. Furthermore, we find individuals have crossed these boundaries, violating their partners' privacy and betraying their trust. We examine how, when, why, and by whom these betrayals occur. We consider the ramifications of these boundary violations in the case of intimate partner violence. Finally, we provide recommendations for design changes to enable technological enforcement of boundaries currently enforced by trust, bringing access control in line with users' sharing preferences.
Aortic root landmark localization with optimal transport loss for heatmap regression
Venue: arXiv cs.CV 2024
Year: 2024

Anatomical landmark localization is gaining attention to ease the burden on physicians. Focusing on aortic root landmark localization, the three hinge points of the aortic valve can reduce the burden by automatically determining the valve size required for transcatheter aortic valve implantation surgery. Existing methods for landmark prediction of the aortic root mainly use time-consuming two-step estimation methods. We propose a highly accurate one-step landmark localization method from even coarse images. The proposed method uses an optimal transport loss to break the trade-off between prediction precision and learning stability in conventional heatmap regression methods. We apply the proposed method to the 3D CT image dataset collected at Sendai Kousei Hospital and show that it significantly improves the estimation error over existing methods and other loss functions. Our code is available on GitHub.
Multi-agent Off-policy Actor-Critic Reinforcement Learning for Partially Observable Environments
Venue: arXiv cs.LG 2024
Year: 2024

This study proposes the use of a social learning method to estimate a global state within a multi-agent off-policy actor-critic algorithm for reinforcement learning (RL) operating in a partially observable environment. We assume that the network of agents operates in a fully-decentralized manner, possessing the capability to exchange variables with their immediate neighbors. The proposed design methodology is supported by an analysis demonstrating that the difference between final outcomes, obtained when the global state is fully observed versus estimated through the social learning method, is $\varepsilon$-bounded when an appropriate number of iterations of social learning updates are implemented. Unlike many existing dec-POMDP-based RL approaches, the proposed algorithm is suitable for model-free multi-agent reinforcement learning as it does not require knowledge of a transition model. Furthermore, experimental results illustrate the efficacy of the algorithm and demonstrate its superiority over the current state-of-the-art methods.
Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games
Venue: arXiv cs.SE 2024
Year: 2024

As games challenge traditional automated white-box test generators, the Neatest approach generates test suites consisting of neural networks that exercise the source code by playing the games. Neatest generates these neural networks using an evolutionary algorithm that is guided by an objective function targeting individual source code statements. This approach works well if the objective function provides sufficient guidance, but deceiving or complex fitness landscapes may inhibit the search. In this paper, we investigate whether the issue of challenging fitness landscapes can be addressed by promoting novel behaviours during the search. Our case study on two Scratch games demonstrates that rewarding novel behaviours is a promising approach for overcoming challenging fitness landscapes, thus enabling future research on how to adapt the search algorithms to best use this information.
The Solution for the sequential task continual learning track of the 2nd Greater Bay Area International Algorithm Competition
Venue: arXiv cs.LG 2024
Year: 2024

This paper presents a data-free, parameter-isolation-based continual learning algorithm we developed for the sequential task continual learning track of the 2nd Greater Bay Area International Algorithm Competition. The method learns an independent parameter subspace for each task within the network's convolutional and linear layers and freezes the batch normalization layers after the first task. Specifically, for domain incremental setting where all domains share a classification head, we freeze the shared classification head after first task is completed, effectively solving the issue of catastrophic forgetting. Additionally, facing the challenge of domain incremental settings without providing a task identity, we designed an inference task identity strategy, selecting an appropriate mask matrix for each sample. Furthermore, we introduced a gradient supplementation strategy to enhance the importance of unselected parameters for the current task, facilitating learning for new tasks. We also implemented an adaptive importance scoring strategy that dynamically adjusts the amount of parameters to optimize single-task performance while reducing parameter usage. Moreover, considering the limitations of storage space and inference time, we designed a mask matrix compression strategy to save storage space and improve the speed of encryption and decryption of the mask matrix. Our approach does not require expanding the core network or using external auxiliary networks or data, and performs well under both task incremental and domain incremental settings. This solution ultimately won a second-place prize in the competition.
Achieving Tool Calling Functionality in LLMs Using Only Prompt Engineering Without Fine-Tuning
Venue: arXiv cs.SE 2024
Year: 2024

Currently, the vast majority of locally deployed open-source large language models (LLMs) and some commercial model interfaces do not support stable tool calling functionality. The existing solution involves fine-tuning LLMs, which results in significant time and computational resource consumption. This paper proposes a method that enables LLMs to achieve stable tool calling capabilities using only prompt engineering and some ingenious code design. We conducted experiments on multiple LLMs that lack tool calling capabilities across various tool calling tasks, achieving a success rate of 100%.
DCitizens Roles Unveiled: SIG Navigating Identities in Digital Civics and the Spectrum of Societal Impact
Venue: arXiv cs.HC 2024
Year: 2024

The DCitizens SIG aims to navigate ethical dimensions in forthcoming Digital Civics projects, ensuring enduring benefits and community resilience. Additionally, it seeks to shape the future landscape of digital civics for ethical and sustainable interventions. As we dive into these interactive processes, a challenge arises of discerning authentic intentions and validating perspectives. This exploration extends to evaluating the sustainability of future interactions and scrutinising biases impacting engaged communities. The commitment is to ensure future outcomes align with genuine community needs and address the ethical imperative of a considerate departure strategy. This dialogue encourages future researchers and practitioners to integrate ethical considerations and community-centric principles, fostering a more sustainable and responsible approach to technology-driven interventions in future urban regeneration and beyond.
Enhance the Robustness of Text-Centric Multimodal Alignments
Venue: arXiv cs.CL 2024
Year: 2024

Converting different modalities into general text, serving as input prompts for large language models (LLMs), is a common method to align multimodal models when there is limited pairwise data. This text-centric approach leverages the unique properties of text as a modality space, transforming diverse inputs into a unified textual representation. This enables downstream models to effectively interpret various modal inputs. This study assesses the quality and robustness of multimodal representations in the presence of missing entries, noise, or absent modalities, revealing that current text-centric alignment methods compromise downstream robustness. To address this issue, we propose a new text-centric approach that achieves superior robustness compared to previous methods across various modalities in different settings. Our findings highlight the potential of this approach to enhance the robustness and adaptability of multimodal representations, offering a promising solution for dynamic and real-world applications.
Smooth Sensitivity Revisited: Towards Optimality
Venue: arXiv cs.CR 2024
Year: 2024

Smooth sensitivity is one of the most commonly used techniques for designing practical differentially private mechanisms. In this approach, one computes the smooth sensitivity of a given query $q$ on the given input $D$ and releases $q(D)$ with noise added proportional to this smooth sensitivity. One question remains: what distribution should we pick the noise from? In this paper, we give a new class of distributions suitable for the use with smooth sensitivity, which we name the PolyPlace distribution. This distribution improves upon the state-of-the-art Student's T distribution in terms of standard deviation by arbitrarily large factors, depending on a "smoothness parameter" $γ$, which one has to set in the smooth sensitivity framework. Moreover, our distribution is defined for a wider range of parameter $γ$, which can lead to significantly better performance. Moreover, we prove that the PolyPlace distribution converges for $γ\rightarrow 0$ to the Laplace distribution and so does its variance. This means that the Laplace mechanism is a limit special case of the PolyPlace mechanism. This implies that out mechanism is in a certain sense optimal for $γ\to 0$.
Exploring agent interaction patterns in the comment sections of fake and real news
Venue: arXiv cs.SI 2024
Year: 2024

User comments on social media have been recognized as a crucial factor in distinguishing between fake and real news, with many studies focusing on the textual content of user reactions. However, the interactions among agents in the comment sections for fake and real news have not been fully explored. In this study, we analyze a dataset comprising both fake and real news from Reddit to investigate agent interaction patterns, considering both the network structure and the sentiment of the nodes. Our findings reveal that (i) comments on fake news are more likely to form groups, (ii) compared to fake news, where users generate more negative sentiment, real news tend to elicit more neutral and positive sentiments. Additionally, nodes with similar sentiments cluster together more tightly than anticipated. From a dynamic perspective, we found that the sentiment distribution among nodes stabilizes early and remains stable over time. These findings have both theoretical and practical implications, particularly for the early detection of real and fake news within social networks.
Database Technology Evolution III: Knowledge Graphs and Linked Data
Venue: arXiv cs.DB 2024
Year: 2024

This paper reviews the changes for database technology represented by the current development of the draft international standard ISO 39075 (Database Languages - GQL), which seeks a unified specification for property graphs and knowledge graphs. This paper examines these current developments as part of our review of the evolution of database technology, and their relation to the longer-term goal of supporting the Semantic Web using relational technology.
Crowdsourced reviews reveal substantial disparities in public perceptions of parking
Venue: arXiv cs.CY 2024
Year: 2024

Due to increased reliance on private vehicles and growing travel demand, parking remains a longstanding urban challenge globally. Quantifying parking perceptions is paramount as it enables decision-makers to identify problematic areas and make informed decisions on parking management. This study introduces a cost-effective and widely accessible data source, crowdsourced online reviews, to investigate public perceptions of parking across the U.S. Specifically, we examine 4,987,483 parking-related reviews for 1,129,460 points of interest (POIs) across 911 core-based statistical areas (CBSAs) sourced from Google Maps. We employ the Bidirectional Encoder Representations from Transformers (BERT) model to classify the parking sentiment and conduct regression analyses to explore its relationships with socio-spatial factors. Findings reveal significant variations in parking sentiment across POI types and CBSAs, with Restaurant POIs showing the most negative. Regression results further indicate that denser urban areas with higher proportions of African Americans and Hispanics and lower socioeconomic status are more likely to exhibit negative parking sentiment. Interestingly, an opposite relationship between parking supply and sentiment is observed, indicating increasing supply does not necessarily improve parking experiences. Finally, our textual analysis identifies keywords associated with positive or negative sentiments and highlights disparities between urban and rural areas. Overall, this study demonstrates the potential of a novel data source and methodological framework in measuring parking sentiment, offering valuable insights that help identify hyperlocal parking issues and guide targeted parking management strategies.
SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention
Venue: arXiv cs.CV 2024
Year: 2024

Channel and spatial attentions have respectively brought significant improvements in extracting feature dependencies and spatial structure relations for various downstream vision tasks. While their combination is more beneficial for leveraging their individual strengths, the synergy between channel and spatial attentions has not been fully explored, lacking in fully harness the synergistic potential of multi-semantic information for feature guidance and mitigation of semantic disparities. Our study attempts to reveal the synergistic relationship between spatial and channel attention at multiple semantic levels, proposing a novel Spatial and Channel Synergistic Attention module (SCSA). Our SCSA consists of two parts: the Shareable Multi-Semantic Spatial Attention (SMSA) and the Progressive Channel-wise Self-Attention (PCSA). SMSA integrates multi-semantic information and utilizes a progressive compression strategy to inject discriminative spatial priors into PCSA's channel self-attention, effectively guiding channel recalibration. Additionally, the robust feature interactions based on the self-attention mechanism in PCSA further mitigate the disparities in multi-semantic information among different sub-features within SMSA. We conduct extensive experiments on seven benchmark datasets, including classification on ImageNet-1K, object detection on MSCOCO 2017, segmentation on ADE20K, and four other complex scene detection datasets. Our results demonstrate that our proposed SCSA not only surpasses the current state-of-the-art attention but also exhibits enhanced generalization capabilities across various task scenarios. The code and models are available at: https://github.com/HZAI-ZJNU/SCSA.
RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models
Venue: arXiv cs.LG 2024
Year: 2024

The recent emergence of Medical Large Vision Language Models (Med-LVLMs) has enhanced medical diagnosis. However, current Med-LVLMs frequently encounter factual issues, often generating responses that do not align with established medical facts. Retrieval-Augmented Generation (RAG), which utilizes external knowledge, can improve the factual accuracy of these models but introduces two major challenges. First, limited retrieved contexts might not cover all necessary information, while excessive retrieval can introduce irrelevant and inaccurate references, interfering with the model's generation. Second, in cases where the model originally responds correctly, applying RAG can lead to an over-reliance on retrieved contexts, resulting in incorrect answers. To address these issues, we propose RULE, which consists of two components. First, we introduce a provably effective strategy for controlling factuality risk through the calibrated selection of the number of retrieved contexts. Second, based on samples where over-reliance on retrieved contexts led to errors, we curate a preference dataset to fine-tune the model, balancing its dependence on inherent knowledge and retrieved contexts for generation. We demonstrate the effectiveness of RULE on medical VQA and report generation tasks across three datasets, achieving an average improvement of 47.4% in factual accuracy. We publicly release our benchmark and code in https://github.com/richard-peng-xia/RULE.
Almost Envy-free Allocation of Indivisible Goods: A Tale of Two Valuations
Venue: arXiv cs.GT 2024
Year: 2024

The existence of $\textsf{EFX}$ allocations stands as one of the main challenges in discrete fair division.In this paper, we present symmetrical results on the existence of $\textsf{EFX}$ and its approximate variations for two distinct valuations: restricted additive valuations and $(p,q)$-bounded valuations introduced by Christodoulou \etal \cite{christodoulou2023fair}. In a $(p,q)$-bounded instance, each good has relevance for at most $p$ agents, and any pair of agents shares at most $q$ common relevant goods. We show that instances with $(\infty,1)$-bounded valuations admit $\textsf{EF2X}$ allocations and $\textsf{EFX}$ allocations with at most $\lfloor {n}/{2} \rfloor - 1$ discarded goods, mirroring results for the restricted additive setting \cite{akrami2022ef2x}. We also present ${({\sqrt{2}}/{2})\textsf{-EFX}}$ algorithms for both restricted additive and $(\infty,1)$-bounded subadditive settings. The symmetry of these results suggests these valuations share symmetric structures. Building on this, we propose an $\textsf{EFX}$ allocation for restricted additive valuations when $p=2$ and $q=\infty$. To achieve these results, we further develop the rank concept introduced by Farhadi \etal \cite{farhadi2021almost} and introduce several new concepts such as virtual value, rankpath, and root, which advance the overall understanding of $\textsf{EFX}$ allocations. In addition, we suggest an updating rule based on the virtual values which we believe will lead to broader and more generalized results on $\textsf{EFX}$.
Learning Velocity-based Humanoid Locomotion: Massively Parallel Learning with Brax and MJX
Venue: arXiv cs.RO 2024
Year: 2024

Humanoid locomotion is a key skill to bring humanoids out of the lab and into the real-world. Many motion generation methods for locomotion have been proposed including reinforcement learning (RL). RL locomotion policies offer great versatility and generalizability along with the ability to experience new knowledge to improve over time. This work presents a velocity-based RL locomotion policy for the REEM-C robot. The policy uses a periodic reward formulation and is implemented in Brax/MJX for fast training. Simulation results for the policy are demonstrated with future experimental results in progress.
DehazeDCT: Towards Effective Non-Homogeneous Dehazing via Deformable Convolutional Transformer
Venue: arXiv cs.CV 2024
Year: 2024

Image dehazing, a pivotal task in low-level vision, aims to restore the visibility and detail from hazy images. Many deep learning methods with powerful representation learning capability demonstrate advanced performance on non-homogeneous dehazing, however, these methods usually struggle with processing high-resolution images (e.g., $4000 \times 6000$) due to their heavy computational demands. To address these challenges, we introduce an innovative non-homogeneous Dehazing method via Deformable Convolutional Transformer-like architecture (DehazeDCT). Specifically, we first design a transformer-like network based on deformable convolution v4, which offers long-range dependency and adaptive spatial aggregation capabilities and demonstrates faster convergence and forward speed. Furthermore, we leverage a lightweight Retinex-inspired transformer to achieve color correction and structure refinement. Extensive experiment results and highly competitive performance of our method in NTIRE 2024 Dense and Non-Homogeneous Dehazing Challenge, ranking second among all 16 submissions, demonstrate the superior capability of our proposed method. The code is available: https://github.com/movingforward100/Dehazing_R.
Toward a Unified Metadata Schema for Ecological Momentary Assessment with Voice-First Virtual Assistants
Venue: arXiv cs.HC 2024
Year: 2024

Ecological momentary assessment (EMA) is used to evaluate subjects' behaviors and moods in their natural environments, yet collecting real-time and self-report data with EMA is challenging due to user burden. Integrating voice into EMA data collection platforms through today's intelligent virtual assistants (IVAs) is promising due to hands-free and eye-free nature. However, efficiently managing conversations and EMAs is non-trivial and time consuming due to the ambiguity of the voice input. We approach this problem by rethinking the data modeling of EMA questions and what is needed to deploy them on voice-first user interfaces. We propose a unified metadata schema that models EMA questions and the necessary attributes to effectively and efficiently integrate voice as a new EMA modality. Our schema allows user experience researchers to write simple rules that can be rendered at run-time, instead of having to edit the source code. We showcase an example EMA survey implemented with our schema, which can run on multiple voice-only and voice-first devices. We believe that our work will accelerate the iterative prototyping and design process of real-world voice-based EMA data collection platforms.