OpenRaiser/Intern-Atlas huggingface dataset:
Towards Better Generalization in Open-Domain Question Answering by Mitigating Context Memorization
Venue: arXiv cs.CL 2024
Year: 2024

Open-domain Question Answering (OpenQA) aims at answering factual questions with an external large-scale knowledge corpus. However, real-world knowledge is not static; it updates and evolves continually. Such a dynamic characteristic of knowledge poses a vital challenge for these models, as the trained models need to constantly adapt to the latest information to make sure that the answers remain accurate. In addition, it is still unclear how well an OpenQA model can transfer to completely new knowledge domains. In this paper, we investigate the generalization performance of a retrieval-augmented QA model in two specific scenarios: 1) adapting to updated versions of the same knowledge corpus; 2) switching to completely different knowledge domains. We observe that the generalization challenges of OpenQA models stem from the reader's over-reliance on memorizing the knowledge from the external corpus, which hinders the model from generalizing to a new knowledge corpus. We introduce Corpus-Invariant Tuning (CIT), a simple but effective training strategy, to mitigate the knowledge over-memorization by controlling the likelihood of retrieved contexts during training. Extensive experimental results on multiple OpenQA benchmarks show that CIT achieves significantly better generalizability without compromising the model's performance in its original corpus and domain.
Incentives in Private Collaborative Machine Learning
Venue: arXiv cs.LG 2024
Year: 2024

Collaborative machine learning involves training models on data from multiple parties but must incentivize their participation. Existing data valuation methods fairly value and reward each party based on shared data or model parameters but neglect the privacy risks involved. To address this, we introduce differential privacy (DP) as an incentive. Each party can select its required DP guarantee and perturb its sufficient statistic (SS) accordingly. The mediator values the perturbed SS by the Bayesian surprise it elicits about the model parameters. As our valuation function enforces a privacy-valuation trade-off, parties are deterred from selecting excessive DP guarantees that reduce the utility of the grand coalition's model. Finally, the mediator rewards each party with different posterior samples of the model parameters. Such rewards still satisfy existing incentives like fairness but additionally preserve DP and a high similarity to the grand coalition's posterior. We empirically demonstrate the effectiveness and practicality of our approach on synthetic and real-world datasets.
HeMeNet: Heterogeneous Multichannel Equivariant Network for Protein Multitask Learning
Venue: arXiv cs.LG 2024
Year: 2024

Understanding and leveraging the 3D structures of proteins is central to a variety of biological and drug discovery tasks. While deep learning has been applied successfully for structure-based protein function prediction tasks, current methods usually employ distinct training for each task. However, each of the tasks is of small size, and such a single-task strategy hinders the models' performance and generalization ability. As some labeled 3D protein datasets are biologically related, combining multi-source datasets for larger-scale multi-task learning is one way to overcome this problem. In this paper, we propose a neural network model to address multiple tasks jointly upon the input of 3D protein structures. In particular, we first construct a standard structure-based multi-task benchmark called Protein-MT, consisting of 6 biologically relevant tasks, including affinity prediction and property prediction, integrated from 4 public datasets. Then, we develop a novel graph neural network for multi-task learning, dubbed Heterogeneous Multichannel Equivariant Network (HeMeNet), which is E(3) equivariant and able to capture heterogeneous relationships between different atoms. Besides, HeMeNet can achieve task-specific learning via the task-aware readout mechanism. Extensive evaluations on our benchmark verify the effectiveness of multi-task learning, and our model generally surpasses state-of-the-art models.
Boosting Visual Recognition in Real-world Degradations via Unsupervised Feature Enhancement Module with Deep Channel Prior
Venue: arXiv cs.CV 2024
Year: 2024

The environmental perception of autonomous vehicles in normal conditions have achieved considerable success in the past decade. However, various unfavourable conditions such as fog, low-light, and motion blur will degrade image quality and pose tremendous threats to the safety of autonomous driving. That is, when applied to degraded images, state-of-the-art visual models often suffer performance decline due to the feature content loss and artifact interference caused by statistical and structural properties disruption of captured images. To address this problem, this work proposes a novel Deep Channel Prior (DCP) for degraded visual recognition. Specifically, we observe that, in the deep representation space of pre-trained models, the channel correlations of degraded features with the same degradation type have uniform distribution even if they have different content and semantics, which can facilitate the mapping relationship learning between degraded and clear representations in high-sparsity feature space. Based on this, a novel plug-and-play Unsupervised Feature Enhancement Module (UFEM) is proposed to achieve unsupervised feature correction, where the multi-adversarial mechanism is introduced in the first stage of UFEM to achieve the latent content restoration and artifact removal in high-sparsity feature space. Then, the generated features are transferred to the second stage for global correlation modulation under the guidance of DCP to obtain high-quality and recognition-friendly features. Evaluations of three tasks and eight benchmark datasets demonstrate that our proposed method can comprehensively improve the performance of pre-trained models in real degradation conditions. The source code is available at https://github.com/liyuhang166/Deep_Channel_Prior
CIRP: Cross-Item Relational Pre-training for Multimodal Product Bundling
Venue: arXiv cs.IR 2024
Year: 2024

Product bundling has been a prevailing marketing strategy that is beneficial in the online shopping scenario. Effective product bundling methods depend on high-quality item representations, which need to capture both the individual items' semantics and cross-item relations. However, previous item representation learning methods, either feature fusion or graph learning, suffer from inadequate cross-modal alignment and struggle to capture the cross-item relations for cold-start items. Multimodal pre-train models could be the potential solutions given their promising performance on various multimodal downstream tasks. However, the cross-item relations have been under-explored in the current multimodal pre-train models. To bridge this gap, we propose a novel and simple framework Cross-Item Relational Pre-training (CIRP) for item representation learning in product bundling. Specifically, we employ a multimodal encoder to generate image and text representations. Then we leverage both the cross-item contrastive loss (CIC) and individual item's image-text contrastive loss (ITC) as the pre-train objectives. Our method seeks to integrate cross-item relation modeling capability into the multimodal encoder, while preserving the in-depth aligned multimodal semantics. Therefore, even for cold-start items that have no relations, their representations are still relation-aware. Furthermore, to eliminate the potential noise and reduce the computational cost, we harness a relation pruning module to remove the noisy and redundant relations. We apply the item representations extracted by CIRP to the product bundling model ItemKNN, and experiments on three e-commerce datasets demonstrate that CIRP outperforms various leading representation learning methods.
Unleash the Potential of CLIP for Video Highlight Detection
Venue: arXiv cs.CV 2024
Year: 2024

Multimodal and large language models (LLMs) have revolutionized the utilization of open-world knowledge, unlocking novel potentials across various tasks and applications. Among these domains, the video domain has notably benefited from their capabilities. In this paper, we present Highlight-CLIP (HL-CLIP), a method designed to excel in the video highlight detection task by leveraging the pre-trained knowledge embedded in multimodal models. By simply fine-tuning the multimodal encoder in combination with our innovative saliency pooling technique, we have achieved the state-of-the-art performance in the highlight detection task, the QVHighlight Benchmark, to the best of our knowledge.
Security for adversarial wiretap channels
Venue: arXiv cs.CR 2024
Year: 2024

We consider the wiretap channel, where the individual channel uses have memory or are influenced by an adversary. We analyze the explicit and computationally efficient construction of information-theoretically secure coding schemes which use the inverse of an extractor and an error-correcting code. These schemes are known to achieve secrecy capacity on a large class of memoryless wiretap channels. We show that this also holds for certain channel types with memory. In particular, they can achieve secrecy capacity on channels where an adversary can pick a sequence of ``states'' governing the channel's behavior, as long as, given every possible state, the channel is strongly symmetric.
Open Experimental Measurements of Sub-6GHz Reconfigurable Intelligent Surfaces
Venue: arXiv cs.IT 2024
Year: 2024

In this paper, we present two datasets that we make publicly available for research. The data is collected in a testbed comprised of a custom-made Reconfigurable Intelligent Surface (RIS) prototype and two regular OFDM transceivers within an anechoic chamber. First, we discuss the details of the testbed and equipment used, including insights about the design and implementation of our RIS prototype. We further present the methodology we employ to gather measurement samples, which consists of letting the RIS electronically steer the signal reflections from an OFDM transmitter toward a specific location. To this end, we evaluate a suitably designed configuration codebook and collect measurement samples of the received power with an OFDM receiver. Finally, we present the resulting datasets, their format, and examples of exploiting this data for research purposes.
On the Algorithmic Recovering of Coefficients in Linearizable Differential Equations
Venue: arXiv cs.SC 2024
Year: 2024

We investigate the problem of recovering coefficients in scalar nonlinear ordinary differential equations that can be exactly linearized. This contribution builds upon prior work by Lyakhov, Gerdt, and Michels, which focused on obtaining a linearizability certificate through point transformations. Our focus is on quasi-linear equations, specifically those solved for the highest derivative with a rational dependence on the variables involved. Our novel algorithm for coefficient recovery relies on basic operations on Lie algebras, such as computing the derived algebra and the dimension of the symmetry algebra. This algorithmic approach is efficient, although finding the linearization transformation necessitates computing at least one solution of the corresponding Bluman-Kumei equation system.
Neuromorphic Wireless Device-Edge Co-Inference via the Directed Information Bottleneck
Venue: arXiv cs.LG 2024
Year: 2024

An important use case of next-generation wireless systems is device-edge co-inference, where a semantic task is partitioned between a device and an edge server. The device carries out data collection and partial processing of the data, while the remote server completes the given task based on information received from the device. It is often required that processing and communication be run as efficiently as possible at the device, while more computing resources are available at the edge. To address such scenarios, we introduce a new system solution, termed neuromorphic wireless device-edge co-inference. According to it, the device runs sensing, processing, and communication units using neuromorphic hardware, while the server employs conventional radio and computing technologies. The proposed system is designed using a transmitter-centric information-theoretic criterion that targets a reduction of the communication overhead, while retaining the most relevant information for the end-to-end semantic task of interest. Numerical results on standard data sets validate the proposed architecture, and a preliminary testbed realization is reported.
Tensorized NeuroEvolution of Augmenting Topologies for GPU Acceleration
Venue: arXiv cs.NE 2024
Year: 2024

The NeuroEvolution of Augmenting Topologies (NEAT) algorithm has received considerable recognition in the field of neuroevolution. Its effectiveness is derived from initiating with simple networks and incrementally evolving both their topologies and weights. Although its capability across various challenges is evident, the algorithm's computational efficiency remains an impediment, limiting its scalability potential. In response, this paper introduces a tensorization method for the NEAT algorithm, enabling the transformation of its diverse network topologies and associated operations into uniformly shaped tensors for computation. This advancement facilitates the execution of the NEAT algorithm in a parallelized manner across the entire population. Furthermore, we develop TensorNEAT, a library that implements the tensorized NEAT algorithm and its variants, such as CPPN and HyperNEAT. Building upon JAX, TensorNEAT promotes efficient parallel computations via automated function vectorization and hardware acceleration. Moreover, the TensorNEAT library supports various benchmark environments including Gym, Brax, and gymnax. Through evaluations across a spectrum of robotics control environments in Brax, TensorNEAT achieves up to 500x speedups compared to the existing implementations such as NEAT-Python. Source codes are available at: https://github.com/EMI-Group/tensorneat.
Sketch3D: Style-Consistent Guidance for Sketch-to-3D Generation
Venue: arXiv cs.CV 2024
Year: 2024

Recently, image-to-3D approaches have achieved significant results with a natural image as input. However, it is not always possible to access these enriched color input samples in practical applications, where only sketches are available. Existing sketch-to-3D researches suffer from limitations in broad applications due to the challenges of lacking color information and multi-view content. To overcome them, this paper proposes a novel generation paradigm Sketch3D to generate realistic 3D assets with shape aligned with the input sketch and color matching the textual description. Concretely, Sketch3D first instantiates the given sketch in the reference image through the shape-preserving generation process. Second, the reference image is leveraged to deduce a coarse 3D Gaussian prior, and multi-view style-consistent guidance images are generated based on the renderings of the 3D Gaussians. Finally, three strategies are designed to optimize 3D Gaussians, i.e., structural optimization via a distribution transfer mechanism, color optimization with a straightforward MSE loss and sketch similarity optimization with a CLIP-based geometric similarity loss. Extensive visual comparisons and quantitative analysis illustrate the advantage of our Sketch3D in generating realistic 3D assets while preserving consistency with the input.
Unmasking the Nuances of Loneliness: Using Digital Biomarkers to Understand Social and Emotional Loneliness in College Students
Venue: arXiv cs.HC 2024
Year: 2024

Background: Loneliness among students is increasing across the world, with potential consequences for mental health and academic success. To address this growing problem, accurate methods of detection are needed to identify loneliness and to differentiate social and emotional loneliness so that intervention can be personalized to individual need. Passive sensing technology provides a unique technique to capture behavioral patterns linked with distinct loneliness forms, allowing for more nuanced understanding and interventions for loneliness. Methods: To differentiate between social and emotional loneliness using digital biomarkers, our study included statistical tests, machine learning for predictive modeling, and SHAP values for feature importance analysis, revealing important factors in loneliness classification. Results: Our analysis revealed significant behavioral differences between socially and emotionally lonely groups, particularly in terms of phone usage and location-based features , with machine learning models demonstrating substantial predictive power in classifying loneliness levels. The XGBoost model, in particular, showed high accuracy and was effective in identifying key digital biomarkers, including phone usage duration and location-based features, as significant predictors of loneliness categories. Conclusion: This study underscores the potential of passive sensing data, combined with machine learning techniques, to provide insights into the behavioral manifestations of social and emotional loneliness among students. The identification of key digital biomarkers paves the way for targeted interventions aimed at mitigating loneliness in this population.
Pairwise Similarity Distribution Clustering for Noisy Label Learning
Venue: arXiv cs.LG 2024
Year: 2024

Noisy label learning aims to train deep neural networks using a large amount of samples with noisy labels, whose main challenge comes from how to deal with the inaccurate supervision caused by wrong labels. Existing works either take the label correction or sample selection paradigm to involve more samples with accurate labels into the training process. In this paper, we propose a simple yet effective sample selection algorithm, termed as Pairwise Similarity Distribution Clustering~(PSDC), to divide the training samples into one clean set and another noisy set, which can power any of the off-the-shelf semi-supervised learning regimes to further train networks for different downstream tasks. Specifically, we take the pairwise similarity between sample pairs to represent the sample structure, and the Gaussian Mixture Model~(GMM) to model the similarity distribution between sample pairs belonging to the same noisy cluster, therefore each sample can be confidently divided into the clean set or noisy set. Even under severe label noise rate, the resulting data partition mechanism has been proved to be more robust in judging the label confidence in both theory and practice. Experimental results on various benchmark datasets, such as CIFAR-10, CIFAR-100 and Clothing1M, demonstrate significant improvements over state-of-the-art methods.
Enhancing Robot Navigation Efficiency Using Cellular Automata with Active Cells
Venue: arXiv cs.RO 2024
Year: 2024

Enhancing robot navigation efficiency is a crucial objective in modern robotics. Robots relying on external navigation systems are often susceptible to electromagnetic interference (EMI) and encounter environmental disturbances, resulting in orientation errors within their surroundings. Therefore, the study employed an internal navigation system to enhance robot navigation efficacy under interference conditions, based on the analysis of the internal parameters and the external signals. This article presents details of the robot's autonomous operation, which allows for setting the robot's trajectory using an embedded map. The robot's navigation process involves counting the number of wheel revolutions as well as adjusting wheel orientation after each straight path section. In this article, an autonomous robot navigation system has been presented that leverages an embedded control navigation map utilising cellular automata with active cells which can effectively navigate in an environment containing various types of obstacles. By analysing the neighbouring cells of the active cell, the cellular environment determines which cell should become active during the robot's next movement step. This approach ensures the robot's independence from external control inputs. Furthermore, the accuracy and speed of the robot's movement have been further enhanced using a hexagonal mosaic for navigation surface mapping. This concept of utilising on cellular automata with active cells has been extended to the navigation of a group of robots on a shared navigation surface, taking into account the intersections of the robots' trajectories over time. To achieve this, a distance control module has been used that records the travelled trajectories in terms of wheel turns and revolutions.
Automatic Derivation of an Optimal Task Frame for Learning and Controlling Contact-Rich Tasks
Venue: arXiv cs.RO 2024
Year: 2024

In previous work on learning and controlling contact-rich tasks, the procedure for choosing a proper reference frame to express learned signals for the motion and the interaction wrench is often implicit, requires expert insight, or starts from proposed frame candidates. This article presents an automatic method to derive the optimal reference frame, referred to as optimal task frame, directly from the recorded motion and wrench data of the demonstration. Using screw theory, several origin and orientation candidates are generated that maximize decoupling in the data. These candidates are then processed probabilistically, without needing hyperparameters, to obtain the optimal task frame. Its origin and orientation are independently fixed to either the world or the robot tool. The method works regardless of whether the task involves translation, rotation, force, or moment, or any combination thereof. The method was validated for various tasks, including surface following and manipulation of articulated objects, showing good agreement between derived and assumed expert task frames. To validate the robot's performance, a constraint-based controller was designed based on the data expressed in the derived task frames. These experiments demonstrated the approach's effectiveness and versatility. The automatic task frame derivation approach supports learning methods to design controllers for a wide range of contact-rich tasks.
Optimizing Offload Performance in Heterogeneous MPSoCs
Venue: arXiv cs.AR 2024
Year: 2024

Heterogeneous multi-core architectures combine a few "host" cores, optimized for single-thread performance, with many small energy-efficient "accelerator" cores for data-parallel processing, on a single chip. Offloading a computation to the many-core acceleration fabric introduces a communication and synchronization cost which reduces the speedup attainable on the accelerator, particularly for small and fine-grained parallel tasks. We demonstrate that by co-designing the hardware and offload routines, we can increase the speedup of an offloaded DAXPY kernel by as much as 47.9%. Furthermore, we show that it is possible to accurately model the runtime of an offloaded application, accounting for the offload overheads, with as low as 1% MAPE error, enabling optimal offload decisions under offload execution time constraints.