OpenRaiser/Intern-Atlas huggingface dataset:
Wearable Device-Based Real-Time Monitoring of Physiological Signals: Evaluating Cognitive Load Across Different Tasks
Venue: arXiv cs.HC 2024
Year: 2024

This study employs cutting-edge wearable monitoring technology to conduct high-precision, high-temporal-resolution (1-second interval) cognitive load assessment on electroencephalogram (EEG) data from the FP1 channel and heart rate variability (HRV) data of secondary vocational students. By jointly analyzing these two critical physiological indicators, the research delves into their application value in assessing cognitive load among secondary vocational students and their utility across various tasks. The study designed two experiments to validate the efficacy of the proposed approach: Initially, a random forest classification model, developed using the N-BACK task, enabled the precise decoding of physiological signal characteristics in secondary vocational students under different levels of cognitive load, achieving a classification accuracy of 97%. Subsequently, this classification model was applied in a cross-task experiment involving the National Computer Rank Examination (Level-1), demonstrating the method's significant applicability and cross-task transferability in diverse learning contexts. Conducted with high portability, this research holds substantial theoretical and practical significance for optimizing teaching resource allocation in secondary vocational education, as well as for cognitive load assessment methods and monitoring. Currently, the research findings are undergoing trial implementation in the school.
Merging Improves Self-Critique Against Jailbreak Attacks
Venue: arXiv cs.CL 2024
Year: 2024

The robustness of large language models (LLMs) against adversarial manipulations, such as jailbreak attacks, remains a significant challenge. In this work, we propose an approach that enhances the self-critique capability of the LLM and further fine-tunes it over sanitized synthetic data. This is done with the addition of an external critic model that can be merged with the original, thus bolstering self-critique capabilities and improving the robustness of the LLMs response to adversarial prompts. Our results demonstrate that the combination of merging and self-critique can reduce the attack success rate of adversaries significantly, thus offering a promising defense mechanism against jailbreak attacks. Code, data and models released at https://github.com/vicgalle/merging-self-critique-jailbreaks .
RGB-Sonar Tracking Benchmark and Spatial Cross-Attention Transformer Tracker
Venue: arXiv cs.CV 2024
Year: 2024

Vision camera and sonar are naturally complementary in the underwater environment. Combining the information from two modalities will promote better observation of underwater targets. However, this problem has not received sufficient attention in previous research. Therefore, this paper introduces a new challenging RGB-Sonar (RGB-S) tracking task and investigates how to achieve efficient tracking of an underwater target through the interaction of RGB and sonar modalities. Specifically, we first propose an RGBS50 benchmark dataset containing 50 sequences and more than 87000 high-quality annotated bounding boxes. Experimental results show that the RGBS50 benchmark poses a challenge to currently popular SOT trackers. Second, we propose an RGB-S tracker called SCANet, which includes a spatial cross-attention module (SCAM) consisting of a novel spatial cross-attention layer and two independent global integration modules. The spatial cross-attention is used to overcome the problem of spatial misalignment of between RGB and sonar images. Third, we propose a SOT data-based RGB-S simulation training method (SRST) to overcome the lack of RGB-S training datasets. It converts RGB images into sonar-like saliency images to construct pseudo-data pairs, enabling the model to learn the semantic structure of RGB-S-like data. Comprehensive experiments show that the proposed spatial cross-attention effectively achieves the interaction between RGB and sonar modalities and SCANet achieves state-of-the-art performance on the proposed benchmark. The code is available at https://github.com/LiYunfengLYF/RGBS50.
MeMSVD: Long-Range Temporal Structure Capturing Using Incremental SVD
Venue: arXiv cs.CV 2024
Year: 2024

This paper is on long-term video understanding where the goal is to recognise human actions over long temporal windows (up to minutes long). In prior work, long temporal context is captured by constructing a long-term memory bank consisting of past and future video features which are then integrated into standard (short-term) video recognition backbones through the use of attention mechanisms. Two well-known problems related to this approach are the quadratic complexity of the attention operation and the fact that the whole feature bank must be stored in memory for inference. To address both issues, we propose an alternative to attention-based schemes which is based on a low-rank approximation of the memory obtained using Singular Value Decomposition. Our scheme has two advantages: (a) it reduces complexity by more than an order of magnitude, and (b) it is amenable to an efficient implementation for the calculation of the memory bases in an incremental fashion which does not require the storage of the whole feature bank in memory. The proposed scheme matches or surpasses the accuracy achieved by attention-based mechanisms while being memory-efficient. Through extensive experiments, we demonstrate that our framework generalises to different architectures and tasks, outperforming the state-of-the-art in three datasets.
OPFData: Large-scale datasets for AC optimal power flow with topological perturbations
Venue: arXiv cs.LG 2024
Year: 2024

Solving the AC optimal power flow problem (AC-OPF) is critical to the efficient and safe planning and operation of power grids. Small efficiency improvements in this domain have the potential to lead to billions of dollars of cost savings, and significant reductions in emissions from fossil fuel generators. Recent work on data-driven solution methods for AC-OPF shows the potential for large speed improvements compared to traditional solvers; however, no large-scale open datasets for this problem exist. We present the largest readily-available collection of solved AC-OPF problems to date. This collection is orders of magnitude larger than existing readily-available datasets, allowing training of high-capacity data-driven models. Uniquely, it includes topological perturbations - a critical requirement for usage in realistic power grid operations. We hope this resource will spur the community to scale research to larger grid sizes with variable topology.
Scholarly Question Answering using Large Language Models in the NFDI4DataScience Gateway
Venue: arXiv cs.CL 2024
Year: 2024

This paper introduces a scholarly Question Answering (QA) system on top of the NFDI4DataScience Gateway, employing a Retrieval Augmented Generation-based (RAG) approach. The NFDI4DS Gateway, as a foundational framework, offers a unified and intuitive interface for querying various scientific databases using federated search. The RAG-based scholarly QA, powered by a Large Language Model (LLM), facilitates dynamic interaction with search results, enhancing filtering capabilities and fostering a conversational engagement with the Gateway search. The effectiveness of both the Gateway and the scholarly QA system is demonstrated through experimental analysis.
Personalisation of d'Hondt's algorithm and its use in recommender ecosystems
Venue: arXiv cs.HC 2024
Year: 2024

In the area of recommender systems, we are dealing with aggregations and potential of personalisation in ecosystems. Personalisation is based on separate aggregation models for each user. This approach reveals differences in user preferences, especially when they are in strict disagreement with global preferences. Hybrid models are based on combination of global and personalised model of weights for d'Hondt's voting algorithm. This paper shows that personalisation combined with hybridisation on case-by-case basis outperforms non-personalised d'Hondt's algorithm on datasets RetailRocket and SLANTour. By taking into account voices of minorities we achieved better click through rate.
Voxel Map to Occupancy Map Conversion Using Free Space Projection for Efficient Map Representation for Aerial and Ground Robots
Venue: arXiv cs.RO 2024
Year: 2024

This article introduces a novel method for converting 3D voxel maps, commonly utilized by robots for localization and navigation, into 2D occupancy maps for both unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs). The generated 2D maps can be used for more efficient global navigation for both UAVs and UGVs, in enabling algorithms developed for 2D maps to be useful in 3D applications, and allowing for faster transfer of maps between multiple agents in bandwidth-limited scenarios. The proposed method uses the free space representation in the UFOMap mapping solution to generate 2D occupancy maps. During the 3D to 2D map conversion, the method conducts safety checks and eliminates free spaces in the map with dimensions (in the height axis) lower than the robot's safety margins. This ensures that an aerial or ground robot can navigate safely, relying primarily on the 2D map generated by the method. Additionally, the method extracts the height of navigable free space and a local estimate of the slope of the floor from the 3D voxel map. The height data is utilized in converting paths generated using the 2D map into paths in 3D space for both UAVs and UGVs. The slope data identifies areas too steep for a ground robot to traverse, marking them as occupied, thus enabling a more accurate representation of the terrain for ground robots. The effectiveness of the proposed method in enabling computationally efficient navigation for both aerial and ground robots is validated in two different environments, over both static maps and in online implementation in an exploration mission. The methods proposed within this article have been implemented in the popular robotics framework ROS and are open-sourced. The code is available at: https://github.com/LTU-RAI/Map-Conversion-3D-Voxel-Map-to-2D-Occupancy-Map.
Exploring Cognitive Bias Triggers in COVID-19 Misinformation Tweets: A Bot vs. Human Perspective
Venue: arXiv cs.SI 2024
Year: 2024

During the COVID-19 pandemic, the proliferation of misinformation on social media has been rapidly increasing. Automated Bot authors are believed to be significant contributors of this surge. It is hypothesized that Bot authors deliberately craft online misinformation aimed at triggering and exploiting human cognitive biases, thereby enhancing tweet engagement and persuasive influence. This study investigates this hypothesis by studying triggers of biases embedded in Bot-authored misinformation and comparing them with their counterparts, Human-authored misinformation. We complied a Misinfo Dataset that contains COVID-19 vaccine-related misinformation tweets annotated by author identities, Bots vs Humans, from Twitter during the vaccination period from July 2020 to July 2021. We developed an algorithm to computationally automate the extraction of triggers for eight cognitive biase. Our analysis revealed that the Availability Bias, Cognitive Dissonance, and Confirmation Bias were most commonly present in misinformation, with Bot-authored tweets exhibiting a greater prevalence, with distinct patterns in utilizing bias triggers between Humans and Bots. We further linked these bias triggers with engagement metrics, inferring their potential influence on tweet engagement and persuasiveness. Overall, our findings indicate that bias-triggering tactics have been more influential on Bot-authored tweets than Human-authored tweets. While certain bias triggers boosted engagement for Bot-authored tweets, some other bias triggers unexpectedly decreased it. Conversely, triggers of most biases appeared to be unrelated to the engagement of Human-authored tweets. Our work sheds light on the differential utilization and effect of persuasion strategies between Bot-authored and Human-authored misinformation from the lens of human biases, offering insights for the development of effective counter-measures.
Enhanced In-Flight Connectivity for Urban Air Mobility via LEO Satellite Networks
Venue: arXiv cs.ET 2024
Year: 2024

Urban Air Mobility (UAM) is the envisioned future of inter-city aerial transportation. This paper presents a novel, in-flight connectivity link allocation method for UAM, which dynamically switches between terrestrial cellular and Low Earth Orbit (LEO) satellite networks based on real-time conditions. Our approach prefers cellular networks for cost efficiency, switching to LEO satellites under poor cellular conditions to ensure continuous UAM connectivity. By integrating real-time metrics like signal strength, network congestion, and flight trajectory into the selection process, our algorithm effectively balances cost, minimum data rate requirements, and continuity of communication. Numerical results validate minimization of data-loss while ensuring an optimal selection from the set of available above-threshold data rates at every time sample. Furthermore, insights derived from our study emphasize the importance of hybrid connectivity solutions in ensuring seamless, uninterrupted communication for future urban aerial vehicles.
CTC-based Non-autoregressive Textless Speech-to-Speech Translation
Venue: arXiv cs.CL 2024
Year: 2024

Direct speech-to-speech translation (S2ST) has achieved impressive translation quality, but it often faces the challenge of slow decoding due to the considerable length of speech sequences. Recently, some research has turned to non-autoregressive (NAR) models to expedite decoding, yet the translation quality typically lags behind autoregressive (AR) models significantly. In this paper, we investigate the performance of CTC-based NAR models in S2ST, as these models have shown impressive results in machine translation. Experimental results demonstrate that by combining pretraining, knowledge distillation, and advanced NAR training techniques such as glancing training and non-monotonic latent alignments, CTC-based NAR models achieve translation quality comparable to the AR model, while preserving up to 26.81$\times$ decoding speedup.
Minimizing Energy Costs in Deep Learning Model Training: The Gaussian Sampling Approach
Venue: arXiv cs.CV 2024
Year: 2024

Computing the loss gradient via backpropagation consumes considerable energy during deep learning (DL) model training. In this paper, we propose a novel approach to efficiently compute DL models' gradients to mitigate the substantial energy overhead associated with backpropagation. Exploiting the over-parameterized nature of DL models and the smoothness of their loss landscapes, we propose a method called {\em GradSamp} for sampling gradient updates from a Gaussian distribution. Specifically, we update model parameters at a given epoch (chosen periodically or randomly) by perturbing the parameters (element-wise) from the previous epoch with Gaussian ``noise''. The parameters of the Gaussian distribution are estimated using the error between the model parameter values from the two previous epochs. {\em GradSamp} not only streamlines gradient computation but also enables skipping entire epochs, thereby enhancing overall efficiency. We rigorously validate our hypothesis across a diverse set of standard and non-standard CNN and transformer-based models, spanning various computer vision tasks such as image classification, object detection, and image segmentation. Additionally, we explore its efficacy in out-of-distribution scenarios such as Domain Adaptation (DA), Domain Generalization (DG), and decentralized settings like Federated Learning (FL). Our experimental results affirm the effectiveness of {\em GradSamp} in achieving notable energy savings without compromising performance, underscoring its versatility and potential impact in practical DL applications.
Large Language Models for Constrained-Based Causal Discovery
Venue: arXiv cs.AI 2024
Year: 2024

Causality is essential for understanding complex systems, such as the economy, the brain, and the climate. Constructing causal graphs often relies on either data-driven or expert-driven approaches, both fraught with challenges. The former methods, like the celebrated PC algorithm, face issues with data requirements and assumptions of causal sufficiency, while the latter demand substantial time and domain knowledge. This work explores the capabilities of Large Language Models (LLMs) as an alternative to domain experts for causal graph generation. We frame conditional independence queries as prompts to LLMs and employ the PC algorithm with the answers. The performance of the LLM-based conditional independence oracle on systems with known causal graphs shows a high degree of variability. We improve the performance through a proposed statistical-inspired voting schema that allows some control over false-positive and false-negative rates. Inspecting the chain-of-thought argumentation, we find causal reasoning to justify its answer to a probabilistic query. We show evidence that knowledge-based CIT could eventually become a complementary tool for data-driven causal discovery.
Politics in Games -- An Overview and Classification
Venue: arXiv cs.HC 2024
Year: 2024

The representation of politics in media influences societal perceptions and attitudes. Video games, as a pervasive form of media, contribute significantly to this phenomenon. In this work, we explore political themes within video games by analyzing politically-themed games on game distribution platforms including Steam. We conducted a statistical examination of games with political context to identify patterns and use this as a basis to introduce a first taxonomy to categorize and better understand the interplay between politics and video games. This taxonomy offers a first framework for analyzing political content in games and also sets a foundation for future research in this field.
World Models with Hints of Large Language Models for Goal Achieving
Venue: arXiv cs.AI 2024
Year: 2024

Reinforcement learning struggles in the face of long-horizon tasks and sparse goals due to the difficulty in manual reward specification. While existing methods address this by adding intrinsic rewards, they may fail to provide meaningful guidance in long-horizon decision-making tasks with large state and action spaces, lacking purposeful exploration. Inspired by human cognition, we propose a new multi-modal model-based RL approach named Dreaming with Large Language Models (DLLM). DLLM integrates the proposed hinting subgoals from the LLMs into the model rollouts to encourage goal discovery and reaching in challenging tasks. By assigning higher intrinsic rewards to samples that align with the hints outlined by the language model during model rollouts, DLLM guides the agent toward meaningful and efficient exploration. Extensive experiments demonstrate that the DLLM outperforms recent methods in various challenging, sparse-reward environments such as HomeGrid, Crafter, and Minecraft by 27.7\%, 21.1\%, and 9.9\%, respectively.
Disrupting Bipartite Trading Networks: Matching for Revenue Maximization
Venue: arXiv cs.GT 2024
Year: 2024

We model the role of an online platform disrupting a market with unit-demand buyers and unit-supply sellers. Each seller can transact with a subset of the buyers whom she already knows, as well as with any additional buyers to whom she is introduced by the platform. Given these constraints on trade, prices and transactions are induced by a competitive equilibrium. The platform's revenue is proportional to the total price of all trades between platform-introduced buyers and sellers. In general, we show that the platform's revenue-maximization problem is computationally intractable. We provide structural results for revenue-optimal matchings and isolate special cases in which the platform can efficiently compute them. Furthermore, in a market where the maximum increase in social welfare that the platform can create is $ΔW$, we prove that the platform can attain revenue $Ω(ΔW/\log(\min\{n,m\}))$, where $n$ and $m$ are the numbers of buyers and sellers, respectively. When $ΔW$ is large compared to welfare without the platform, this gives a polynomial-time algorithm that guarantees a logarithmic approximation of the optimal welfare as revenue. We also show that even when the platform optimizes for revenue, the social welfare is at least an $O(\log(\min\{n,m\}))$-approximation to the optimal welfare. Finally, we prove significantly stronger bounds for revenue and social welfare in homogeneous-goods markets.
Machine Learning-Based Channel Prediction for RIS-assisted MIMO Systems With Channel Aging
Venue: arXiv cs.IT 2024
Year: 2024

Reconfigurable intelligent surfaces (RISs) have emerged as a promising technology to enhance the performance of sixth-generation (6G) and beyond communication systems. The passive nature of RISs and their large number of reflecting elements pose challenges to the channel estimation process. The associated complexity further escalates when the channel coefficients are fast-varying as in scenarios with user mobility. In this paper, we propose an extended channel estimation framework for RIS-assisted multiple-input multiple-output (MIMO) systems based on a convolutional neural network (CNN) integrated with an autoregressive (AR) predictor. The implemented framework is designed for identifying the aging pattern and predicting enhanced estimates of the wireless channels in correlated fast-fading environments. Insightful simulation results demonstrate that our proposed CNN-AR approach is robust to channel aging, exhibiting a high-precision estimation accuracy. The results also show that our approach can achieve high spectral efficiency and low pilot overhead compared to traditional methods.
Large Language Models are Limited in Out-of-Context Knowledge Reasoning
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) possess extensive knowledge and strong capabilities in performing in-context reasoning. However, previous work challenges their out-of-context reasoning ability, i.e., the ability to infer information from their training data, instead of from the context or prompt. This paper focuses on a significant aspect of out-of-context reasoning: Out-of-Context Knowledge Reasoning (OCKR), which is to combine multiple knowledge to infer new knowledge. We designed a synthetic dataset with seven representative OCKR tasks to systematically assess the OCKR capabilities of LLMs. Using this dataset, we evaluated several LLMs and discovered that their proficiency in this aspect is limited, regardless of whether the knowledge is trained in a separate or adjacent training settings. Moreover, training the model to reason with reasoning examples does not result in significant improvement, while training the model to perform explicit knowledge retrieval helps for retrieving attribute knowledge but not the relation knowledge, indicating that the model's limited OCKR capabilities are due to difficulties in knowledge retrieval. Furthermore, we treat cross-lingual knowledge transfer as a distinct form of OCKR, and evaluate this ability. Our results show that the evaluated model also exhibits limited ability in transferring knowledge across languages.