OpenRaiser/Intern-Atlas huggingface dataset:
Self-Consistency Boosts Calibration for Math Reasoning
Venue: arXiv cs.CL 2024
Year: 2024

Calibration, which establishes the correlation between accuracy and model confidence, is important for LLM development. We design three off-the-shelf calibration methods based on self-consistency (Wang et al., 2022) for math reasoning tasks. Evaluation on two popular benchmarks (GSM8K and MathQA) using strong open-source LLMs (Mistral and LLaMA2), our methods better bridge model confidence and accuracy than existing methods based on p(True) (Kadavath et al., 2022) or logit (Kadavath et al., 2022).
FakeWatch: A Framework for Detecting Fake News to Ensure Credible Elections
Venue: arXiv cs.CL 2024
Year: 2024

In today's technologically driven world, the rapid spread of fake news, particularly during critical events like elections, poses a growing threat to the integrity of information. To tackle this challenge head-on, we introduce FakeWatch, a comprehensive framework carefully designed to detect fake news. Leveraging a newly curated dataset of North American election-related news articles, we construct robust classification models. Our framework integrates a model hub comprising of both traditional machine learning (ML) techniques, and state-of-the-art Language Models (LMs) to discern fake news effectively. Our objective is to provide the research community with adaptable and precise classification models adept at identifying fake news for the elections agenda. Quantitative evaluations of fake news classifiers on our dataset reveal that, while state-of-the-art LMs exhibit a slight edge over traditional ML models, classical models remain competitive due to their balance of accuracy and computational efficiency. Additionally, qualitative analyses shed light on patterns within fake news articles. We provide our labeled data at https://huggingface.co/datasets/newsmediabias/fake_news_elections_labelled_data and model https://huggingface.co/newsmediabias/FakeWatch for reproducibility and further research.
Towards White Box Deep Learning
Venue: arXiv cs.LG 2024
Year: 2024

Deep neural networks learn fragile "shortcut" features, rendering them difficult to interpret (black box) and vulnerable to adversarial attacks. This paper proposes semantic features as a general architectural solution to this problem. The main idea is to make features locality-sensitive in the adequate semantic topology of the domain, thus introducing a strong regularization. The proof of concept network is lightweight, inherently interpretable and achieves almost human-level adversarial test metrics - with no adversarial training! These results and the general nature of the approach warrant further research on semantic features. The code is available at https://github.com/314-Foundation/white-box-nn
Sabiá-2: A New Generation of Portuguese Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

We introduce Sabiá-2, a family of large language models trained on Portuguese texts. The models are evaluated on a diverse range of exams, including entry-level tests for Brazilian universities, professional certification exams, and graduate-level exams for various disciplines such as accounting, economics, engineering, law and medicine. Our results reveal that our best model so far, Sabiá-2 Medium, matches or surpasses GPT-4's performance in 23 out of 64 exams and outperforms GPT-3.5 in 58 out of 64 exams. Notably, specialization has a significant impact on a model's performance without the need to increase its size, allowing us to offer Sabiá-2 Medium at a price per token that is 10 times cheaper than GPT-4. Finally, we identified that math and coding are key abilities that need improvement.
Overcoming the cohesive zone limit in the modelling of composites delamination with TUBA cohesive elements
Venue: arXiv cs.CE 2024
Year: 2024

The wide adoption of composite structures in the aerospace industry requires reliable numerical methods to account for the effects of various damage mechanisms, including delamination. Cohesive elements are a versatile and physically representative way of modelling delamination. However, using their standard form which conforms to solid substrate elements, multiple elements are required in the narrow cohesive zone, thereby requiring an excessively fine mesh and hindering the applicability in practical scenarios. The present work focuses on the implementation and testing of triangular thin plate substrate elements and compatible cohesive elements, which satisfy C1-continuity in the domain. The improved regularity meets the continuity requirement coming from the Kirchhoff Plate Theory and the triangular shape allows for conformity to complex geometries. The overall model is validated for mode I delamination, the case with the smallest cohesive zone. Very accurate predictions of the limit load and crack propagation phase are achieved, using elements as large as 11 times the cohesive zone.
TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting
Venue: arXiv cs.LG 2024
Year: 2024

Long-term time-series forecasting remains challenging due to the difficulty in capturing long-term dependencies, achieving linear scalability, and maintaining computational efficiency. We introduce TimeMachine, an innovative model that leverages Mamba, a state-space model, to capture long-term dependencies in multivariate time series data while maintaining linear scalability and small memory footprints. TimeMachine exploits the unique properties of time series data to produce salient contextual cues at multi-scales and leverage an innovative integrated quadruple-Mamba architecture to unify the handling of channel-mixing and channel-independence situations, thus enabling effective selection of contents for prediction against global and local contexts at different scales. Experimentally, TimeMachine achieves superior performance in prediction accuracy, scalability, and memory efficiency, as extensively validated using benchmark datasets. Code availability: https://github.com/Atik-Ahamed/TimeMachine
Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection
Venue: arXiv cs.CL 2024
Year: 2024

Self-detection for Large Language Models (LLMs) seeks to evaluate the trustworthiness of the LLM's output by leveraging its own capabilities, thereby alleviating the issue of output hallucination. However, existing self-detection approaches only retrospectively evaluate answers generated by LLM, typically leading to the over-trust in incorrectly generated answers. To tackle this limitation, we propose a novel self-detection paradigm that considers the comprehensive answer space beyond LLM-generated answers. It thoroughly compares the trustworthiness of multiple candidate answers to mitigate the over-trust in LLM-generated incorrect answers. Building upon this paradigm, we introduce a two-step framework, which firstly instructs LLM to reflect and provide justifications for each candidate answer, and then aggregates the justifications for comprehensive target answer evaluation. This framework can be seamlessly integrated with existing approaches for superior self-detection. Extensive experiments on six datasets spanning three tasks demonstrate the effectiveness of the proposed framework.
DDPS: Dynamic Differential Pricing-based Edge Offloading System with Energy Harvesting Devices
Venue: arXiv cs.GT 2024
Year: 2024

Mobile edge computing (MEC) paves the way to alleviate the burden of energy and computation of mobile users (MUs) by offloading tasks to the network edge. To enhance the MEC server utilization by optimizing its resource allocation, a well-designed pricing strategy is indispensable. In this paper, we consider the edge offloading scenario with energy harvesting devices, and propose a dynamic differential pricing system (DDPS), which determines the price per unit time according to the usage of computing resources to improve the edge server utilization. Firstly, we propose an offloading decision algorithm to decide whether to conduct the offloading operation and how much data to be offloaded if conducted, the algorithm determines offloading operation by balancing the energy harvested with the energy consumed. Secondly, for the offloading case, we formulate the game between the MUs and the server as a Stackelberg game, and propose a differential pricing algorithm to determine the optimal computing resources required by MUs. Furthermore, the proposed algorithm also reallocates computing resources for delay-sensitive devices while server resources are surplus after the initial allocation, aiming to make full use of the server computing resources. Extensive simulations are conducted to demonstrate the effectiveness of the proposed DDPS scheme.
Identifying Health Risks from Family History: A Survey of Natural Language Processing Techniques
Venue: arXiv cs.CL 2024
Year: 2024

Electronic health records include information on patients' status and medical history, which could cover the history of diseases and disorders that could be hereditary. One important use of family history information is in precision health, where the goal is to keep the population healthy with preventative measures. Natural Language Processing (NLP) and machine learning techniques can assist with identifying information that could assist health professionals in identifying health risks before a condition is developed in their later years, saving lives and reducing healthcare costs. We survey the literature on the techniques from the NLP field that have been developed to utilise digital health records to identify risks of familial diseases. We highlight that rule-based methods are heavily investigated and are still actively used for family history extraction. Still, more recent efforts have been put into building neural models based on large-scale pre-trained language models. In addition to the areas where NLP has successfully been utilised, we also identify the areas where more research is needed to unlock the value of patients' records regarding data collection, task formulation and downstream applications.
Securing Federated Learning with Control-Flow Attestation: A Novel Framework for Enhanced Integrity and Resilience against Adversarial Attacks
Venue: arXiv cs.CR 2024
Year: 2024

The advent of Federated Learning (FL) as a distributed machine learning paradigm has introduced new cybersecurity challenges, notably adversarial attacks that threaten model integrity and participant privacy. This study proposes an innovative security framework inspired by Control-Flow Attestation (CFA) mechanisms, traditionally used in cybersecurity, to ensure software execution integrity. By integrating digital signatures and cryptographic hashing within the FL framework, we authenticate and verify the integrity of model updates across the network, effectively mitigating risks associated with model poisoning and adversarial interference. Our approach, novel in its application of CFA principles to FL, ensures contributions from participating nodes are authentic and untampered, thereby enhancing system resilience without compromising computational efficiency or model performance. Empirical evaluations on benchmark datasets, MNIST and CIFAR-10, demonstrate our framework's effectiveness, achieving a 100\% success rate in integrity verification and authentication and notable resilience against adversarial attacks. These results validate the proposed security enhancements and open avenues for more secure, reliable, and privacy-conscious distributed machine learning solutions. Our work bridges a critical gap between cybersecurity and distributed machine learning, offering a foundation for future advancements in secure FL.
Graph Enhanced Reinforcement Learning for Effective Group Formation in Collaborative Problem Solving
Venue: arXiv cs.CY 2024
Year: 2024

This study addresses the challenge of forming effective groups in collaborative problem-solving environments. Recognizing the complexity of human interactions and the necessity for efficient collaboration, we propose a novel approach leveraging graph theory and reinforcement learning. Our methodology involves constructing a graph from a dataset where nodes represent participants, and edges signify the interactions between them. We conceptualize each participant as an agent within a reinforcement learning framework, aiming to learn an optimal graph structure that reflects effective group dynamics. Clustering techniques are employed to delineate clear group structures based on the learned graph. Our approach provides theoretical solutions based on evaluation metrics and graph measurements, offering insights into potential improvements in group effectiveness and reductions in conflict incidences. This research contributes to the fields of collaborative work and educational psychology by presenting a data-driven, analytical approach to group formation. It has practical implications for organizational team building, classroom settings, and any collaborative scenario where group dynamics are crucial. The study opens new avenues for exploring the application of graph theory and reinforcement learning in social and behavioral sciences, highlighting the potential for empirical validation in future work.
Representing Domain-Mixing Optical Degradation for Real-World Computational Aberration Correction via Vector Quantization
Venue: arXiv cs.CV 2024
Year: 2024

Relying on paired synthetic data, existing learning-based Computational Aberration Correction (CAC) methods are confronted with the intricate and multifaceted synthetic-to-real domain gap, which leads to suboptimal performance in real-world applications. In this paper, in contrast to improving the simulation pipeline, we deliver a novel insight into real-world CAC from the perspective of Unsupervised Domain Adaptation (UDA). By incorporating readily accessible unpaired real-world data into training, we formalize the Domain Adaptive CAC (DACAC) task, and then introduce a comprehensive Real-world aberrated images (Realab) dataset to benchmark it. The setup task presents a formidable challenge due to the intricacy of understanding the target optical degradation domain. To this intent, we propose a novel Quantized Domain-Mixing Representation (QDMR) framework as a potent solution to the issue. Centering around representing and quantizing the optical degradation which is consistent across different images, QDMR adapts the CAC model to the target domain from three key aspects: (1) reconstructing aberrated images of both domains by a VQGAN to learn a Domain-Mixing Codebook (DMC) characterizing the optical degradation; (2) modulating the deep features in CAC model with DMC to transfer the target domain knowledge; and (3) leveraging the trained VQGAN to generate pseudo target aberrated images from the source ones for convincing target domain supervision. Extensive experiments on both synthetic and real-world benchmarks reveal that the models with QDMR consistently surpass the competitive methods in mitigating the synthetic-to-real gap, which produces visually pleasant real-world CAC results with fewer artifacts. Codes and datasets are made publicly available at https://github.com/zju-jiangqi/QDMR.
HeR-DRL:Heterogeneous Relational Deep Reinforcement Learning for Decentralized Multi-Robot Crowd Navigation
Venue: arXiv cs.RO 2024
Year: 2024

Crowd navigation has received significant research attention in recent years, especially DRL-based methods. While single-robot crowd scenarios have dominated research, they offer limited applicability to real-world complexities. The heterogeneity of interaction among multiple agent categories, like in decentralized multi-robot pedestrian scenarios, are frequently disregarded. This "interaction blind spot" hinders generalizability and restricts progress towards robust navigation algorithms. In this paper, we propose a heterogeneous relational deep reinforcement learning(HeR-DRL), based on customised heterogeneous GNN, in order to improve navigation strategies in decentralized multi-robot crowd navigation. Firstly, we devised a method for constructing robot-crowd heterogenous relation graph that effectively simulates the heterogeneous pair-wise interaction relationships. We proposed a new heterogeneous graph neural network for transferring and aggregating the heterogeneous state information. Finally, we incorporate the encoded information into deep reinforcement learning to explore the optimal policy. HeR-DRL are rigorously evaluated through comparing it to state-of-the-art algorithms in both single-robot and multi-robot circle crowssing scenario. The experimental results demonstrate that HeR-DRL surpasses the state-of-the-art approaches in overall performance, particularly excelling in safety and comfort metrics. This underscores the significance of interaction heterogeneity for crowd navigation. The source code will be publicly released in https://github.com/Zhouxy-Debugging-Den/HeR-DRL.
Efficient Multiplayer Battle Game Optimizer for Adversarial Robust Neural Architecture Search
Venue: arXiv cs.NE 2024
Year: 2024

This paper introduces a novel metaheuristic algorithm, known as the efficient multiplayer battle game optimizer (EMBGO), specifically designed for addressing complex numerical optimization tasks. The motivation behind this research stems from the need to rectify identified shortcomings in the original MBGO, particularly in search operators during the movement phase, as revealed through ablation experiments. EMBGO mitigates these limitations by integrating the movement and battle phases to simplify the original optimization framework and improve search efficiency. Besides, two efficient search operators: differential mutation and Lévy flight are introduced to increase the diversity of the population. To evaluate the performance of EMBGO comprehensively and fairly, numerical experiments are conducted on benchmark functions such as CEC2017, CEC2020, and CEC2022, as well as engineering problems. Twelve well-established MA approaches serve as competitor algorithms for comparison. Furthermore, we apply the proposed EMBGO to the complex adversarial robust neural architecture search (ARNAS) tasks and explore its robustness and scalability. The experimental results and statistical analyses confirm the efficiency and effectiveness of EMBGO across various optimization tasks. As a potential optimization technique, EMBGO holds promise for diverse applications in real-world problems and deep learning scenarios. The source code of EMBGO is made available in \url{https://github.com/RuiZhong961230/EMBGO}.
CyberMoraba: A game-based approach enhancing cybersecurity awareness
Venue: arXiv cs.HC 2024
Year: 2024

Numerous studies confirm Cybersecurity Awareness Games (CAGs) effectively bolster organisational security against cyberattacks. This article introduces a serious CAG, integrating the traditional South African Morabaraba board game into cybersecurity education. Players adopt roles of defenders or attackers, strategically placing tokens to enhance awareness. Evaluation shows positive outcomes, enhancing understanding and enjoyment among participants.
Taiyi: A high-performance CKKS accelerator for Practical Fully Homomorphic Encryption
Venue: arXiv cs.CR 2024
Year: 2024

Fully Homomorphic Encryption (FHE), a novel cryptographic theory enabling computation directly on ciphertext data, offers significant security benefits but is hampered by substantial performance overhead. In recent years, a series of accelerator designs have significantly enhanced the performance of FHE applications, bringing them closer to real-world applicability. However, these accelerators face challenges related to large on-chip memory and area. Additionally, FHE algorithms undergo rapid development, rendering the previous accelerator designs less perfectly adapted to the evolving landscape of optimized FHE applications. In this paper, we conducted a detailed analysis of existing applications with the new FHE method, making two key observations: 1) the bottleneck of FHE applications shifts from NTT to the inner-product operation, and 2) the optimal α of KeySwitch changes with the decrease in multiplicative level. Based on these observations, we designed an accelerator named Taiyi, which includes specific hardware for the inner-product operation and optimizes the NTT and BConv operations through algorithmic derivation. A comparative evaluation of Taiyi against previous state-of-the-art designs reveals an average performance improvement of 1.5x and reduces the area overhead by 15.7%.
A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges
Venue: arXiv cs.AI 2024
Year: 2024

The swift evolution of Large-scale Models (LMs), either language-focused or multi-modal, has garnered extensive attention in both academy and industry. But despite the surge in interest in this rapidly evolving area, there are scarce systematic reviews on their capabilities and potential in distinct impactful scenarios. This paper endeavours to help bridge this gap, offering a thorough examination of the current landscape of LM usage in regards to complex game playing scenarios and the challenges still open. Here, we seek to systematically review the existing architectures of LM-based Agents (LMAs) for games and summarize their commonalities, challenges, and any other insights. Furthermore, we present our perspective on promising future research avenues for the advancement of LMs in games. We hope to assist researchers in gaining a clear understanding of the field and to generate more interest in this highly impactful research direction. A corresponding resource, continuously updated, can be found in our GitHub repository.
A Vocabulary of Board Game Dynamics
Venue: arXiv cs.SE 2024
Year: 2024

In recent years, significant advances have been made in the field of game research. However, there has been a noticeable dearth of scholarly research focused on the domain of dynamics, despite the widespread recognition among researchers of its existence and importance. The objective of this paper is to address this research gap by presenting a vocabulary dedicated to boardgame dynamics. To achieve this goal, we employ a focus group to generate a set of dynamic concepts that are subsequently subjected to validation and refinement through a survey. The resulting concepts are then organized into a vocabulary using a taxonomic structure, allowing the grouping of these concepts into broader and more general ideas.
RACH-less Handover with Early Timing Advance Acquisition for Outage Reduction
Venue: arXiv cs.NI 2024
Year: 2024

For fifth-generation (5G) and 5G-Advanced networks, outage reduction within the context of reliability is a key objective since outage denotes the time period when a user equipment (UE) cannot communicate with the network. Earlier studies have shown that in the experimental high mobility scenario considered, outage is dominated by the interruption time that stems from the random access channel (RACH)-based handover process from the serving cell to the target cell. A handover by itself is a necessary mobility process to prevent mobility failures and their associated outage. This paper proposes a RACH-less handover signaling scheme for the 3rd Generation Partnership Project (3GPP) conditional handover (CHO) mechanism. The proposed scheme exploits the decoupling between the CHO preparation and execution phases to establish initial synchronization between the UE and the target cell through an early acquisition of the timing advance. This significantly curtails the RACH process and therefore the handover interruption time. Results based on a system-level simulation-based mobility study have shown that the proposed scheme significantly reduces the outage and its constituent handover interruption time relatively by 18.7% and 43.2%, respectively.