OpenRaiser/Intern-Atlas huggingface dataset:
Enhanced Classroom Dialogue Sequences Analysis with a Hybrid AI Agent: Merging Expert Rule-Base with Large Language Models
Venue: arXiv cs.AI 2024
Year: 2024

Classroom dialogue plays a crucial role in fostering student engagement and deeper learning. However, analysing dialogue sequences has traditionally relied on either theoretical frameworks or empirical descriptions of practice, with limited integration between the two. This study addresses this gap by developing a comprehensive rule base of dialogue sequences and an Artificial Intelligence (AI) agent that combines expert-informed rule-based systems with a large language model (LLM). The agent applies expert knowledge while adapting to the complexities of natural language, enabling accurate and flexible categorisation of classroom dialogue sequences. By synthesising findings from over 30 studies, we established a comprehensive framework for dialogue analysis. The agent was validated against human expert coding, achieving high levels of precision and reliability. The results demonstrate that the agent provides theory-grounded and adaptive functions, tremendously enhancing the efficiency and scalability of classroom dialogue analysis, offering significant potential in improving classroom teaching practices and supporting teacher professional development.
Machine Unlearning on Pre-trained Models by Residual Feature Alignment Using LoRA
Venue: arXiv cs.LG 2024
Year: 2024

Machine unlearning is new emerged technology that removes a subset of the training data from a trained model without affecting the model performance on the remaining data. This topic is becoming increasingly important in protecting user privacy and eliminating harmful or outdated data. The key challenge lies in effectively and efficiently unlearning specific information without compromising the model's utility on the retained data. For the pre-trained models, fine-tuning is an important way to achieve the unlearning target. Previous work typically fine-tuned the entire model's parameters, which incurs significant computation costs. In addition, the fine-tuning process may cause shifts in the intermediate layer features, affecting the model's overall utility. In this work, we propose a novel and efficient machine unlearning method on pre-trained models. We term the method as Residual Feature Alignment Unlearning. Specifically, we leverage LoRA (Low-Rank Adaptation) to decompose the model's intermediate features into pre-trained features and residual features. By adjusting the residual features, we align the unlearned model with the pre-trained model at the intermediate feature level to achieve both unlearning and remaining targets. The method aims to learn the zero residuals on the retained set and shifted residuals on the unlearning set. Extensive experiments on numerous datasets validate the effectiveness of our approach.
Variable-Length Feedback Codes via Deep Learning
Venue: arXiv cs.IT 2024
Year: 2024

Variable-length feedback coding has the potential to significantly enhance communication reliability in finite block length scenarios by adapting coding strategies based on real-time receiver feedback. Designing such codes, however, is challenging. While deep learning (DL) has been employed to design sophisticated feedback codes, existing DL-aided feedback codes are predominantly fixed-length and suffer performance degradation in the high code rate regime, limiting their adaptability and efficiency. This paper introduces deep variable-length feedback (DeepVLF) code, a novel DL-aided variable-length feedback coding scheme. By segmenting messages into multiple bit groups and employing a threshold-based decoding mechanism for independent decoding of each bit group across successive communication rounds, DeepVLF outperforms existing DL-based feedback codes and establishes a new benchmark in feedback channel coding.
On the Design of Variable Modulation and Adaptive Modulation for Uplink Sparse Code Multiple Access
Venue: arXiv cs.IT 2024
Year: 2024

Sparse code multiple access (SCMA) is a promising non-orthogonal multiple access scheme for enabling massive connectivity in next generation wireless networks. However, current SCMA codebooks are designed with the same size, leading to inflexibility of user grouping and supporting diverse data rates. To address this issue, we propose a variable modulation SCMA (VM-SCMA) that allows users to employ codebooks with different modulation orders. To guide the VM-SCMA design, a VM matrix (VMM) that assigns modulation orders based on the SCMA factor graph is first introduced. We formulate the VM-SCMA design using the proposed average inverse product distance and the asymptotic upper bound of sum-rate, and jointly optimize the VMM, VM codebooks, power and codebook allocations. The proposed VM-SCMA not only enables diverse date rates but also supports different modulation order combinations for each rate. Leveraging these distinct advantages, we further propose an adaptive VM-SCMA (AVM-SCMA) scheme which adaptively selects the rate and the corresponding VM codebooks to adapt to the users' channel conditions by maximizing the proposed effective throughput. Simulation results show that the overall designs are able to simultaneously achieve a high-level system flexibility, enhanced error rate results, and significantly improved throughput performance, when compared to conventional SCMA schemes.
The EU AI Act is a good start but falls short
Venue: arXiv cs.SE 2024
Year: 2024

The EU AI Act was created to ensure ethical and safe Artificial Intelligence (AI) development and deployment across the EU. This study aims to identify key challenges and strategies for helping enterprises focus on resources effectively. To achieve this aim, we conducted a Multivocal Literature Review (MLR) to explore the sentiments of both the industry and the academia. From 130 articles, 56 met the criteria. Our key findings are three-fold. First, liability. Second, discrimination. Third, tool adequacy. Additionally, some negative sentiments were expressed by industry and academia regarding regulatory interpretations, specific requirements, and transparency issues. Next, our findings are three essential themes for enterprises. First, risk-based regulatory compliance. Second, ethical frameworks and principles in technology development. Third, policies and systems for regulatory risk management. These results identify the key challenges and strategies and provide less commonly discussed themes, enabling enterprises to align with the requirements and minimize their distance from the EU market.
Learning Locally Adaptive Metrics that Enhance Structural Representation with $\texttt{LAMINAR}$
Venue: arXiv cs.LG 2024
Year: 2024

We present $\texttt{LAMINAR}$, a novel unsupervised machine learning pipeline designed to enhance the representation of structure within data via producing a more-informative distance metric. Analysis methods in the physical sciences often rely on standard metrics to define geometric relationships in data, which may fail to capture the underlying structure of complex data sets. $\texttt{LAMINAR}$ addresses this by using a continuous-normalising-flow and inverse-transform-sampling to define a Riemannian manifold in the data space without the need for the user to specify a metric over the data a-priori. The result is a locally-adaptive-metric that produces structurally-informative density-based distances. We demonstrate the utility of $\texttt{LAMINAR}$ by comparing its output to the Euclidean metric for structured data sets.
NavAgent: Multi-scale Urban Street View Fusion For UAV Embodied Vision-and-Language Navigation
Venue: arXiv cs.CV 2024
Year: 2024

Vision-and-Language Navigation (VLN), as a widely discussed research direction in embodied intelligence, aims to enable embodied agents to navigate in complicated visual environments through natural language commands. Most existing VLN methods focus on indoor ground robot scenarios. However, when applied to UAV VLN in outdoor urban scenes, it faces two significant challenges. First, urban scenes contain numerous objects, which makes it challenging to match fine-grained landmarks in images with complex textual descriptions of these landmarks. Second, overall environmental information encompasses multiple modal dimensions, and the diversity of representations significantly increases the complexity of the encoding process. To address these challenges, we propose NavAgent, the first urban UAV embodied navigation model driven by a large Vision-Language Model. NavAgent undertakes navigation tasks by synthesizing multi-scale environmental information, including topological maps (global), panoramas (medium), and fine-grained landmarks (local). Specifically, we utilize GLIP to build a visual recognizer for landmark capable of identifying and linguisticizing fine-grained landmarks. Subsequently, we develop dynamically growing scene topology map that integrate environmental information and employ Graph Convolutional Networks to encode global environmental data. In addition, to train the visual recognizer for landmark, we develop NavAgent-Landmark2K, the first fine-grained landmark dataset for real urban street scenes. In experiments conducted on the Touchdown and Map2seq datasets, NavAgent outperforms strong baseline models. The code and dataset will be released to the community to facilitate the exploration and development of outdoor VLN.
I Can Embrace and Avoid Vagueness Myself: Supporting the Design Process by Balancing Vagueness through Text-to-Image Generative AI
Venue: arXiv cs.HC 2024
Year: 2024

This study examines the role of vagueness in the design process and its strategic management for the effective human-AI interaction. While vagueness in the generation of design ideas promotes diverse interpretations and prevents fixation, excessive vagueness can lead to scattered results. Designers attempt to use image search tools or generative AIs (e.g., Dall-E) for their work but often fail to achieve satisfactory results because the level of vagueness is not properly managed in these technologies. In this work, we identified how designers coordinate vagueness in their design process and applied key components of the process to the design of CLAY, an interactive system that balances vagueness through iterative prompt refinement by integrating the strengths of text-to-image generative AI. Results from our user study with 10 fashion designers showed that CLAY effectively supported their design process, reducing design time, and expanding creative possibilities compared to their existing practice, by allowing them to both embrace and avoid vagueness as needed. Our study highlights the importance of identifying key characteristics of the target user and domain, and exploring ways to incorporate them into the design of an AI-based interactive tool.
Theoretical Analysis of Byte-Pair Encoding
Venue: arXiv cs.DS 2024
Year: 2024

Byte-Pair Encoding (BPE) is a widely used method for subword tokenization, with origins in grammar-based text compression. It is employed in a variety of language processing tasks such as machine translation or large language model (LLM) pretraining, to create a token dictionary of a prescribed size. Most evaluations of BPE to date are empirical, and the reasons for its good practical performance are not well understood. In this paper we focus on the optimization problem underlying BPE: finding a pair encoding that achieves optimal compression utility. We show that this problem is APX-complete, indicating that it is unlikely to admit a polynomial-time approximation scheme. This answers, in a stronger form, a question recently raised by Zouhar et al. On the positive side, we show that BPE approximates the compression utility of the optimal pair encoding to a worst-case factor between $0.333$ and $0.625$. Our results aim to explain the ongoing success of BPE and are, to our knowledge, the first rigorous guarantees on its compression utility that hold for all inputs.
Scholarly Wikidata: Population and Exploration of Conference Data in Wikidata using LLMs
Venue: arXiv cs.DL 2024
Year: 2024

Several initiatives have been undertaken to conceptually model the domain of scholarly data using ontologies and to create respective Knowledge Graphs. Yet, the full potential seems unleashed, as automated means for automatic population of said ontologies are lacking, and respective initiatives from the Semantic Web community are not necessarily connected: we propose to make scholarly data more sustainably accessible by leveraging Wikidata's infrastructure and automating its population in a sustainable manner through LLMs by tapping into unstructured sources like conference Web sites and proceedings texts as well as already existing structured conference datasets. While an initial analysis shows that Semantic Web conferences are only minimally represented in Wikidata, we argue that our methodology can help to populate, evolve and maintain scholarly data as a community within Wikidata. Our main contributions include (a) an analysis of ontologies for representing scholarly data to identify gaps and relevant entities/properties in Wikidata, (b) semi-automated extraction -- requiring (minimal) manual validation -- of conference metadata (e.g., acceptance rates, organizer roles, programme committee members, best paper awards, keynotes, and sponsors) from websites and proceedings texts using LLMs. Finally, we discuss (c) extensions to visualization tools in the Wikidata context for data exploration of the generated scholarly data. Our study focuses on data from 105 Semantic Web-related conferences and extends/adds more than 6000 entities in Wikidata. It is important to note that the method can be more generally applicable beyond Semantic Web-related conferences for enhancing Wikidata's utility as a comprehensive scholarly resource. Source Repository: https://github.com/scholarly-wikidata/ DOI: https://doi.org/10.5281/zenodo.10989709 License: Creative Commons CC0 (Data), MIT (Code)
Rethinking negative sampling in content-based news recommendation
Venue: arXiv cs.IR 2024
Year: 2024

News recommender systems are hindered by the brief lifespan of articles, as they undergo rapid relevance decay. Recent studies have demonstrated the potential of content-based neural techniques in tackling this problem. However, these models often involve complex neural architectures and often lack consideration for negative examples. In this study, we posit that the careful sampling of negative examples has a big impact on the model's outcome. We devise a negative sampling technique that not only improves the accuracy of the model but also facilitates the decentralization of the recommendation system. The experimental results obtained using the MIND dataset demonstrate that the accuracy of the method under consideration can compete with that of State-of-the-Art models. The utilization of the sampling technique is essential in reducing model complexity and accelerating the training process, while maintaining a high level of accuracy. Finally, we discuss how decentralized models can help improve privacy and scalability.
Can sparse autoencoders be used to decompose and interpret steering vectors?
Venue: arXiv cs.LG 2024
Year: 2024

Steering vectors are a promising approach to control the behaviour of large language models. However, their underlying mechanisms remain poorly understood. While sparse autoencoders (SAEs) may offer a potential method to interpret steering vectors, recent findings show that SAE-reconstructed vectors often lack the steering properties of the original vectors. This paper investigates why directly applying SAEs to steering vectors yields misleading decompositions, identifying two reasons: (1) steering vectors fall outside the input distribution for which SAEs are designed, and (2) steering vectors can have meaningful negative projections in feature directions, which SAEs are not designed to accommodate. These limitations hinder the direct use of SAEs for interpreting steering vectors.
An alignment problem
Venue: arXiv cs.DS 2024
Year: 2024

This work concerns an alignment problem that has applications in many geospatial problems such as resource allocation and building reliable disease maps. Here, we introduce the problem of optimally aligning $k$ collections of $m$ spatial supports over $n$ spatial units in a $d$-dimensional Euclidean space. We show that the 1-dimensional case is solvable in time polynomial in $k$, $m$ and $n$. We then show that the 2-dimensional case is NP-hard for 2 collections of 2 supports. Finally, we devise a heuristic for aligning a set of collections in the 2-dimensional case.
Evaluating World Models with LLM for Decision Making
Venue: arXiv cs.AI 2024
Year: 2024

World model emerges as a key module in decision making, where MuZero and Dreamer achieve remarkable successes in complex tasks. Recent work leverages Large Language Models (LLMs) as general world simulators to simulate the dynamics of the world due to their generalizability. LLMs also serve as the world model for deliberative reasoning in Reasoning via Planning (RAP) and Tree of Thought (ToT). However, the world models are either evaluated as a general world simulator, or as a functional module of the agent, i.e., predicting the transitions to assist the planning. In this work, we propose a comprehensive evaluation of the world models with LLMs from the decision making perspective. Specifically, we leverage the 31 diverse environments from (Wang et al., 2023;2024) and curate the rule-based policy of each environment for the diverse evaluation. Then, we design three main tasks, i.e., policy verification, action proposal, and policy planning, where the world models can be used for decision making solely. Finally, we conduct the comprehensive evaluation of the advanced LLMs, i.e., GPT-4o and GPT-4o-mini, on the environments for the three main tasks under various settings. The key observations include: i) GPT-4o significantly outperforms GPT-4o-mini on the three main tasks, especially for the tasks which require the domain knowledge, ii) the performance of the world model with LLM will be decreased for long-term decision-making tasks, and iii) the combination of different functionalities of the world model will brings additional unstabilities of the performance.
Spotlight Session on Autonomous Weapons Systems at ICRC 34th International Conference
Venue: arXiv cs.CY 2024
Year: 2024

Autonomous weapons systems (AWS) change the way humans make decisions, the effect of those decisions and who is accountable for decisions made. We must remain vigilant, informed and human-centred as we tackle our deliberations on developing norms regarding their development, use and justification. Ways to enhance compliance in international humanitarian law (IHL) include: Training weapons decision makers in IHL; developing best practice in weapons reviews including requirements for industry to ensure that any new weapon, means or method of warfare is capable of being used lawfully; develop human-centred test and evaluation methods; invest in digital infrastructure to increase knowledge of the civilian environment in a conflict and its dynamics; invest in research on the real effects and consequences of civilian harms to the achievement of military and political objectives; improve secure communications between stakeholders in a conflict; and finally to upskill governments and NGOs in what is technically achievable with emerging technologies so that they can contribute to system requirements, test and evaluation protocols and operational rules of use and engagement. Governments are responsible for setting requirements for weapons systems. They are responsible for driving ethicality as well as lethality. Governments can require systems to be made and used to better protect civilians and protected objects. The UN can advocate for compliance with IHL, human rights, human-centred use of weapons systems and improved mechanisms to monitor and trace military decision making including those decisions affected by autonomous functionality.
Auto-assessment of assessment: A conceptual framework towards fulfilling the policy gaps in academic assessment practices
Venue: arXiv cs.CY 2024
Year: 2024

Education is being transformed by rapid advances in Artificial Intelligence (AI), including emerging Generative Artificial Intelligence (GAI). Such technology can significantly support academics and students by automating monotonous tasks and making personalised suggestions. However, despite the potential of the technology, there are significant concerns regarding AI misuse, particularly by students in assessments. There are two schools of thought: one advocates for a complete ban on it, while the other views it as a valuable educational tool, provided it is governed by a robust usage policy. This contradiction clearly indicates a major policy gap in academic practices, and new policies are required to uphold academic standards while enabling staff and students to benefit from technological advancements. We surveyed 117 academics from three countries (UK, UAE, and Iraq), and identified that most academics retain positive opinions regarding AI in education. For example, the majority of experienced academics do not favour complete bans, and they see the potential benefits of AI for students, teaching staff, and academic institutions. Importantly, academics specifically identified the particular benefits of AI for autonomous assessment (71.79% of respondents agreed). Therefore, for the first time, we propose a novel AI framework for autonomously evaluating students' work (e.g., reports, coursework, etc.) and automatically assigning grades based on their knowledge and in-depth understanding of the submitted content. The survey results further highlight a significant lack of awareness of modern AI-based tools (e.g., ChatGPT) among experienced academics, a gap that must be addressed to uphold educational standards.
SoccerGuard: Investigating Injury Risk Factors for Professional Soccer Players with Machine Learning
Venue: arXiv cs.HC 2024
Year: 2024

We present SoccerGuard, a novel framework for predicting injuries in women's soccer using Machine Learning (ML). This framework can ingest data from multiple sources, including subjective wellness and training load reports from players, objective GPS sensor measurements, third-party player statistics, and injury reports verified by medical personnel. We experiment with a number of different settings related to synthetic data generation, input and output window sizes, and ML models for prediction. Our results show that, given the right configurations and feature combinations, injury event prediction can be undertaken with considerable accuracy. The optimal results are achieved when input windows are reduced and larger combined output windows are defined, in combination with an ideally balanced data set. The framework also includes a dashboard with a user-friendly Graphical User Interface (GUI) to support interactive analysis and visualization.