OpenRaiser/Intern-Atlas huggingface dataset:
Relax and Merge: A Simple Yet Effective Framework for Solving Fair $k$-Means and $k$-sparse Wasserstein Barycenter Problems
Venue: arXiv cs.LG 2024
Year: 2024

The fairness of clustering algorithms has gained widespread attention across various areas, including machine learning, In this paper, we study fair $k$-means clustering in Euclidean space. Given a dataset comprising several groups, the fairness constraint requires that each cluster should contain a proportion of points from each group within specified lower and upper bounds. Due to these fairness constraints, determining the optimal locations of $k$ centers is a quite challenging task. We propose a novel ``Relax and Merge'' framework that returns a $(1+4ρ+ O(ε))$-approximate solution, where $ρ$ is the approximate ratio of an off-the-shelf vanilla $k$-means algorithm and $O(ε)$ can be an arbitrarily small positive number. If equipped with a PTAS of $k$-means, our solution can achieve an approximation ratio of $(5+O(ε))$ with only a slight violation of the fairness constraints, which improves the current state-of-the-art approximation guarantee. Furthermore, using our framework, we can also obtain a $(1+4ρ+O(ε))$-approximate solution for the $k$-sparse Wasserstein Barycenter problem, which is a fundamental optimization problem in the field of optimal transport, and a $(2+6ρ)$-approximate solution for the strictly fair $k$-means clustering with no violation, both of which are better than the current state-of-the-art methods. In addition, the empirical results demonstrate that our proposed algorithm can significantly outperform baseline approaches in terms of clustering cost.
Hedging and Pricing Structured Products Featuring Multiple Underlying Assets
Venue: arXiv cs.CE 2024
Year: 2024

Hedging a portfolio containing autocallable notes presents unique challenges due to the complex risk profile of these financial instruments. In addition to hedging, pricing these notes, particularly when multiple underlying assets are involved, adds another layer of complexity. Pricing autocallable notes involves intricate considerations of various risk factors, including underlying assets, interest rates, and volatility. Traditional pricing methods, such as sample-based Monte Carlo simulations, are often time-consuming and impractical for long maturities, particularly when there are multiple underlying assets. In this paper, we explore autocallable structured notes with three underlying assets and proposes a machine learning-based pricing method that significantly improves efficiency, computing prices 250 times faster than traditional Monte Carlo simulation based method. Additionally, we introduce a Distributional Reinforcement Learning (RL) algorithm to hedge a portfolio containing an autocallable structured note. Our distributional RL based hedging strategy provides better PnL compared to traditional Delta-neutral and Delta-Gamma neutral hedging strategies. The VaR 5% (PnL value) of our RL agent based hedging is 33.95, significantly outperforming both the Delta neutral strategy, which has a VaR 5% of -0.04, and the Delta-Gamma neutral strategy, which has a VaR 5% of 13.05. It also provides the hedging action with better left tail PnL, such as 95% and 99% value-at-risk (VaR) and conditional value-at-risk (CVaR), highlighting its potential for front-office hedging and risk management.
Data movement limits to frontier model training
Venue: arXiv cs.DC 2024
Year: 2024

We present a theoretical model of distributed training, and use it to analyze how far dense and sparse training runs can be scaled. Under our baseline assumptions, given a three month training duration, data movement bottlenecks begin to significantly lower hardware utilization for training runs exceeding about $10^{28}$ FLOP, two orders of magnitude above the largest training run to date, suggesting the arrival of fundamental barriers to scaling in three years given recent rates of growth. A training run exceeding about $10^{31}$ FLOP is infeasible even at low utilization. However, more aggressive batch size scaling and/or shorter and fatter model shapes, if achievable, have the potential to permit much larger training runs.
Real-Time Spatio-Temporal Reconstruction of Dynamic Endoscopic Scenes with 4D Gaussian Splatting
Venue: arXiv cs.CV 2024
Year: 2024

Dynamic scene reconstruction is essential in robotic minimally invasive surgery, providing crucial spatial information that enhances surgical precision and outcomes. However, existing methods struggle to address the complex, temporally dynamic nature of endoscopic scenes. This paper presents ST-Endo4DGS, a novel framework that models the spatio-temporal volume of dynamic endoscopic scenes using unbiased 4D Gaussian Splatting (4DGS) primitives, parameterized by anisotropic ellipses with flexible 4D rotations. This approach enables precise representation of deformable tissue dynamics, capturing intricate spatial and temporal correlations in real time. Additionally, we extend spherindrical harmonics to represent time-evolving appearance, achieving realistic adaptations to lighting and view changes. A new endoscopic normal alignment constraint (ENAC) further enhances geometric fidelity by aligning rendered normals with depth-derived geometry. Extensive evaluations show that ST-Endo4DGS outperforms existing methods in both visual quality and real-time performance, establishing a new state-of-the-art in dynamic scene reconstruction for endoscopic surgery.
TDS Simulator: A MATLAB App to model temperature-programmed hydrogen desorption
Venue: arXiv cs.CE 2024
Year: 2024

We present TDS Simulator, a new software tool aimed at modelling thermal desorption spectroscopy (TDS) experiments. TDS is a widely used technique for quantifying key characteristics of hydrogen-material interactions, such as diffusivity and trapping. However, interpreting the output of TDS experiments is non-trivial and requires appropriate post-processing tools. This work introduces the first software tool capable of simulating TDS curves for arbitrary choices of material parameters and hydrogen trap characteristics, using the primary hydrogen diffusion and trapping models (Oriani, McNabb-Foster). Moreover, TDS Simulator contains a specific functionality for loading experimental TDS data and conducting the inverse calibration of a selected transport model, providing automatic estimates of the density and binding energy of each hydrogen trap type in the material. In its first version, TDS Simulator is provided as a MATLAB App, which is made freely available to the community and provides a simple graphical user interface (GUI) to make use of TDS Simulator straightforward. As reported in the present manuscript, the outputs of TDS Simulator have been extensively validated against literature data. Demonstrations of automatic determination of trap characteristics from experimental data through the optimisation tool are also provided. The present work enables an efficient and straightforward characterisation of hydrogen-material characteristics relevant to multiple applications, from nuclear fusion to the development of hydrogen-compatible materials for the hydrogen economy. TDS Simulator can be downloaded from https://mechmat.web.ox.ac.uk/codes.
AutoPT: How Far Are We from the End2End Automated Web Penetration Testing?
Venue: arXiv cs.CR 2024
Year: 2024

Penetration testing is essential to ensure Web security, which can detect and fix vulnerabilities in advance, and prevent data leakage and serious consequences. The powerful inference capabilities of large language models (LLMs) have made significant progress in various fields, and the development potential of LLM-based agents can revolutionize the cybersecurity penetration testing industry. In this work, we establish a comprehensive end-to-end penetration testing benchmark using a real-world penetration testing environment to explore the capabilities of LLM-based agents in this domain. Our results reveal that the agents are familiar with the framework of penetration testing tasks, but they still face limitations in generating accurate commands and executing complete processes. Accordingly, we summarize the current challenges, including the difficulty of maintaining the entire message history and the tendency for the agent to become stuck. Based on the above insights, we propose a Penetration testing State Machine (PSM) that utilizes the Finite State Machine (FSM) methodology to address these limitations. Then, we introduce AutoPT, an automated penetration testing agent based on the principle of PSM driven by LLMs, which utilizes the inherent inference ability of LLM and the constraint framework of state machines. Our evaluation results show that AutoPT outperforms the baseline framework ReAct on the GPT-4o mini model and improves the task completion rate from 22% to 41% on the benchmark target. Compared with the baseline framework and manual work, AutoPT also reduces time and economic costs further. Hence, our AutoPT has facilitated the development of automated penetration testing and significantly impacted both academia and industry.
An Innovative CGL-MHA Model for Sarcasm Sentiment Recognition Using the MindSpore Framework
Venue: arXiv cs.CL 2024
Year: 2024

The pervasive use of the Internet and social media introduces significant challenges to automated sentiment analysis, particularly for sarcastic expressions in user-generated content. Sarcasm conveys negative emotions through ostensibly positive or exaggerated language, complicating its detection within natural language processing tasks. To address this, we propose an innovative sarcasm detection model integrating Convolutional Neural Networks (CNN), Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM), and Multi-Head Attention mechanisms. The CNN component captures local n-gram features, while GRU and LSTM layers model sequential dependencies and contextual information. Multi-Head Attention enhances the model's focus on relevant parts of the input, improving interpretability. Experiments on two sarcasm detection datasets, Headlines and Riloff, demonstrate that the model achieves an accuracy of 81.20% and an F1 score of 80.77% on Headlines, and an accuracy of 79.72% with an F1 score of 61.39% on Riloff, outperforming traditional models. These results validate the effectiveness of our hybrid approach for sarcasm detection in social media texts.
Conformalized High-Density Quantile Regression via Dynamic Prototypes-based Probability Density Estimation
Venue: arXiv cs.LG 2024
Year: 2024

Recent methods in quantile regression have adopted a classification perspective to handle challenges posed by heteroscedastic, multimodal, or skewed data by quantizing outputs into fixed bins. Although these regression-as-classification frameworks can capture high-density prediction regions and bypass convex quantile constraints, they are restricted by quantization errors and the curse of dimensionality due to a constant number of bins per dimension. To address these limitations, we introduce a conformalized high-density quantile regression approach with a dynamically adaptive set of prototypes. Our method optimizes the set of prototypes by adaptively adding, deleting, and relocating quantization bins throughout the training process. Moreover, our conformal scheme provides valid coverage guarantees, focusing on regions with the highest probability density. Experiments across diverse datasets and dimensionalities confirm that our method consistently achieves high-quality prediction regions with enhanced coverage and robustness, all while utilizing fewer prototypes and memory, ensuring scalability to higher dimensions. The code is available at https://github.com/batuceng/max_quantile .
NLP and Education: using semantic similarity to evaluate filled gaps in a large-scale Cloze test in the classroom
Venue: arXiv cs.CL 2024
Year: 2024

This study examines the applicability of the Cloze test, a widely used tool for assessing text comprehension proficiency, while highlighting its challenges in large-scale implementation. To address these limitations, an automated correction approach was proposed, utilizing Natural Language Processing (NLP) techniques, particularly word embeddings (WE) models, to assess semantic similarity between expected and provided answers. Using data from Cloze tests administered to students in Brazil, WE models for Brazilian Portuguese (PT-BR) were employed to measure the semantic similarity of the responses. The results were validated through an experimental setup involving twelve judges who classified the students' answers. A comparative analysis between the WE models' scores and the judges' evaluations revealed that GloVe was the most effective model, demonstrating the highest correlation with the judges' assessments. This study underscores the utility of WE models in evaluating semantic similarity and their potential to enhance large-scale Cloze test assessments. Furthermore, it contributes to educational assessment methodologies by offering a more efficient approach to evaluating reading proficiency.
Task-Oriented Hierarchical Object Decomposition for Visuomotor Control
Venue: arXiv cs.RO 2024
Year: 2024

Good pre-trained visual representations could enable robots to learn visuomotor policy efficiently. Still, existing representations take a one-size-fits-all-tasks approach that comes with two important drawbacks: (1) Being completely task-agnostic, these representations cannot effectively ignore any task-irrelevant information in the scene, and (2) They often lack the representational capacity to handle unconstrained/complex real-world scenes. Instead, we propose to train a large combinatorial family of representations organized by scene entities: objects and object parts. This hierarchical object decomposition for task-oriented representations (HODOR) permits selectively assembling different representations specific to each task while scaling in representational capacity with the complexity of the scene and the task. In our experiments, we find that HODOR outperforms prior pre-trained representations, both scene vector representations and object-centric representations, for sample-efficient imitation learning across 5 simulated and 5 real-world manipulation tasks. We further find that the invariances captured in HODOR are inherited into downstream policies, which can robustly generalize to out-of-distribution test conditions, permitting zero-shot skill chaining. Appendix, code, and videos: https://sites.google.com/view/hodor-corl24.
Regret of exploratory policy improvement and $q$-learning
Venue: arXiv cs.LG 2024
Year: 2024

We study the convergence of $q$-learning and related algorithms introduced by Jia and Zhou (J. Mach. Learn. Res., 24 (2023), 161) for controlled diffusion processes. Under suitable conditions on the growth and regularity of the model parameters, we provide a quantitative error and regret analysis of both the exploratory policy improvement algorithm and the $q$-learning algorithm.
Advancing Biomedical Signal Security: Real-Time ECG Monitoring with Chaotic Encryption
Venue: arXiv cs.CR 2024
Year: 2024

The real time analysis and secure transmission of electrocardiogram (ECG) signals are critical for ensuring both effective medical diagnosis and patient data privacy. In this study, we developed a real time ECG monitoring system that integrates chaotic encryption to protect the integrity and confidentiality of ECG signals during acquisition, transmission, and storage. By leveraging the logistic map as the chaotic function for encryption, our system offers a highly secure framework that dynamically encrypts ECG signals without adding significant latency. To validate the system's reliability, we applied a series of security tests. The results demonstrate that chaotic encryption is effective in enhancing data security, as evidenced by high entropy values and strong key sensitivity, ensuring protection against common cryptographic attacks. Additionally, the system's real time disease detection model, based on deep learning, operates seamlessly with encrypted data, providing accurate diagnosis without compromising security. Our findings indicate that chaotic encryption, paired with real time analysis, is a powerful method for protecting sensitive medical data, making this approach particularly relevant for telemedicine and remote patient monitoring applications. The success of this system highlights its potential for broader application to other biomedical signals, providing a secure infrastructure for the future of digital health.
From Federated Learning to Quantum Federated Learning for Space-Air-Ground Integrated Networks
Venue: arXiv cs.LG 2024
Year: 2024

6G wireless networks are expected to provide seamless and data-based connections that cover space-air-ground and underwater networks. As a core partition of future 6G networks, Space-Air-Ground Integrated Networks (SAGIN) have been envisioned to provide countless real-time intelligent applications. To realize this, promoting AI techniques into SAGIN is an inevitable trend. Due to the distributed and heterogeneous architecture of SAGIN, federated learning (FL) and then quantum FL are emerging AI model training techniques for enabling future privacy-enhanced and computation-efficient SAGINs. In this work, we explore the vision of using FL/QFL in SAGINs. We present a few representative applications enabled by the integration of FL and QFL in SAGINs. A case study of QFL over UAV networks is also given, showing the merit of quantum-enabled training approach over the conventional FL benchmark. Research challenges along with standardization for QFL adoption in future SAGINs are also highlighted.
FEED: Fairness-Enhanced Meta-Learning for Domain Generalization
Venue: arXiv cs.LG 2024
Year: 2024

Generalizing to out-of-distribution data while being aware of model fairness is a significant and challenging problem in meta-learning. The goal of this problem is to find a set of fairness-aware invariant parameters of classifier that is trained using data drawn from a family of related training domains with distribution shift on non-sensitive features as well as different levels of dependence between model predictions and sensitive features so that the classifier can achieve good generalization performance on unknown but distinct test domains. To tackle this challenge, existing state-of-the-art methods either address the domain generalization problem but completely ignore learning with fairness or solely specify shifted domains with various fairness levels. This paper introduces an approach to fairness-aware meta-learning that significantly enhances domain generalization capabilities. Our framework, Fairness-Enhanced Meta-Learning for Domain Generalization (FEED), disentangles latent data representations into content, style, and sensitive vectors. This disentanglement facilitates the robust generalization of machine learning models across diverse domains while adhering to fairness constraints. Unlike traditional methods that focus primarily on domain invariance or sensitivity to shifts, our model integrates a fairness-aware invariance criterion directly into the meta-learning process. This integration ensures that the learned parameters uphold fairness consistently, even when domain characteristics vary widely. We validate our approach through extensive experiments across multiple benchmarks, demonstrating not only superior performance in maintaining high accuracy and fairness but also significant improvements over existing state-of-the-art methods in domain generalization tasks.
The Case for an Industrial Policy Approach to AI Sector of Pakistan for Growth and Autonomy
Venue: arXiv cs.CY 2024
Year: 2024

This paper argues for the strategic treatment of artificial intelligence as a key industry within broader industrial policy framework of Pakistan, underscoring the importance of aligning it with national goals such as economic resilience and preservation of autonomy. The paper starts with defining industrial policy as a set of targeted government interventions to shape specific sectors for strategic outcomes and argues for its application to AI in Pakistan due to its huge potential, the risks of unregulated adoption, and prevailing market inefficiencies. The paper conceptualizes AI as a layered ecosystem, comprising foundational infrastructure, core computing, development platforms, and service and product layers, supported by education, government policy, and research and development. The analysis highlights that AI sector of Pakistan is predominantly service oriented, with limited product innovation and dependence on foreign technologies, posing risks to economic independence, national security, and employment. To address these challenges, the paper recommends educational reforms, support for local AI product development, initiatives for indigenous cloud and hardware capabilities, and public-private collaborations on foundational models. Additionally, it advocates for public procurement policies and infrastructure incentives to foster local solutions and reduce reliance on foreign providers. This strategy aims to position Pakistan as a competitive, autonomous player in the global AI ecosystem.
Artificial Intelligence Driven Course Generation: A Case Study Using ChatGPT
Venue: arXiv cs.CL 2024
Year: 2024

This study explores Artificial Intelligence use, specifically ChatGPT, in creating educational content. The study aims to elaborate on using ChatGPT to create course materials. The main objective is to assess the efficiency, quality, and impact of AI-driven course generation, and to create a Multimedia Databases course as a case study. The study highlights the potential of AI to revolutionize educational content creation, making it more accessible, personalized, and efficient. The course content was generated in less than one day through iterative methods, using prompts for translation, content expansion, practical examples, assignments, supplementary materials, and LaTeX formatting. Each part was verified immediately after generation to ensure accuracy. Post-generation analysis with Detectia and Turnitin showed similarity rates of 8.7% and 13%, indicating high originality. Experts and university committees reviewed and approved the course, with English university teachers praising its language quality. ChatGPT also created a well-structured and diversified exam for the module. Key findings reveal significant time efficiency, comprehensive content coverage, and high flexibility. The study underscores AI's transformative potential in education, addressing challenges related to data privacy, technology dependence, content accuracy, and algorithmic biases. The conclusions emphasize the need for collaboration between educators, policymakers, and technology developers to harness AI's benefits in education fully.