OpenRaiser/Intern-Atlas huggingface dataset:
Y-MAP-Net: Real-time depth, normals, segmentation, multi-label captioning and 2D human pose in RGB images
Venue: arXiv cs.CV 2024
Year: 2024

We present Y-MAP-Net, a Y-shaped neural network architecture designed for real-time multi-task learning on RGB images. Y-MAP-Net, simultaneously predicts depth, surface normals, human pose, semantic segmentation and generates multi-label captions, all from a single network evaluation. To achieve this, we adopt a multi-teacher, single-student training paradigm, where task-specific foundation models supervise the network's learning, enabling it to distill their capabilities into a lightweight architecture suitable for real-time applications. Y-MAP-Net, exhibits strong generalization, simplicity and computational efficiency, making it ideal for robotics and other practical scenarios. To support future research, we will release our code publicly.
Interactive Image-Based Aphid Counting in Yellow Water Traps under Stirring Actions
Venue: arXiv cs.CV 2024
Year: 2024

The current vision-based aphid counting methods in water traps suffer from undercounts caused by occlusions and low visibility arising from dense aggregation of insects and other objects. To address this problem, we propose a novel aphid counting method through interactive stirring actions. We use interactive stirring to alter the distribution of aphids in the yellow water trap and capture a sequence of images which are then used for aphid detection and counting through an optimized small object detection network based on Yolov5. We also propose a counting confidence evaluation system to evaluate the confidence of count-ing results. The final counting result is a weighted sum of the counting results from all sequence images based on the counting confidence. Experimental results show that our proposed aphid detection network significantly outperforms the original Yolov5, with improvements of 33.9% in AP@0.5 and 26.9% in AP@[0.5:0.95] on the aphid test set. In addition, the aphid counting test results using our proposed counting confidence evaluation system show significant improvements over the static counting method, closely aligning with manual counting results.
Deep Learning for Micro-Scale Crack Detection on Imbalanced Datasets Using Key Point Localization
Venue: arXiv cs.CV 2024
Year: 2024

Internal crack detection has been a subject of focus in structural health monitoring. By focusing on crack detection in structural datasets, it is demonstrated that deep learning (DL) methods can effectively analyze seismic wave fields interacting with micro-scale cracks, which are beyond the resolution of conventional visual inspection. This work explores a novel application of DL-based key point detection technique, where cracks are localized by predicting the coordinates of four key points that define a bounding region of the crack. The study not only opens new research directions for non-visual applications but also effectively mitigates the impact of imbalanced data which poses a challenge for previous DL models, as it can be biased toward predicting the majority class (non-crack regions). Popular DL techniques, such as the Inception blocks, are used and investigated. The model shows an overall reduction in loss when applied to micro-scale crack detection and is reflected in the lower average deviation between the location of actual and predicted cracks, with an average Intersection over Union (IoU) being 0.511 for all micro cracks (greater than 0.00 micrometers) and 0.631 for larger micro cracks (greater than 4 micrometers).
Fair Division via the Cake-Cutting Share
Venue: arXiv cs.GT 2024
Year: 2024

In this paper, we consider the classic fair division problem of allocating $m$ divisible items to $n$ agents with linear valuations over the items. We define novel notions of fair shares from the perspective of individual agents via the cake-cutting process. These shares generalize the notion of proportionality by taking into account the valuations of other agents via constraints capturing envy. We study what fraction (approximation) of these shares are achievable in the worst case, and present tight and non-trivial approximation bounds as a function of $n$ and $m$. In particular, we show a tight approximation bound of $Θ(\sqrt{n})$ for various notions of such shares. We show this bound via a novel application of dual fitting, which may be of independent interest. We also present a bound of $O(m^{2/3})$ for a strict notion of share, with an almost matching lower bound. We further develop weaker notions of shares whose approximation bounds interpolate smoothly between proportionality and the shares described above. We finally present empirical results showing that our definitions lead to more reasonable shares than the standard fair share notion of proportionality.
LLM-itation is the Sincerest Form of Data: Generating Synthetic Buggy Code Submissions for Computing Education
Venue: arXiv cs.CY 2024
Year: 2024

There is a great need for data in computing education research. Data is needed to understand how students behave, to train models of student behavior to optimally support students, and to develop and validate new assessment tools and learning analytics techniques. However, relatively few computing education datasets are shared openly, often due to privacy regulations and issues in making sure the data is anonymous. Large language models (LLMs) offer a promising approach to create large-scale, privacy-preserving synthetic data, which can be used to explore various aspects of student learning, develop and test educational technologies, and support research in areas where collecting real student data may be challenging or impractical. This work explores generating synthetic buggy code submissions for introductory programming exercises using GPT-4o. We compare the distribution of test case failures between synthetic and real student data from two courses to analyze the accuracy of the synthetic data in mimicking real student data. Our findings suggest that LLMs can be used to generate synthetic incorrect submissions that are not significantly different from real student data with regard to test case failure distributions. Our research contributes to the development of reliable synthetic datasets for computing education research and teaching, potentially accelerating progress in the field while preserving student privacy.
IUMENTA: A generic framework for animal digital twins within the Open Digital Twin Platform
Venue: arXiv cs.OH 2024
Year: 2024

IUMENTA (Latin for livestock) is an innovative software framework designed to construct and simulate digital twins of animals. By leveraging the powerful capability of the Open Digital Twin Platform (ODTP) alongside advanced software sensors, IUMENTA offers researchers a user-friendly tool to seamlessly develop adaptive digital replicas of animal-based processes. This framework establishes a dynamic ecosystem that integrates insights from diverse experiments, consequently enhancing our understanding of animal behavioural and physiological responses. Through real-time tracking of an animal's energy balance. IUMENTA provides valuable insights into metabolic rates, nutritional needs, emotional states, and overall well-being of animals. In this article, we explore the application of the IUMENTA framework in developing a digital twin focused on the animal's energy balance. IUMENTA includes the EnergyTag system, a state-of-the-art wearable software sensor, which facilitates real-time monitoring of energy expenditure, allowing for continuous updates and personalisation of the energy balance digital twin.
PhDGPT: Introducing a psychometric and linguistic dataset about how large language models perceive graduate students and professors in psychology
Venue: arXiv cs.HC 2024
Year: 2024

Machine psychology aims to reconstruct the mindset of Large Language Models (LLMs), i.e. how these artificial intelligences perceive and associate ideas. This work introduces PhDGPT, a prompting framework and synthetic dataset that encapsulates the machine psychology of PhD researchers and professors as perceived by OpenAI's GPT-3.5. The dataset consists of 756,000 datapoints, counting 300 iterations repeated across 15 academic events, 2 biological genders, 2 career levels and 42 unique item responses of the Depression, Anxiety, and Stress Scale (DASS-42). PhDGPT integrates these psychometric scores with their explanations in plain language. This synergy of scores and texts offers a dual, comprehensive perspective on the emotional well-being of simulated academics, e.g. male/female PhD students or professors. By combining network psychometrics and psycholinguistic dimensions, this study identifies several similarities and distinctions between human and LLM data. The psychometric networks of simulated male professors do not differ between physical and emotional anxiety subscales, unlike humans. Other LLMs' personification can reconstruct human DASS factors with a purity up to 80%. Furthemore, LLM-generated personifications across different scenarios are found to elicit explanations lower in concreteness and imageability in items coding for anxiety, in agreement with past studies about human psychology. Our findings indicate an advanced yet incomplete ability for LLMs to reproduce the complexity of human psychometric data, unveiling convenient advantages and limitations in using LLMs to replace human participants. PhDGPT also intriguingly capture the ability for LLMs to adapt and change language patterns according to prompted mental distress contextual features, opening new quantitative opportunities for assessing the machine psychology of these artificial intelligences.
Hateful Meme Detection through Context-Sensitive Prompting and Fine-Grained Labeling
Venue: arXiv cs.CV 2024
Year: 2024

The prevalence of multi-modal content on social media complicates automated moderation strategies. This calls for an enhancement in multi-modal classification and a deeper understanding of understated meanings in images and memes. Although previous efforts have aimed at improving model performance through fine-tuning, few have explored an end-to-end optimization pipeline that accounts for modalities, prompting, labeling, and fine-tuning. In this study, we propose an end-to-end conceptual framework for model optimization in complex tasks. Experiments support the efficacy of this traditional yet novel framework, achieving the highest accuracy and AUROC. Ablation experiments demonstrate that isolated optimizations are not ineffective on their own.
Architectural Patterns for Designing Quantum Artificial Intelligence Systems
Venue: arXiv cs.SE 2024
Year: 2024

Utilising quantum computing technology to enhance artificial intelligence systems is expected to improve training and inference times, increase robustness against noise and adversarial attacks, and reduce the number of parameters without compromising accuracy. However, moving beyond proof-of-concept or simulations to develop practical applications of these systems while ensuring high software quality faces significant challenges due to the limitations of quantum hardware and the underdeveloped knowledge base in software engineering for such systems. In this work, we have conducted a systematic mapping study to identify the challenges and solutions associated with the software architecture of quantum-enhanced artificial intelligence systems. The results of the systematic mapping study reveal several architectural patterns that describe how quantum components can be integrated into inference engines, as well as middleware patterns that facilitate communication between classical and quantum components. Each pattern realises a trade-off between various software quality attributes, such as efficiency, scalability, trainability, simplicity, portability, and deployability. The outcomes of this work have been compiled into a catalogue of architectural patterns.
Guided Learning: Lubricating End-to-End Modeling for Multi-stage Decision-making
Venue: arXiv cs.LG 2024
Year: 2024

Multi-stage decision-making is crucial in various real-world artificial intelligence applications, including recommendation systems, autonomous driving, and quantitative investment systems. In quantitative investment, for example, the process typically involves several sequential stages such as factor mining, alpha prediction, portfolio optimization, and sometimes order execution. While state-of-the-art end-to-end modeling aims to unify these stages into a single global framework, it faces significant challenges: (1) training such a unified neural network consisting of multiple stages between initial inputs and final outputs often leads to suboptimal solutions, or even collapse, and (2) many decision-making scenarios are not easily reducible to standard prediction problems. To overcome these challenges, we propose Guided Learning, a novel methodological framework designed to enhance end-to-end learning in multi-stage decision-making. We introduce the concept of a ``guide'', a function that induces the training of intermediate neural network layers towards some phased goals, directing gradients away from suboptimal collapse. For decision scenarios lacking explicit supervisory labels, we incorporate a utility function that quantifies the ``reward'' of the throughout decision. Additionally, we explore the connections between Guided Learning and classic machine learning paradigms such as supervised, unsupervised, semi-supervised, multi-task, and reinforcement learning. Experiments on quantitative investment strategy building demonstrate that guided learning significantly outperforms both traditional stage-wise approaches and existing end-to-end methods.
Debias-CLR: A Contrastive Learning Based Debiasing Method for Algorithmic Fairness in Healthcare Applications
Venue: arXiv cs.LG 2024
Year: 2024

Artificial intelligence based predictive models trained on the clinical notes can be demographically biased. This could lead to adverse healthcare disparities in predicting outcomes like length of stay of the patients. Thus, it is necessary to mitigate the demographic biases within these models. We proposed an implicit in-processing debiasing method to combat disparate treatment which occurs when the machine learning model predict different outcomes for individuals based on the sensitive attributes like gender, ethnicity, race, and likewise. For this purpose, we used clinical notes of heart failure patients and used diagnostic codes, procedure reports and physiological vitals of the patients. We used Clinical BERT to obtain feature embeddings within the diagnostic codes and procedure reports, and LSTM autoencoders to obtain feature embeddings within the physiological vitals. Then, we trained two separate deep learning contrastive learning frameworks, one for gender and the other for ethnicity to obtain debiased representations within those demographic traits. We called this debiasing framework Debias-CLR. We leveraged clinical phenotypes of the patients identified in the diagnostic codes and procedure reports in the previous study to measure fairness statistically. We found that Debias-CLR was able to reduce the Single-Category Word Embedding Association Test (SC-WEAT) effect size score when debiasing for gender and ethnicity. We further found that to obtain fair representations in the embedding space using Debias-CLR, the accuracy of the predictive models on downstream tasks like predicting length of stay of the patients did not get reduced as compared to using the un-debiased counterparts for training the predictive models. Hence, we conclude that our proposed approach, Debias-CLR is fair and representative in mitigating demographic biases and can reduce health disparities.
A Combined Environmental Monitoring Framework based on WSN Clustering and VANET Edge Computation Offloading
Venue: arXiv cs.NI 2024
Year: 2024

Wireless Sensor Networks (WSN) and Vehicular Ad-hoc Networks (VANET) have been extensively used in IoT applications for environmental monitoring, especially in rural and agricultural areas. In this paper we present a novel combined approach which uses both WSN and VANET clustered structures for the efficient gathering and processing of environmental parameters in long eNodeB/RSU-enabled roads/highways, which cross large rural areas. The former (WSNs) is used for traditional sensing and data gathering, whereas the latter (VANET) is used for both (a) performing intermediate processing based on modern edge computation offloading techniques, and (b) propagating the data (either raw data or computed results) to the residing eNodeB/RSUs. Extended experimental measurements, taken via the combined use of SUMO and Veins simulation platforms (and focusing in the air quality monitoring experimental case), demonstrate the high efficiency and scalability of the presented approach over very large deployment areas.
The Sixth Generation of the Perseus Digital Library and a Workflow for Open Philology -- DRAFT
Venue: arXiv cs.DL 2024
Year: 2024

We report here on the workflow that we needed to develop in order to integrate the growing range of openly licensed, born-digital and, increasingly, machine actionable publications. Our developmental work focused upon textual data for Ancient Greek, Latin, Old English, Classical Arabic and Classical Persian but the challenges that we have had to address are relevant to sources in a wide range of languages, ancient and modern. Perseus 6 was designed to be a publishing workflow that organizes complementary data into an integrated reading environment. This document focuses on the ways in which we have organized the data and describes the current state of ATLAS (Aligned Text and Linguistic Annotation Server) architecture. While this is the sixth version of the Perseus Digital Library, Perseus 6 represents a major step beyond its predecessors. Where Perseus 5 (described below) can represent and integrate digital versions of print editions (e.g., critical editions with interactive textual notes, links to lexicon and commentary entries), Perseus 6 was designed to bring together an expandable range of born-digital classes of annotation. An online ATLAS server with some initial functionality is now online and public services will expand during the rest of 2024. Most of the ATLAS data is, however, now available on Github and that data will be the focus on this paper in its current version.
KAT to KANs: A Review of Kolmogorov-Arnold Networks and the Neural Leap Forward
Venue: arXiv cs.LG 2024
Year: 2024

The curse of dimensionality poses a significant challenge to modern multilayer perceptron-based architectures, often causing performance stagnation and scalability issues. Addressing this limitation typically requires vast amounts of data. In contrast, Kolmogorov-Arnold Networks have gained attention in the machine learning community for their bold claim of being unaffected by the curse of dimensionality. This paper explores the Kolmogorov-Arnold representation theorem and the mathematical principles underlying Kolmogorov-Arnold Networks, which enable their scalability and high performance in high-dimensional spaces. We begin with an introduction to foundational concepts necessary to understand Kolmogorov-Arnold Networks, including interpolation methods and Basis-splines, which form their mathematical backbone. This is followed by an overview of perceptron architectures and the Universal approximation theorem, a key principle guiding modern machine learning. This is followed by an overview of the Kolmogorov-Arnold representation theorem, including its mathematical formulation and implications for overcoming dimensionality challenges. Next, we review the architecture and error-scaling properties of Kolmogorov-Arnold Networks, demonstrating how these networks achieve true freedom from the curse of dimensionality. Finally, we discuss the practical viability of Kolmogorov-Arnold Networks, highlighting scenarios where their unique capabilities position them to excel in real-world applications. This review aims to offer insights into Kolmogorov-Arnold Networks' potential to redefine scalability and performance in high-dimensional learning tasks.
Understanding Learning with Sliced-Wasserstein Requires Rethinking Informative Slices
Venue: arXiv cs.LG 2024
Year: 2024

The practical applications of Wasserstein distances (WDs) are constrained by their sample and computational complexities. Sliced-Wasserstein distances (SWDs) provide a workaround by projecting distributions onto one-dimensional subspaces, leveraging the more efficient, closed-form WDs for one-dimensional distributions. However, in high dimensions, most random projections become uninformative due to the concentration of measure phenomenon. Although several SWD variants have been proposed to focus on \textit{informative} slices, they often introduce additional complexity, numerical instability, and compromise desirable theoretical (metric) properties of SWD. Amidst the growing literature that focuses on directly modifying the slicing distribution, which often face challenges, we revisit the classical Sliced-Wasserstein and propose instead to rescale the 1D Wasserstein to make all slices equally informative. Importantly, we show that with an appropriate data assumption and notion of \textit{slice informativeness}, rescaling for all individual slices simplifies to \textbf{a single global scaling factor} on the SWD. This, in turn, translates to the standard learning rate search for gradient-based learning in common machine learning workflows. We perform extensive experiments across various machine learning tasks showing that the classical SWD, when properly configured, can often match or surpass the performance of more complex variants. We then answer the following question: "Is Sliced-Wasserstein all you need for common learning tasks?"
AutoIoT: Automated IoT Platform Using Large Language Models
Venue: arXiv cs.CR 2024
Year: 2024

IoT platforms, particularly smart home platforms providing significant convenience to people's lives such as Apple HomeKit and Samsung SmartThings, allow users to create automation rules through trigger-action programming. However, some users may lack the necessary knowledge to formulate automation rules, thus preventing them from fully benefiting from the conveniences offered by smart home technology. To address this, smart home platforms provide pre-defined automation policies based on the smart home devices registered by the user. Nevertheless, these policies, being pre-generated and relatively simple, fail to adequately cover the diverse needs of users. Furthermore, conflicts may arise between automation rules, and integrating conflict detection into the IoT platform increases the burden on developers. In this paper, we propose AutoIoT, an automated IoT platform based on Large Language Models (LLMs) and formal verification techniques, designed to achieve end-to-end automation through device information extraction, LLM-based rule generation, conflict detection, and avoidance. AutoIoT can help users generate conflict-free automation rules and assist developers in generating codes for conflict detection, thereby enhancing their experience. A code adapter has been designed to separate logical reasoning from the syntactic details of code generation, enabling LLMs to generate code for programming languages beyond their training data. Finally, we evaluated the performance of AutoIoT and presented a case study demonstrating how AutoIoT can integrate with existing IoT platforms.