OpenRaiser/Intern-Atlas huggingface dataset:
Automated Detection of Inter-Language Design Smells in Multi-Language Deep Learning Frameworks
Venue: arXiv cs.SE 2024
Year: 2024

Nowadays, most DL frameworks (DLFs) use multilingual programming of Python and C/C++, facilitating the flexibility and performance of the DLF. However, inappropriate interlanguage interaction may introduce design smells involving multiple programming languages (PLs), i.e., Inter-Language Design Smells (ILDS). Despite the negative impact of ILDS on multi-language DLFs, there is a lack of an automated approach for detecting ILDS in multi-language DLFs and a comprehensive understanding on ILDS in such DLFs. This work automatically detects ILDS in multi-language DLFs written in the combination of Python and C/C++, and to obtain a understanding on such ILDS in DLFs. We first developed an approach to automatically detecting ILDS in the multi-language DLFs written in the combination of Python and C/C++, including a number of ILDS and their detection rules defined based on inter-language communication mechanisms and code analysis. We then developed the CPSMELL tool that implements detection rules for automatically detecting such ILDS, and manually validated the accuracy of the tool. Finally, we performed a study to evaluate the ILDS in multi-language DLFs. We proposed seven ILDS and achieved an accuracy of 98.17% in the manual validation of CPSMELL in 5 popular multi-language DLFs. The study results revealed that among the 5 DLFs, TensorFlow, PyTorch, and PaddlePaddle exhibit relatively high prevalence of ILDS; each smelly file contains around 5 ILDS instances on average, with ILDS Long Lambda Function For Inter-language Binding and Unused Native Entity being relatively prominent; throughout the evolution process of the 5 DLFs, some ILDS were resolved to a certain extent, but the overall count of ILDS instances shows an upward trend. The automated detection of the proposed ILDS achieved a high accuracy, and the study provides a comprehensive understanding on ILDS in the multi-language DLFs.
Can Language Models Rival Mathematics Students? Evaluating Mathematical Reasoning through Textual Manipulation and Human Experiments
Venue: arXiv cs.CL 2024
Year: 2024

In this paper we look at the ability of recent large language models (LLMs) at solving mathematical problems in combinatorics. We compare models LLaMA-2, LLaMA-3.1, GPT-4, and Mixtral against each other and against human pupils and undergraduates with prior experience in mathematical olympiads. To facilitate these comparisons we introduce the Combi-Puzzles dataset, which contains 125 problem variants based on 25 combinatorial reasoning problems. Each problem is presented in one of five distinct forms, created by systematically manipulating the problem statements through adversarial additions, numeric parameter changes, and linguistic obfuscation. Our variations preserve the mathematical core and are designed to measure the generalisability of LLM problem-solving abilities, while also increasing confidence that problems are submitted to LLMs in forms that have not been seen as training instances. We found that a model based on GPT-4 outperformed all other models in producing correct responses, and performed significantly better in the mathematical variation of the problems than humans. We also found that modifications to problem statements significantly impact the LLM's performance, while human performance remains unaffected.
Lightweight Decentralized Neural Network-Based Strategies for Multi-Robot Patrolling
Venue: arXiv cs.RO 2024
Year: 2024

The problem of decentralized multi-robot patrol has previously been approached primarily with hand-designed strategies for minimization of 'idlenes' over the vertices of a graph-structured environment. Here we present two lightweight neural network-based strategies to tackle this problem, and show that they significantly outperform existing strategies in both idleness minimization and against an intelligent intruder model, as well as presenting an examination of robustness to communication failure. Our results also indicate important considerations for future strategy design.
Coconut Palm Tree Counting on Drone Images with Deep Object Detection and Synthetic Training Data
Venue: arXiv cs.CV 2024
Year: 2024

Drones have revolutionized various domains, including agriculture. Recent advances in deep learning have propelled among other things object detection in computer vision. This study utilized YOLO, a real-time object detector, to identify and count coconut palm trees in Ghanaian farm drone footage. The farm presented has lost track of its trees due to different planting phases. While manual counting would be very tedious and error-prone, accurately determining the number of trees is crucial for efficient planning and management of agricultural processes, especially for optimizing yields and predicting production. We assessed YOLO for palm detection within a semi-automated framework, evaluated accuracy augmentations, and pondered its potential for farmers. Data was captured in September 2022 via drones. To optimize YOLO with scarce data, synthetic images were created for model training and validation. The YOLOv7 model, pretrained on the COCO dataset (excluding coconut palms), was adapted using tailored data. Trees from footage were repositioned on synthetic images, with testing on distinct authentic images. In our experiments, we adjusted hyperparameters, improving YOLO's mean average precision (mAP). We also tested various altitudes to determine the best drone height. From an initial mAP@.5 of $0.65$, we achieved 0.88, highlighting the value of synthetic images in agricultural scenarios.
Memory-Reduced Meta-Learning with Guaranteed Convergence
Venue: arXiv cs.LG 2024
Year: 2024

The optimization-based meta-learning approach is gaining increased traction because of its unique ability to quickly adapt to a new task using only small amounts of data. However, existing optimization-based meta-learning approaches, such as MAML, ANIL and their variants, generally employ backpropagation for upper-level gradient estimation, which requires using historical lower-level parameters/gradients and thus increases computational and memory overhead in each iteration. In this paper, we propose a meta-learning algorithm that can avoid using historical parameters/gradients and significantly reduce memory costs in each iteration compared to existing optimization-based meta-learning approaches. In addition to memory reduction, we prove that our proposed algorithm converges sublinearly with the iteration number of upper-level optimization, and the convergence error decays sublinearly with the batch size of sampled tasks. In the specific case in terms of deterministic meta-learning, we also prove that our proposed algorithm converges to an exact solution. Moreover, we quantify that the computational complexity of the algorithm is on the order of $\mathcal{O}(ε^{-1})$, which matches existing convergence results on meta-learning even without using any historical parameters/gradients. Experimental results on meta-learning benchmarks confirm the efficacy of our proposed algorithm.
RepFace: Refining Closed-Set Noise with Progressive Label Correction for Face Recognition
Venue: arXiv cs.CV 2024
Year: 2024

Face recognition has made remarkable strides, driven by the expanding scale of datasets, advancements in various backbone and discriminative losses. However, face recognition performance is heavily affected by the label noise, especially closed-set noise. While numerous studies have focused on handling label noise, addressing closed-set noise still poses challenges. This paper identifies this challenge as training isn't robust to noise at the early-stage training, and necessitating an appropriate learning strategy for samples with low confidence, which are often misclassified as closed-set noise in later training phases. To address these issues, we propose a new framework to stabilize the training at early stages and split the samples into clean, ambiguous and noisy groups which are devised with separate training strategies. Initially, we employ generated auxiliary closed-set noisy samples to enable the model to identify noisy data at the early stages of training. Subsequently, we introduce how samples are split into clean, ambiguous and noisy groups by their similarity to the positive and nearest negative centers. Then we perform label fusion for ambiguous samples by incorporating accumulated model predictions. Finally, we apply label smoothing within the closed set, adjusting the label to a point between the nearest negative class and the initially assigned label. Extensive experiments validate the effectiveness of our method on mainstream face datasets, achieving state-of-the-art results. The code will be released upon acceptance.
An Algorithmic Approach to Line Construction in Existing Transit Networks
Venue: arXiv cs.NI 2024
Year: 2024

Transit networks often have existing infrastructure that cannot be modified when designing new lines for the network. This paper provides an algorithm to generate a line within a transit network without changing any existing lines or connections between stations. Additionally, a method of analyzing the efficiency of a transit line and network is provided, and used within the algorithm presented. An analysis of the effects of different parameters and objectives on the location of a new line is performed. We find that under most cases, a new line generated improves the overall efficiency of the network, while under certain circumstances, an unsuitable combination of pathfinding algorithm and efficiency evaluation method or an increase in construction and maintenance cost can cause the algorithm to create a less efficient network.
Hyperbolic Hypergraph Neural Networks for Multi-Relational Knowledge Hypergraph Representation
Venue: arXiv cs.LG 2024
Year: 2024

Knowledge hypergraphs generalize knowledge graphs using hyperedges to connect multiple entities and depict complicated relations. Existing methods either transform hyperedges into an easier-to-handle set of binary relations or view hyperedges as isolated and ignore their adjacencies. Both approaches have information loss and may potentially lead to the creation of sub-optimal models. To fix these issues, we propose the Hyperbolic Hypergraph Neural Network (H2GNN), whose essential component is the hyper-star message passing, a novel scheme motivated by a lossless expansion of hyperedges into hierarchies. It implements a direct embedding that consciously incorporates adjacent entities, hyper-relations, and entity position-aware information. As the name suggests, H2GNN operates in the hyperbolic space, which is more adept at capturing the tree-like hierarchy. We compare H2GNN with 15 baselines on knowledge hypergraphs, and it outperforms state-of-the-art approaches in both node classification and link prediction tasks.
Personalized Sleep Staging Leveraging Source-free Unsupervised Domain Adaptation
Venue: arXiv cs.LG 2024
Year: 2024

Sleep staging is crucial for assessing sleep quality and diagnosing related disorders. Recent deep learning models for automatic sleep staging using polysomnography often suffer from poor generalization to new subjects because they are trained and tested on the same labeled datasets, overlooking individual differences. To tackle this issue, we propose a novel Source-Free Unsupervised Individual Domain Adaptation (SF-UIDA) framework. This two-step adaptation scheme allows the model to effectively adjust to new unlabeled individuals without needing source data, facilitating personalized customization in clinical settings. Our framework has been applied to three established sleep staging models and tested on three public datasets, achieving state-of-the-art performance.
Towards LLM-based optimization compilers. Can LLMs learn how to apply a single peephole optimization? Reasoning is all LLMs need!
Venue: arXiv cs.LG 2024
Year: 2024

Large Language Models (LLMs) have demonstrated great potential in various language processing tasks, and recent studies have explored their application in compiler optimizations. However, all these studies focus on the conventional open-source LLMs, such as Llama2, which lack enhanced reasoning mechanisms. In this study, we investigate the errors produced by the fine-tuned 7B-parameter Llama2 model as it attempts to learn and apply a simple peephole optimization for the AArch64 assembly code. We provide an analysis of the errors produced by the LLM and compare it with state-of-the-art OpenAI models which implement advanced reasoning logic, including GPT-4o and GPT-o1 (preview). We demonstrate that OpenAI GPT-o1, despite not being fine-tuned, outperforms the fine-tuned Llama2 and GPT-4o. Our findings indicate that this advantage is largely due to the chain-of-thought reasoning implemented in GPT-o1. We hope our work will inspire further research on using LLMs with enhanced reasoning mechanisms and chain-of-thought for code generation and optimization.
Performance of a large language model-Artificial Intelligence based chatbot for counseling patients with sexually transmitted infections and genital diseases
Venue: arXiv cs.CL 2024
Year: 2024

Introduction: Global burden of sexually transmitted infections (STIs) is rising out of proportion to specialists. Current chatbots like ChatGPT are not tailored for handling STI-related concerns out of the box. We developed Otiz, an Artificial Intelligence-based (AI-based) chatbot platform designed specifically for STI detection and counseling, and assessed its performance. Methods: Otiz employs a multi-agent system architecture based on GPT4-0613, leveraging large language model (LLM) and Deterministic Finite Automaton principles to provide contextually relevant, medically accurate, and empathetic responses. Its components include modules for general STI information, emotional recognition, Acute Stress Disorder detection, and psychotherapy. A question suggestion agent operates in parallel. Four STIs (anogenital warts, herpes, syphilis, urethritis/cervicitis) and 2 non-STIs (candidiasis, penile cancer) were evaluated using prompts mimicking patient language. Each prompt was independently graded by two venereologists conversing with Otiz as patient actors on 6 criteria using Numerical Rating Scale ranging from 0 (poor) to 5 (excellent). Results: Twenty-three venereologists did 60 evaluations of 30 prompts. Across STIs, Otiz scored highly on diagnostic accuracy (4.1-4.7), overall accuracy (4.3-4.6), correctness of information (5.0), comprehensibility (4.2-4.4), and empathy (4.5-4.8). However, relevance scores were lower (2.9-3.6), suggesting some redundancy. Diagnostic scores for non-STIs were lower (p=0.038). Inter-observer agreement was strong, with differences greater than 1 point occurring in only 12.7% of paired evaluations. Conclusions: AI conversational agents like Otiz can provide accurate, correct, discrete, non-judgmental, readily accessible and easily understandable STI-related information in an empathetic manner, and can alleviate the burden on healthcare systems.
PickLLM: Context-Aware RL-Assisted Large Language Model Routing
Venue: arXiv cs.LG 2024
Year: 2024

Recently, the number of off-the-shelf Large Language Models (LLMs) has exploded with many open-source options. This creates a diverse landscape regarding both serving options (e.g., inference on local hardware vs remote LLM APIs) and model heterogeneous expertise. However, it is hard for the user to efficiently optimize considering operational cost (pricing structures, expensive LLMs-as-a-service for large querying volumes), efficiency, or even per-case specific measures such as response accuracy, bias, or toxicity. Also, existing LLM routing solutions focus mainly on cost reduction, with response accuracy optimizations relying on non-generalizable supervised training, and ensemble approaches necessitating output computation for every considered LLM candidate. In this work, we tackle the challenge of selecting the optimal LLM from a model pool for specific queries with customizable objectives. We propose PickLLM, a lightweight framework that relies on Reinforcement Learning (RL) to route on-the-fly queries to available models. We introduce a weighted reward function that considers per-query cost, inference latency, and model response accuracy by a customizable scoring function. Regarding the learning algorithms, we explore two alternatives: PickLLM router acting as a learning automaton that utilizes gradient ascent to select a specific LLM, or utilizing stateless Q-learning to explore the set of LLMs and perform selection with a $ε$-greedy approach. The algorithm converges to a single LLM for the remaining session queries. To evaluate, we utilize a pool of four LLMs and benchmark prompt-response datasets with different contexts. A separate scoring function is assessing response accuracy during the experiment. We demonstrate the speed of convergence for different learning rates and improvement in hard metrics such as cost per querying session and overall response latency.
AI Adoption to Combat Financial Crime: Study on Natural Language Processing in Adverse Media Screening of Financial Services in English and Bangla multilingual interpretation
Venue: arXiv cs.CL 2024
Year: 2024

This document explores the potential of employing Artificial Intelligence (AI), specifically Natural Language Processing (NLP), to strengthen the detection and prevention of financial crimes within the Mobile Financial Services(MFS) of Bangladesh with multilingual scenario. The analysis focuses on the utilization of NLP for adverse media screening, a vital aspect of compliance with anti-money laundering (AML) and combating financial terrorism (CFT) regulations. Additionally, it investigates the overall reception and obstacles related to the integration of AI in Bangladeshi banks. This report measures the effectiveness of NLP is promising with an accuracy around 94\%. NLP algorithms display substantial promise in accurately identifying adverse media content linked to financial crimes. The lack of progress in this aspect is visible in Bangladesh, whereas globally the technology is already being used to increase effectiveness and efficiency. Hence, it is clear there is an issue with the acceptance of AI in Bangladesh. Some AML \& CFT concerns are already being addressed by AI technology. For example, Image Recognition OCR technology are being used in KYC procedures. Primary hindrances to AI integration involve a lack of technical expertise, high expenses, and uncertainties surrounding regulations. This investigation underscores the potential of AI-driven NLP solutions in fortifying efforts to prevent financial crimes in Bangladesh.
A NotSo Simple Way to Beat Simple Bench
Venue: arXiv cs.CL 2024
Year: 2024

This paper presents a novel framework for enhancing reasoning capabilities in large language models (LLMs) by leveraging iterative reasoning and feedback-driven methodologies. Building on the limitations identified in the SimpleBench benchmark, a dataset designed to evaluate logical coherence and real-world reasoning, we propose a multi-step prompting strategy coupled with global consistency checks to improve model accuracy and robustness. Through comparative analysis of state-of-the-art models, including Claude 3 Opus, Claude 3.5, GPT- 4o, and o1-preview, we demonstrate that iterative reasoning significantly enhances model performance, with improvements observed in both standard accuracy metrics (AVG@5) and a newly introduced metric, Extreme Averaging (EAG@5). Our results reveal model-specific strengths: Claude excels in maintaining logical consistency, while GPT-4o exhibits exploratory creativity but struggles with ambiguous prompts. By analyzing case studies and identifying gaps in spatial and temporal reasoning, we highlight areas for further refinement. The findings underscore the potential of structured reasoning frameworks to address inherent model limitations, irrespective of pretraining methodologies. This study lays the groundwork for integrating dynamic feedback mechanisms, adaptive restart strategies, and diverse evaluation metrics to advance LLM reasoning capabilities across complex and multi-domain problem spaces.
SprayCraft: Graph-Based Route Optimization for Variable Rate Precision Spraying
Venue: arXiv cs.OH 2024
Year: 2024

To efficiently manage plant diseases, Agriculture Cyber-Physical Systems (A-CPS) have been developed to detect and localize disease infestations by integrating the Internet of Agro-Things (IoAT). By the nature of plant and pathogen interactions, the spread of a disease appears as a focus with density of infected plants and intensity of infection diminishing outwards. This gradient of infection needs variable rate and precision pesticide spraying to efficiently utilize resources and effectively handle the diseases. This article, SprayCraft presents a graph based method for disease management A-CPS to identify disease hotspots and compute near optimal path for a spraying drone to perform variable rate precision spraying. It uses graph to represent the diseased locations and their spatial relation, Message Passing is performed over the graph to compute the probability of a location to be a disease hotspot. These probabilities also serve as disease intensity measures and are used for variable rate spraying at each location. Whereas, the graph is utilized to compute tour path by considering it as Traveling Salesman Problem (TSP) for precision spraying by the drone. Proposed method has been validated on synthetic data of locations of diseased locations in a farmland.
Adopting Explainable-AI to investigate the impact of urban morphology design on energy and environmental performance in dry-arid climates
Venue: arXiv cs.LG 2024
Year: 2024

In rapidly urbanizing regions, designing climate-responsive urban forms is crucial for sustainable development, especially in dry arid-climates where urban morphology has a significant impact on energy consumption and environmental performance. This study advances urban morphology evaluation by combining Urban Building Energy Modeling (UBEM) with machine learning methods (ML) and Explainable AI techniques, specifically Shapley Additive Explanations (SHAP). Using Tehran's dense urban landscape as a case study, this research assesses and ranks the impact of 30 morphology parameters at the urban block level on key energy metrics (cooling, heating, and lighting demand) and environmental performance (sunlight exposure, photovoltaic generation, and Sky View Factor). Among seven ML algorithms evaluated, the XGBoost model was the most effective predictor, achieving high accuracy (R2: 0.92) and a training time of 3.64 seconds. Findings reveal that building shape, window-to-wall ratio, and commercial ratio are the most critical parameters affecting energy efficiency, while the heights and distances of neighboring buildings strongly influence cooling demand and solar access. By evaluating urban blocks with varied densities and configurations, this study offers generalizable insights applicable to other dry-arid regions. Moreover, the integration of UBEM and Explainable AI offers a scalable, data-driven framework for developing climate-responsive urban designs adaptable to high-density environments worldwide.