OpenRaiser/Intern-Atlas huggingface dataset:
Predictive Health Analysis in Industry 5.0: A Scientometric and Systematic Review of Motion Capture in Construction
Venue: arXiv cs.CY 2024
Year: 2024

In an era of rapid technological advancement, the rise of Industry 4.0 has prompted industries to pursue innovative improvements in their processes. As we advance towards Industry 5.0, which focuses more on collaboration between humans and intelligent systems, there is a growing requirement for better sensing technologies for healthcare and safety purposes. Consequently, Motion Capture (MoCap) systems have emerged as critical enablers in this technological evolution by providing unmatched precision and versatility in various workplaces, including construction. As the construction workplace requires physically demanding tasks, leading to work-related musculoskeletal disorders (WMSDs) and health issues, the study explores the increasing relevance of MoCap systems within the concept of Industry 4.0 and 5.0. Despite the growing significance, there needs to be more comprehensive research, a scientometric review that quantitatively assesses the role of MoCap systems in construction. Our study combines bibliometric, scientometric, and systematic review approaches to address this gap, analyzing articles sourced from the Scopus database. A total of 52 papers were carefully selected from a pool of 962 papers for a quantitative study using a scientometric approach and a qualitative, indepth examination. Results showed that MoCap systems are employed to improve worker health and safety and reduce occupational hazards.The in-depth study also finds the most tested construction tasks are masonry, lifting, training, and climbing, with a clear preference for markerless systems.
HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification
Venue: arXiv cs.CL 2024
Year: 2024

Hierarchical text classification (HTC) is a complex subtask under multi-label text classification, characterized by a hierarchical label taxonomy and data imbalance. The best-performing models aim to learn a static representation by combining document and hierarchical label information. However, the relevance of document sections can vary based on the hierarchy level, necessitating a dynamic document representation. To address this, we propose HiGen, a text-generation-based framework utilizing language models to encode dynamic text representations. We introduce a level-guided loss function to capture the relationship between text and label name semantics. Our approach incorporates a task-specific pretraining strategy, adapting the language model to in-domain knowledge and significantly enhancing performance for classes with limited examples. Furthermore, we present a new and valuable dataset called ENZYME, designed for HTC, which comprises articles from PubMed with the goal of predicting Enzyme Commission (EC) numbers. Through extensive experiments on the ENZYME dataset and the widely recognized WOS and NYT datasets, our methodology demonstrates superior performance, surpassing existing approaches while efficiently handling data and mitigating class imbalance. The data and code will be released publicly.
APT-Pipe: A Prompt-Tuning Tool for Social Data Annotation using ChatGPT
Venue: arXiv cs.CL 2024
Year: 2024

Recent research has highlighted the potential of LLM applications, like ChatGPT, for performing label annotation on social computing text. However, it is already well known that performance hinges on the quality of the input prompts. To address this, there has been a flurry of research into prompt tuning -- techniques and guidelines that attempt to improve the quality of prompts. Yet these largely rely on manual effort and prior knowledge of the dataset being annotated. To address this limitation, we propose APT-Pipe, an automated prompt-tuning pipeline. APT-Pipe aims to automatically tune prompts to enhance ChatGPT's text classification performance on any given dataset. We implement APT-Pipe and test it across twelve distinct text classification datasets. We find that prompts tuned by APT-Pipe help ChatGPT achieve higher weighted F1-score on nine out of twelve experimented datasets, with an improvement of 7.01% on average. We further highlight APT-Pipe's flexibility as a framework by showing how it can be extended to support additional tuning mechanisms.
Fluent dreaming for language models
Venue: arXiv cs.CL 2024
Year: 2024

Feature visualization, also known as "dreaming", offers insights into vision models by optimizing the inputs to maximize a neuron's activation or other internal component. However, dreaming has not been successfully applied to language models because the input space is discrete. We extend Greedy Coordinate Gradient, a method from the language model adversarial attack literature, to design the Evolutionary Prompt Optimization (EPO) algorithm. EPO optimizes the input prompt to simultaneously maximize the Pareto frontier between a chosen internal feature and prompt fluency, enabling fluent dreaming for language models. We demonstrate dreaming with neurons, output logits and arbitrary directions in activation space. We measure the fluency of the resulting prompts and compare language model dreaming with max-activating dataset examples. Critically, fluent dreaming allows automatically exploring the behavior of model internals in reaction to mildly out-of-distribution prompts. Code for running EPO is available at https://github.com/Confirm-Solutions/dreamy. A companion page demonstrating code usage is at https://confirmlabs.org/posts/dreamy.html
Revitalizing Sex Education for Chinese Children: A Formative Study
Venue: arXiv cs.CY 2024
Year: 2024

Sex education helps children obtain knowledge and awareness of sexuality, and protects them against sexually transmitted diseases, pregnancy, and sexual abuse. Sex education is not well taught to children in China -- both school-based education and parental communication on this topic are limited. To interrogate the status quo of sex education in China and explore suitable interventions, we conducted a series of formative studies including interviews and social media analysis. Multiple stakeholders such as children, parents, education practitioners, and the general public were engaged for an in-depth understanding of their unique needs regarding teaching and learning sex education. We found that school-based sex education for Chinese children was currently insufficient and restrictive. Involving parents in sex education posed several challenges, such as a lack of sexuality and pedagogy knowledge, and embarrassment in initiating sex education conversations. Culture and politics were major hurdles to effective sex education. Based on the findings, we reflect on the complex interactions between culture, politics, education policy, and pedagogy, and discuss situated design of sex education in broader cultural and social contexts.
Socially Aware Synthetic Data Generation for Suicidal Ideation Detection Using Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Suicidal ideation detection is a vital research area that holds great potential for improving mental health support systems. However, the sensitivity surrounding suicide-related data poses challenges in accessing large-scale, annotated datasets necessary for training effective machine learning models. To address this limitation, we introduce an innovative strategy that leverages the capabilities of generative AI models, such as ChatGPT, Flan-T5, and Llama, to create synthetic data for suicidal ideation detection. Our data generation approach is grounded in social factors extracted from psychology literature and aims to ensure coverage of essential information related to suicidal ideation. In our study, we benchmarked against state-of-the-art NLP classification models, specifically, those centered around the BERT family structures. When trained on the real-world dataset, UMD, these conventional models tend to yield F1-scores ranging from 0.75 to 0.87. Our synthetic data-driven method, informed by social factors, offers consistent F1-scores of 0.82 for both models, suggesting that the richness of topics in synthetic data can bridge the performance gap across different model complexities. Most impressively, when we combined a mere 30% of the UMD dataset with our synthetic data, we witnessed a substantial increase in performance, achieving an F1-score of 0.88 on the UMD test set. Such results underscore the cost-effectiveness and potential of our approach in confronting major challenges in the field, such as data scarcity and the quest for diversity in data representation.
Integration of Artificial Intelligence in Educational Measurement: Efficacy of ChatGPT in Data Generation within the Scope of Item Response Theory
Venue: arXiv cs.CY 2024
Year: 2024

The aim of this study is to investigate the effectiveness of ChatGPT 3.5 in developing algorithms for data generation within the framework of Item Response Theory (IRT) using the R programming language. In this context, validity examinations were conducted on data sets generated according to the Two-Parameter Logistic Model (2PLM) with algorithms written by ChatGPT 3.5 and researchers. These examinations considered whether the data sets met the IRT assumptions and the simulation conditions of the item parameters. As a result, it was determined that while ChatGPT 3.5 was quite successful in generating data that met the IRT assumptions, it was less effective in meeting the simulation conditions of the item parameters compared to the algorithm developed by the researchers. In this regard, ChatGPT 3.5 is recommended as a useful tool that researchers can use in developing data generation algorithms for IRT.
The Reasoning Under Uncertainty Trap: A Structural AI Risk
Venue: arXiv cs.CY 2024
Year: 2024

This report examines a novel risk associated with current (and projected) AI tools. Making effective decisions about future actions requires us to reason under uncertainty (RUU), and doing so is essential to many critical real world problems. Overfaced by this challenge, there is growing demand for AI tools like LLMs to assist decision-makers. Having evidenced this demand and the incentives behind it, we expose a growing risk: we 1) do not currently sufficiently understand LLM capabilities in this regard, and 2) have no guarantees of performance given fundamental computational explosiveness and deep uncertainty constraints on accuracy. This report provides an exposition of what makes RUU so challenging for both humans and machines, and relates these difficulties to prospective AI timelines and capabilities. Having established this current potential misuse risk, we go on to expose how this seemingly additive risk (more misuse additively contributed to potential harm) in fact has multiplicative properties. Specifically, we detail how this misuse risk connects to a wider network of underlying structural risks (e.g., shifting incentives, limited transparency, and feedback loops) to produce non-linear harms. We go on to provide a solutions roadmap that targets multiple leverage points in the structure of the problem. This includes recommendations for all involved actors (prospective users, developers, and policy-makers) and enfolds insights from areas including Decision-making Under Deep Uncertainty and complex systems theory. We argue this report serves not only to raise awareness (and subsequently mitigate/correct) of a current, novel AI risk, but also awareness of the underlying class of structural risks by illustrating how their interconnected nature poses twin-dangers of camouflaging their presence, whilst amplifying their potential effects.
Systematic Literature Review: Computational Approaches for Humour Style Classification
Venue: arXiv cs.CL 2024
Year: 2024

Understanding various humour styles is essential for comprehending the multifaceted nature of humour and its impact on fields such as psychology and artificial intelligence. This understanding has revealed that humour, depending on the style employed, can either have therapeutic or detrimental effects on an individual's health and relationships. Although studies dedicated exclusively to computational-based humour style analysis remain somewhat rare, an expansive body of research thrives within related task, particularly binary humour and sarcasm recognition. In this systematic literature review (SLR), we survey the landscape of computational techniques applied to these related tasks and also uncover their fundamental relevance to humour style analysis. Through this study, we unveil common approaches, illuminate various datasets and evaluation metrics, and effectively navigate the complex terrain of humour research. Our efforts determine potential research gaps and outlined promising directions. Furthermore, the SLR identifies a range of features and computational models that can seamlessly transition from related tasks like binary humour and sarcasm detection to invigorate humour style classification. These features encompass incongruity, sentiment and polarity analysis, ambiguity detection, acoustic nuances, visual cues, contextual insights, and more. The computational models that emerge contain traditional machine learning paradigms, neural network architectures, transformer-based models, and specialised models attuned to the nuances of humour. Finally, the SLR provides access to existing datasets related to humour and sarcasm, facilitating the work of future researchers.
XP2021 Experience Report: Five Strategies for the Future of Work: Accelerating Innovation through Tech Transfer
Venue: arXiv cs.CY 2024
Year: 2024

This experience report outlines five tech transfer strategies developed over a period of 25 years at four Global 1000 companies (HP, Cisco, Qualcomm, and Nortel) to mitigate R&D challenges associated with duplicated effort, product quality, and time-to-market. The five strategies accelerate innovation through open knowledge sharing, rather than licensing intellectual property rights (IPR) such as patents, trade secrets, and copyrights. The strategies are based on corporate tech forums, conference panels, exploratory workshops, research reviews (at universities and companies), and talent exchanges. While the initial objective was to foster the corporate adoption of software best practices, over time the strategies had broader impact on company innovation, including incubating cross-company R&D collaborations, capturing organizational memory, cultivating and leveraging external research partnerships, and feeding company talent pipelines.
HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA
Venue: arXiv cs.CL 2024
Year: 2024

Retrieval-augmented generation (RAG) has rapidly advanced the language model field, particularly in question-answering (QA) systems. By integrating external documents during the response generation phase, RAG significantly enhances the accuracy and reliability of language models. This method elevates the quality of responses and reduces the frequency of hallucinations, where the model generates incorrect or misleading information. However, these methods exhibit limited retrieval accuracy when faced with numerous indistinguishable documents, presenting notable challenges in their practical application. In response to these emerging challenges, we present HiQA, an advanced multi-document question-answering (MDQA) framework that integrates cascading metadata into content and a multi-route retrieval mechanism. We also release a benchmark called MasQA to evaluate and research in MDQA. Finally, HiQA demonstrates the state-of-the-art performance in multi-document environments.
Disentangling the Roles of Target-Side Transfer and Regularization in Multilingual Machine Translation
Venue: arXiv cs.CL 2024
Year: 2024

Multilingual Machine Translation (MMT) benefits from knowledge transfer across different language pairs. However, improvements in one-to-many translation compared to many-to-one translation are only marginal and sometimes even negligible. This performance discrepancy raises the question of to what extent positive transfer plays a role on the target-side for one-to-many MT. In this paper, we conduct a large-scale study that varies the auxiliary target side languages along two dimensions, i.e., linguistic similarity and corpus size, to show the dynamic impact of knowledge transfer on the main language pairs. We show that linguistically similar auxiliary target languages exhibit strong ability to transfer positive knowledge. With an increasing size of similar target languages, the positive transfer is further enhanced to benefit the main language pairs. Meanwhile, we find distant auxiliary target languages can also unexpectedly benefit main language pairs, even with minimal positive transfer ability. Apart from transfer, we show distant auxiliary target languages can act as a regularizer to benefit translation performance by enhancing the generalization and model inference calibration.
Creating a Synthesizer from Schrödinger's Equation
Venue: arXiv cs.SD 2024
Year: 2024

Our project offers an alternative approach to the sensory perception of the Schrödinger equation (an elementary model of quantum phenomena) by interpreting it as a sound wave. We are building a synthesizer plugin that simulates a quantum mechanical state that evolves over time. Thus, our tool allows the creation of unique sounds that are in motion and feel alive. These can be used in professional music production without any knowledge of physics, while at the same time providing insight into a chapter of quantum mechanics. The goal is to lower the threshold for entering complex theory by first developing an intuition for the subject; but the tool can also be used purely as a musical instrument. The user is encouraged, but not forced, to learn more about the underlying physics. Simulation parameters are adjustable in real-time, allowing intuitive experimentation. Despite the approximate calculations, real physical effects such as quantum tunneling can be observed acoustically and visually.
Hierarchical Multi-Label Classification of Online Vaccine Concerns
Venue: arXiv cs.CL 2024
Year: 2024

Vaccine concerns are an ever-evolving target, and can shift quickly as seen during the COVID-19 pandemic. Identifying longitudinal trends in vaccine concerns and misinformation might inform the healthcare space by helping public health efforts strategically allocate resources or information campaigns. We explore the task of detecting vaccine concerns in online discourse using large language models (LLMs) in a zero-shot setting without the need for expensive training datasets. Since real-time monitoring of online sources requires large-scale inference, we explore cost-accuracy trade-offs of different prompting strategies and offer concrete takeaways that may inform choices in system designs for current applications. An analysis of different prompting strategies reveals that classifying the concerns over multiple passes through the LLM, each consisting a boolean question whether the text mentions a vaccine concern or not, works the best. Our results indicate that GPT-4 can strongly outperform crowdworker accuracy when compared to ground truth annotations provided by experts on the recently introduced VaxConcerns dataset, achieving an overall F1 score of 78.7%.
Robust support vector machines via conic optimization
Venue: arXiv cs.LG 2024
Year: 2024

We consider the problem of learning support vector machines robust to uncertainty. It has been established in the literature that typical loss functions, including the hinge loss, are sensible to data perturbations and outliers, thus performing poorly in the setting considered. In contrast, using the 0-1 loss or a suitable non-convex approximation results in robust estimators, at the expense of large computational costs. In this paper we use mixed-integer optimization techniques to derive a new loss function that better approximates the 0-1 loss compared with existing alternatives, while preserving the convexity of the learning problem. In our computational results, we show that the proposed estimator is competitive with the standard SVMs with the hinge loss in outlier-free regimes and better in the presence of outliers.
Can LLMs perform structured graph reasoning?
Venue: arXiv cs.CL 2024
Year: 2024

Pretrained Large Language Models (LLMs) have demonstrated various reasoning capabilities through language-based prompts alone, particularly in unstructured task settings (tasks purely based on language semantics). However, LLMs often struggle with structured tasks, because of the inherent incompatibility of input representation. Reducing structured tasks to uni-dimensional language semantics often renders the problem trivial. Keeping the trade-off between LLM compatibility and structure complexity in mind, we design various graph reasoning tasks as a proxy to semi-structured tasks in this paper, in order to test the ability to navigate through representations beyond plain text in various LLMs. Particularly, we design 10 distinct problems of graph traversal, each representing increasing levels of complexity, and benchmark 5 different instruct-finetuned LLMs (GPT-4, GPT-3.5, Claude-2, Llama-2 and Palm-2) on the aforementioned tasks. Further, we analyse the performance of models across various settings such as varying sizes of graphs as well as different forms of k-shot prompting. We highlight various limitations, biases and properties of LLMs through this benchmarking process, such as an inverse relation to the average degrees of freedom of traversal per node in graphs, the overall negative impact of k-shot prompting on graph reasoning tasks, and a positive response bias which prevents LLMs from identifying the absence of a valid solution. Finally, we introduce a new prompting technique specially designed for graph traversal tasks (PathCompare), which demonstrates a notable increase in the performance of LLMs in comparison to standard prompting techniques such as Chain-of-Thought (CoT).
Simple Symmetric Sustainable Sorting -- the greeNsort article
Venue: arXiv cs.DS 2024
Year: 2024

We explored an uncharted part of the solution space for sorting algorithms: the role of symmetry in divide&conquer algorithms. We found/designed novel simple binary Quicksort and Mergesort algorithms operating in contiguous space which achieve improved trade-offs between worst-case CPU-efficiency, best-case adaptivity and RAM-requirements. The 'greeNsort' algorithms need less hardware (RAM) and/or less energy (CPU) compared to the prior art. The new algorithms fit a theoretical framework: 'Footprint' KPIs allow to compare algorithms with different RAM-requirements, a new 'definition' of sorting API-targets simplifies construction of stable algorithms with mirrored scan directions, and our ordinal machine model encourages robust algorithms that minimize access 'distance'. Unlike earlier 'Quicksorts', our 'Zacksort', 'Zucksort' and 'Ducksort' algorithms optimally marry CPU-efficiency and tie-adaptivity. Unlike earlier 'Mergesorts' which required 100% distant buffer, our 'Frogsort' and 'Geckosort' algorithms achieve similar CPU-efficiency with 50% or less local buffer. Unlike natural Mergesorts such as 'Timsort' which are optimized for the best case of full-presorting, our 'Octosort' and 'Squidsort' algorithms achieve excellent bi-adaptivity to presorted best-cases without sacrificing worst-case efficiency in real sorting tasks. Our 'Walksort' and 'Jumpsort' have lower Footprint than the impressive low-memory 'Grailsort' and 'Sqrtsort' of Astrelin. Given the current climate-emergency, this is a call to action for all maintainers of sorting libraries, all software-engineers using custom sorting code, all professors teaching algorithms, all IT professionals designing programming languages, compilers and CPUs: check for better algorithms and consider symmetric code-mirroring.