OpenRaiser/Intern-Atlas huggingface dataset:
Design of a multimodal device to improve well-being of autistic workers interacting with collaborative robots
Venue: arXiv cs.HC 2023
Year: 2023

Several studies report that people diagnosed with autism spectrum disorder (ASD) have low employment rates and major difficulties in maintaining occupation (Hendricks, 2010). Adopting smart and interactive technology, promoted by Industry 4.0, can be a valid solution to improve social inclusion in the workplace. The present work describes the design and development process of the multimodal device (A)MICO (acronym for 'A Multimodal device to improve inclusive Interaction between Cobot and Operator'), aimed to improve the user experience of ASD workers interacting with collaborative robots (cobots) in production lines. (A)MICO proposes a new intuitive mode of communication in which information about the cobot activity is transferred through acoustic and visual feedback. Lights and sounds are designed to make the user aware of the cobot's activity and gain control of the situation. To define the optimal feedback configuration, a co-design process has been performed involving five users with high functioning autism (HFA). The inclusion of users with HFA in the design process allows to analyse the system from different perspectives and increase the acceptance of the final product (Francis et al., 2009). In addition, (A)MICO is designed to be integrated into a pre-existing cobot system and its feedback can be customized making job tasks more human-friendly and accessible to all. To achieve this purpose, Design for All principles were taken into consideration since the preliminary stages of the study, in order to develop a device that could represent a valuable aid in improving the well-being and productivity for both neurotypical and neurodiverse users.
Timing Analysis of Embedded Software Updates
Venue: arXiv cs.SE 2023
Year: 2023

We present RETA (Relative Timing Analysis), a differential timing analysis technique to verify the impact of an update on the execution time of embedded software. Timing analysis is computationally expensive and labor intensive. Software updates render repeating the analysis from scratch a waste of resources and time, because their impact is inherently confined. To determine this boundary, in RETA we apply a slicing procedure that identifies all relevant code segments and a statement categorization that determines how to analyze each such line of code. We adapt a subset of RETA for integration into aiT, an industrial timing analysis tool, and also develop a complete implementation in a tool called DELTA. Based on staple benchmarks and realistic code updates from official repositories, we test the accuracy by analyzing the worst-case execution time (WCET) before and after an update, comparing the measures with the use of the unmodified aiT as well as real executions on embedded hardware. DELTA returns WCET information that ranges from exactly the WCET of real hardware to 148% of the new version's measured WCET. With the same benchmarks, the unmodified aiT estimates are 112% and 149% of the actual executions; therefore, even when DELTA is pessimistic, an industry-strength tool such as aiT cannot do better. Crucially, we also show that RETA decreases aiT's analysis time by 45% and its memory consumption by 8.9%, whereas removing RETA from DELTA, effectively rendering it a regular timing analysis tool, increases its analysis time by 27%.
AI, write an essay for me: A large-scale comparison of human-written versus ChatGPT-generated essays
Venue: arXiv cs.CL 2023
Year: 2023

Background: Recently, ChatGPT and similar generative AI models have attracted hundreds of millions of users and become part of the public discourse. Many believe that such models will disrupt society and will result in a significant change in the education system and information generation in the future. So far, this belief is based on either colloquial evidence or benchmarks from the owners of the models -- both lack scientific rigour. Objective: Through a large-scale study comparing human-written versus ChatGPT-generated argumentative student essays, we systematically assess the quality of the AI-generated content. Methods: A large corpus of essays was rated using standard criteria by a large number of human experts (teachers). We augment the analysis with a consideration of the linguistic characteristics of the generated essays. Results: Our results demonstrate that ChatGPT generates essays that are rated higher for quality than human-written essays. The writing style of the AI models exhibits linguistic characteristics that are different from those of the human-written essays, e.g., it is characterized by fewer discourse and epistemic markers, but more nominalizations and greater lexical diversity. Conclusions: Our results clearly demonstrate that models like ChatGPT outperform humans in generating argumentative essays. Since the technology is readily available for anyone to use, educators must act immediately. We must re-invent homework and develop teaching concepts that utilize these AI models in the same way as math utilized the calculator: teach the general concepts first and then use AI tools to free up time for other learning objectives.
Deep Imitation Learning for Automated Drop-In Gamma Probe Manipulation
Venue: arXiv cs.RO 2023
Year: 2023

The increasing prevalence of prostate cancer has led to the widespread adoption of Robotic-Assisted Surgery (RAS) as a treatment option. Sentinel lymph node biopsy (SLNB) is a crucial component of prostate cancer surgery and requires accurate diagnostic evidence. This procedure can be improved by using a drop-in gamma probe, SENSEI system, to distinguish cancerous tissue from normal tissue. However, manual control of the probe using live gamma level display and audible feedback could be challenging for inexperienced surgeons, leading to the potential for missed detections. In this study, a deep imitation training workflow was proposed to automate the radioactive node detection procedure. The proposed training workflow uses simulation data to train an end-to-end vision-based gamma probe manipulation agent. The evaluation results showed that the proposed approach was capable to predict the next-step action and holds promise for further improvement and extension to a hardware setup.
Industrial Engineering with Large Language Models: A case study of ChatGPT's performance on Oil & Gas problems
Venue: arXiv cs.CL 2023
Year: 2023

Large Language Models (LLMs) have shown great potential in solving complex problems in various fields, including oil and gas engineering and other industrial engineering disciplines like factory automation, PLC programming etc. However, automatic identification of strong and weak solutions to fundamental physics equations governing several industrial processes remain a challenging task. This paper identifies the limitation of current LLM approaches, particularly ChatGPT in selected practical problems native to oil and gas engineering but not exclusively. The performance of ChatGPT in solving complex problems in oil and gas engineering is discussed and the areas where LLMs are most effective are presented.
Functional Diffusion Maps
Venue: arXiv cs.LG 2023
Year: 2023

Nowadays many real-world datasets can be considered as functional, in the sense that the processes which generate them are continuous. A fundamental property of this type of data is that in theory they belong to an infinite-dimensional space. Although in practice we usually receive finite observations, they are still high-dimensional and hence dimensionality reduction methods are crucial. In this vein, the main state-of-the-art method for functional data analysis is Functional PCA. Nevertheless, this classic technique assumes that the data lie in a linear manifold, and hence it could have problems when this hypothesis is not fulfilled. In this research, attention has been placed on a non-linear manifold learning method: Diffusion Maps. The article explains how to extend this multivariate method to functional data and compares its behavior against Functional PCA over different simulated and real examples.
SLoMo: A General System for Legged Robot Motion Imitation from Casual Videos
Venue: arXiv cs.RO 2023
Year: 2023

We present SLoMo: a first-of-its-kind framework for transferring skilled motions from casually captured "in the wild" video footage of humans and animals to legged robots. SLoMo works in three stages: 1) synthesize a physically plausible reconstructed key-point trajectory from monocular videos; 2) optimize a dynamically feasible reference trajectory for the robot offline that includes body and foot motion, as well as contact sequences that closely tracks the key points; 3) track the reference trajectory online using a general-purpose model-predictive controller on robot hardware. Traditional motion imitation for legged motor skills often requires expert animators, collaborative demonstrations, and/or expensive motion capture equipment, all of which limits scalability. Instead, SLoMo only relies on easy-to-obtain monocular video footage, readily available in online repositories such as YouTube. It converts videos into motion primitives that can be executed reliably by real-world robots. We demonstrate our approach by transferring the motions of cats, dogs, and humans to example robots including a quadruped (on hardware) and a humanoid (in simulation). To the best knowledge of the authors, this is the first attempt at a general-purpose motion transfer framework that imitates animal and human motions on legged robots directly from casual videos without artificial markers or labels.
ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding System
Venue: arXiv cs.CV 2023
Year: 2023

Existing deep video models are limited by specific tasks, fixed input-output spaces, and poor generalization capabilities, making it difficult to deploy them in real-world scenarios. In this paper, we present our vision for multimodal and versatile video understanding and propose a prototype system, \system. Our system is built upon a tracklet-centric paradigm, which treats tracklets as the basic video unit and employs various Video Foundation Models (ViFMs) to annotate their properties e.g., appearance, motion, \etc. All the detected tracklets are stored in a database and interact with the user through a database manager. We have conducted extensive case studies on different types of in-the-wild videos, which demonstrates the effectiveness of our method in answering various video-related problems. Our project is available at https://www.wangjunke.info/ChatVideo/
How Can Bar Robots Enhance the Well-being of Guests?
Venue: arXiv cs.HC 2023
Year: 2023

This paper addresses the question of how bar robots can contribute to the well-being of guests. It first develops the basics of service robots and social robots. It gives a brief overview of which gastronomy robots are on the market. It then presents examples of bar robots and describes two models used in Switzerland. A research project at the School of Business FHNW collected empirical data on them, which is used for this article. The authors then discuss how the robots could be improved to increase the well-being of customers and guests and better address their individual wishes and requirements. Artificial intelligence can play an important role in this. Finally, ethical and social problems in the use of bar robots are discussed and possible solutions are suggested to counter these.
Adversary Aware Continual Learning
Venue: arXiv cs.LG 2023
Year: 2023

Class incremental learning approaches are useful as they help the model to learn new information (classes) sequentially, while also retaining the previously acquired information (classes). However, it has been shown that such approaches are extremely vulnerable to the adversarial backdoor attacks, where an intelligent adversary can introduce small amount of misinformation to the model in the form of imperceptible backdoor pattern during training to cause deliberate forgetting of a specific task or class at test time. In this work, we propose a novel defensive framework to counter such an insidious attack where, we use the attacker's primary strength-hiding the backdoor pattern by making it imperceptible to humans-against it, and propose to learn a perceptible (stronger) pattern (also during the training) that can overpower the attacker's imperceptible (weaker) pattern. We demonstrate the effectiveness of the proposed defensive mechanism through various commonly used Replay-based (both generative and exact replay-based) class incremental learning algorithms using continual learning benchmark variants of CIFAR-10, CIFAR-100, and MNIST datasets. Most noteworthy, our proposed defensive framework does not assume that the attacker's target task and target class is known to the defender. The defender is also unaware of the shape, size, and location of the attacker's pattern. We show that our proposed defensive framework considerably improves the performance of class incremental learning algorithms with no knowledge of the attacker's target task, attacker's target class, and attacker's imperceptible pattern. We term our defensive framework as Adversary Aware Continual Learning (AACL).
Inter-sphere consistency-based method for camera-projector pair calibration
Venue: arXiv cs.CV 2023
Year: 2023

We construct constraints from consistency between estimated parameters from different spheres, termed inter-sphere consistency. It facilitates more flexible calibration using only two spheres, which has been considered a challenging and not well addressed ill-posed problem.
Ultra-Fast Zernike Moments using FFT and GPU
Venue: arXiv cs.CV 2023
Year: 2023

Zernike moments can be used to generate invariant features that are applied in various machine vision applications. They, however, suffer from slow implementation and numerical stability problems. We propose a novel method for computing Zernike using Fast Fourier Transform (FFT) and GPU computing. The method can be used to generate accurate moments up to high orders, and can compute Zernike moments of 4K resolution images in real-time. Numerical accuracies of Zernike moments computed with the proposed FFT approach have been analyzed using the orthogonality property and the results show that they beat other methods in numerical stability. The proposed method is simple and fast and can make use of the huge GPU-FFT libraries that are available in several programming frameworks.
Symmetry and Complexity in Object-Centric Deep Active Inference Models
Venue: arXiv cs.CV 2023
Year: 2023

Humans perceive and interact with hundreds of objects every day. In doing so, they need to employ mental models of these objects and often exploit symmetries in the object's shape and appearance in order to learn generalizable and transferable skills. Active inference is a first principles approach to understanding and modeling sentient agents. It states that agents entertain a generative model of their environment, and learn and act by minimizing an upper bound on their surprisal, i.e. their Free Energy. The Free Energy decomposes into an accuracy and complexity term, meaning that agents favor the least complex model, that can accurately explain their sensory observations. In this paper, we investigate how inherent symmetries of particular objects also emerge as symmetries in the latent state space of the generative model learnt under deep active inference. In particular, we focus on object-centric representations, which are trained from pixels to predict novel object views as the agent moves its viewpoint. First, we investigate the relation between model complexity and symmetry exploitation in the state space. Second, we do a principal component analysis to demonstrate how the model encodes the principal axis of symmetry of the object in the latent space. Finally, we also demonstrate how more symmetrical representations can be exploited for better generalization in the context of manipulation.
Hybrid Deepfake Detection Utilizing MLP and LSTM
Venue: arXiv cs.CV 2023
Year: 2023

The growing reliance of society on social media for authentic information has done nothing but increase over the past years. This has only raised the potential consequences of the spread of misinformation. One of the growing methods in popularity is to deceive users using a deepfake. A deepfake is an invention that has come with the latest technological advancements, which enables nefarious online users to replace their face with a computer generated, synthetic face of numerous powerful members of society. Deepfake images and videos now provide the means to mimic important political and cultural figures to spread massive amounts of false information. Models that can detect these deepfakes to prevent the spread of misinformation are now of tremendous necessity. In this paper, we propose a new deepfake detection schema utilizing two deep learning algorithms: long short term memory and multilayer perceptron. We evaluate our model using a publicly available dataset named 140k Real and Fake Faces to detect images altered by a deepfake with accuracies achieved as high as 74.7%
3D Patient-specific Modelling and Characterisation of Muscle-Skeletal Districts
Venue: arXiv cs.CV 2023
Year: 2023

This work addresses the patient-specific characterisation of the morphology and pathologies of muscle-skeletal districts (e.g., wrist, spine) to support diagnostic activities and follow-up exams through the integration of morphological and tissue information. We propose different methods for the integration of morphological information, retrieved from the geometrical analysis of 3D surface models, with tissue information extracted from volume images. For the qualitative and quantitative validation, we will discuss the localisation of bone erosion sites on the wrists to monitor rheumatic diseases and the characterisation of the three functional regions of the spinal vertebrae to study the presence of osteoporotic fractures. The proposed approach supports the quantitative and visual evaluation of possible damages, surgery planning, and early diagnosis or follow-up studies. Finally, our analysis is general enough to be applied to different districts.
pyBibX -- A Python Library for Bibliometric and Scientometric Analysis Powered with Artificial Intelligence Tools
Venue: arXiv cs.DL 2023
Year: 2023

Bibliometric and Scientometric analyses offer invaluable perspectives on the complex research terrain and collaborative dynamics spanning diverse academic disciplines. This paper presents pyBibX, a python library devised to conduct comprehensive bibliometric and scientometric analyses on raw data files sourced from Scopus, Web of Science, and PubMed, seamlessly integrating state of the art AI capabilities into its core functionality. The library executes a comprehensive EDA, presenting outcomes via visually appealing graphical illustrations. Network capabilities have been deftly integrated, encompassing Citation, Collaboration, and Similarity Analysis. Furthermore, the library incorporates AI capabilities, including Embedding vectors, Topic Modeling, Text Summarization, and other general Natural Language Processing tasks, employing models such as Sentence-BERT, BerTopic, BERT, chatGPT, and PEGASUS. As a demonstration, we have analyzed 184 documents associated with multiple-criteria decision analysis published between 1984 and 2023. The EDA emphasized a growing fascination with decision-making and fuzzy logic methodologies. Next, Network Analysis further accentuated the significance of central authors and intra-continental collaboration, identifying Canada and China as crucial collaboration hubs. Finally, AI Analysis distinguished two primary topics and chatGPT preeminence in Text Summarization. It also proved to be an indispensable instrument for interpreting results, as our library enables researchers to pose inquiries to chatGPT regarding bibliometric outcomes. Even so, data homogeneity remains a daunting challenge due to database inconsistencies. PyBibX is the first application integrating cutting-edge AI capabilities for analyzing scientific publications, enabling researchers to examine and interpret these outcomes more effectively.
A Critique of Czerwinski's "Separation of ${\rm PSPACE}$ and ${\rm EXP}$"
Venue: arXiv cs.CC 2023
Year: 2023

Czerwinski's paper "Separation of ${\rm PSPACE}$ and ${\rm EXP}$" [Cze21] claims to prove that ${\rm PSPACE} \neq {\rm EXP}$ by showing there is no length-increasing polynomial-time reduction from a given ${\rm EXP}$-complete set to a given ${\rm PSPACE}$-complete set. However, in this critique, we show that there are fundamental flaws within the paper's approach and provide a counterexample to one of the paper's theorems, which makes the proposed proof that ${\rm PSPACE} \neq {\rm EXP}$ insufficient.
A logical word embedding for learning grammar
Venue: arXiv cs.CL 2023
Year: 2023

We introduce the logical grammar emdebbing (LGE), a model inspired by pregroup grammars and categorial grammars to enable unsupervised inference of lexical categories and syntactic rules from a corpus of text. LGE produces comprehensible output summarizing its inferences, has a completely transparent process for producing novel sentences, and can learn from as few as a hundred sentences.
Barriers and Self-Efficacy: A Large-Scale Study on the Impact of OSS Courses on Student Perceptions
Venue: arXiv cs.SE 2023
Year: 2023

Open source software (OSS) development offers a unique opportunity for students in Software Engineering to experience and participate in large-scale software development, however, the impact of such courses on students' self-efficacy and the challenges faced by students are not well understood. This paper aims to address this gap by analyzing data from multiple instances of OSS development courses at universities in different countries and reporting on how students' self-efficacy changed as a result of taking the course, as well as the barriers and challenges faced by students.
Towards Robust Text-Prompted Semantic Criterion for In-the-Wild Video Quality Assessment
Venue: arXiv cs.CV 2023
Year: 2023

The proliferation of videos collected during in-the-wild natural settings has pushed the development of effective Video Quality Assessment (VQA) methodologies. Contemporary supervised opinion-driven VQA strategies predominantly hinge on training from expensive human annotations for quality scores, which limited the scale and distribution of VQA datasets and consequently led to unsatisfactory generalization capacity of methods driven by these data. On the other hand, although several handcrafted zero-shot quality indices do not require training from human opinions, they are unable to account for the semantics of videos, rendering them ineffective in comprehending complex authentic distortions (e.g., white balance, exposure) and assessing the quality of semantic content within videos. To address these challenges, we introduce the text-prompted Semantic Affinity Quality Index (SAQI) and its localized version (SAQI-Local) using Contrastive Language-Image Pre-training (CLIP) to ascertain the affinity between textual prompts and visual features, facilitating a comprehensive examination of semantic quality concerns without the reliance on human quality annotations. By amalgamating SAQI with existing low-level metrics, we propose the unified Blind Video Quality Index (BVQI) and its improved version, BVQI-Local, which demonstrates unprecedented performance, surpassing existing zero-shot indices by at least 24\% on all datasets. Moreover, we devise an efficient fine-tuning scheme for BVQI-Local that jointly optimizes text prompts and final fusion weights, resulting in state-of-the-art performance and superior generalization ability in comparison to prevalent opinion-driven VQA methods. We conduct comprehensive analyses to investigate different quality concerns of distinct indices, demonstrating the effectiveness and rationality of our design.