OpenRaiser/Intern-Atlas huggingface dataset:
Deep Learning Model Reuse in the HuggingFace Community: Challenges, Benefit and Trends
Venue: arXiv cs.SE 2024
Year: 2024

The ubiquity of large-scale Pre-Trained Models (PTMs) is on the rise, sparking interest in model hubs, and dedicated platforms for hosting PTMs. Despite this trend, a comprehensive exploration of the challenges that users encounter and how the community leverages PTMs remains lacking. To address this gap, we conducted an extensive mixed-methods empirical study by focusing on discussion forums and the model hub of HuggingFace, the largest public model hub. Based on our qualitative analysis, we present a taxonomy of the challenges and benefits associated with PTM reuse within this community. We then conduct a quantitative study to track model-type trends and model documentation evolution over time. Our findings highlight prevalent challenges such as limited guidance for beginner users, struggles with model output comprehensibility in training or inference, and a lack of model understanding. We also identified interesting trends among models where some models maintain high upload rates despite a decline in topics related to them. Additionally, we found that despite the introduction of model documentation tools, its quantity has not increased over time, leading to difficulties in model comprehension and selection among users. Our study sheds light on new challenges in reusing PTMs that were not reported before and we provide recommendations for various stakeholders involved in PTM reuse.
Why People Still Fall for Phishing Emails: An Empirical Investigation into How Users Make Email Response Decisions
Venue: arXiv cs.CR 2024
Year: 2024

Despite technical and non-technical countermeasures, humans continue to be tricked by phishing emails. How users make email response decisions is a missing piece in the puzzle to identifying why people still fall for phishing emails. We conducted an empirical study using a think-aloud method to investigate how people make 'response decisions' while reading emails. The grounded theory analysis of the in-depth qualitative data has enabled us to identify different elements of email users' decision-making that influence their email response decisions. Furthermore, we developed a theoretical model that explains how people could be driven to respond to emails based on the identified elements of users' email decision-making processes and the relationships uncovered from the data. The findings provide deeper insights into phishing email susceptibility due to people's email response decision-making behavior. We also discuss the implications of our findings for designers and researchers working in anti-phishing training, education, and awareness interventions
A modular architecture for IMU-based data gloves
Venue: arXiv cs.AR 2024
Year: 2024

The flexibility and range of motion in human hands play a crucial role in human interaction with the environment and have been studied across different fields. Researchers explored various technological solutions for gathering information from the hands. These solutions include tracking hand motion through cameras or wearable sensors and using wearable sensors to measure the position and pressure of contact points. Data gloves can collect both types of information by utilizing inertial measurement units, flex sensors, magnetic trackers for motion tracking, and force resistors or touch sensors for contact measurement. Although there are commercially available data gloves, researchers often create custom data gloves to achieve the desired flexibility and control over the hardware. However, the existing literature lacks standardization and the reuse of previously designed data gloves. As a result, many gloves with unclear characteristics exist, which makes replication challenging and negatively impacts the reproducibility of studies. This work proposes a modular, open hardware and software architecture for creating customized data gloves based on IMU technology. We also provide an architecture implementation along with an experimental protocol to evaluate device performance.
SpecLLM: Exploring Generation and Review of VLSI Design Specification with Large Language Model
Venue: arXiv cs.AR 2024
Year: 2024

The development of architecture specifications is an initial and fundamental stage of the integrated circuit (IC) design process. Traditionally, architecture specifications are crafted by experienced chip architects, a process that is not only time-consuming but also error-prone. Mistakes in these specifications may significantly affect subsequent stages of chip design. Despite the presence of advanced electronic design automation (EDA) tools, effective solutions to these specification-related challenges remain scarce. Since writing architecture specifications is naturally a natural language processing (NLP) task, this paper pioneers the automation of architecture specification development with the advanced capabilities of large language models (LLMs). Leveraging our definition and dataset, we explore the application of LLMs in two key aspects of architecture specification development: (1) Generating architecture specifications, which includes both writing specifications from scratch and converting RTL code into detailed specifications. (2) Reviewing existing architecture specifications. We got promising results indicating that LLMs may revolutionize how these critical specification documents are developed in IC design nowadays. By reducing the effort required, LLMs open up new possibilities for efficiency and accuracy in this crucial aspect of chip design.
Audio-Infused Automatic Image Colorization by Exploiting Audio Scene Semantics
Venue: arXiv cs.CV 2024
Year: 2024

Automatic image colorization is inherently an ill-posed problem with uncertainty, which requires an accurate semantic understanding of scenes to estimate reasonable colors for grayscale images. Although recent interaction-based methods have achieved impressive performance, it is still a very difficult task to infer realistic and accurate colors for automatic colorization. To reduce the difficulty of semantic understanding of grayscale scenes, this paper tries to utilize corresponding audio, which naturally contains extra semantic information about the same scene. Specifically, a novel and pluggable audio-infused automatic image colorization (AIAIC) method is proposed, which consists of three stages. First, we take color image semantics as a bridge and pretrain a colorization network guided by color image semantics. Second, the natural co-occurrence of audio and video is utilized to learn the color semantic correlations between audio and visual scenes. Third, the implicit audio semantic representation is fed into the pretrained network to finally realize the audio-guided colorization. The whole process is trained in a self-supervised manner without human annotation. Experiments demonstrate that audio guidance can effectively improve the performance of automatic colorization, especially for some scenes that are difficult to understand only from visual modality.
A Lagrange-Newton Approach to Smoothing-and-Mapping
Venue: arXiv cs.RO 2024
Year: 2024

In this report we explore the application of the Lagrange-Newton method to the SAM (smoothing-and-mapping) problem in mobile robotics. In Lagrange-Newton SAM, the angular component of each pose vector is expressed by orientation vectors and treated through Lagrange constraints. This is different from the typical Gauss-Newton approach where variations need to be mapped back and forth between Euclidean space and a manifold suitable for rotational components. We derive equations for five different types of measurements between robot poses: translation, distance, and rotation from odometry in the plane, as well as home-vector angle and compass angle from visual homing. We demonstrate the feasibility of the Lagrange-Newton approach for a simple example related to a cleaning robot scenario.
Do You Guys Want to Dance: Zero-Shot Compositional Human Dance Generation with Multiple Persons
Venue: arXiv cs.CV 2024
Year: 2024

Human dance generation (HDG) aims to synthesize realistic videos from images and sequences of driving poses. Despite great success, existing methods are limited to generating videos of a single person with specific backgrounds, while the generalizability for real-world scenarios with multiple persons and complex backgrounds remains unclear. To systematically measure the generalizability of HDG models, we introduce a new task, dataset, and evaluation protocol of compositional human dance generation (cHDG). Evaluating the state-of-the-art methods on cHDG, we empirically find that they fail to generalize to real-world scenarios. To tackle the issue, we propose a novel zero-shot framework, dubbed MultiDance-Zero, that can synthesize videos consistent with arbitrary multiple persons and background while precisely following the driving poses. Specifically, in contrast to straightforward DDIM or null-text inversion, we first present a pose-aware inversion method to obtain the noisy latent code and initialization text embeddings, which can accurately reconstruct the composed reference image. Since directly generating videos from them will lead to severe appearance inconsistency, we propose a compositional augmentation strategy to generate augmented images and utilize them to optimize a set of generalizable text embeddings. In addition, consistency-guided sampling is elaborated to encourage the background and keypoints of the estimated clean image at each reverse step to be close to those of the reference image, further improving the temporal consistency of generated videos. Extensive qualitative and quantitative results demonstrate the effectiveness and superiority of our approach.
Organizing Scientific Knowledge From Energy System Research Using the Open Research Knowledge Graph
Venue: arXiv cs.DL 2024
Year: 2024

Engineering sciences, such as energy system research, play an important role in developing solutions to technical, environmental, economic, and social challenges of our modern society. In this context, the transformation of energy systems into climate-neutral systems is one of the key strategies for mitigating climate change. For the transformation of energy systems, engineers model, simulate and analyze scenarios and transformation pathways to initiate debates about possible transformation strategies. For these debates and research in general, all steps of the research process must be traceable to guarantee the trustworthiness of published results, avoid redundancies, and ensure their social acceptance. However, the analysis of energy systems is an interdisciplinary field as the investigations of large, complex energy systems often require the use of different software applications and large amounts of heterogeneous data. Engineers must therefore communicate, understand, and (re)use heterogeneous scientific knowledge and data. Although the importance of FAIR scientific knowledge and data in the engineering sciences and energy system research is increasing, little research has been conducted on this topic. When it comes to publishing scientific knowledge and data from publications, software, and datasets (such as models, scenarios, and simulations) openly available and transparent, energy system research lags behind other research domains. According to Schmitt et al. and Nieße et al., engineers need technical support in the form of infrastructures, services, and terminologies to improve communication, understanding, and (re)use of scientific knowledge and data.
Determining hulls of generalized Reed-Solomon codes from algebraic geometry codes
Venue: arXiv cs.IT 2024
Year: 2024

In this paper, we provide conditions that hulls of generalized Reed-Solomon (GRS) codes are also GRS codes from algebraic geometry codes. If the conditions are not satisfied, we provide a method of linear algebra to find the bases of hulls of GRS codes and give formulas to compute their dimensions. Besides, we explain that the conditions are too good to be improved by some examples. Moreover, we show self-orthogonal and self-dual GRS codes.
Text Categorization Can Enhance Domain-Agnostic Stopword Extraction
Venue: arXiv cs.CL 2024
Year: 2024

This paper investigates the role of text categorization in streamlining stopword extraction in natural language processing (NLP), specifically focusing on nine African languages alongside French. By leveraging the MasakhaNEWS, African Stopwords Project, and MasakhaPOS datasets, our findings emphasize that text categorization effectively identifies domain-agnostic stopwords with over 80% detection success rate for most examined languages. Nevertheless, linguistic variances result in lower detection rates for certain languages. Interestingly, we find that while over 40% of stopwords are common across news categories, less than 15% are unique to a single category. Uncommon stopwords add depth to text but their classification as stopwords depends on context. Therefore combining statistical and linguistic approaches creates comprehensive stopword lists, highlighting the value of our hybrid method. This research enhances NLP for African languages and underscores the importance of text categorization in stopword extraction.
Characterizing Perspective Error in Voxel-Based Lidar Scan Matching
Venue: arXiv cs.RO 2024
Year: 2024

This paper quantifies an error source that limits the accuracy of lidar scan matching, particularly for voxel-based methods. Lidar scan matching, which is used in dead reckoning (also known as lidar odometry) and mapping, computes the rotation and translation that best align a pair of point clouds. Perspective errors occur when a scene is viewed from different angles, with different surfaces becoming visible or occluded from each viewpoint. To explain perspective anomalies observed in data, this paper models perspective errors for two objects representative of urban landscapes: a cylindrical column and a dual-wall corner. For each object, we provide an analytical model of the perspective error for voxel-based lidar scan matching. We then analyze how perspective errors accumulate as a lidar-equipped vehicle moves past these objects.
Symbolic Equation Solving via Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

Machine-learning methods are gradually being adopted in a wide variety of social, economic, and scientific contexts, yet they are notorious for struggling with exact mathematics. A typical example is computer algebra, which includes tasks like simplifying mathematical terms, calculating formal derivatives, or finding exact solutions of algebraic equations. Traditional software packages for these purposes are commonly based on a huge database of rules for how a specific operation (e.g., differentiation) transforms a certain term (e.g., sine function) into another one (e.g., cosine function). These rules have usually needed to be discovered and subsequently programmed by humans. Efforts to automate this process by machine-learning approaches are faced with challenges like the singular nature of solutions to mathematical problems, when approximations are unacceptable, as well as hallucination effects leading to flawed reasoning. We propose a novel deep-learning interface involving a reinforcement-learning agent that operates a symbolic stack calculator to explore mathematical relations. By construction, this system is capable of exact transformations and immune to hallucination. Using the paradigmatic example of solving linear equations in symbolic form, we demonstrate how our reinforcement-learning agent autonomously discovers elementary transformation rules and step-by-step solutions.
Multi-Agent Diagnostics for Robustness via Illuminated Diversity
Venue: arXiv cs.LG 2024
Year: 2024

In the rapidly advancing field of multi-agent systems, ensuring robustness in unfamiliar and adversarial settings is crucial. Notwithstanding their outstanding performance in familiar environments, these systems often falter in new situations due to overfitting during the training phase. This is especially pronounced in settings where both cooperative and competitive behaviours are present, encapsulating a dual nature of overfitting and generalisation challenges. To address this issue, we present Multi-Agent Diagnostics for Robustness via Illuminated Diversity (MADRID), a novel approach for generating diverse adversarial scenarios that expose strategic vulnerabilities in pre-trained multi-agent policies. Leveraging the concepts from open-ended learning, MADRID navigates the vast space of adversarial settings, employing a target policy's regret to gauge the vulnerabilities of these settings. We evaluate the effectiveness of MADRID on the 11vs11 version of Google Research Football, one of the most complex environments for multi-agent reinforcement learning. Specifically, we employ MADRID for generating a diverse array of adversarial settings for TiZero, the state-of-the-art approach which "masters" the game through 45 days of training on a large-scale distributed infrastructure. We expose key shortcomings in TiZero's tactical decision-making, underlining the crucial importance of rigorous evaluation in multi-agent systems.
Visualization of rank-citation curves for fast detection of h-index anomalies in university metrics
Venue: arXiv cs.DL 2024
Year: 2024

University rankings, despite facing criticism, continue to maintain their popularity. In the 2023 Scopus Ranking of Ukrainian Universities, certain institutions stood out due to their high h-index, despite modest publication and citation numbers. This phenomenon can be attributed to influential research topics or involvement in international collaborative research. However, these results may also be due to the authors' own efforts to increase the number of citations of their publications in order to improve their h-index. To investigate this, the publications from the top 30 universities in the ranking were analysed, revealing humpback rank-citation curves for two universities. These humpbacks indicate unusual trends in the citation data, especially considering the high percentage of self-citations and FWCI of analysed papers. While quantitative analysis has limitations, the combination of humped rank-citation curves, self-citations, FWCI, and previous research findings raises concerns about the possible causes of these anomalies in the citation data of the analysed universities. The method presented in this paper can aid ranking compilers and citation databases managers in identifying potential instances of citation data anomalies, emphasizing the importance of expert assessment for accurate conclusions.
SpeechGPT-Gen: Scaling Chain-of-Information Speech Generation
Venue: arXiv cs.CL 2024
Year: 2024

Benefiting from effective speech modeling, current Speech Large Language Models (SLLMs) have demonstrated exceptional capabilities in in-context speech generation and efficient generalization to unseen speakers. However, the prevailing information modeling process is encumbered by certain redundancies, leading to inefficiencies in speech generation. We propose Chain-of-Information Generation (CoIG), a method for decoupling semantic and perceptual information in large-scale speech generation. Building on this, we develop SpeechGPT-Gen, an 8-billion-parameter SLLM efficient in semantic and perceptual information modeling. It comprises an autoregressive model based on LLM for semantic information modeling and a non-autoregressive model employing flow matching for perceptual information modeling. Additionally, we introduce the novel approach of infusing semantic information into the prior distribution to enhance the efficiency of flow matching. Extensive experimental results demonstrate that SpeechGPT-Gen markedly excels in zero-shot text-to-speech, zero-shot voice conversion, and speech-to-speech dialogue, underscoring CoIG's remarkable proficiency in capturing and modeling speech's semantic and perceptual dimensions. Code and models are available at https://github.com/0nutation/SpeechGPT.
RIS Empowered Near-Field Covert Communications
Venue: arXiv cs.IT 2024
Year: 2024

This paper studies an extremely large-scale reconfigurable intelligent surface (XL-RIS) empowered covert communication system in the near-field region. Alice covertly transmits messages to Bob with the assistance of the XL-RIS, while evading detection by Willie. To enhance the covert communication performance, we maximize the achievable covert rate by jointly optimizing the hybrid analog and digital beamformers at Alice, as well as the reflection coefficient matrix at the XL-RIS. An alternating optimization algorithm is proposed to solve the joint beamforming design problem. For the hybrid beamformer design, a semi-closed-form solution for fully digital beamformer is first obtained by a weighted minimum mean-square error based algorithm, then the baseband digital and analog beamformers at Alice are designed by approximating the fully digital beamformer via manifold optimization. For the XL-RIS's reflection coefficient matrix design, a low-complexity alternating direction method of multipliers based algorithm is proposed to address the challenge of large-scale variables and unit-modulus constraints. Numerical results unveil that i) the near-field communications can achieve a higher covert rate than the far-field covert communications in general, and still realize covert transmission even if Willie is located at the same direction as Bob and closer to the XL-RIS; ii) the proposed algorithm can enhance the covert rate significantly compared to the benchmark schemes; iii) the proposed algorithm leads to a beam diffraction pattern that can bypass Willie and achieve high-rate covert transmission to Bob.
Investigating the Performance of Soft Robotic Adaptive Feet with Longitudinal and Transverse Arches
Venue: arXiv cs.RO 2024
Year: 2024

Biped robots usually adopt feet with a rigid structure that simplifies walking on flat grounds and yet hinders ground adaptation in unstructured environments, thus jeopardizing stability. We recently explored in the SoftFoot the idea of adapting a robotic foot to ground irregularities along the sagittal plane. Building on the previous results, we propose in this paper a novel robotic foot able to adapt both in the sagittal and frontal planes, similarly to the human foot. It features five parallel modules with intrinsic longitudinal adaptability that can be combined in many possible designs through optional rigid or elastic connections. By following a methodological design approach, we narrow down the design space to five candidate foot designs and implement them on a modular system. Prototypes are tested experimentally via controlled application of force, through a robotic arm, onto a sensorized plate endowed with different obstacles. Their performance is compared, using also a rigid foot and the previous SoftFoot as a baseline. Analysis of footprint stability shows that the introduction of the transverse arch, by elastically connecting the five parallel modules, is advantageous for obstacle negotiation, especially when obstacles are located under the forefoot. In addition to biped robots' locomotion, this finding might also benefit lower-limb prostheses design.