OpenRaiser/Intern-Atlas huggingface dataset:
Signformer is all you need: Towards Edge AI for Sign Language
Venue: arXiv cs.CL 2024
Year: 2024

Sign language translation, especially in gloss-free paradigm, is confronting a dilemma of impracticality and unsustainability due to growing resource-intensive methodologies. Contemporary state-of-the-arts (SOTAs) have significantly hinged on pretrained sophiscated backbones such as Large Language Models (LLMs), embedding sources, or extensive datasets, inducing considerable parametric and computational inefficiency for sustainable use in real-world scenario. Despite their success, following this research direction undermines the overarching mission of this domain to create substantial value to bridge hard-hearing and common populations. Committing to the prevailing trend of LLM and Natural Language Processing (NLP) studies, we pursue a profound essential change in architecture to achieve ground-up improvements without external aid from pretrained models, prior knowledge transfer, or any NLP strategies considered not-from-scratch. Introducing Signformer, a from-scratch Feather-Giant transforming the area towards Edge AI that redefines extremities of performance and efficiency with LLM-competence and edgy-deployable compactness. In this paper, we present nature analysis of sign languages to inform our algorithmic design and deliver a scalable transformer pipeline with convolution and attention novelty. We achieve new 2nd place on leaderboard with a parametric reduction of 467-1807x against the finests as of 2024 and outcompete almost every other methods in a lighter configuration of 0.57 million parameters.
Veryl: A New Hardware Description Language as an Altarnative to SystemVerilog
Venue: arXiv cs.AR 2024
Year: 2024

Veryl, a hardware description language based on SystemVerilog, offers optimized syntax tailored for logic design, ensuring synthesizability and simplifying common constructs. It prioritizes interoperability with SystemVerilog, allowing for smooth integration with existing projects while maintaining high readability. Additionally, Veryl includes a comprehensive set of development support tools, such as package managers and real-time checkers, to boost productivity and streamline the design process. These features empower designers to conduct high-quality hardware design efficiently.
HapKnob -- A Motorized Shape-changing Haptic Knob Interface
Venue: arXiv cs.HC 2024
Year: 2024

The absence of physical interfaces creates challenges when interacting with touchscreen technology. This study aims to investigate an innovative haptic solution for interacting with graphical user interfaces. A motorized shape-changing rotary knob interface, HapKnob, has been developed, achieving seven distinctive shape configurations and various force feedback renderings. HapKnob presents a compact design and can provide additional configurations and combinations based on user needs. The anticipated results hold the potential to advance the development of future user interfaces, especially in situations where visual interaction is unavailable.
Evaluating LLMs Capabilities Towards Understanding Social Dynamics
Venue: arXiv cs.LG 2024
Year: 2024

Social media discourse involves people from different backgrounds, beliefs, and motives. Thus, often such discourse can devolve into toxic interactions. Generative Models, such as Llama and ChatGPT, have recently exploded in popularity due to their capabilities in zero-shot question-answering. Because these models are increasingly being used to ask questions of social significance, a crucial research question is whether they can understand social media dynamics. This work provides a critical analysis regarding generative LLM's ability to understand language and dynamics in social contexts, particularly considering cyberbullying and anti-cyberbullying (posts aimed at reducing cyberbullying) interactions. Specifically, we compare and contrast the capabilities of different large language models (LLMs) to understand three key aspects of social dynamics: language, directionality, and the occurrence of bullying/anti-bullying messages. We found that while fine-tuned LLMs exhibit promising results in some social media understanding tasks (understanding directionality), they presented mixed results in others (proper paraphrasing and bullying/anti-bullying detection). We also found that fine-tuning and prompt engineering mechanisms can have positive effects in some tasks. We believe that a understanding of LLM's capabilities is crucial to design future models that can be effectively used in social applications.
AMaze: An intuitive benchmark generator for fast prototyping of generalizable agents
Venue: arXiv cs.RO 2024
Year: 2024

Traditional approaches to training agents have generally involved a single, deterministic environment of minimal complexity to solve various tasks such as robot locomotion or computer vision. However, agents trained in static environments lack generalization capabilities, limiting their potential in broader scenarios. Thus, recent benchmarks frequently rely on multiple environments, for instance, by providing stochastic noise, simple permutations, or altogether different settings. In practice, such collections result mainly from costly human-designed processes or the liberal use of random number generators. In this work, we introduce AMaze, a novel benchmark generator in which embodied agents must navigate a maze by interpreting visual signs of arbitrary complexities and deceptiveness. This generator promotes human interaction through the easy generation of feature-specific mazes and an intuitive understanding of the resulting agents' strategies. As a proof-of-concept, we demonstrate the capabilities of the generator in a simple, fully discrete case with limited deceptiveness. Agents were trained under three different regimes (one-shot, scaffolding, interactive), and the results showed that the latter two cases outperform direct training in terms of generalization capabilities. Indeed, depending on the combination of generalization metric, training regime, and algorithm, the median gain ranged from 50% to 100% and maximal performance was achieved through interactive training, thereby demonstrating the benefits of a controllable human-in-the-loop benchmark generator.
Incremental Label Distribution Learning with Scalable Graph Convolutional Networks
Venue: arXiv cs.LG 2024
Year: 2024

Label Distribution Learning (LDL) is an effective approach for handling label ambiguity, as it can analyze all labels at once and indicate the extent to which each label describes a given sample. Most existing LDL methods consider the number of labels to be static. However, in various LDL-specific contexts (e.g., disease diagnosis), the label count grows over time (such as the discovery of new diseases), a factor that existing methods overlook. Learning samples with new labels directly means learning all labels at once, thus wasting more time on the old labels and even risking overfitting the old labels. At the same time, learning new labels by the LDL model means reconstructing the inter-label relationships. How to make use of constructed relationships is also a crucial challenge. To tackle these challenges, we introduce Incremental Label Distribution Learning (ILDL), analyze its key issues regarding training samples and inter-label relationships, and propose Scalable Graph Label Distribution Learning (SGLDL) as a practical framework for implementing ILDL. Specifically, in SGLDL, we develop a New-label-aware Gradient Compensation Loss to speed up the learning of new labels and represent inter-label relationships as a graph to reduce the time required to reconstruct inter-label relationships. Experimental results on the classical LDL dataset show the clear advantages of unique algorithms and illustrate the importance of a dedicated design for the ILDL problem.
Globally Correlation-Aware Hard Negative Generation
Venue: arXiv cs.CV 2024
Year: 2024

Hard negative generation aims to generate informative negative samples that help to determine the decision boundaries and thus facilitate advancing deep metric learning. Current works select pair/triplet samples, learn their correlations, and fuse them to generate hard negatives. However, these works merely consider the local correlations of selected samples, ignoring global sample correlations that would provide more significant information to generate more informative negatives. In this work, we propose a Globally Correlation-Aware Hard Negative Generation (GCA-HNG) framework, which first learns sample correlations from a global perspective and exploits these correlations to guide generating hardness-adaptive and diverse negatives. Specifically, this approach begins by constructing a structured graph to model sample correlations, where each node represents a specific sample and each edge represents the correlations between corresponding samples. Then, we introduce an iterative graph message propagation to propagate the messages of node and edge through the whole graph and thus learn the sample correlations globally. Finally, with the guidance of the learned global correlations, we propose a channel-adaptive manner to combine an anchor and multiple negatives for HNG. Compared to current methods, GCA-HNG allows perceiving sample correlations with numerous negatives from a global and comprehensive perspective and generates the negatives with better hardness and diversity. Extensive experiment results demonstrate that the proposed GCA-HNG is superior to related methods on four image retrieval benchmark datasets. Codes and trained models are available at \url{https://github.com/PWenJay/GCA-HNG}.
Long-term Detection System for Six Kinds of Abnormal Behavior of the Elderly Living Alone
Venue: arXiv cs.LG 2024
Year: 2024

The proportion of elderly people is increasing worldwide, particularly those living alone in Japan. As elderly people get older, their risks of physical disabilities and health issues increase. To automatically discover these issues at a low cost in daily life, sensor-based detection in a smart home is promising. As part of the effort towards early detection of abnormal behaviors, we propose a simulator-based detection systems for six typical anomalies: being semi-bedridden, being housebound, forgetting, wandering, fall while walking and fall while standing. Our detection system can be customized for various room layout, sensor arrangement and resident's characteristics by training detection classifiers using the simulator with the parameters fitted to individual cases. Considering that the six anomalies that our system detects have various occurrence durations, such as being housebound for weeks or lying still for seconds after a fall, the detection classifiers of our system produce anomaly labels depending on each anomaly's occurrence duration, e.g., housebound per day and falls per second. We propose a method that standardizes the processing of sensor data, and uses a simple detection approach. Although the validity depends on the realism of the simulation, numerical evaluations using sensor data that includes a variety of resident behavior patterns over nine years as test data show that (1) the methods for detecting wandering and falls are comparable to previous methods, and (2) the methods for detecting being semi-bedridden, being housebound, and forgetting achieve a sensitivity of over 0.9 with fewer than one false alarm every 50 days.
DMQR-RAG: Diverse Multi-Query Rewriting for RAG
Venue: arXiv cs.IR 2024
Year: 2024

Large language models often encounter challenges with static knowledge and hallucinations, which undermine their reliability. Retrieval-augmented generation (RAG) mitigates these issues by incorporating external information. However, user queries frequently contain noise and intent deviations, necessitating query rewriting to improve the relevance of retrieved documents. In this paper, we introduce DMQR-RAG, a Diverse Multi-Query Rewriting framework designed to improve the performance of both document retrieval and final responses in RAG. Specifically, we investigate how queries with varying information quantities can retrieve a diverse array of documents, presenting four rewriting strategies that operate at different levels of information to enhance the performance of baseline approaches. Additionally, we propose an adaptive strategy selection method that minimizes the number of rewrites while optimizing overall performance. Our methods have been rigorously validated through extensive experiments conducted in both academic and industry settings.
MecQaBot: A Modular Robot Sensing and Wireless Mechatronics Framework for Education and Research
Venue: arXiv cs.RO 2024
Year: 2024

We introduce MecQaBot, an open-source, affordable, and modular autonomous mobile robotics framework developed for education and research at Macquarie University, School of Engineering, since 2019. This platform aims to provide students and researchers with an accessible means for exploring autonomous robotics and fostering hands-on learning and innovation. Over the five years, the platform has engaged more than 240 undergraduate and postgraduate students across various engineering disciplines. The framework addresses the growing need for practical robotics training in response to the expanding robotics field and its increasing relevance in industry and academia. The platform facilitates teaching critical concepts in sensing, programming, hardware-software integration, and autonomy within real-world contexts, igniting student interest and engagement. We describe the design and evolution of the MecQaBot framework and the underlying principles of scalability and flexibility, which are keys to its success. Complete documentation: https://github.com/AliceJames-1/MecQaBot
Writing Style Matters: An Examination of Bias and Fairness in Information Retrieval Systems
Venue: arXiv cs.IR 2024
Year: 2024

The rapid advancement of Language Model technologies has opened new opportunities, but also introduced new challenges related to bias and fairness. This paper explores the uncharted territory of potential biases in state-of-the-art universal text embedding models towards specific document and query writing styles within Information Retrieval (IR) systems. Our investigation reveals that different embedding models exhibit different preferences of document writing style, while more informal and emotive styles are less favored by most embedding models. In terms of query writing styles, many embedding models tend to match the style of the query with the style of the retrieved documents, but some show a consistent preference for specific styles. Text embedding models fine-tuned on synthetic data generated by LLMs display a consistent preference for certain style of generated data. These biases in text embedding based IR systems can inadvertently silence or marginalize certain communication styles, thereby posing a significant threat to fairness in information retrieval. Finally, we also compare the answer styles of Retrieval Augmented Generation (RAG) systems based on different LLMs and find out that most text embedding models are biased towards LLM's answer styles when used as evaluation metrics for answer correctness. This study sheds light on the critical issue of writing style based bias in IR systems, offering valuable insights for the development of more fair and robust models.
ViSTa Dataset: Do vision-language models understand sequential tasks?
Venue: arXiv cs.CV 2024
Year: 2024

Using vision-language models (VLMs) as reward models in reinforcement learning holds promise for reducing costs and improving safety. So far, VLM reward models have only been used for goal-oriented tasks, where the agent must reach a particular final outcome. We explore VLMs' potential to supervise tasks that cannot be scored by the final state alone. To this end, we introduce ViSTa, a dataset for evaluating Vision-based understanding of Sequential Tasks. ViSTa comprises over 4,000 videos with step-by-step descriptions in virtual home, Minecraft, and real-world environments. Its novel hierarchical structure -- basic single-step tasks composed into more and more complex sequential tasks -- allows a fine-grained understanding of how well VLMs can judge tasks with varying complexity. To illustrate this, we use ViSTa to evaluate state-of-the-art VLMs, including CLIP, ViCLIP, and GPT-4o. We find that, while they are all good at object recognition, they fail to understand sequential tasks, with only GPT-4o achieving non-trivial performance.
On Minimal and Minimum Cylindrical Algebraic Decompositions
Venue: arXiv cs.SC 2024
Year: 2024

We consider cylindrical algebraic decompositions (CADs) as a tool for representing semi-algebraic subsets of $\mathbb{R}^n$. In this framework, a CAD $\mathscr{C}$ is adapted to a given set $S$ if $S$ is a union of cells of $\mathscr{C}$. Different algorithms computing an adapted CAD may produce different outputs, usually with redundant cell divisions. In this paper we analyse the possibility to remove the superfluous data. More precisely we consider the set CAD$(S)$ of CADs that are adapted to $S$, endowed with the refinement partial order and we study the existence of minimal and minimum elements in this poset. We show that for every semi-algebraic set $S$ of $\mathbb{R}^n$ and every CAD $\mathscr{C}$ adapted to $S$, there is a minimal CAD adapted to $S$ and smaller (i.e. coarser) than or equal to $\mathscr{C}$. Moreover, when $n=1$ or $n=2$, we strengthen this result by proving the existence of a minimum element in CAD$(S)$. Astonishingly for $n \geq 3$, there exist semi-algebraic sets whose associated poset of adapted CADs does not admit a minimum. We prove this result by providing explicit examples. We finally use a reduction relation on CAD$(S)$ to define an algorithm for the computation of minimal CADs. We conclude with a characterization of those semi-algebraic sets $S$ for which CAD$(S)$ has a minimum by means of confluence of the associated reduction system.
Advanced Plaque Modeling for Atherosclerosis Detection Using Molecular Communication
Venue: arXiv cs.ET 2024
Year: 2024

As one of the most prevalent diseases worldwide, plaque formation in human arteries, known as atherosclerosis, is the focus of many research efforts. Previously, molecular communication (MC) models have been proposed to capture and analyze the natural processes inside the human body and to support the development of diagnosis and treatment methods. In the future, synthetic MC networks are envisioned to span the human body as part of the Internet of Bio-Nano Things (IoBNT), turning blood vessels into physical communication channels. By observing and characterizing changes in these channels, MC networks could play an active role in detecting diseases like atherosclerosis. In this paper, building on previous preliminary work for simulating an MC scenario in a plaque-obstructed blood vessel, we evaluate different analytical models for non-Newtonian flow and derive associated channel impulse responses (CIRs). Additionally, we add the crucial factor of flow pulsatility to our simulation model and investigate the effect of the systole-diastole cycle on the received particles across the plaque channel. We observe a significant influence of the plaque on the channel in terms of the flow profile and CIR across different emission times in the cycle. These metrics could act as crucial indicators for early non-invasive plaque detection in advanced future MC methods.
A Resource Efficient Fusion Network for Object Detection in Bird's-Eye View using Camera and Raw Radar Data
Venue: arXiv cs.CV 2024
Year: 2024

Cameras can be used to perceive the environment around the vehicle, while affordable radar sensors are popular in autonomous driving systems as they can withstand adverse weather conditions unlike cameras. However, radar point clouds are sparser with low azimuth and elevation resolution that lack semantic and structural information of the scenes, resulting in generally lower radar detection performance. In this work, we directly use the raw range-Doppler (RD) spectrum of radar data, thus avoiding radar signal processing. We independently process camera images within the proposed comprehensive image processing pipeline. Specifically, first, we transform the camera images to Bird's-Eye View (BEV) Polar domain and extract the corresponding features with our camera encoder-decoder architecture. The resultant feature maps are fused with Range-Azimuth (RA) features, recovered from the RD spectrum input from the radar decoder to perform object detection. We evaluate our fusion strategy with other existing methods not only in terms of accuracy but also on computational complexity metrics on RADIal dataset.
CAFE A Novel Code switching Dataset for Algerian Dialect French and English
Venue: arXiv cs.SD 2024
Year: 2024

The paper introduces and publicly releases (Data download link available after acceptance) CAFE -- the first Code-switching dataset between Algerian dialect, French, and english languages. The CAFE speech data is unique for (a) its spontaneous speaking style in vivo human-human conversation capturing phenomena like code-switching and overlapping speech, (b) addresses distinct linguistic challenges in North African Arabic dialect; (c) the CAFE captures dialectal variations from various parts of Algeria within different sociolinguistic contexts. CAFE data contains approximately 37 hours of speech, with a subset, CAFE-small, of 2 hours and 36 minutes released with manual human annotation including speech segmentation, transcription, explicit annotation of code-switching points, overlapping speech, and other events such as noises, and laughter among others. The rest approximately 34.58 hours contain pseudo label transcriptions. In addition to the data release, the paper also highlighted the challenges of using state-of-the-art Automatic Speech Recognition (ASR) models such as Whisper large-v2,3 and PromptingWhisper to handle such content. Following, we benchmark CAFE data with the aforementioned Whisper models and show how well-designed data processing pipelines and advanced decoding techniques can improve the ASR performance in terms of Mixed Error Rate (MER) of 0.310, Character Error Rate (CER) of 0.329 and Word Error Rate (WER) of 0.538.
Sampling and Integration of Logconcave Functions by Algorithmic Diffusion
Venue: arXiv cs.DS 2024
Year: 2024

We study the complexity of sampling, rounding, and integrating arbitrary logconcave functions. Our new approach provides the first complexity improvements in nearly two decades for general logconcave functions for all three problems, and matches the best-known complexities for the special case of uniform distributions on convex bodies. For the sampling problem, our output guarantees are significantly stronger than previously known, and lead to a streamlined analysis of statistical estimation based on dependent random samples.