OpenRaiser/Intern-Atlas huggingface dataset:
Design and Fabrication of Soft Locomotion Robots based on Spatial Compliant Mechanisms
Venue: arXiv cs.RO 2024
Year: 2024

Soft robotics has emerged as a promising technology that holds great potential for various application areas. This is due to soft materials unique properties, including flexibility, safety, and shock absorption, among others. Despite many advancement in the field, the development of effective design methodologies and production techniques for soft robots remains a challenge. Although numerous robot prototypes have been proposed in recent years, their designs are often complex and difficult to produce. As such, there is a need for more efficient and unified design approaches that can facilitate the production of soft robots with desirable properties. In this paper, we propose a method for designing soft robots using elastic beams and spatial compliant mechanisms. The method is based on an evolutionary approach that enables the creation of designs with both high motion and force transmission ratios. Specifically, we focus on the development of locomotion mechanisms using a central linear actuator. Our approach involves the use of commonly available plastic materials and a 3D printer to manufacture the designs. We demonstrate the feasibility of our approach by presenting experimental results that show successful production and real world operation. Overall, our findings suggest that the use of elastic beams and an evolutionary approach can facilitate the creation of soft robots with desirable locomotion properties, including fast locomotion up to 3.7 body lengths per second, locomotion with a payload, and underwater locomotion. This method has the potential to enable the development of more efficient and practical soft robots for various applications.
AirChain: A Novel Blockchain Framework and Low-Cost Device for Democratized Air Quality Data Aggregation
Venue: arXiv cs.DC 2024
Year: 2024

Air pollutant exposure kills over 6,700,000 people it per annum, yet there remains a systemic lack of accurate ground level data reporting the concentrations of the leading causes of such fatalities. Ambient particulate matter is a primary driver of this effect. Namely, PM1.0, PM2.5, and PM10.0 display a systemic lack of accurate and high-definition reporting. This project suggests and implements a prototype for a distributed and low cost model for reporting such data and designs a novel framework in order to remedy three main shortfalls of previously implemented systems. First, their central operation and distribution, and therefore their requirement of trust in a central governing body. Second, their requirement of the purchase of comparatively high-cost devices for ordinary consumers. Finally, their high degree of error and accordingly low functional certainty. This project explores the creation of AirChain, a prototype system utilizing blockchain technology that will demonstrate the effectiveness of low-cost sensors when paired with simple microcontroller devices.
Implementing a Restricted Function Space Class in Firedrake
Venue: arXiv cs.MS 2024
Year: 2024

The implementation process of a $\texttt{RestrictedFunctionSpace}$ class in Firedrake, a Python library which numerically solves partial differential equations through the use of the finite element method, is documented. This includes an introduction to the current $\texttt{FunctionSpace}$ class in Firedrake, and the key features that it has. With the current $\texttt{FunctionSpace}$ class, the limitations of the capabilities of the solvers in Firedrake when imposing Dirichlet boundary conditions are explored, as well as what the $\texttt{RestrictedFunctionSpace}$ class does differently to remove these issues. These will be considered in both a mathematical way, and in the code as an abstraction of the mathematical ideas presented. Finally, the benefits to the user of the $\texttt{RestrictedFunctionSpace}$ class are considered, and demonstrated through tests and comparisons. This leads to the conclusion that in particular, the eigensolver in Firedrake is improved through the use of the $\texttt{RestrictedFunctionSpace}$, through the removal of eigenvalues associated with the Dirichlet boundary conditions for a system.
LightPHE: Integrating Partially Homomorphic Encryption into Python with Extensive Cloud Environment Evaluations
Venue: arXiv cs.CR 2024
Year: 2024

Homomorphic encryption enables computations on encrypted data without accessing private keys, enhancing security in cloud environments. Without this technology, updates need to be performed on-premises or require transmitting private keys to the cloud, increasing security risks. Fully homomorphic encryption (FHE) supports both additive and multiplicative operations on ciphertexts, while partially homomorphic encryption (PHE) supports either addition or multiplication, offering a more efficient and practical solution. This study introduces LightPHE, a lightweight hybrid PHE framework for Python, designed to address the lack of existing PHE libraries. LightPHE integrates multiple PHE algorithms with a modular and extensible design, ensuring robustness and usability for rapid prototyping and secure application development. Cloud-based experiments were conducted on Google Colab (Normal, A100 GPU, L4 GPU, T4 High RAM, TPU2) and Microsoft Azure Spark to evaluate LightPHE's performance and scalability. Key metrics such as key generation, encryption, decryption, and homomorphic operations were assessed. Results showed LightPHE's superior performance in high-computation environments like Colab A100 GPU and TPU2, while also offering viable options for cost-effective setups like Colab Normal and Azure Spark. Comparative analyses demonstrated LightPHE's efficiency and scalability, making it suitable for various applications. The benchmarks offer insights into selecting appropriate cloud environments based on performance needs, highlighting LightPHE's potential to advance homomorphic encryption for secure and efficient cloud-based data processing.
MUSE: Multi-Knowledge Passing on the Edges, Boosting Knowledge Graph Completion
Venue: arXiv cs.CL 2024
Year: 2024

Knowledge Graph Completion (KGC) aims to predict the missing information in the (head entity)-[relation]-(tail entity) triplet. Deep Neural Networks have achieved significant progress in the relation prediction task. However, most existing KGC methods focus on single features (e.g., entity IDs) and sub-graph aggregation, which cannot fully explore all the features in the Knowledge Graph (KG), and neglect the external semantic knowledge injection. To address these problems, we propose MUSE, a knowledge-aware reasoning model to learn a tailored embedding space in three dimensions for missing relation prediction through a multi-knowledge representation learning mechanism. Our MUSE consists of three parallel components: 1) Prior Knowledge Learning for enhancing the triplets' semantic representation by fine-tuning BERT; 2) Context Message Passing for enhancing the context messages of KG; 3) Relational Path Aggregation for enhancing the path representation from the head entity to the tail entity. Our experimental results show that MUSE significantly outperforms other baselines on four public datasets, such as over 5.50% improvement in H@1 and 4.20% improvement in MRR on the NELL995 dataset. The code and all datasets will be released via https://github.com/NxxTGT/MUSE.
Omobot: a low-cost mobile robot for autonomous search and fall detection
Venue: arXiv cs.RO 2024
Year: 2024

Detecting falls among the elderly and alerting their community responders can save countless lives. We design and develop a low-cost mobile robot that periodically searches the house for the person being monitored and sends an email to a set of designated responders if a fall is detected. In this project, we make three novel design decisions and contributions. First, our custom-designed low-cost robot has advanced features like omnidirectional wheels, the ability to run deep learning models, and autonomous wireless charging. Second, we improve the accuracy of fall detection for the YOLOv8-Pose-nano object detection network by 6% and YOLOv8-Pose-large by 12%. We do so by transforming the images captured from the robot viewpoint (camera height 0.15m from the ground) to a typical human viewpoint (1.5m above the ground) using a principally computed Homography matrix. This improves network accuracy because the training dataset MS-COCO on which YOLOv8-Pose is trained is captured from a human-height viewpoint. Lastly, we improve the robot controller by learning a model that predicts the robot velocity from the input signal to the motor controller.
SHIELD: LLM-Driven Schema Induction for Predictive Analytics in EV Battery Supply Chain Disruptions
Venue: arXiv cs.AI 2024
Year: 2024

The electric vehicle (EV) battery supply chain's vulnerability to disruptions necessitates advanced predictive analytics. We present SHIELD (Schema-based Hierarchical Induction for EV supply chain Disruption), a system integrating Large Language Models (LLMs) with domain expertise for EV battery supply chain risk assessment. SHIELD combines: (1) LLM-driven schema learning to construct a comprehensive knowledge library, (2) a disruption analysis system utilizing fine-tuned language models for event extraction, multi-dimensional similarity matching for schema matching, and Graph Convolutional Networks (GCNs) with logical constraints for prediction, and (3) an interactive interface for visualizing results and incorporating expert feedback to enhance decision-making. Evaluated on 12,070 paragraphs from 365 sources (2022-2023), SHIELD outperforms baseline GCNs and LLM+prompt methods (e.g., GPT-4o) in disruption prediction. These results demonstrate SHIELD's effectiveness in combining LLM capabilities with domain expertise for enhanced supply chain risk assessment.
MindSpeech: Continuous Imagined Speech Decoding using High-Density fNIRS and Prompt Tuning for Advanced Human-AI Interaction
Venue: arXiv cs.HC 2024
Year: 2024

In the coming decade, artificial intelligence systems will continue to improve and revolutionise every industry and facet of human life. Designing effective, seamless and symbiotic communication paradigms between humans and AI agents is increasingly important. This paper reports a novel method for human-AI interaction by developing a direct brain-AI interface. We discuss a novel AI model, called MindSpeech, which enables open-vocabulary, continuous decoding for imagined speech. This study focuses on enhancing human-AI communication by utilising high-density functional near-infrared spectroscopy (fNIRS) data to develop an AI model capable of decoding imagined speech non-invasively. We discuss a new word cloud paradigm for data collection, improving the quality and variety of imagined sentences generated by participants and covering a broad semantic space. Utilising a prompt tuning-based approach, we employed the Llama2 large language model (LLM) for text generation guided by brain signals. Our results show significant improvements in key metrics, such as BLEU-1 and BERT P scores, for three out of four participants, demonstrating the method's effectiveness. Additionally, we demonstrate that combining data from multiple participants enhances the decoder performance, with statistically significant improvements in BERT scores for two participants. Furthermore, we demonstrated significantly above-chance decoding accuracy for imagined speech versus resting conditions and the identified activated brain regions during imagined speech tasks in our study are consistent with the previous studies on brain regions involved in speech encoding. This study underscores the feasibility of continuous imagined speech decoding. By integrating high-density fNIRS with advanced AI techniques, we highlight the potential for non-invasive, accurate communication systems with AI in the near future.
Enhancing Representation Learning of EEG Data with Masked Autoencoders
Venue: arXiv cs.HC 2024
Year: 2024

Self-supervised learning has been a powerful training paradigm to facilitate representation learning. In this study, we design a masked autoencoder (MAE) to guide deep learning models to learn electroencephalography (EEG) signal representation. Our MAE includes an encoder and a decoder. A certain proportion of input EEG signals are randomly masked and sent to our MAE. The goal is to recover these masked signals. After this self-supervised pre-training, the encoder is fine-tuned on downstream tasks. We evaluate our MAE on EEGEyeNet gaze estimation task. We find that the MAE is an effective brain signal learner. It also significantly improves learning efficiency. Compared to the model without MAE pre-training, the pre-trained one achieves equal performance with 1/3 the time of training and outperforms it in half the training time. Our study shows that self-supervised learning is a promising research direction for EEG-based applications as other fields (natural language processing, computer vision, robotics, etc.), and thus we expect foundation models to be successful in EEG domain.
SAM-FNet: SAM-Guided Fusion Network for Laryngo-Pharyngeal Tumor Detection
Venue: arXiv cs.CV 2024
Year: 2024

Laryngo-pharyngeal cancer (LPC) is a highly fatal malignant disease affecting the head and neck region. Previous studies on endoscopic tumor detection, particularly those leveraging dual-branch network architectures, have shown significant advancements in tumor detection. These studies highlight the potential of dual-branch networks in improving diagnostic accuracy by effectively integrating global and local (lesion) feature extraction. However, they are still limited in their capabilities to accurately locate the lesion region and capture the discriminative feature information between the global and local branches. To address these issues, we propose a novel SAM-guided fusion network (SAM-FNet), a dual-branch network for laryngo-pharyngeal tumor detection. By leveraging the powerful object segmentation capabilities of the Segment Anything Model (SAM), we introduce the SAM into the SAM-FNet to accurately segment the lesion region. Furthermore, we propose a GAN-like feature optimization (GFO) module to capture the discriminative features between the global and local branches, enhancing the fusion feature complementarity. Additionally, we collect two LPC datasets from the First Affiliated Hospital (FAHSYSU) and the Sixth Affiliated Hospital (SAHSYSU) of Sun Yat-sen University. The FAHSYSU dataset is used as the internal dataset for training the model, while the SAHSYSU dataset is used as the external dataset for evaluating the model's performance. Extensive experiments on both datasets of FAHSYSU and SAHSYSU demonstrate that the SAM-FNet can achieve competitive results, outperforming the state-of-the-art counterparts. The source code of SAM-FNet is available at the URL of https://github.com/VVJia/SAM-FNet.
Simple and Nearly-Optimal Sampling for Rank-1 Tensor Completion via Gauss-Jordan
Venue: arXiv cs.DS 2024
Year: 2024

We revisit the sample and computational complexity of completing a rank-1 tensor in $\otimes_{i=1}^{N} \mathbb{R}^{d}$, given a uniformly sampled subset of its entries. We present a characterization of the problem (i.e. nonzero entries) which admits an algorithm amounting to Gauss-Jordan on a pair of random linear systems. For example, when $N = Θ(1)$, we prove it uses no more than $m = O(d^2 \log d)$ samples and runs in $O(md^2)$ time. Moreover, we show any algorithm requires $Ω(d\log d)$ samples. By contrast, existing upper bounds on the sample complexity are at least as large as $d^{1.5} μ^{Ω(1)} \log^{Ω(1)} d$, where $μ$ can be $Θ(d)$ in the worst case. Prior work obtained these looser guarantees in higher rank versions of our problem, and tend to involve more complicated algorithms.
Convergence Guarantee of Dynamic Programming for LTL Surrogate Reward
Venue: arXiv cs.RO 2024
Year: 2024

Linear Temporal Logic (LTL) is a formal way of specifying complex objectives for planning problems modeled as Markov Decision Processes (MDPs). The planning problem aims to find the optimal policy that maximizes the satisfaction probability of the LTL objective. One way to solve the planning problem is to use the surrogate reward with two discount factors and dynamic programming, which bypasses the graph analysis used in traditional model-checking. The surrogate reward is designed such that its value function represents the satisfaction probability. However, in some cases where one of the discount factors is set to $1$ for higher accuracy, the computation of the value function using dynamic programming is not guaranteed. This work shows that a multi-step contraction always exists during dynamic programming updates, guaranteeing that the approximate value function will converge exponentially to the true value function. Thus, the computation of satisfaction probability is guaranteed.
Navigating Weight Prediction with Diet Diary
Venue: arXiv cs.MM 2024
Year: 2024

Current research in food analysis primarily concentrates on tasks such as food recognition, recipe retrieval and nutrition estimation from a single image. Nevertheless, there is a significant gap in exploring the impact of food intake on physiological indicators (e.g., weight) over time. This paper addresses this gap by introducing the DietDiary dataset, which encompasses daily dietary diaries and corresponding weight measurements of real users. Furthermore, we propose a novel task of weight prediction with a dietary diary that aims to leverage historical food intake and weight to predict future weights. To tackle this task, we propose a model-agnostic time series forecasting framework. Specifically, we introduce a Unified Meal Representation Learning (UMRL) module to extract representations for each meal. Additionally, we design a diet-aware loss function to associate food intake with weight variations. By conducting experiments on the DietDiary dataset with two state-of-the-art time series forecasting models, NLinear and iTransformer, we demonstrate that our proposed framework achieves superior performance compared to the original models. We make our dataset, code, and models publicly available at: https://yxg1005.github.io/weight-prediction/.
Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness
Venue: arXiv cs.CV 2024
Year: 2024

Adversarial examples pose a significant challenge to the robustness, reliability and alignment of deep neural networks. We propose a novel, easy-to-use approach to achieving high-quality representations that lead to adversarial robustness through the use of multi-resolution input representations and dynamic self-ensembling of intermediate layer predictions. We demonstrate that intermediate layer predictions exhibit inherent robustness to adversarial attacks crafted to fool the full classifier, and propose a robust aggregation mechanism based on Vickrey auction that we call \textit{CrossMax} to dynamically ensemble them. By combining multi-resolution inputs and robust ensembling, we achieve significant adversarial robustness on CIFAR-10 and CIFAR-100 datasets without any adversarial training or extra data, reaching an adversarial accuracy of $\approx$72% (CIFAR-10) and $\approx$48% (CIFAR-100) on the RobustBench AutoAttack suite ($L_\infty=8/255)$ with a finetuned ImageNet-pretrained ResNet152. This represents a result comparable with the top three models on CIFAR-10 and a +5 % gain compared to the best current dedicated approach on CIFAR-100. Adding simple adversarial training on top, we get $\approx$78% on CIFAR-10 and $\approx$51% on CIFAR-100, improving SOTA by 5 % and 9 % respectively and seeing greater gains on the harder dataset. We validate our approach through extensive experiments and provide insights into the interplay between adversarial robustness, and the hierarchical nature of deep representations. We show that simple gradient-based attacks against our model lead to human-interpretable images of the target classes as well as interpretable image changes. As a byproduct, using our multi-resolution prior, we turn pre-trained classifiers and CLIP models into controllable image generators and develop successful transferable attacks on large vision language models.
Mathematical Models of Computation in Superposition
Venue: arXiv cs.LG 2024
Year: 2024

Superposition -- when a neural network represents more ``features'' than it has dimensions -- seems to pose a serious challenge to mechanistically interpreting current AI systems. Existing theory work studies \emph{representational} superposition, where superposition is only used when passing information through bottlenecks. In this work, we present mathematical models of \emph{computation} in superposition, where superposition is actively helpful for efficiently accomplishing the task. We first construct a task of efficiently emulating a circuit that takes the AND of the $\binom{m}{2}$ pairs of each of $m$ features. We construct a 1-layer MLP that uses superposition to perform this task up to $\varepsilon$-error, where the network only requires $\tilde{O}(m^{\frac{2}{3}})$ neurons, even when the input features are \emph{themselves in superposition}. We generalize this construction to arbitrary sparse boolean circuits of low depth, and then construct ``error correction'' layers that allow deep fully-connected networks of width $d$ to emulate circuits of width $\tilde{O}(d^{1.5})$ and \emph{any} polynomial depth. We conclude by providing some potential applications of our work for interpreting neural networks that implement computation in superposition.
Trajectory Planning for Teleoperated Space Manipulators Using Deep Reinforcement Learning
Venue: arXiv cs.RO 2024
Year: 2024

Trajectory planning for teleoperated space manipulators involves challenges such as accurately modeling system dynamics, particularly in free-floating modes with non-holonomic constraints, and managing time delays that increase model uncertainty and affect control precision. Traditional teleoperation methods rely on precise dynamic models requiring complex parameter identification and calibration, while data-driven methods do not require prior knowledge but struggle with time delays. A novel framework utilizing deep reinforcement learning (DRL) is introduced to address these challenges. The framework incorporates three methods: Mapping, Prediction, and State Augmentation, to handle delays when delayed state information is received at the master end. The Soft Actor Critic (SAC) algorithm processes the state information to compute the next action, which is then sent to the remote manipulator for environmental interaction. Four environments are constructed using the MuJoCo simulation platform to account for variations in base and target fixation: fixed base and target, fixed base with rotated target, free-floating base with fixed target, and free-floating base with rotated target. Extensive experiments with both constant and random delays are conducted to evaluate the proposed methods. Results demonstrate that all three methods effectively address trajectory planning challenges, with State Augmentation showing superior efficiency and robustness.