OpenRaiser/Intern-Atlas huggingface dataset:
Haptic-Enhanced Virtual Reality Simulator for Robot-Assisted Femur Fracture Surgery
Venue: arXiv cs.RO 2023
Year: 2023

In this paper, we develop a virtual reality (VR) simulator for the Robossis robot-assisted femur fracture surgery. Due to the steep learning curve for such procedures, a VR simulator is essential for training surgeon(s) and staff. The Robossis Surgical Simulator (RSS) is designed to immerse user(s) in a realistic surgery setting using the Robossis system as completed in a previous real-world cadaveric procedure. The RSS is designed to interface the Sigma-7 Haptic Controller with the Robossis Surgical Robot (RSR) and the Meta Quest VR headset. Results show that the RSR follows user commands in 6 DOF and prevents the overlapping of bone segments. This development demonstrates a promising avenue for future implementation of the Robossis system.
A Survey on Watching Social Issue Videos among YouTube and TikTok Users
Venue: arXiv cs.HC 2023
Year: 2023

The openness and influence of video-sharing platforms (VSPs) such as YouTube and TikTok attracted creators to share videos on various social issues. Although social issue videos (SIVs) affect public opinions and breed misinformation, how VSP users obtain information and interact with SIVs is under-explored. This work surveyed 659 YouTube and 127 TikTok users to understand the motives for consuming SIVs on VSPs. We found that VSP users are primarily motivated by the information and entertainment gratifications to use the platform. VSP users use SIVs for information-seeking purposes and find YouTube and TikTok convenient to interact with SIVs. VSP users moderately watch SIVs for entertainment and inactively engage in social interactions. SIV consumption is associated with information and socialization gratifications of the platform. VSP users appreciate the diversity of information and opinions but would also do their own research and are concerned about the misinformation and echo chamber problems.
Stochastic Configuration Machines: FPGA Implementation
Venue: arXiv cs.LG 2023
Year: 2023

Neural networks for industrial applications generally have additional constraints such as response speed, memory size and power usage. Randomized learners can address some of these issues. However, hardware solutions can provide better resource reduction whilst maintaining the model's performance. Stochastic configuration networks (SCNs) are a prime choice in industrial applications due to their merits and feasibility for data modelling. Stochastic Configuration Machines (SCMs) extend this to focus on reducing the memory constraints by limiting the randomized weights to a binary value with a scalar for each node and using a mechanism model to improve the learning performance and result interpretability. This paper aims to implement SCM models on a field programmable gate array (FPGA) and introduce binary-coded inputs to the algorithm. Results are reported for two benchmark and two industrial datasets, including SCM with single-layer and deep architectures.
Knolling bot 2.0: Enhancing Object Organization with Self-supervised Graspability Estimation
Venue: arXiv cs.RO 2023
Year: 2023

Building on recent advancements in transformer based approaches for domestic robots performing knolling, the art of organizing scattered items into neat arrangements. This paper introduces Knolling bot 2.0. Recognizing the challenges posed by piles of objects or items situated closely together, this upgraded system incorporates a self-supervised graspability estimation model. If objects are deemed ungraspable, an additional behavior will be executed to separate the objects before knolling the table. By integrating this grasp prediction mechanism with existing visual perception and transformer based knolling models, an advanced system capable of decluttering and organizing even more complex and densely populated table settings is demonstrated. Experimental evaluations demonstrate the effectiveness of this module, yielding a graspability prediction accuracy of 95.7%.
Adapter Pruning using Tropical Characterization
Venue: arXiv cs.CL 2023
Year: 2023

Adapters are widely popular parameter-efficient transfer learning approaches in natural language processing that insert trainable modules in between layers of a pre-trained language model. Apart from several heuristics, however, there has been a lack of studies analyzing the optimal number of adapter parameters needed for downstream applications. In this paper, we propose an adapter pruning approach by studying the tropical characteristics of trainable modules. We cast it as an optimization problem that aims to prune parameters from the adapter layers without changing the orientation of underlying tropical hypersurfaces. Our experiments on five NLP datasets show that tropical geometry tends to identify more relevant parameters to prune when compared with the magnitude-based baseline, while a combined approach works best across the tasks.
Free from Bellman Completeness: Trajectory Stitching via Model-based Return-conditioned Supervised Learning
Venue: arXiv cs.LG 2023
Year: 2023

Off-policy dynamic programming (DP) techniques such as $Q$-learning have proven to be important in sequential decision-making problems. In the presence of function approximation, however, these techniques often diverge due to the absence of Bellman completeness in the function classes considered, a crucial condition for the success of DP-based methods. In this paper, we show how off-policy learning techniques based on return-conditioned supervised learning (RCSL) are able to circumvent these challenges of Bellman completeness, converging under significantly more relaxed assumptions inherited from supervised learning. We prove there exists a natural environment in which if one uses two-layer multilayer perceptron as the function approximator, the layer width needs to grow linearly with the state space size to satisfy Bellman completeness while a constant layer width is enough for RCSL. These findings take a step towards explaining the superior empirical performance of RCSL methods compared to DP-based methods in environments with near-optimal datasets. Furthermore, in order to learn from sub-optimal datasets, we propose a simple framework called MBRCSL, granting RCSL methods the ability of dynamic programming to stitch together segments from distinct trajectories. MBRCSL leverages learned dynamics models and forward sampling to accomplish trajectory stitching while avoiding the need for Bellman completeness that plagues all dynamic programming algorithms. We propose both theoretical analysis and experimental evaluation to back these claims, outperforming state-of-the-art model-free and model-based offline RL algorithms across several simulated robotics problems.
USSR is in P/poly
Venue: arXiv cs.CC 2023
Year: 2023

The Sum of Square Roots (SSR) problem is the following computational problem: Given positive integers $a_1, \dots, a_k$, and signs $δ_1, \dots, δ_k \in \{-1, 1\}$, check if $\sum_{i=1}^k δ_i \sqrt{a_i} > 0$. The problem is known to have a polynomial time algorithm on the real RAM model of computation, however no sub-exponential time algorithm is known in the bit or Turing model of computation. The precise computational complexity of SSR has been a notorious open problem \cite{ggj} over the last four decades. The problem is known to admit an upper bound in the third level of the \emph{Counting Hierarchy}, i.e., $\CHtwo$ and no non-trivial lower bounds are known. Even when the input numbers are \emph{small}, i.e., given in \emph{unary}, no better complexity bound was known prior to our work. In this paper, we show that the unary variant (USSR) of the sum of square roots problem is considerably easier by giving a $P/poly$ upper bound.
Considerations for the Control Design of Augmentative Robots
Venue: arXiv cs.RO 2023
Year: 2023

Robotic systems that are intended to augment human capabilities commonly require the use of semi-autonomous control and artificial sensing, while at the same time aiming to empower the user to make decisions and take actions. This work identifies principles and techniques from the literature that can help to resolve this apparent contradiction. It is postulated that augmentative robots must function as tools that have partial agency, as collaborative agents that provide conditional transparency, and ideally, serve as extensions of the human body.
Linear Realisability and Cobordisms
Venue: arXiv cs.LO 2023
Year: 2023

Cobordism categories are known to be compact closed. They can therefore be used to define non-degenerate models of multiplicative linear logic by combining the Int construction with double glueing. In this work we detail such construction in the case of low-dimensional cobordisms, and exhibit a connexion between those models and the model of Interaction graphs introduced by Seiller. In particular, we exhibit how the so-called trefoil property is a consequence of the associativity of composition of higher structures, providing a first step toward establishing models as obtained from a double glueing construction. We discuss possible extensions to higher-dimensional cobordisms categories
Introducing instance label correlation in multiple instance learning. Application to cancer detection on histopathological images
Venue: arXiv cs.CV 2023
Year: 2023

In the last years, the weakly supervised paradigm of multiple instance learning (MIL) has become very popular in many different areas. A paradigmatic example is computational pathology, where the lack of patch-level labels for whole-slide images prevents the application of supervised models. Probabilistic MIL methods based on Gaussian Processes (GPs) have obtained promising results due to their excellent uncertainty estimation capabilities. However, these are general-purpose MIL methods that do not take into account one important fact: in (histopathological) images, the labels of neighboring patches are expected to be correlated. In this work, we extend a state-of-the-art GP-based MIL method, which is called VGPMIL-PR, to exploit such correlation. To do so, we develop a novel coupling term inspired by the statistical physics Ising model. We use variational inference to estimate all the model parameters. Interestingly, the VGPMIL-PR formulation is recovered when the weight that regulates the strength of the Ising term vanishes. The performance of the proposed method is assessed in two real-world problems of prostate cancer detection. We show that our model achieves better results than other state-of-the-art probabilistic MIL methods. We also provide different visualizations and analysis to gain insights into the influence of the novel Ising term. These insights are expected to facilitate the application of the proposed model to other research areas.
RGB-X Object Detection via Scene-Specific Fusion Modules
Venue: arXiv cs.CV 2023
Year: 2023

Multimodal deep sensor fusion has the potential to enable autonomous vehicles to visually understand their surrounding environments in all weather conditions. However, existing deep sensor fusion methods usually employ convoluted architectures with intermingled multimodal features, requiring large coregistered multimodal datasets for training. In this work, we present an efficient and modular RGB-X fusion network that can leverage and fuse pretrained single-modal models via scene-specific fusion modules, thereby enabling joint input-adaptive network architectures to be created using small, coregistered multimodal datasets. Our experiments demonstrate the superiority of our method compared to existing works on RGB-thermal and RGB-gated datasets, performing fusion using only a small amount of additional parameters. Our code is available at https://github.com/dsriaditya999/RGBXFusion.
A numerical tool for efficient analysis and optimization of offshore wind turbine jacket substructure considering realistic boundary and loading conditions
Venue: arXiv cs.CE 2023
Year: 2023

The jacket substructure is a critical component of the offshore wind turbine (OWT) that is the interface between the transition piece at the top and the grouted connection. This paper presents a comprehensive study on the optimization of a jacket substructure to achieve greater cost efficiency while maintain acceptable structural performance. A fast parametric finite element modelling (FEM) approach for jacket substructures was firstly proposed. The generated models took into account realistic loading conditions, including self-weight, wind load and section-dependent wave load, and soil-pile interaction. Parametric studies were conducted afterwards to investigate the trends of the mass and response of the jacket substructure with respect to the variation of geometric and sectional parameters. Optimizations of the jacket substructure were carried out using parametric optimization and numerical genetic algorithm (GA) optimization under three different optimization strategies corresponding to three groups of objective and constraint functions. The trends obtained by parametric analysis were used to guide the parameter selection in parametric optimization, while a rank-based mutation GA was established with the proposed efficient FEM embedded in as the solver to the optimization objective and constraint functions. Parametric optimization gained its advantage in computational efficiency, and the mass reduction were 6.2%,10% and 14.8% for the three strategies respectively. GA optimization was more aggressive as the mass reductions were 16.8%,22.3% and 34.3% for the three strategies, but was relatively more computational intense. The two proposed optimization methods and the three optimization strategies are both expected to be applied in practical engineering design of OWT jacket substructure with good optimization output and high computational efficiency.
PlayTest: A Gamified Test Generator for Games
Venue: arXiv cs.SE 2023
Year: 2023

Games are usually created incrementally, requiring repeated testing of the same scenarios, which is a tedious and error-prone task for game developers. Therefore, we aim to alleviate this game testing process by encapsulating it into a game called Playtest, which transforms the tiring testing process into a competitive game with a purpose. Playtest automates the generation of valuable test cases based on player actions, without the players even realising it. We envision the use of Playtest to crowdsource the task of testing games by giving players access to the respective games through our tool in the playtesting phases during the development process.
Dynamic Gaussian Splatting from Markerless Motion Capture can Reconstruct Infants Movements
Venue: arXiv cs.CV 2023
Year: 2023

Easy access to precise 3D tracking of movement could benefit many aspects of rehabilitation. A challenge to achieving this goal is that while there are many datasets and pretrained algorithms for able-bodied adults, algorithms trained on these datasets often fail to generalize to clinical populations including people with disabilities, infants, and neonates. Reliable movement analysis of infants and neonates is important as spontaneous movement behavior is an important indicator of neurological function and neurodevelopmental disability, which can help guide early interventions. We explored the application of dynamic Gaussian splatting to sparse markerless motion capture (MMC) data. Our approach leverages semantic segmentation masks to focus on the infant, significantly improving the initialization of the scene. Our results demonstrate the potential of this method in rendering novel views of scenes and tracking infant movements. This work paves the way for advanced movement analysis tools that can be applied to diverse clinical populations, with a particular emphasis on early detection in infants.
Towards Grouping in Large Scenes with Occlusion-aware Spatio-temporal Transformers
Venue: arXiv cs.CV 2023
Year: 2023

Group detection, especially for large-scale scenes, has many potential applications for public safety and smart cities. Existing methods fail to cope with frequent occlusions in large-scale scenes with multiple people, and are difficult to effectively utilize spatio-temporal information. In this paper, we propose an end-to-end framework,GroupTransformer, for group detection in large-scale scenes. To deal with the frequent occlusions caused by multiple people, we design an occlusion encoder to detect and suppress severely occluded person crops. To explore the potential spatio-temporal relationship, we propose spatio-temporal transformers to simultaneously extract trajectory information and fuse inter-person features in a hierarchical manner. Experimental results on both large-scale and small-scale scenes demonstrate that our method achieves better performance compared with state-of-the-art methods. On large-scale scenes, our method significantly boosts the performance in terms of precision and F1 score by more than 10%. On small-scale scenes, our method still improves the performance of F1 score by more than 5%. The project page with code can be found at http://cic.tju.edu.cn/faculty/likun/projects/GroupTrans.
Large-Scale Application of Fault Injection into PyTorch Models -- an Extension to PyTorchFI for Validation Efficiency
Venue: arXiv cs.AI 2023
Year: 2023

Transient or permanent faults in hardware can render the output of Neural Networks (NN) incorrect without user-specific traces of the error, i.e. silent data errors (SDE). On the other hand, modern NNs also possess an inherent redundancy that can tolerate specific faults. To establish a safety case, it is necessary to distinguish and quantify both types of corruptions. To study the effects of hardware (HW) faults on software (SW) in general and NN models in particular, several fault injection (FI) methods have been established in recent years. Current FI methods focus on the methodology of injecting faults but often fall short of accounting for large-scale FI tests, where many fault locations based on a particular fault model need to be analyzed in a short time. Results need to be concise, repeatable, and comparable. To address these requirements and enable fault injection as the default component in a machine learning development cycle, we introduce a novel fault injection framework called PyTorchALFI (Application Level Fault Injection for PyTorch) based on PyTorchFI. PyTorchALFI provides an efficient way to define randomly generated and reusable sets of faults to inject into PyTorch models, defines complex test scenarios, enhances data sets, and generates test KPIs while tightly coupling fault-free, faulty, and modified NN. In this paper, we provide details about the definition of test scenarios, software architecture, and several examples of how to use the new framework to apply iterative changes in fault location and number, compare different model modifications, and analyze test results.
Trust, Accountability, and Autonomy in Knowledge Graph-based AI for Self-determination
Venue: arXiv cs.AI 2023
Year: 2023

Knowledge Graphs (KGs) have emerged as fundamental platforms for powering intelligent decision-making and a wide range of Artificial Intelligence (AI) services across major corporations such as Google, Walmart, and AirBnb. KGs complement Machine Learning (ML) algorithms by providing data context and semantics, thereby enabling further inference and question-answering capabilities. The integration of KGs with neuronal learning (e.g., Large Language Models (LLMs)) is currently a topic of active research, commonly named neuro-symbolic AI. Despite the numerous benefits that can be accomplished with KG-based AI, its growing ubiquity within online services may result in the loss of self-determination for citizens as a fundamental societal issue. The more we rely on these technologies, which are often centralised, the less citizens will be able to determine their own destinies. To counter this threat, AI regulation, such as the European Union (EU) AI Act, is being proposed in certain regions. The regulation sets what technologists need to do, leading to questions concerning: How can the output of AI systems be trusted? What is needed to ensure that the data fuelling and the inner workings of these artefacts are transparent? How can AI be made accountable for its decision-making? This paper conceptualises the foundational topics and research pillars to support KG-based AI for self-determination. Drawing upon this conceptual framework, challenges and opportunities for citizen self-determination are illustrated and analysed in a real-world scenario. As a result, we propose a research agenda aimed at accomplishing the recommended objectives.
Analysing Multi-Agent Systems using 1-safe Petri Nets
Venue: arXiv cs.MA 2023
Year: 2023

In the modelling and analysis of large, real systems, the main problem in their efficient processing is the size of the global model. One of the popular approaches that address this issue is the decomposition of such global model into much smaller submodels and interaction between them. In this paper we discuss the translation of multi-agent systems with the common-action-based synchronization to 1-safe Petri nets. We prove that the composition in terms of transition systems is equivalent to the transition-based fusion of nets modelling different agents. We also address the issue of permanent disabling of some parts of the system by constraints implied by the synchronization and discuss the methods of solving it without the computation of the entire global model.
SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity
Venue: arXiv cs.AI 2023
Year: 2023

To address the challenge of increasing network size, researchers have developed sparse models through network pruning. However, maintaining model accuracy while achieving significant speedups on general computing devices remains an open problem. In this paper, we present a novel mobile inference acceleration framework SparseByteNN, which leverages fine-grained kernel sparsity to achieve real-time execution as well as high accuracy. Our framework consists of two parts: (a) A fine-grained kernel sparsity schema with a sparsity granularity between structured pruning and unstructured pruning. It designs multiple sparse patterns for different operators. Combined with our proposed whole network rearrangement strategy, the schema achieves a high compression rate and high precision at the same time. (b) Inference engine co-optimized with the sparse pattern. The conventional wisdom is that this reduction in theoretical FLOPs does not translate into real-world efficiency gains. We aim to correct this misconception by introducing a family of efficient sparse kernels for ARM and WebAssembly. Equipped with our efficient implementation of sparse primitives, we show that sparse versions of MobileNet-v1 outperform strong dense baselines on the efficiency-accuracy curve. Experimental results on Qualcomm 855 show that for 30% sparse MobileNet-v1, SparseByteNN achieves 1.27x speedup over the dense version and 1.29x speedup over the state-of-the-art sparse inference engine MNN with a slight accuracy drop of 0.224%. The source code of SparseByteNN will be available at https://github.com/lswzjuer/SparseByteNN