OpenRaiser/Intern-Atlas huggingface dataset:
Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers
Venue: arXiv cs.LG 2023
Year: 2023

We propose a method for learning dynamical systems from high-dimensional empirical data that combines variational autoencoders and (spatio-)temporal attention within a framework designed to enforce certain scientifically-motivated invariances. We focus on the setting in which data are available from multiple different instances of a system whose underlying dynamical model is entirely unknown at the outset. The approach rests on a separation into an instance-specific encoding (capturing initial conditions, constants etc.) and a latent dynamics model that is itself universal across all instances/realizations of the system. The separation is achieved in an automated, data-driven manner and only empirical data are required as inputs to the model. The approach allows effective inference of system behaviour at any continuous time but does not require an explicit neural ODE formulation, which makes it efficient and highly scalable. We study behaviour through simple theoretical analyses and extensive experiments on synthetic and real-world datasets. The latter investigate learning the dynamics of complex systems based on finite data and show that the proposed approach can outperform state-of-the-art neural-dynamical models. We study also more general inductive bias in the context of transfer to data obtained under entirely novel system interventions. Overall, our results provide a promising new framework for efficiently learning dynamical models from heterogeneous data with potential applications in a wide range of fields including physics, medicine, biology and engineering.
A Comprehensive Study on the Robustness of Image Classification and Object Detection in Remote Sensing: Surveying and Benchmarking
Venue: arXiv cs.CV 2023
Year: 2023

Deep neural networks (DNNs) have found widespread applications in interpreting remote sensing (RS) imagery. However, it has been demonstrated in previous works that DNNs are vulnerable to different types of noises, particularly adversarial noises. Surprisingly, there has been a lack of comprehensive studies on the robustness of RS tasks, prompting us to undertake a thorough survey and benchmark on the robustness of image classification and object detection in RS. To our best knowledge, this study represents the first comprehensive examination of both natural robustness and adversarial robustness in RS tasks. Specifically, we have curated and made publicly available datasets that contain natural and adversarial noises. These datasets serve as valuable resources for evaluating the robustness of DNNs-based models. To provide a comprehensive assessment of model robustness, we conducted meticulous experiments with numerous different classifiers and detectors, encompassing a wide range of mainstream methods. Through rigorous evaluation, we have uncovered insightful and intriguing findings, which shed light on the relationship between adversarial noise crafting and model training, yielding a deeper understanding of the susceptibility and limitations of various models, and providing guidance for the development of more resilient and robust models
PrivSketch: A Private Sketch-based Frequency Estimation Protocol for Data Streams
Venue: arXiv cs.CR 2023
Year: 2023

Local differential privacy (LDP) has recently become a popular privacy-preserving data collection technique protecting users' privacy. The main problem of data stream collection under LDP is the poor utility due to multi-item collection from a very large domain. This paper proposes PrivSketch, a high-utility frequency estimation protocol taking advantage of sketches, suitable for private data stream collection. Combining the proposed background information and a decode-first collection-side workflow, PrivSketch improves the utility by reducing the errors introduced by the sketching algorithm and the privacy budget utilization when collecting multiple items. We analytically prove the superior accuracy and privacy characteristics of PrivSketch, and also evaluate them experimentally. Our evaluation, with several diverse synthetic and real datasets, demonstrates that PrivSketch is 1-3 orders of magnitude better than the competitors in terms of utility in both frequency estimation and frequent item estimation, while being up to ~100x faster.
Fast Segment Anything
Venue: arXiv cs.CV 2023
Year: 2023

The recently proposed segment anything model (SAM) has made a significant influence in many computer vision tasks. It is becoming a foundation step for many high-level tasks, like image segmentation, image caption, and image editing. However, its huge computation costs prevent it from wider applications in industry scenarios. The computation mainly comes from the Transformer architecture at high-resolution inputs. In this paper, we propose a speed-up alternative method for this fundamental task with comparable performance. By reformulating the task as segments-generation and prompting, we find that a regular CNN detector with an instance segmentation branch can also accomplish this task well. Specifically, we convert this task to the well-studied instance segmentation task and directly train the existing instance segmentation method using only 1/50 of the SA-1B dataset published by SAM authors. With our method, we achieve a comparable performance with the SAM method at 50 times higher run-time speed. We give sufficient experimental results to demonstrate its effectiveness. The codes and demos will be released at https://github.com/CASIA-IVA-Lab/FastSAM.
Static-Equilibrium Oriented Interaction Force Modeling and Control of Aerial Manipulation with Uni-Directional Thrust Multirotors
Venue: arXiv cs.RO 2023
Year: 2023

This paper presents a static-equilibrium oriented interaction force modeling and control approach of aerial manipulation employing uni-directional thrust (UDT) multirotors interacting with variously defined environments. First, a simplified system model for a quadrotor-based aerial manipulator is introduced considering parameterized work surfaces under assumptions, and then a range of meaningful manipulation tasks are utilized to explore the system properties in a quasi-static equilibrium state. An explicit interaction force model in relation with the aerial manipulator pose configuration and the environment parameter is derived from the static equilibrium analysis, based on which singularity is pointed out. Then a hybrid attitude/force interaction control strategy is presented to verify the proposed interaction force model, which involves high gain attitude control and feedforward plus feedback force control. This paper represents preliminary results. We study the properties of UDT-based aerial manipulators via specific tasks, and propose a novel framework for interaction force modeling and control aiming at maximizing the commercial values of UDT platforms for aerial manipulation purpose.
Feature Interactions Reveal Linguistic Structure in Language Models
Venue: arXiv cs.CL 2023
Year: 2023

We study feature interactions in the context of feature attribution methods for post-hoc interpretability. In interpretability research, getting to grips with feature interactions is increasingly recognised as an important challenge, because interacting features are key to the success of neural networks. Feature interactions allow a model to build up hierarchical representations for its input, and might provide an ideal starting point for the investigation into linguistic structure in language models. However, uncovering the exact role that these interactions play is also difficult, and a diverse range of interaction attribution methods has been proposed. In this paper, we focus on the question which of these methods most faithfully reflects the inner workings of the target models. We work out a grey box methodology, in which we train models to perfection on a formal language classification task, using PCFGs. We show that under specific configurations, some methods are indeed able to uncover the grammatical rules acquired by a model. Based on these findings we extend our evaluation to a case study on language models, providing novel insights into the linguistic structure that these models have acquired.
Adaptive DNN Surgery for Selfish Inference Acceleration with On-demand Edge Resource
Venue: arXiv cs.GT 2023
Year: 2023

Deep Neural Networks (DNNs) have significantly improved the accuracy of intelligent applications on mobile devices. DNN surgery, which partitions DNN processing between mobile devices and multi-access edge computing (MEC) servers, can enable real-time inference despite the computational limitations of mobile devices. However, DNN surgery faces a critical challenge: determining the optimal computing resource demand from the server and the corresponding partition strategy, while considering both inference latency and MEC server usage costs. This problem is compounded by two factors: (1) the finite computing capacity of the MEC server, which is shared among multiple devices, leading to inter-dependent demands, and (2) the shift in modern DNN architecture from chains to directed acyclic graphs (DAGs), which complicates potential solutions. In this paper, we introduce a novel Decentralized DNN Surgery (DDS) framework. We formulate the partition strategy as a min-cut and propose a resource allocation game to adaptively schedule the demands of mobile devices in an MEC environment. We prove the existence of a Nash Equilibrium (NE), and develop an iterative algorithm to efficiently reach the NE for each device. Our extensive experiments demonstrate that DDS can effectively handle varying MEC scenarios, achieving up to 1.25$\times$ acceleration compared to the state-of-the-art algorithm.
Python Framework for Modular and Parametric SPICE Netlists Generation
Venue: arXiv cs.AR 2023
Year: 2023

Due to the complex specifications of current electronic systems, design decisions need to be explored automatically. However, the exploration process is a complex task given the plethora of design choices such as the selection of components, number of components, operating modes of each of the components, connections between the components and variety of ways in which the same functionality can be implemented. To tackle these issues, scripts are used generate designs based on high abstract constructions. Still, this approach is usually ad-hoc and platform dependent, making the whole procedure hardly reusable, scalable and versatile. We propose a generic, open-source framework tackling rapid design exploration for the generation of modular and parametric electronic designs that is able to work on any major simulator.
Knowledge-based Multimodal Music Similarity
Venue: arXiv cs.SD 2023
Year: 2023

Music similarity is an essential aspect of music retrieval, recommendation systems, and music analysis. Moreover, similarity is of vital interest for music experts, as it allows studying analogies and influences among composers and historical periods. Current approaches to musical similarity rely mainly on symbolic content, which can be expensive to produce and is not always readily available. Conversely, approaches using audio signals typically fail to provide any insight about the reasons behind the observed similarity. This research addresses the limitations of current approaches by focusing on the study of musical similarity using both symbolic and audio content. The aim of this research is to develop a fully explainable and interpretable system that can provide end-users with more control and understanding of music similarity and classification systems.
An efficient mass lumping scheme for isogeometric analysis based on approximate dual basis functions
Venue: arXiv cs.CE 2023
Year: 2023

In this contribution, we provide a new mass lumping scheme for explicit dynamics in isogeometric analysis (IGA). To this end, an element formulation based on the idea of dual functionals is developed. Non-Uniform Rational B-splines (NURBS) are applied as shape functions and their corresponding dual basis functions are applied as test functions in the variational form, where two kinds of dual basis functions are compared. The first type are approximate dual basis functions (AD) with varying degree of reproduction, resulting in banded mass matrices. Dual basis functions derived from the inversion of the Gram matrix (IG) are the second type and already yield diagonal mass matrices. We will show that it is possible to apply the dual scheme as a transformation of the resulting system of equations based on NURBS as shape and test functions. Hence, it can be easily implemented into existing IGA routines. Treating the application of dual test functions as preconditioner reduces the additional computational effort, but it cannot entirely erase it and the density of the stiffness matrix still remains higher than in standard Bubnov-Galerkin formulations. In return applying additional row-sum lumping to the mass matrices is either not necessary for IG or the caused loss of accuracy is lowered to a reasonable magnitude in the case of AD. Numerical examples show a significantly better approximation of the dynamic behavior for the dual lumping scheme compared to standard NURBS approaches making use of row-sum lumping. Applying IG yields accurate numerical results without additional lumping. But as result of the global support of the IG dual basis functions, fully populated stiffness matrices occur, which are entirely unsuitable for explicit dynamic simulations. Combining AD and row-sum lumping leads to an efficient computation regarding effort and accuracy.
Decoupled Boundary Handling in SPH
Venue: arXiv cs.GR 2023
Year: 2023

Particle-based boundary representations are frequently used in Smoothed Particle Hydrodynamics (SPH) due to their simple integration into fluid solvers. Commonly, incompressible fluid solvers estimate the current density and corresponding forces in case the current density exceeds the rest density to push fluid particles apart. Close to the boundary, the calculation of the fluid particles' density involves both, neighboring fluid and neighboring boundary particles, yielding an overestimation of density, and, subsequently, wrong pressure forces and wrong velocities leading to the disturbed fluid particles' behavior in the vicinity of the boundary. In this paper, we present a detailed explanation of this disturbed fluid particle behavior, which is mainly due to the combined or coupled handling of the fluid-fluid particle and the fluid-boundary particle interaction. We propose the decoupled handling of both interaction types, leading to two densities for a given fluid particle, i.e., fluid-induced density and boundary-induced density. In our approach, we alternately apply the corresponding fluid-induced and boundary-induced forces during pressure estimation. This separation avoids force overestimation and reduces unintended fluid dynamics near the boundary, as well as a consistent fluid-boundary distance across different fluid amounts and different particle-based boundary handling methods. We compare our method with two regular state-of-the-art methods in different experiments and show how our method handles detailed boundary shapes.
Computationally efficient human body modelling for real time motion comfort assessment
Venue: arXiv cs.HC 2023
Year: 2023

Due to the complexity of the human body and its neuromuscular stabilization, it has been challenging to efficiently and accurately predict human motion and capture posture while being driven. Existing simple models of the seated human body are mostly two-dimensional and developed in the mid-sagittal plane ex-posed to in-plane excitation. Such models capture fore-aft and vertical motion but not the more complex 3D motions due to lateral loading. Advanced 3D full-body active human models (AHMs), such as in MADYMO, can be used for comfort analysis and to investigate how vibrations influence the human body while being driven. However, such AHMs are very time-consuming due to their complexity. To effectively analyze motion comfort, a computationally efficient and accurate three dimensional (3D) human model, which runs faster than real-time, is presented. The model's postural stabilization parameters are tuned using available 3D vibration data for head, trunk and pelvis translation and rotation. A comparison between AHM and EHM is conducted regarding human body kinematics. According to the results, the EHM model configuration with two neck joints, two torso bending joints, and a spinal compression joint accurately predicts body kinematics.
SIFTER: A Task-specific Alignment Strategy for Enhancing Sentence Embeddings
Venue: arXiv cs.CL 2023
Year: 2023

The paradigm of pre-training followed by fine-tuning on downstream tasks has become the mainstream method in natural language processing tasks. Although pre-trained models have the advantage of generalization, their performance may still vary significantly across different domain tasks. This is because the data distribution in different domains varies. For example, the different parts of the sentence 'He married Smt. Dipali Ghosh in 1947 and led a very happy married life' may have different impact for downstream tasks. For similarity calculations, words such as 'led' and 'life' are more important. On the other hand, for sentiment analysis, the word 'happy' is crucial. This indicates that different downstream tasks have different levels of sensitivity to sentence components. Our starting point is to scale information of the model and data according to the specifics of downstream tasks, enhancing domain information of relevant parts for these tasks and reducing irrelevant elements for different domain tasks, called SIFTER. In the experimental part, we use the SIFTER to improve SimCSE by constructing positive sample pairs based on enhancing the sentence stem and reducing the unimportant components in the sentence, and maximize the similarity between three sentences. Similarly, SIFTER can improve the gate mechanism of the LSTM model by short-circuiting the input gate of important words so that the LSTM model remembers the important parts of the sentence. Our experiments demonstrate that SIFTER outperforms the SimCSE and LSTM baselines.
Tackling the Awkward Squad for Reactive Programming: The Actor-Reactor Model
Venue: arXiv cs.PL 2023
Year: 2023

Reactive programming is a programming paradigm whereby programs are internally represented by a dependency graph, which is used to automatically (re)compute parts of a program whenever its input changes. In practice reactive programming can only be used for some parts of an application: a reactive program is usually embedded in an application that is still written in ordinary imperative languages such as JavaScript or Scala. In this paper we investigate this embedding and we distill "the awkward squad for reactive programming" as 3 concerns that are essential for real-world software development, but that do not fit within reactive programming. They are related to long lasting computations, side-effects, and the coordination between imperative and reactive code. To solve these issues we design a new programming model called the Actor-Reactor Model in which programs are split up in a number of actors and reactors. Actors and reactors enforce a strict separation of imperative and reactive code, and they can be composed via a number of composition operators that make use of data streams. We demonstrate the model via our own implementation in a language called Stella.
Iterated Piecewise Affine (IPA) Approximation for Language Modeling
Venue: arXiv cs.CL 2023
Year: 2023

In this work, we demonstrate the application of a first-order Taylor expansion to approximate a generic function $F: R^{n \times m} \to R^{n \times m}$ and utilize it in language modeling. To enhance the basic Taylor expansion, we introduce iteration and piecewise modeling, leading us to name the algorithm the Iterative Piecewise Affine (IPA) approximation. The final algorithm exhibits interesting resemblances to the Transformers decoder architecture. By comparing parameter arrangements in IPA and Transformers, we observe a strikingly similar performance, with IPA outperforming Transformers by 1.5\% in the next token prediction task with cross-entropy loss for smaller sequence lengths.
Linear and Non-Linear Barrier Coverage in Deterministic and Uncertain environment in WSNs: A New Classification
Venue: arXiv cs.NI 2023
Year: 2023

This paper Various studies cited in the literature deal with the classic problem of obstacle coverage, where the deployment environment, sensor nodes, and base stations have characteristics that are considered perfect but suffer from various flaws in the real world. This paper presents other barrier coverage types ranked in a new classification based on linear and nonlinear barrier coverage according to deterministic and insecure environments, and enumerates some of the different current and future challenges of these coverage types and connectivity in WSNs.
Solving Dialogue Grounding Embodied Task in a Simulated Environment using Further Masked Language Modeling
Venue: arXiv cs.CL 2023
Year: 2023

Enhancing AI systems with efficient communication skills that align with human understanding is crucial for their effective assistance to human users. Proactive initiatives from the system side are needed to discern specific circumstances and interact aptly with users to solve these scenarios. In this research, we opt for a collective building assignment taken from the Minecraft dataset. Our proposed method employs language modeling to enhance task understanding through state-of-the-art (SOTA) methods using language models. These models focus on grounding multi-modal understandinging and task-oriented dialogue comprehension tasks. This focus aids in gaining insights into how well these models interpret and respond to a variety of inputs and tasks. Our experimental results provide compelling evidence of the superiority of our proposed method. This showcases a substantial improvement and points towards a promising direction for future research in this domain.
PalmGazer: Unimanual Eye-hand Menus in Augmented Reality
Venue: arXiv cs.HC 2023
Year: 2023

How can we design the user interfaces for augmented reality (AR) so that we can interact as simple, flexible and expressive as we can with smartphones in one hand? To explore this question, we propose PalmGazer as an interaction concept integrating eye-hand interaction to establish a singlehandedly operable menu system. In particular, PalmGazer is designed to support quick and spontaneous digital commands -- such as to play a music track, check notifications or browse visual media -- through our devised three-way interaction model: hand opening to summon the menu UI, eye-hand input for selection of items, and dragging gesture for navigation. A key aspect is that it remains always-accessible and movable to the user, as the menu supports meaningful hand and head based reference frames. We demonstrate the concept in practice through a prototypical personal UI with application probes, and describe technique designs specifically-tailored to the application UI. A qualitative evaluation highlights the system's design benefits and drawbacks, e.g., that common 2D scroll and selection tasks are simple to operate, but higher degrees of freedom may be reserved for two hands. Our work contributes interaction techniques and design insights to expand AR's uni-manual capabilities.