OpenRaiser/Intern-Atlas huggingface dataset:
Variational Bayes Inference for Data Detection in Cell-Free Massive MIMO
Venue: arXiv cs.IT 2023
Year: 2023

Cell-free massive MIMO is a promising technology for beyond-5G networks. Through the deployment of many cooperating access points (AP), the technology can significantly enhance user coverage and spectral efficiency compared to traditional cellular systems. Since the APs are distributed over a large area, the level of favorable propagation in cell-free massive MIMO is less than the one in colocated massive MIMO. As a result, the current linear processing schemes are not close to the optimal ones when the number of AP antennas is not very large. The aim of this paper is to develop nonlinear variational Bayes (VB) methods for data detection in cell-free massive MIMO systems. Contrary to existing work in the literature, which only attained point estimates of the transmit data symbols, the proposed methods aim to obtain the posterior distribution and the Bayes estimate of the data symbols. We develop the VB methods accordingly to the levels of cooperation among the APs. Simulation results show significant performance advantages of the developed VB methods over the linear processing techniques.
Towards Microstructural State Variables in Materials Systems
Venue: arXiv cs.LG 2023
Year: 2023

The vast combination of material properties seen in nature are achieved by the complexity of the material microstructure. Advanced characterization and physics based simulation techniques have led to generation of extremely large microstructural datasets. There is a need for machine learning techniques that can manage data complexity by capturing the maximal amount of information about the microstructure using the least number of variables. This paper aims to formulate dimensionality and state variable estimation techniques focused on reducing microstructural image data. It is shown that local dimensionality estimation based on nearest neighbors tend to give consistent dimension estimates for natural images for all p-Minkowski distances. However, it is found that dimensionality estimates have a systematic error for low-bit depth microstructural images. The use of Manhattan distance to alleviate this issue is demonstrated. It is also shown that stacked autoencoders can reconstruct the generator space of high dimensional microstructural data and provide a sparse set of state variables to fully describe the variability in material microstructures.
Linear Time Online Algorithms for Constructing Linear-size Suffix Trie
Venue: arXiv cs.DS 2023
Year: 2023

The suffix trees are fundamental data structures for various kinds of string processing. The suffix tree of a text string $T$ of length $n$ has $O(n)$ nodes and edges, and the string label of each edge is encoded by a pair of positions in $T$. Thus, even after the tree is built, the input string $T$ needs to be kept stored and random access to $T$ is still needed. The \emph{linear-size suffix tries} (\emph{LSTs}), proposed by Crochemore et al. [Linear-size suffix tries, TCS 638:171-178, 2016], are a "stand-alone" alternative to the suffix trees. Namely, the LST of an input text string $T$ of length $n$ occupies $O(n)$ total space, and supports pattern matching and other tasks with the same efficiency as the suffix tree without the need to store the input text string $T$. Crochemore et al. proposed an \emph{offline} algorithm which transforms the suffix tree of $T$ into the LST of $T$ in $O(n \log σ)$ time and $O(n)$ space, where $σ$ is the alphabet size. In this paper, we present two types of \emph{online} algorithms which "directly" construct the LST, from right to left, and from left to right, without constructing the suffix tree as an intermediate structure. Both algorithms construct the LST incrementally when a new symbol is read, and do not access the previously read symbols. Both of the right-to-left construction algorithm and the left-to-right construction algorithm work in $O(n \log σ)$ time and $O(n)$ space. The main feature of our algorithms is that the input text string does not need to be stored.
Age of Information in Deep Learning-Driven Task-Oriented Communications
Venue: arXiv cs.IT 2023
Year: 2023

This paper studies the notion of age in task-oriented communications that aims to execute a task at a receiver utilizing the data at its transmitter. The transmitter-receiver operations are modeled as an encoder-decoder pair that is jointly trained while considering channel effects. The encoder converts data samples into feature vectors of small dimension and transmits them with a small number of channel uses thereby reducing the number of transmissions and latency. Instead of reconstructing input samples, the decoder performs a task, e.g., classification, on the received signals. Applying different deep neural networks of encoder-decoder pairs on MNIST and CIFAR-10 image datasets, the classifier accuracy is shown to increase with the number of channel uses at the expense of longer service time. The peak age of task information (PAoTI) is introduced to analyze this accuracy-latency tradeoff when the age grows unless a received signal is classified correctly. By incorporating channel and traffic effects, design guidelines are obtained for task-oriented communications by characterizing how the PAoTI first decreases and then increases with the number of channel uses. A dynamic update mechanism is presented to adapt the number of channel uses to channel and traffic conditions, and reduce the PAoTI in task-oriented communications.
ML-FEED: Machine Learning Framework for Efficient Exploit Detection
Venue: arXiv cs.CR 2023
Year: 2023

Machine learning (ML)-based methods have recently become attractive for detecting security vulnerability exploits. Unfortunately, state-of-the-art ML models like long short-term memories (LSTMs) and transformers incur significant computation overheads. This overhead makes it infeasible to deploy them in real-time environments. We propose a novel ML-based exploit detection model, ML-FEED, that enables highly efficient inference without sacrificing performance. We develop a novel automated technique to extract vulnerability patterns from the Common Weakness Enumeration (CWE) and Common Vulnerabilities and Exposures (CVE) databases. This feature enables ML-FEED to be aware of the latest cyber weaknesses. Second, it is not based on the traditional approach of classifying sequences of application programming interface (API) calls into exploit categories. Such traditional methods that process entire sequences incur huge computational overheads. Instead, ML-FEED operates at a finer granularity and predicts the exploits triggered by every API call of the program trace. Then, it uses a state table to update the states of these potential exploits and track the progress of potential exploit chains. ML-FEED also employs a feature engineering approach that uses natural language processing-based word embeddings, frequency vectors, and one-hot encoding to detect semantically-similar instruction calls. Then, it updates the states of the predicted exploit categories and triggers an alarm when a vulnerability fingerprint executes. Our experiments show that ML-FEED is 72.9x and 75,828.9x faster than state-of-the-art lightweight LSTM and transformer models, respectively. We trained and tested ML-FEED on 79 real-world exploit categories. It predicts categories of exploit in real-time with 98.2% precision, 97.4% recall, and 97.8% F1 score. These results also outperform the LSTM and transformer baselines.
Fast Kinodynamic Planning on the Constraint Manifold with Deep Neural Networks
Venue: arXiv cs.RO 2023
Year: 2023

Motion planning is a mature area of research in robotics with many well-established methods based on optimization or sampling the state space, suitable for solving kinematic motion planning. However, when dynamic motions under constraints are needed and computation time is limited, fast kinodynamic planning on the constraint manifold is indispensable. In recent years, learning-based solutions have become alternatives to classical approaches, but they still lack comprehensive handling of complex constraints, such as planning on a lower-dimensional manifold of the task space while considering the robot's dynamics. This paper introduces a novel learning-to-plan framework that exploits the concept of constraint manifold, including dynamics, and neural planning methods. Our approach generates plans satisfying an arbitrary set of constraints and computes them in a short constant time, namely the inference time of a neural network. This allows the robot to plan and replan reactively, making our approach suitable for dynamic environments. We validate our approach on two simulated tasks and in a demanding real-world scenario, where we use a Kuka LBR Iiwa 14 robotic arm to perform the hitting movement in robotic Air Hockey.
An Overview of Artificial Intelligence-based Soft Upper Limb Exoskeleton for Rehabilitation: A Descriptive Review
Venue: arXiv cs.RO 2023
Year: 2023

The upper limb robotic exoskeleton is an electromechanical device which use to recover a patients motor dysfunction in the rehabilitation field. It can provide repetitive, comprehensive, focused, positive, and precise training to regain the joints and muscles capability. It has been shown that existing robotic exoskeletons are generally used rigid motors and mechanical structures. Soft robotic devices can be a correct substitute for rigid ones. Soft exosuits are flexible, portable, comfortable, user-friendly, low-cost, and travel-friendly. Somehow, they need expertise or therapist to assist those devices. Also, they cannot be adaptable to different patients with non-identical physical parameters and various rehabilitation needs. For that reason, nowadays we need intelligent exoskeletons during rehabilitation which have to learn from patients previous data and act according to it with patients intention. There also has a big gap between theoretical and practical applications for using those exoskeletons. Most of the intelligent exoskeletons are prototype in manner. To solve this problem, the robotic exoskeleton should be made both criteria as ergonomic and portable. The exoskeletons have to the power of decision-making to avoid the presence of expertise. In this growing field, the present trend is to make the exoskeleton intelligent and make it more reliable to use in clinical practice.
Counteracts: Testing Stereotypical Representation in Pre-trained Language Models
Venue: arXiv cs.CL 2023
Year: 2023

Recently, language models have demonstrated strong performance on various natural language understanding tasks. Language models trained on large human-generated corpus encode not only a significant amount of human knowledge, but also the human stereotype. As more and more downstream tasks have integrated language models as part of the pipeline, it is necessary to understand the internal stereotypical representation in order to design the methods for mitigating the negative effects. In this paper, we use counterexamples to examine the internal stereotypical knowledge in pre-trained language models (PLMs) that can lead to stereotypical preference. We mainly focus on gender stereotypes, but the method can be extended to other types of stereotype. We evaluate 7 PLMs on 9 types of cloze-style prompt with different information and base knowledge. The results indicate that PLMs show a certain amount of robustness against unrelated information and preference of shallow linguistic cues, such as word position and syntactic structure, but a lack of interpreting information by meaning. Such findings shed light on how to interact with PLMs in a neutral approach for both finetuning and evaluation.
Graph based Environment Representation for Vision-and-Language Navigation in Continuous Environments
Venue: arXiv cs.CV 2023
Year: 2023

Vision-and-Language Navigation in Continuous Environments (VLN-CE) is a navigation task that requires an agent to follow a language instruction in a realistic environment. The understanding of environments is a crucial part of the VLN-CE task, but existing methods are relatively simple and direct in understanding the environment, without delving into the relationship between language instructions and visual environments. Therefore, we propose a new environment representation in order to solve the above problems. First, we propose an Environment Representation Graph (ERG) through object detection to express the environment in semantic level. This operation enhances the relationship between language and environment. Then, the relational representations of object-object, object-agent in ERG are learned through GCN, so as to obtain a continuous expression about ERG. Sequentially, we combine the ERG expression with object label embeddings to obtain the environment representation. Finally, a new cross-modal attention navigation framework is proposed, incorporating our environment representation and a special loss function dedicated to training ERG. Experimental result shows that our method achieves satisfactory performance in terms of success rate on VLN-CE tasks. Further analysis explains that our method attains better cross-modal matching and strong generalization ability.
Order-Preserving Database Encryption with Secret Sharing
Venue: arXiv cs.CR 2023
Year: 2023

The order-preserving encryption (OPE) problem was initially formulated by the database community in 2004 soon after the paradigm database-as-a-service (DaaS) was coined in 2002. Over the past two decades, OPE has drawn tremendous research interest from communities of databases, cryptography, and security; we have witnessed significant advances in OPE schemes both theoretically and systematically. All existing OPE schemes assume that the outsourced database is modeled as a single semi-honest adversary who should learn nothing more than the order information of plaintext messages up to a negligible probability. This paper addresses the OPE problem from a new perspective: instead of modeling the outsourced database as a single semi-honest adversary, we assume the outsourced database \textit{service} compromises a cluster of non-colluding servers, which is a practical assumption as all major cloud vendors support multiple database instances deployed to exclusive sub-networks or even to distinct data centers. This assumption allows us to design a new stateless OPE protocol, namely order-preserving database encryption with secret sharing (ODES), by employing secret-sharing schemes among those presumably non-colluding servers. We will demonstrate that ODES guarantees the latest security level, namely IND-FAOCPA, and outperforms the state-of-the-art scheme by orders of magnitude.
TBV Radar SLAM -- trust but verify loop candidates
Venue: arXiv cs.RO 2023
Year: 2023

Robust SLAM in large-scale environments requires fault resilience and awareness at multiple stages, from sensing and odometry estimation to loop closure. In this work, we present TBV (Trust But Verify) Radar SLAM, a method for radar SLAM that introspectively verifies loop closure candidates. TBV Radar SLAM achieves a high correct-loop-retrieval rate by combining multiple place-recognition techniques: tightly coupled place similarity and odometry uncertainty search, creating loop descriptors from origin-shifted scans, and delaying loop selection until after verification. Robustness to false constraints is achieved by carefully verifying and selecting the most likely ones from multiple loop constraints. Importantly, the verification and selection are carried out after registration when additional sources of loop evidence can easily be computed. We integrate our loop retrieval and verification method with a fault-resilient odometry pipeline within a pose graph framework. By evaluating on public benchmarks we found that TBV Radar SLAM achieves 65% lower error than the previous state of the art. We also show that it's generalizing across environments without needing to change any parameters.
A prediction and behavioural analysis of machine learning methods for modelling travel mode choice
Venue: arXiv cs.LG 2023
Year: 2023

The emergence of a variety of Machine Learning (ML) approaches for travel mode choice prediction poses an interesting question to transport modellers: which models should be used for which applications? The answer to this question goes beyond simple predictive performance, and is instead a balance of many factors, including behavioural interpretability and explainability, computational complexity, and data efficiency. There is a growing body of research which attempts to compare the predictive performance of different ML classifiers with classical random utility models. However, existing studies typically analyse only the disaggregate predictive performance, ignoring other aspects affecting model choice. Furthermore, many studies are affected by technical limitations, such as the use of inappropriate validation schemes, incorrect sampling for hierarchical data, lack of external validation, and the exclusive use of discrete metrics. We address these limitations by conducting a systematic comparison of different modelling approaches, across multiple modelling problems, in terms of the key factors likely to affect model choice (out-of-sample predictive performance, accuracy of predicted market shares, extraction of behavioural indicators, and computational efficiency). We combine several real world datasets with synthetic datasets, where the data generation function is known. The results indicate that the models with the highest disaggregate predictive performance (namely extreme gradient boosting and random forests) provide poorer estimates of behavioural indicators and aggregate mode shares, and are more expensive to estimate, than other models, including deep neural networks and Multinomial Logit (MNL). It is further observed that the MNL model performs robustly in a variety of situations, though ML techniques can improve the estimates of behavioural indices such as Willingness to Pay.
Allo-centric Occupancy Grid Prediction for Urban Traffic Scene Using Video Prediction Networks
Venue: arXiv cs.CV 2023
Year: 2023

Prediction of dynamic environment is crucial to safe navigation of an autonomous vehicle. Urban traffic scenes are particularly challenging to forecast due to complex interactions between various dynamic agents, such as vehicles and vulnerable road users. Previous approaches have used egocentric occupancy grid maps to represent and predict dynamic environments. However, these predictions suffer from blurriness, loss of scene structure at turns, and vanishing of agents over longer prediction horizon. In this work, we propose a novel framework to make long-term predictions by representing the traffic scene in a fixed frame, referred as allo-centric occupancy grid. This allows for the static scene to remain fixed and to represent motion of the ego-vehicle on the grid like other agents'. We study the allo-centric grid prediction with different video prediction networks and validate the approach on the real-world Nuscenes dataset. The results demonstrate that the allo-centric grid representation significantly improves scene prediction, in comparison to the conventional ego-centric grid approach.
Fast spline detection in high density microscopy data
Venue: arXiv cs.CV 2023
Year: 2023

Computer-aided analysis of biological microscopy data has seen a massive improvement with the utilization of general-purpose deep learning techniques. Yet, in microscopy studies of multi-organism systems, the problem of collision and overlap remains challenging. This is particularly true for systems composed of slender bodies such as crawling nematodes, swimming spermatozoa, or the beating of eukaryotic or prokaryotic flagella. Here, we develop a novel end-to-end deep learning approach to extract precise shape trajectories of generally motile and overlapping splines. Our method works in low resolution settings where feature keypoints are hard to define and detect. Detection is fast and we demonstrate the ability to track thousands of overlapping organisms simultaneously. While our approach is agnostic to area of application, we present it in the setting of and exemplify its usability on dense experiments of crawling Caenorhabditis elegans. The model training is achieved purely on synthetic data, utilizing a physics-based model for nematode motility, and we demonstrate the model's ability to generalize from simulations to experimental videos.
BINN: A deep learning approach for computational mechanics problems based on boundary integral equations
Venue: arXiv cs.LG 2023
Year: 2023

We proposed the boundary-integral type neural networks (BINN) for the boundary value problems in computational mechanics. The boundary integral equations are employed to transfer all the unknowns to the boundary, then the unknowns are approximated using neural networks and solved through a training process. The loss function is chosen as the residuals of the boundary integral equations. Regularization techniques are adopted to efficiently evaluate the weakly singular and Cauchy principle integrals in boundary integral equations. Potential problems and elastostatic problems are mainly concerned in this article as a demonstration. The proposed method has several outstanding advantages: First, the dimensions of the original problem are reduced by one, thus the freedoms are greatly reduced. Second, the proposed method does not require any extra treatment to introduce the boundary conditions, since they are naturally considered through the boundary integral equations. Therefore, the method is suitable for complex geometries. Third, BINN is suitable for problems on the infinite or semi-infinite domains. Moreover, BINN can easily handle heterogeneous problems with a single neural network without domain decomposition.
The SeaLiT Ontology -- An Extension of CIDOC-CRM for the Modeling and Integration of Maritime History Information
Venue: arXiv cs.DB 2023
Year: 2023

We describe the construction and use of the SeaLiT Ontology, an extension of the ISO standard CIDOC-CRM for the modelling and integration of maritime history information. The ontology has been developed gradually, following a bottom-up approach that required the analysis of large amounts of real primary data (archival material) as well as knowledge and validation by domain experts (maritime historians). We present the specification of the ontology, RDFS and OWL implementations, as well as knowledge graphs that make use of this data model for integrating information originating from a large and diverse set of archival documents, such as crew lists, sailors registers, naval ship registers and payrolls. We also describe an application that operates over these knowledge graphs and which supports historians in exploring and quantitatively analysing the integrated data through a user-friendly interface. Finally, we discuss aspects related to the use, evolution and sustainability of the ontology.
The Role of Interactive Visualization in Explaining (Large) NLP Models: from Data to Inference
Venue: arXiv cs.CL 2023
Year: 2023

With a constant increase of learned parameters, modern neural language models become increasingly more powerful. Yet, explaining these complex model's behavior remains a widely unsolved problem. In this paper, we discuss the role interactive visualization can play in explaining NLP models (XNLP). We motivate the use of visualization in relation to target users and common NLP pipelines. We also present several use cases to provide concrete examples on XNLP with visualization. Finally, we point out an extensive list of research opportunities in this field.