OpenRaiser/Intern-Atlas huggingface dataset:
An optimization-based IMU/Lidar/Camera Co-calibration method
Venue: arXiv cs.RO 2022
Year: 2022

Recently, multi-sensors fusion has achieved significant progress in the field of automobility to improve navigation and position performance. As the prerequisite of the fusion algorithm, the demand for the extrinsic calibration of multi-sensors is growing. To calculate the extrinsic parameter, many researches have been dedicated to the two-step method, which integrates the respective calibration in pairs. It is inefficient and incompact because of losing sight of the constrain of all sensors. With regard to remove this burden, an optimization-based IMU/Lidar/Camera co-calibration method is proposed in the paper. Firstly, the IMU/camera and IMU/lidar online calibrations are conducted, respectively. Then, the corner and surface feature points in the chessboard are associated with the coarse result and the camera/lidar constraint is constructed. Finally, construct the co-calibration optimization to refine all extrinsic parameters. We evaluate the performance of the proposed scheme in simulation and the result demonstrates that our proposed method outperforms the two-step method.
Deep Convolutional Neural Network and Transfer Learning for Locomotion Intent Prediction
Venue: arXiv cs.RO 2022
Year: 2022

Powered prosthetic legs must anticipate the user's intent when switching between different locomotion modes (e.g., level walking, stair ascent/descent, ramp ascent/descent). Numerous data-driven classification techniques have demonstrated promising results for predicting user intent, but the performance of these intent prediction models on novel subjects remains undesirable. In other domains (e.g., image classification), transfer learning has improved classification accuracy by using previously learned features from a large dataset (i.e., pre-trained models) and then transferring this learned model to a new task where a smaller dataset is available. In this paper, we develop a deep convolutional neural network with intra-subject (subject-dependent) and inter-subject (subject-independent) validations based on a human locomotion dataset. We then apply transfer learning for the subject-independent model using a small portion (10%) of the data from the left-out subject. We compare the performance of these three models. Our results indicate that the transfer learning (TL) model outperforms the subject-independent (IND) model and is comparable to the subject-dependent (DEP) model (DEP Error: 0.74 $\pm$ 0.002%, IND Error: 11.59 $\pm$ 0.076%, TL Error: 3.57 $\pm$ 0.02% with 10% data). Moreover, as expected, transfer learning accuracy increases with the availability of more data from the left-out subject. We also evaluate the performance of the intent prediction system in various sensor configurations that may be available in a prosthetic leg application. Our results suggest that a thigh IMU on the the prosthesis is sufficient to predict locomotion intent in practice.
Low Complexity SLP: An Inversion-Free, Parallelizable ADMM Approach
Venue: arXiv cs.IT 2022
Year: 2022

We propose a parallel constructive interference (CI)-based symbol-level precoding (SLP) approach for massive connectivity in the downlink of multiuser multiple-input single-output (MU-MISO) systems, with only local channel state information (CSI) used at each processor unit and limited information exchange between processor units. By reformulating the power minimization (PM) SLP problem and exploiting the separability of the corresponding reformulation, the original problem is decomposed into several parallel subproblems via the ADMM framework with closed-form solutions, leading to a substantial reduction in computational complexity. The sufficient condition for guaranteeing the convergence of the proposed approach is derived, based on which an adaptive parameter tuning strategy is proposed to accelerate the convergence rate. To avoid the large-dimension matrix inverse operation, an efficient algorithm is proposed by employing the standard proximal term and by leveraging the singular value decomposition (SVD). Furthermore, a prox-linear proximal term is adopted to fully eliminate the matrix inversion, and a parallel inverse-free SLP (PIF-SLP) algorithm is finally obtained. Numerical results validate our derivations above, and demonstrate that the proposed PIF-SLP algorithm can significantly reduce the computational complexity compared to the state-of-the-arts.
Convergence of score-based generative modeling for general data distributions
Venue: arXiv cs.LG 2022
Year: 2022

Score-based generative modeling (SGM) has grown to be a hugely successful method for learning to generate samples from complex data distributions such as that of images and audio. It is based on evolving an SDE that transforms white noise into a sample from the learned distribution, using estimates of the score function, or gradient log-pdf. Previous convergence analyses for these methods have suffered either from strong assumptions on the data distribution or exponential dependencies, and hence fail to give efficient guarantees for the multimodal and non-smooth distributions that arise in practice and for which good empirical performance is observed. We consider a popular kind of SGM -- denoising diffusion models -- and give polynomial convergence guarantees for general data distributions, with no assumptions related to functional inequalities or smoothness. Assuming $L^2$-accurate score estimates, we obtain Wasserstein distance guarantees for any distribution of bounded support or sufficiently decaying tails, as well as TV guarantees for distributions with further smoothness assumptions.
FastStamp: Accelerating Neural Steganography and Digital Watermarking of Images on FPGAs
Venue: arXiv cs.CV 2022
Year: 2022

Steganography and digital watermarking are the tasks of hiding recoverable data in image pixels. Deep neural network (DNN) based image steganography and watermarking techniques are quickly replacing traditional hand-engineered pipelines. DNN based watermarking techniques have drastically improved the message capacity, imperceptibility and robustness of the embedded watermarks. However, this improvement comes at the cost of increased computational overhead of the watermark encoder neural network. In this work, we design the first accelerator platform FastStamp to perform DNN based steganography and digital watermarking of images on hardware. We first propose a parameter efficient DNN model for embedding recoverable bit-strings in image pixels. Our proposed model can match the success metrics of prior state-of-the-art DNN based watermarking methods while being significantly faster and lighter in terms of memory footprint. We then design an FPGA based accelerator framework to further improve the model throughput and power consumption by leveraging data parallelism and customized computation paths. FastStamp allows embedding hardware signatures into images to establish media authenticity and ownership of digital media. Our best design achieves 68 times faster inference as compared to GPU implementations of prior DNN based watermark encoder while consuming less power.
A 3D Mesh Based Approach to In Home Safe Walking Spaces for Older Adults
Venue: arXiv cs.HC 2022
Year: 2022

Falling continues to be a significant risk factor for older adults and other mobility limited individuals. Monitoring and maintaining clear, tripping hazard free pathways in living spaces is invaluable in helping people live independently and safely in their home. This paper proposes and demonstrates a Microsoft HoloLens-based technology for monitoring the open walking areas in living spaces. The system is based on the 3D mesh produced by the HoloLens augmented reality device. Several algorithms were implemented and evaluated to handle the noisy raw mesh to help identify all possible floor spaces. The resulting 3D model of the home was then processed to find United States Occupational Safety and Health Administration (OSHA) approved pathways through the homes. The long term goals of these technologies are to monitor the living spaces clutter and clear walkways over time. The information it generates shall be provided to the residents and their caregivers during environmental home assessments. It will inform them about how well the home is being maintained so proactive interventions may be taken before a fall occurs.
Feature-based model selection for object detection from point cloud data
Venue: arXiv cs.CV 2022
Year: 2022

Smart monitoring using three-dimensional (3D) image sensors has been attracting attention in the context of smart cities. In smart monitoring, object detection from point cloud data acquired by 3D image sensors is implemented for detecting moving objects such as vehicles and pedestrians to ensure safety on the road. However, the features of point cloud data are diversified due to the characteristics of light detection and ranging (LIDAR) units used as 3D image sensors or the install position of the 3D image sensors. Although a variety of deep learning (DL) models for object detection from point cloud data have been studied to date, no research has considered how to use multiple DL models in accordance with the features of the point cloud data. In this work, we propose a feature-based model selection framework that creates various DL models by using multiple DL methods and by utilizing training data with pseudo incompleteness generated by two artificial techniques: sampling and noise adding. It selects the most suitable DL model for the object detection task in accordance with the features of the point cloud data acquired in the real environment. To demonstrate the effectiveness of the proposed framework, we compare the performance of multiple DL models using benchmark datasets created from the KITTI dataset and present example results of object detection obtained through a real outdoor experiment. Depending on the situation, the detection accuracy varies up to 32% between DL models, which confirms the importance of selecting an appropriate DL model according to the situation.
Self-Supervised Guided Segmentation Framework for Unsupervised Anomaly Detection
Venue: arXiv cs.CV 2022
Year: 2022

Unsupervised anomaly detection is a challenging task in industrial applications since it is impracticable to collect sufficient anomalous samples. In this paper, a novel Self-Supervised Guided Segmentation Framework (SGSF) is proposed by jointly exploring effective generation method of forged anomalous samples and the normal sample features as the guidance information of segmentation for anomaly detection. Specifically, to ensure that the generated forged anomaly samples are conducive to model training, the Saliency Augmentation Module (SAM) is proposed. SAM introduces a saliency map to generate saliency Perlin noise map, and develops an adaptive segmentation strategy to generate irregular masks in the saliency region. Then, the masks are utilized to generate forged anomalous samples as negative samples for training. Unfortunately, the distribution gap between forged and real anomaly samples makes it difficult for models trained based on forged samples to effectively locate real anomalies. Towards this end, the Self-supervised Guidance Network (SGN) is proposed. It leverages the self-supervised module to extract features that are noise-free and contain normal semantic information as the prior knowledge of the segmentation module. The segmentation module with the knowledge of normal patterns segments out the abnormal regions that are different from the guidance features. To evaluate the effectiveness of SGSF for anomaly detection, extensive experiments are conducted on three anomaly detection datasets. The experimental results show that SGSF achieves state-of-the-art anomaly detection results.
Self-supervised similarity models based on well-logging data
Venue: arXiv cs.LG 2022
Year: 2022

Adopting data-based approaches leads to model improvement in numerous Oil&Gas logging data processing problems. These improvements become even more sound due to new capabilities provided by deep learning. However, usage of deep learning is limited to areas where researchers possess large amounts of high-quality data. We present an approach that provides universal data representations suitable for solutions to different problems for different oil fields with little additional data. Our approach relies on the self-supervised methodology for sequential logging data for intervals from well, so it also doesn't require labelled data from the start. For validation purposes of the received representations, we consider classification and clusterization problems. We as well consider the transfer learning scenario. We found out that using the variational autoencoder leads to the most reliable and accurate models. approach We also found that a researcher only needs a tiny separate data set for the target oil field to solve a specific problem on top of universal representations.
Delayed Geometric Discounts: An Alternative Criterion for Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

The endeavor of artificial intelligence (AI) is to design autonomous agents capable of achieving complex tasks. Namely, reinforcement learning (RL) proposes a theoretical background to learn optimal behaviors. In practice, RL algorithms rely on geometric discounts to evaluate this optimality. Unfortunately, this does not cover decision processes where future returns are not exponentially less valuable. Depending on the problem, this limitation induces sample-inefficiency (as feed-backs are exponentially decayed) and requires additional curricula/exploration mechanisms (to deal with sparse, deceptive or adversarial rewards). In this paper, we tackle these issues by generalizing the discounted problem formulation with a family of delayed objective functions. We investigate the underlying RL problem to derive: 1) the optimal stationary solution and 2) an approximation of the optimal non-stationary control. The devised algorithms solved hard exploration problems on tabular environment and improved sample-efficiency on classic simulated robotics benchmarks.
Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval
Venue: arXiv cs.LG 2022
Year: 2022

Zero-shot cross-modal retrieval (ZS-CMR) deals with the retrieval problem among heterogenous data from unseen classes. Typically, to guarantee generalization, the pre-defined class embeddings from natural language processing (NLP) models are used to build a common space. In this paper, instead of using an extra NLP model to define a common space beforehand, we consider a totally different way to construct (or learn) a common hamming space from an information-theoretic perspective. We term our model the Information-Theoretic Hashing (ITH), which is composed of two cascading modules: an Adaptive Information Aggregation (AIA) module; and a Semantic Preserving Encoding (SPE) module. Specifically, our AIA module takes the inspiration from the Principle of Relevant Information (PRI) to construct a common space that adaptively aggregates the intrinsic semantics of different modalities of data and filters out redundant or irrelevant information. On the other hand, our SPE module further generates the hashing codes of different modalities by preserving the similarity of intrinsic semantics with the element-wise Kullback-Leibler (KL) divergence. A total correlation regularization term is also imposed to reduce the redundancy amongst different dimensions of hash codes. Sufficient experiments on three benchmark datasets demonstrate the superiority of the proposed ITH in ZS-CMR. Source code is available in the supplementary material.
Constrained Multi-Agent Path Finding on Directed Graphs
Venue: arXiv cs.MA 2022
Year: 2022

We discuss C-MP and C-MAPF, generalizations of the classical Motion Planning (MP) and Multi-Agent Path Finding (MAPF) problems on a directed graph G. Namely, we enforce an upper bound on the number of agents that occupy each member of a family of vertex subsets. For instance, this constraint allows maintaining a safety distance between agents. We prove that finding a feasible solution of C-MP and C-MAPF is NP-hard, and we propose a reduction method to convert them to standard MP and MAPF. This reduction method consists in finding a subset of nodes W and a reduced graph G/W, such that a solution of MAPF on G/W provides a solution of C-MAPF on G. Moreover, we study the problem of finding W of maximum cardinality, which is strongly NP-hard.
Spatio-temporal Keyframe Control of Traffic Simulation using Coarse-to-Fine Optimization
Venue: arXiv cs.GR 2022
Year: 2022

We present a novel traffic trajectory editing method which uses spatio-temporal keyframes to control vehicles during the simulation to generate desired traffic trajectories. By taking self-motivation, path following and collision avoidance into account, the proposed force-based traffic simulation framework updates vehicle's motions in both the Frenet coordinates and the Cartesian coordinates. With the way-points from users, lane-level navigation can be generated by reference path planning. With a given keyframe, the coarse-to-fine optimization is proposed to efficiently generate the plausible trajectory which can satisfy the spatio-temporal constraints. At first, a directed state-time graph constructed along the reference path is used to search for a coarse-grained trajectory by mapping the keyframe as the goal. Then, using the information extracted from the coarse trajectory as initialization, adjoint-based optimization is applied to generate a finer trajectory with smooth motions based on our force-based simulation. We validate our method with extensive experiments.
Multi-Task Adversarial Training Algorithm for Multi-Speaker Neural Text-to-Speech
Venue: arXiv cs.SD 2022
Year: 2022

We propose a novel training algorithm for a multi-speaker neural text-to-speech (TTS) model based on multi-task adversarial training. A conventional generative adversarial network (GAN)-based training algorithm significantly improves the quality of synthetic speech by reducing the statistical difference between natural and synthetic speech. However, the algorithm does not guarantee the generalization performance of the trained TTS model in synthesizing voices of unseen speakers who are not included in the training data. Our algorithm alternatively trains two deep neural networks: multi-task discriminator and multi-speaker neural TTS model (i.e., generator of GANs). The discriminator is trained not only to distinguish between natural and synthetic speech but also to verify the speaker of input speech is existent or non-existent (i.e., newly generated by interpolating seen speakers' embedding vectors). Meanwhile, the generator is trained to minimize the weighted sum of the speech reconstruction loss and adversarial loss for fooling the discriminator, which achieves high-quality multi-speaker TTS even if the target speaker is unseen. Experimental evaluation shows that our algorithm improves the quality of synthetic speech better than a conventional GANSpeech algorithm.
Experiential Benefits of Interactive Conflict Negotiation Practices in Computer-Supported Shift Planning
Venue: arXiv cs.HC 2022
Year: 2022

Shift planning plays a key role for the health and well-being of healthcare workers. It determines when they work and when they can take time off to recover or engage in social activities. Current computer-support in shift planning is typically designed from a managerial perspective and focuses on process efficiency, with the long-term goal of full automation. This implies automatic resolutions of emotionally charged scheduling conflicts. In the present study, we measured the effects of such a fully automated process on workers' well-being, fairness, and team spirit, and compared them with a more interactive process that directly involves workers in the decision-making. In our experimental online study (n = 94), we found positive effects of the more interactive process on all measures. Our findings indicate that full automation may not be desirable from the worker perspective. We close with concrete suggestions to design more worker-centered, hybrid shift planning systems by optimizing worker control, considering the worker experience, and embedding shift planning in the broader work context.
Challenges and Opportunities for Inclusive Space Missions
Venue: arXiv cs.HC 2022
Year: 2022

People with impairments are not able to participate in space missions. However, this is not because they cannot; instead, spacecraft have not been designed for them. Therefore, instead of simply excluding people with impairments, they should be included in the design process. This may also help astronauts suffering from fatigue or accidents. Indeed, several impairments can occur due to the effects of microgravity on the body and psychological factors during long space missions. In this paper, we describe the idea of including people with all types of impairments in the design process of interactive space systems to obtain, as a result, systems that can also be used by astronauts suffering from a temporary or situational impairment during a long space flight. To this end, we have described solutions for some types of impairments to ensure the use of interactive systems with permanent, temporary, or situational impairments. The benefits from the participation of people with impairments also bring the idea of inclusion in space a bit closer: supporting people with impairments in space by designing appropriate systems that make space accessible.
Deep Manifold Hashing: A Divide-and-Conquer Approach for Semi-Paired Unsupervised Cross-Modal Retrieval
Venue: arXiv cs.CV 2022
Year: 2022

Hashing that projects data into binary codes has shown extraordinary talents in cross-modal retrieval due to its low storage usage and high query speed. Despite their empirical success on some scenarios, existing cross-modal hashing methods usually fail to cross modality gap when fully-paired data with plenty of labeled information is nonexistent. To circumvent this drawback, motivated by the Divide-and-Conquer strategy, we propose Deep Manifold Hashing (DMH), a novel method of dividing the problem of semi-paired unsupervised cross-modal retrieval into three sub-problems and building one simple yet efficiency model for each sub-problem. Specifically, the first model is constructed for obtaining modality-invariant features by complementing semi-paired data based on manifold learning, whereas the second model and the third model aim to learn hash codes and hash functions respectively. Extensive experiments on three benchmarks demonstrate the superiority of our DMH compared with the state-of-the-art fully-paired and semi-paired unsupervised cross-modal hashing methods.
T-NER: An All-Round Python Library for Transformer-based Named Entity Recognition
Venue: arXiv cs.CL 2022
Year: 2022

Language model (LM) pretraining has led to consistent improvements in many NLP downstream tasks, including named entity recognition (NER). In this paper, we present T-NER (Transformer-based Named Entity Recognition), a Python library for NER LM finetuning. In addition to its practical utility, T-NER facilitates the study and investigation of the cross-domain and cross-lingual generalization ability of LMs finetuned on NER. Our library also provides a web app where users can get model predictions interactively for arbitrary text, which facilitates qualitative model evaluation for non-expert programmers. We show the potential of the library by compiling nine public NER datasets into a unified format and evaluating the cross-domain and cross-lingual performance across the datasets. The results from our initial experiments show that in-domain performance is generally competitive across datasets. However, cross-domain generalization is challenging even with a large pretrained LM, which has nevertheless capacity to learn domain-specific features if fine-tuned on a combined dataset. To facilitate future research, we also release all our LM checkpoints via the Hugging Face model hub.