OpenRaiser/Intern-Atlas huggingface dataset:
OCID-Ref: A 3D Robotic Dataset with Embodied Language for Clutter Scene Grounding
Venue: arXiv cs.CL 2021
Year: 2021

To effectively apply robots in working environments and assist humans, it is essential to develop and evaluate how visual grounding (VG) can affect machine performance on occluded objects. However, current VG works are limited in working environments, such as offices and warehouses, where objects are usually occluded due to space utilization issues. In our work, we propose a novel OCID-Ref dataset featuring a referring expression segmentation task with referring expressions of occluded objects. OCID-Ref consists of 305,694 referring expressions from 2,300 scenes with providing RGB image and point cloud inputs. To resolve challenging occlusion issues, we argue that it's crucial to take advantage of both 2D and 3D signals to resolve challenging occlusion issues. Our experimental results demonstrate the effectiveness of aggregating 2D and 3D signals but referring to occluded objects still remains challenging for the modern visual grounding systems. OCID-Ref is publicly available at https://github.com/lluma/OCID-Ref
Personalized Human-Swarm Interaction through Hand Motion
Venue: arXiv cs.RO 2021
Year: 2021

The control of collective robotic systems, such as drone swarms, is often delegated to autonomous navigation algorithms due to their high dimensionality. However, like other robotic entities, drone swarms can still benefit from being teleoperated by human operators, whose perception and decision-making capabilities are still out of the reach of autonomous systems. Drone swarm teleoperation is only at its dawn, and a standard human-swarm interface (HRI) is missing to date. In this study, we analyzed the spontaneous interaction strategies of naive users with a swarm of drones. We implemented a machine-learning algorithm to define a personalized Body-Machine Interface (BoMI) based only on a short calibration procedure. During this procedure, the human operator is asked to move spontaneously as if they were in control of a simulated drone swarm. We assessed that hands are the most commonly adopted body segment, and thus we chose a LEAP Motion controller to track them to let the users control the aerial drone swarm. This choice makes our interface portable since it does not rely on a centralized system for tracking the human body. We validated our algorithm to define personalized HRIs for a set of participants in a realistic simulated environment, showing promising results in performance and user experience. Our method leaves unprecedented freedom to the user to choose between position and velocity control only based on their body motion preferences.
On Bounds for Ring-Based Coding Theory
Venue: arXiv cs.IT 2021
Year: 2021

Coding Theory where the alphabet is identified with the elements of a ring or a module has become an important research topic over the last 30 years. Such codes over rings had important applications and many interesting mathematical problems are related to this line of research. It has been well established, that with the generalization of the algebraic structure to rings there is a need to also generalize the underlying metric beyond the usual Hamming weight used in traditional coding theory over finite fields. This paper introduces a new weight, called the overweight, which can be seen as a generalization of the Lee weight on the integers modulo $4$. For this new weight we provide a number of well-known bounds, like a Plotkin bound, a sphere-packing bound, and a Gilbert-Varshamov bound. A further highlight is the proof of a Johnson bound for the homogeneous weight on a general finite Frobenius ring.
A Methodology for Bi-Directional Knowledge-Based Assessment of Compliance to Continuous Application of Clinical Guidelines
Venue: arXiv cs.AI 2021
Year: 2021

Clinicians often do not sufficiently adhere to evidence-based clinical guidelines in a manner sensitive to the context of each patient. It is important to detect such deviations, typically including redundant or missing actions, even when the detection is performed retrospectively, so as to inform both the attending clinician and policy makers. Furthermore, it would be beneficial to detect such deviations in a manner proportional to the level of the deviation, and not to simply use arbitrary cut-off values. In this study, we introduce a new approach for automated guideline-based quality assessment of the care process, the bidirectional knowledge-based assessment of compliance (BiKBAC) method. Our BiKBAC methodology assesses the degree of compliance when applying clinical guidelines, with respect to multiple different aspects of the guideline (e.g., the guideline's process and outcome objectives). The assessment is performed through a highly detailed, automated quality-assessment retrospective analysis, which compares a formal representation of the guideline and of its process and outcome intentions (we use the Asbru language for that purpose) with the longitudinal electronic medical record of its continuous application over a significant time period, using both a top-down and a bottom-up approach, which we explain in detail. Partial matches of the data to the process and to the outcome objectives are resolved using fuzzy temporal logic. We also introduce the DiscovErr system, which implements the BiKBAC approach, and present its detailed architecture. The DiscovErr system was evaluated in a separate study in the type 2 diabetes management domain, by comparing its performance to a panel of three clinicians, with highly encouraging results with respect to the completeness and correctness of its comments.
A `Sourceful' Twist: Emoji Prediction Based on Sentiment, Hashtags and Application Source
Venue: arXiv cs.CL 2021
Year: 2021

We widely use emojis in social networking to heighten, mitigate or negate the sentiment of the text. Emoji suggestions already exist in many cross-platform applications but an emoji is predicted solely based a few prominent words instead of understanding the subject and substance of the text. Through this paper, we showcase the importance of using Twitter features to help the model understand the sentiment involved and hence to predict the most suitable emoji for the text. Hashtags and Application Sources like Android, etc. are two features which we found to be important yet underused in emoji prediction and Twitter sentiment analysis on the whole. To approach this shortcoming and to further understand emoji behavioral patterns, we propose a more balanced dataset by crawling additional Twitter data, including timestamp, hashtags, and application source acting as additional attributes to the tweet. Our data analysis and neural network model performance evaluations depict that using hashtags and application sources as features allows to encode different information and is effective in emoji prediction.
High speed and reconfigurable optronic neural network with digital nonlinear activation
Venue: arXiv cs.ET 2021
Year: 2021

With its unique parallel processing capability, optical neural network has shown low-power consumption in image recognition and speech processing. At present, the manufacturing technology of programmable photonic chip is not mature, and the realization of optical neural network in free-space is still a hot spot of intelligent optical computing. In this article, based on MNIST datasets and 4f system, three-layer optical neural networks are constructed, whose recognition accuracy can reach 93.66%. Our network is programmable, high speed, reconfigurable and is better than the existing free-space optical neural network in terms of spatial complexity.
Learning a Word-Level Language Model with Sentence-Level Noise Contrastive Estimation for Contextual Sentence Probability Estimation
Venue: arXiv cs.CL 2021
Year: 2021

Inferring the probability distribution of sentences or word sequences is a key process in natural language processing. While word-level language models (LMs) have been widely adopted for computing the joint probabilities of word sequences, they have difficulty in capturing a context long enough for sentence probability estimation (SPE). To overcome this, recent studies introduced training methods using sentence-level noise-contrastive estimation (NCE) with recurrent neural networks (RNNs). In this work, we attempt to extend it for contextual SPE, which aims to estimate a conditional sentence probability given a previous text. The proposed NCE samples negative sentences independently of a previous text so that the trained model gives higher probabilities to the sentences that are more consistent with \textcolor{blue}{the} context. We apply our method to a simple word-level RNN LM to focus on the effect of the sentence-level NCE training rather than on the network architecture. The quality of estimation was evaluated against multiple-choice cloze-style questions including both human and automatically generated questions. The experimental results show that the proposed method improved the SPE quality for the word-level RNN LM.
Principled Ultrasound Data Augmentation for Classification of Standard Planes
Venue: arXiv cs.CV 2021
Year: 2021

Deep learning models with large learning capacities often overfit to medical imaging datasets. This is because training sets are often relatively small due to the significant time and financial costs incurred in medical data acquisition and labelling. Data augmentation is therefore often used to expand the availability of training data and to increase generalization. However, augmentation strategies are often chosen on an ad-hoc basis without justification. In this paper, we present an augmentation policy search method with the goal of improving model classification performance. We include in the augmentation policy search additional transformations that are often used in medical image analysis and evaluate their performance. In addition, we extend the augmentation policy search to include non-linear mixed-example data augmentation strategies. Using these learned policies, we show that principled data augmentation for medical image model training can lead to significant improvements in ultrasound standard plane detection, with an an average F1-score improvement of 7.0% overall over naive data augmentation strategies in ultrasound fetal standard plane classification. We find that the learned representations of ultrasound images are better clustered and defined with optimized data augmentation.
GVINS: Tightly Coupled GNSS-Visual-Inertial Fusion for Smooth and Consistent State Estimation
Venue: arXiv cs.RO 2021
Year: 2021

Visual-Inertial odometry (VIO) is known to suffer from drifting especially over long-term runs. In this paper, we present GVINS, a non-linear optimization based system that tightly fuses GNSS raw measurements with visual and inertial information for real-time and drift-free state estimation. Our system aims to provide accurate global 6-DoF estimation under complex indoor-outdoor environment where GNSS signals may be intermittent or even totally unavailable. To connect global measurements with local states, a coarse-to-fine initialization procedure is proposed to efficiently calibrate the transformation online and initialize GNSS states from only a short window of measurements. The GNSS code pseudorange and Doppler shift measurements, along with visual and inertial information, are then modelled and used to constrain the system states in a factor graph framework. For complex and GNSS-unfriendly areas, the degenerate cases are discussed and carefully handled to ensure robustness. Thanks to the tightly-coupled multi-sensor approach and system design, our system fully exploits the merits of three types of sensors and is capable to seamlessly cope with the transition between indoor and outdoor environments, where satellites are lost and reacquired. We extensively evaluate the proposed system by both simulation and real-world experiments, and the result demonstrates that our system substantially eliminates the drift of VIO and preserves the local accuracy in spite of noisy GNSS measurements. The challenging indoor-outdoor and urban driving experiments verify the availability and robustness of GVINS in complex environments. In addition, experiments also show that our system can gain from even a single satellite while conventional GNSS algorithms need four at least.
An SPH framework for fluid-solid and contact interaction problems including thermo-mechanical coupling and reversible phase transitions
Venue: arXiv cs.CE 2021
Year: 2021

The present work proposes an approach for fluid-solid and contact interaction problems including thermo-mechanical coupling and reversible phase transitions. The solid field is assumed to consist of several arbitrarily-shaped, undeformable but mobile rigid bodies, that are evolved in time individually and allowed to get into mechanical contact with each other. The fluid field generally consists of multiple liquid or gas phases. All fields are spatially discretized using the method of smoothed particle hydrodynamics (SPH). This approach is especially suitable in the context of continually changing interface topologies and dynamic phase transitions without the need for additional methodological and computational effort for interface tracking as compared to mesh- or grid-based methods. Proposing a concept for the parallelization of the computational framework, in particular concerning a computationally efficient evaluation of rigid body motion, is an essential part of this work. Finally, the accuracy and robustness of the proposed framework is demonstrated by several numerical examples in two and three dimensions, involving multiple rigid bodies, two-phase flow, and reversible phase transitions, with a focus on two potential application scenarios in the fields of engineering and biomechanics: powder bed fusion additive manufacturing (PBFAM) and disintegration of food boluses in the human stomach. The efficiency of the parallel computational framework is demonstrated by a strong scaling analysis.
Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systems
Venue: arXiv cs.LG 2021
Year: 2021

Active network management (ANM) of electricity distribution networks include many complex stochastic sequential optimization problems. These problems need to be solved for integrating renewable energies and distributed storage into future electrical grids. In this work, we introduce Gym-ANM, a framework for designing reinforcement learning (RL) environments that model ANM tasks in electricity distribution networks. These environments provide new playgrounds for RL research in the management of electricity networks that do not require an extensive knowledge of the underlying dynamics of such systems. Along with this work, we are releasing an implementation of an introductory toy-environment, ANM6-Easy, designed to emphasize common challenges in ANM. We also show that state-of-the-art RL algorithms can already achieve good performance on ANM6-Easy when compared against a model predictive control (MPC) approach. Finally, we provide guidelines to create new Gym-ANM environments differing in terms of (a) the distribution network topology and parameters, (b) the observation space, (c) the modelling of the stochastic processes present in the system, and (d) a set of hyperparameters influencing the reward signal. Gym-ANM can be downloaded at https://github.com/robinhenry/gym-anm.
Learning needle insertion from sample task executions
Venue: arXiv cs.RO 2021
Year: 2021

Automating a robotic task, e.g., robotic suturing can be very complex and time-consuming. Learning a task model to autonomously perform the task is invaluable making the technology, robotic surgery, accessible for a wider community. The data of robotic surgery can be easily logged where the collected data can be used to learn task models. This will result in reduced time and cost of robotic surgery in which a surgeon can supervise the robot operation or give high-level commands instead of low-level control of the tools. We present a data-set of needle insertion in soft tissue with two arms where Arm 1 inserts the needle into the tissue and Arm 2 actively manipulate the soft tissue to ensure the desired and actual exit points are the same. This is important in real-surgery because suturing without active manipulation of tissue may yield failure of the suturing as the stitch may not grip enough tissue to resist the force applied for the suturing. We present a needle insertion dataset including 60 successful trials recorded by 3 pair of stereo cameras. Moreover, we present Deep-robot Learning from Demonstrations that predicts the desired state of the robot at the time step after t (which the optimal action taken at t yields) by looking at the video of the past time steps, i.e. n step time history where N is the memory time window, of the task execution. The experimental results illustrate our proposed deep model architecture is outperforming the existing methods. Although the solution is not yet ready to be deployed on a real robot, the results indicate the possibility of future development for real robot deployment.
Pre-interpolation loss behaviour in neural networks
Venue: arXiv cs.LG 2021
Year: 2021

When training neural networks as classifiers, it is common to observe an increase in average test loss while still maintaining or improving the overall classification accuracy on the same dataset. In spite of the ubiquity of this phenomenon, it has not been well studied and is often dismissively attributed to an increase in borderline correct classifications. We present an empirical investigation that shows how this phenomenon is actually a result of the differential manner by which test samples are processed. In essence: test loss does not increase overall, but only for a small minority of samples. Large representational capacities allow losses to decrease for the vast majority of test samples at the cost of extreme increases for others. This effect seems to be mainly caused by increased parameter values relating to the correctly processed sample features. Our findings contribute to the practical understanding of a common behaviour of deep neural networks. We also discuss the implications of this work for network optimisation and generalisation.
Deep Tiling: Texture Tile Synthesis Using a Deep Learning Approach
Venue: arXiv cs.CV 2021
Year: 2021

Texturing is a fundamental process in computer graphics. Texture is leveraged to enhance the visualization outcome for a 3D scene. In many cases a texture image cannot cover a large 3D model surface because of its small resolution. Conventional techniques like repeating, mirror repeating or clamp to edge do not yield visually acceptable results. Deep learning based texture synthesis has proven to be very effective in such cases. All deep texture synthesis methods trying to create larger resolution textures are limited in terms of GPU memory resources. In this paper, we propose a novel approach to example-based texture synthesis by using a robust deep learning process for creating tiles of arbitrary resolutions that resemble the structural components of an input texture. In this manner, our method is firstly much less memory limited owing to the fact that a new texture tile of small size is synthesized and merged with the original texture and secondly can easily produce missing parts of a large texture.
Crowdsourced Phrase-Based Tokenization for Low-Resourced Neural Machine Translation: The Case of Fon Language
Venue: arXiv cs.CL 2021
Year: 2021

Building effective neural machine translation (NMT) models for very low-resourced and morphologically rich African indigenous languages is an open challenge. Besides the issue of finding available resources for them, a lot of work is put into preprocessing and tokenization. Recent studies have shown that standard tokenization methods do not always adequately deal with the grammatical, diacritical, and tonal properties of some African languages. That, coupled with the extremely low availability of training samples, hinders the production of reliable NMT models. In this paper, using Fon language as a case study, we revisit standard tokenization methods and introduce Word-Expressions-Based (WEB) tokenization, a human-involved super-words tokenization strategy to create a better representative vocabulary for training. Furthermore, we compare our tokenization strategy to others on the Fon-French and French-Fon translation tasks.
Cloth Manipulation Planning on Basis of Mesh Representations with Incomplete Domain Knowledge and Voxel-to-Mesh Estimation
Venue: arXiv cs.RO 2021
Year: 2021

We consider the problem of open-goal planning for robotic cloth manipulation. Core of our system is a neural network trained as a forward model of cloth behaviour under manipulation, with planning performed through backpropagation. We introduce a neural network-based routine for estimating mesh representations from voxel input, and perform planning in mesh format internally. We address the problem of planning with incomplete domain knowledge by means of an explicit epistemic uncertainty signal. This signal is calculated from prediction divergence between two instances of the forward model network and used to avoid epistemic uncertainty during planning. Finally, we introduce logic for handling restriction of grasp points to a discrete set of candidates, in order to accommodate graspability constraints imposed by robotic hardware. We evaluate the system's mesh estimation, prediction, and planning ability on simulated cloth for sequences of one to three manipulations. Comparative experiments confirm that planning on basis of estimated meshes improves accuracy compared to voxel-based planning, and that epistemic uncertainty avoidance improves performance under conditions of incomplete domain knowledge. Planning time cost is a few seconds. We additionally present qualitative results on robot hardware.
Eigen Space of Mesh Distortion Energy Hessian
Venue: arXiv cs.GR 2021
Year: 2021

Mesh distortion optimization is a popular research topic and has wide range of applications in computer graphics, including geometry modeling, variational shape interpolation, UV parameterization, elastoplastic simulation, etc. In recent years, many solvers have been proposed to solve this nonlinear optimization efficiently, among which projected Newton has been shown to have best convergence rate and work well in both 2D and 3D applications. Traditional Newton approach suffers from ill conditioning and indefiniteness of local energy approximation. A crucial step in projected Newton is to fix this issue by projecting energy Hessian onto symmetric positive definite (SPD) cone so as to guarantee the search direction always pointing to decrease the energy locally. Such step relies on time consuming eigen decomposition of element Hessian, which has been addressed by several work before on how to obtain a conjugacy that is as diagonal as possible. In this report, we demonstrate an analytic form of Hessian eigen system for distortion energy defined using principal stretches, which is the most general representation. Compared with existing projected Newton diagonalization approaches, our formulation is more general as it doesn't require the energy to be representable by tensor invariants. In this report, we will only show the derivation for 3D and the extension to 2D case is straightforward.