OpenRaiser/Intern-Atlas huggingface dataset:
Transmit Correlation Diversity: Generalization, New Techniques, and Improved Bounds
Venue: arXiv cs.IT 2021
Year: 2021

When the users in a MIMO broadcast channel experience different spatial transmit correlation matrices, a class of gains is produced that is denoted transmit correlation diversity. This idea was conceived for channels in which transmit correlation matrices have mutually exclusive eigenspaces, allowing non-interfering training and transmission. This paper broadens the scope of transmit correlation diversity to the case of partially and fully overlapping eigenspaces and introduces techniques to harvest these generalized gains. For the two-user MIMO broadcast channel, we derive achievable degrees of freedom (DoF) and achievable rate regions with/without channel state information at the receiver (CSIR). When CSIR is available, the proposed achievable DoF region is tight in some configurations of the number of receive antennas and the channel correlation ranks. We then extend the DoF results to the $K$-user case by analyzing the interference graph that characterizes the overlapping structure of the eigenspaces. Our achievability results employ a combination of product superposition in the common part of the eigenspaces, and pre-beamforming (rate splitting) to create multiple data streams in non-overlapping parts of the eigenspaces. Massive MIMO is a natural example in which spatially correlated link gains are likely to occur. We study the achievable downlink sum rate for a frequency-division duplex massive MIMO system under transmit correlation diversity.
Mitigating Temporal-Drift: A Simple Approach to Keep NER Models Crisp
Venue: arXiv cs.CL 2021
Year: 2021

Performance of neural models for named entity recognition degrades over time, becoming stale. This degradation is due to temporal drift, the change in our target variables' statistical properties over time. This issue is especially problematic for social media data, where topics change rapidly. In order to mitigate the problem, data annotation and retraining of models is common. Despite its usefulness, this process is expensive and time-consuming, which motivates new research on efficient model updating. In this paper, we propose an intuitive approach to measure the potential trendiness of tweets and use this metric to select the most informative instances to use for training. We conduct experiments on three state-of-the-art models on the Temporal Twitter Dataset. Our approach shows larger increases in prediction accuracy with less training data than the alternatives, making it an attractive, practical solution.
Computing Arlequin coupling coefficient for concurrent FE-MD approaches
Venue: arXiv cs.CE 2021
Year: 2021

Arlequin coupling coefficient is essential for concurrent FE-MD models with overlapping domains, but the calculation of its value is quite difficult when the geometry of the coupling region is complicated. In this work, we introduce a general procedure for the preprocessing of a concurrent FE-MD model, given that the mesh and atoms have already been created. The procedure is independent of the geometry of the coupling region and can be used for both 2D and 3D problems. The procedure includes steps of determining the relative positions of atoms inside the FE elements in the coupling region, as well as computing the Arlequin coupling coefficient for an arbitrary point inside the coupling region or on its boundary. Two approaches are provided for determining the coefficient: the direct approach and the temperature approach.
Flow-based Video Segmentation for Human Head and Shoulders
Venue: arXiv cs.CV 2021
Year: 2021

Video segmentation for the human head and shoulders is essential in creating elegant media for videoconferencing and virtual reality applications. The main challenge is to process high-quality background subtraction in a real-time manner and address the segmentation issues under motion blurs, e.g., shaking the head or waving hands during conference video. To overcome the motion blur problem in video segmentation, we propose a novel flow-based encoder-decoder network (FUNet) that combines both traditional Horn-Schunck optical-flow estimation technique and convolutional neural networks to perform robust real-time video segmentation. We also introduce a video and image segmentation dataset: ConferenceVideoSegmentationDataset. Code and pre-trained models are available on our GitHub repository: \url{https://github.com/kuangzijian/Flow-Based-Video-Matting}.
An Efficient Approach for Anomaly Detection in Traffic Videos
Venue: arXiv cs.CV 2021
Year: 2021

Due to its relevance in intelligent transportation systems, anomaly detection in traffic videos has recently received much interest. It remains a difficult problem due to a variety of factors influencing the video quality of a real-time traffic feed, such as temperature, perspective, lighting conditions, and so on. Even though state-of-the-art methods perform well on the available benchmark datasets, they need a large amount of external training data as well as substantial computational resources. In this paper, we propose an efficient approach for a video anomaly detection system which is capable of running at the edge devices, e.g., on a roadside camera. The proposed approach comprises a pre-processing module that detects changes in the scene and removes the corrupted frames, a two-stage background modelling module and a two-stage object detector. Finally, a backtracking anomaly detection algorithm computes a similarity statistic and decides on the onset time of the anomaly. We also propose a sequential change detection algorithm that can quickly adapt to a new scene and detect changes in the similarity statistic. Experimental results on the Track 4 test set of the 2021 AI City Challenge show the efficacy of the proposed framework as we achieve an F1-score of 0.9157 along with 8.4027 root mean square error (RMSE) and are ranked fourth in the competition.
An Analysis of Indexing and Querying Strategies on a Technologically Assisted Review Task
Venue: arXiv cs.IR 2021
Year: 2021

This paper presents a preliminary experimentation study using the CLEF 2017 eHealth Task 2 collection for evaluating the effectiveness of different indexing methodologies of documents and query parsing techniques. Furthermore, it is an attempt to advance and share the efforts of observing the characteristics and helpfulness of various methodologies for indexing PubMed documents and for different topic parsing techniques to produce queries. For this purpose, my research includes experimentation with different document indexing methodologies, by utilising existing tools, such as the Lucene4IR (L4IR) information retrieval system, the Technology Assisted Reviews for Empirical Medicine tool for parsing topics of the CLEF collection and the TREC evaluation tool to appraise system's performance. The results showed that including a greater number of fields to the PubMed indexer of L4IR is a decisive factor for the retrieval effectiveness of L4IR.
Privacy-aware VR streaming
Venue: arXiv cs.MM 2021
Year: 2021

Proactive tile-based virtual reality (VR) video streaming employs the current tracking data of a user to predict future requested tiles, then renders and delivers the predicted tiles to be requested before playback. The quality of experience (QoE) depends on the overall performance of prediction, computing (i.e., rendering) and communication. All prior works neglect that users may have privacy requirement, i.e., not all the current tracking data are allowed to be uploaded. In this paper, we investigate the privacy-aware VR streaming. We first establish a dataset that collects the privacy requirement of 66 users among 18 panoramic videos. The dataset shows that the privacy requirements of 360$^{\circ}$ videos are heterogeneous. Only 41\% of the total watched videos have no privacy requirement. Based on these findings, we formulate the privacy requirement as the \textit{degree of privacy} (DoP), and investigate the impact of DoP on the proactive VR streaming. First, we find that with DoP, the length of the observation window and prediction window of a tile predictor should be variable. Then, we jointly optimize the durations for computing and transmitting the selected tiles as well as the computing and communication capability, aimed at maximizing the QoE given arbitrary predictor and configured resources. From the obtained optimal closed-form solution, we find a resource-saturated region where DoP has no impact on the QoE and a resource-unsaturated region where the two-fold impacts of DoP are contradictory. On the one hand, the increase of DoP will degrade the prediction performance and thus degrade the QoE. On the other hand, the increase of DoP will improve the capability of computing and communication and thus improve the QoE. Simulation results using two predictors and a real dataset validate the analysis and demonstrate the overall impact of DoP on the QoE.
Tactile Perception based on Injected Vibration in Soft Sensor
Venue: arXiv cs.RO 2021
Year: 2021

Tactile perception using vibration sensation helps robots recognize their environment's physical properties and perform complex tasks. A sliding motion is applied to target objects to generate tactile vibration data. However, situations exist where such a sliding motion is infeasible due to geometrical constraints in the environment or an object's fragility which cannot resist friction forces. This paper explores a novel approach to achieve vibration-based tactile perception without a sliding motion. To this end, our key idea is injecting a mechanical vibration into a soft tactile sensor system and measuring the propagated vibration inside it by a sensor. Soft tactile sensors are deformed by the contact state, and the touched objects' shape or texture should change the characteristics of the vibration propagation. Therefore, the propagated-vibration data are expected to contain useful information for recognizing touched environments. We developed a prototype system for a proof-of-concept: a mechanical vibration is applied to a biomimetic (soft and vibration-based) tactile sensor from a small, mounted piezoelectric actuator. As a verification experiment, we performed two classification tasks for sandpaper's grit size and a slit's gap widths using our approach and compared their accuracies with that of using sliding motions. Our approach resulted in 70% accuracy for the grit size classification and 99% accuracy for the gap width classification. These results are comparable to or better than the comparison methods with a sliding motion.
DeepHunter: A Graph Neural Network Based Approach for Robust Cyber Threat Hunting
Venue: arXiv cs.CR 2021
Year: 2021

Cyber Threat hunting is a proactive search for known attack behaviors in the organizational information system. It is an important component to mitigate advanced persistent threats (APTs). However, the attack behaviors recorded in provenance data may not be completely consistent with the known attack behaviors. In this paper, we propose DeepHunter, a graph neural network (GNN) based graph pattern matching approach that can match provenance data against known attack behaviors in a robust way. Specifically, we design a graph neural network architecture with two novel networks: attribute embedding networks that could incorporate Indicators of Compromise (IOCs) information, and graph embedding networks that could capture the relationships between IOCs. To evaluate DeepHunter, we choose five real and synthetic APT attack scenarios. Results show that DeepHunter can hunt all attack behaviors, and the accuracy and robustness of DeepHunter outperform the state-of-the-art method, Poirot.
Measuring the Ripeness of Fruit with Hyperspectral Imaging and Deep Learning
Venue: arXiv cs.CV 2021
Year: 2021

We present a system to measure the ripeness of fruit with a hyperspectral camera and a suitable deep neural network architecture. This architecture did outperform competitive baseline models on the prediction of the ripeness state of fruit. For this, we recorded a data set of ripening avocados and kiwis, which we make public. We also describe the process of data collection in a manner that the adaption for other fruit is easy. The trained network is validated empirically, and we investigate the trained features. Furthermore, a technique is introduced to visualize the ripening process.
A simple vision-based navigation and control strategy for autonomous drone racing
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we present a control system that allows a drone to fly autonomously through a series of gates marked with ArUco tags. A simple and low-cost DJI Tello EDU quad-rotor platform was used. Based on the API provided by the manufacturer, we have created a Python application that enables the communication with the drone over WiFi, realises drone positioning based on visual feedback, and generates control. Two control strategies were proposed, compared, and critically analysed. In addition, the accuracy of the positioning method used was measured. The application was evaluated on a laptop computer (about 40 fps) and a Nvidia Jetson TX2 embedded GPU platform (about 25 fps). We provide the developed code on GitHub.
Fluid-beam interaction: Capturing the effect of embedded slender bodies on global fluid flow and vice versa
Venue: arXiv cs.CE 2021
Year: 2021

This work addresses research questions arising from the application of geometrically exact beam theory in the context of fluid-structure interaction (FSI). Geometrically exact beam theory has proven to be a computationally efficient way to model the behavior of slender structures while leading to rather well-posed problem descriptions. In particular, we propose a mixed-dimensional embedded finite element approach for the coupling of one-dimensional geometrically exact beam equations to a three-dimensional background fluid mesh, referred to as fluid-beam interaction (FBI) in analogy to the well-established notion of FSI. Here, the fluid is described by the incompressible isothermal Navier-Stokes equations for Newtonian fluids. In particular, we present algorithmic aspects regarding the solution of the resulting one-way coupling schemes and, through selected numerical examples, analyze their spatial convergence behavior as well as their suitability not only as stand-alone methods but also for an extension to a full two-way coupling scheme.
Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit
Venue: arXiv cs.RO 2021
Year: 2021

Robotic-assisted surgery is now well-established in clinical practice and has become the gold standard clinical treatment option for several clinical indications. The field of robotic-assisted surgery is expected to grow substantially in the next decade with a range of new robotic devices emerging to address unmet clinical needs across different specialities. A vibrant surgical robotics research community is pivotal for conceptualizing such new systems as well as for developing and training the engineers and scientists to translate them into practice. The da Vinci Research Kit (dVRK), an academic and industry collaborative effort to re-purpose decommissioned da Vinci surgical systems (Intuitive Surgical Inc, CA, USA) as a research platform for surgical robotics research, has been a key initiative for addressing a barrier to entry for new research groups in surgical robotics. In this paper, we present an extensive review of the publications that have been facilitated by the dVRK over the past decade. We classify research efforts into different categories and outline some of the major challenges and needs for the robotics community to maintain this initiative and build upon it.
Shadow Neural Radiance Fields for Multi-view Satellite Photogrammetry
Venue: arXiv cs.CV 2021
Year: 2021

We present a new generic method for shadow-aware multi-view satellite photogrammetry of Earth Observation scenes. Our proposed method, the Shadow Neural Radiance Field (S-NeRF) follows recent advances in implicit volumetric representation learning. For each scene, we train S-NeRF using very high spatial resolution optical images taken from known viewing angles. The learning requires no labels or shape priors: it is self-supervised by an image reconstruction loss. To accommodate for changing light source conditions both from a directional light source (the Sun) and a diffuse light source (the sky), we extend the NeRF approach in two ways. First, direct illumination from the Sun is modeled via a local light source visibility field. Second, indirect illumination from a diffuse light source is learned as a non-local color field as a function of the position of the Sun. Quantitatively, the combination of these factors reduces the altitude and color errors in shaded areas, compared to NeRF. The S-NeRF methodology not only performs novel view synthesis and full 3D shape estimation, it also enables shadow detection, albedo synthesis, and transient object filtering, without any explicit shape supervision.
GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passing
Venue: arXiv cs.LG 2021
Year: 2021

In recent studies, neural message passing has proved to be an effective way to design graph neural networks (GNNs), which have achieved state-of-the-art performance in many graph-based tasks. However, current neural-message passing architectures typically need to perform an expensive recursive neighborhood expansion in multiple rounds and consequently suffer from a scalability issue. Moreover, most existing neural-message passing schemes are inflexible since they are restricted to fixed-hop neighborhoods and insensitive to the actual demands of different nodes. We circumvent these limitations by a novel feature-message passing framework, called Graph Multi-layer Perceptron (GMLP), which separates the neural update from the message passing. With such separation, GMLP significantly improves the scalability and efficiency by performing the message passing procedure in a pre-compute manner, and is flexible and adaptive in leveraging node feature messages over various levels of localities. We further derive novel variants of scalable GNNs under this framework to achieve the best of both worlds in terms of performance and efficiency. We conduct extensive evaluations on 11 benchmark datasets, including large-scale datasets like ogbn-products and an industrial dataset, demonstrating that GMLP achieves not only the state-of-art performance, but also high training scalability and efficiency.
Comparing Representations in Tracking for Event Camera-based SLAM
Venue: arXiv cs.CV 2021
Year: 2021

This paper investigates two typical image-type representations for event camera-based tracking: time surface (TS) and event map (EM). Based on the original TS-based tracker, we make use of these two representations' complementary strengths to develop an enhanced version. The proposed tracker consists of a general strategy to evaluate the optimization problem's degeneracy online and then switch proper representations. Both TS and EM are motion- and scene-dependent, and thus it is important to figure out their limitations in tracking. We develop six tracker variations and conduct a thorough comparison of them on sequences covering various scenarios and motion complexities. We release our implementations and detailed results to benefit the research community on event cameras: https: //github.com/gogojjh/ESVO_extension.
SALAD: Self-Adaptive Lightweight Anomaly Detection for Real-time Recurrent Time Series
Venue: arXiv cs.LG 2021
Year: 2021

Real-world time series data often present recurrent or repetitive patterns and it is often generated in real time, such as transportation passenger volume, network traffic, system resource consumption, energy usage, and human gait. Detecting anomalous events based on machine learning approaches in such time series data has been an active research topic in many different areas. However, most machine learning approaches require labeled datasets, offline training, and may suffer from high computation complexity, consequently hindering their applicability. Providing a lightweight self-adaptive approach that does not need offline training in advance and meanwhile is able to detect anomalies in real time could be highly beneficial. Such an approach could be immediately applied and deployed on any commodity machine to provide timely anomaly alerts. To facilitate such an approach, this paper introduces SALAD, which is a Self-Adaptive Lightweight Anomaly Detection approach based on a special type of recurrent neural networks called Long Short-Term Memory (LSTM). Instead of using offline training, SALAD converts a target time series into a series of average absolute relative error (AARE) values on the fly and predicts an AARE value for every upcoming data point based on short-term historical AARE values. If the difference between a calculated AARE value and its corresponding forecast AARE value is higher than a self-adaptive detection threshold, the corresponding data point is considered anomalous. Otherwise, the data point is considered normal. Experiments based on two real-world open-source time series datasets demonstrate that SALAD outperforms five other state-of-the-art anomaly detection approaches in terms of detection accuracy. In addition, the results also show that SALAD is lightweight and can be deployed on a commodity machine.
Robustness Tests of NLP Machine Learning Models: Search and Semantically Replace
Venue: arXiv cs.CL 2021
Year: 2021

This paper proposes a strategy to assess the robustness of different machine learning models that involve natural language processing (NLP). The overall approach relies upon a Search and Semantically Replace strategy that consists of two steps: (1) Search, which identifies important parts in the text; (2) Semantically Replace, which finds replacements for the important parts, and constrains the replaced tokens with semantically similar words. We introduce different types of Search and Semantically Replace methods designed specifically for particular types of machine learning models. We also investigate the effectiveness of this strategy and provide a general framework to assess a variety of machine learning models. Finally, an empirical comparison is provided of robustness performance among three different model types, each with a different text representation.
Efficient Online Transfer Learning for 3D Object Classification in Autonomous Driving
Venue: arXiv cs.RO 2021
Year: 2021

Autonomous driving has achieved rapid development over the last few decades, including the machine perception as an important issue of it. Although object detection based on conventional cameras has achieved remarkable results in 2D/3D, non-visual sensors such as 3D LiDAR still have incomparable advantages in the accuracy of object position detection. However, the challenge also exists with the difficulty in properly interpreting point cloud generated by LiDAR. This paper presents a multi-modal-based online learning system for 3D LiDAR-based object classification in urban environments, including cars, cyclists and pedestrians. The proposed system aims to effectively transfer the mature detection capabilities based on visual sensors to the new model learning based on non-visual sensors through a multi-target tracker (i.e. using one sensor to train another). In particular, it integrates the Online Random Forests (ORF) [1] method, which inherently has the abilities of fast and multi-class learning. Through experiments, we show that our system is capable of learning a high-performance model for LiDAR-based 3D object classification on-the-fly, which is especially suitable for robotics in-situ deployment while responding to the widespread challenge of insufficient detector generalization capabilities.