OpenRaiser/Intern-Atlas huggingface dataset:
GymFG: A Framework with a Gym Interface for FlightGear
Venue: arXiv cs.AI 2020
Year: 2020

Over the past decades, progress in deployable autonomous flight systems has slowly stagnated. This is reflected in today's production air-crafts, where pilots only enable simple physics-based systems such as autopilot for takeoff, landing, navigation, and terrain/traffic avoidance. Evidently, autonomy has not gained the trust of the community where higher problem complexity and cognitive workload are required. To address trust, we must revisit the process for developing autonomous capabilities: modeling and simulation. Given the prohibitive costs for live tests, we need to prototype and evaluate autonomous aerial agents in a high fidelity flight simulator with autonomous learning capabilities applicable to flight systems: such a open-source development platform is not available. As a result, we have developed GymFG: GymFG couples and extends a high fidelity, open-source flight simulator and a robust agent learning framework to facilitate learning of more complex tasks. Furthermore, we have demonstrated the use of GymFG to train an autonomous aerial agent using Imitation Learning. With GymFG, we can now deploy innovative ideas to address complex problems and build the trust necessary to move prototypes to the real-world.
How Much Should I Pay? An Empirical Analysis on Monetary Prize in TopCoder
Venue: arXiv cs.SE 2020
Year: 2020

It is reported that task monetary prize is one of the most important motivating factors to attract crowd workers. While using expert-based methods to price Crowdsourcing tasks is a common practice, the challenge of validating the associated prices across different tasks is a constant issue. To address this issue, three different classifications of multiple linear regression, logistic regression, and K-nearest neighbor were compared to find the most accurate predicted price, using a dataset from the TopCoder website. The result of comparing chosen algorithms showed that the logistics regression model will provide the highest accuracy of 90% to predict the associated price to tasks and KNN ranked the second with an accuracy of 64% for K = 7. Also, applying PCA wouldn't lead to any better prediction accuracy as data components are not correlated.
Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning continues to revolutionize an ever-growing number of critical application areas including healthcare, transportation, finance, and basic sciences. Despite their increased predictive power, model transparency and human explainability remain a significant challenge due to the "black box" nature of modern deep learning models. In many cases the desired balance between interpretability and performance is predominately task specific. Human-centric domains such as healthcare necessitate a renewed focus on understanding how and why these frameworks are arriving at critical and potentially life-or-death decisions. Given the quantity of research and empirical successes of deep learning for computer vision, most of the existing interpretability research has focused on image processing techniques. Comparatively, less attention has been paid to interpreting deep learning frameworks using sequential data. Given recent deep learning advancements in highly sequential domains such as natural language processing and physiological signal processing, the need for deep sequential explanations is at an all-time high. In this paper, we review current techniques for interpreting deep learning techniques involving sequential data, identify similarities to non-sequential methods, and discuss current limitations and future avenues of sequential interpretability research.
Spatial-Temporal Mitosis Detection in Phase-Contrast Microscopy via Likelihood Map Estimation by 3DCNN
Venue: arXiv cs.LG 2020
Year: 2020

Automated mitotic detection in time-lapse phasecontrast microscopy provides us much information for cell behavior analysis, and thus several mitosis detection methods have been proposed. However, these methods still have two problems; 1) they cannot detect multiple mitosis events when there are closely placed. 2) they do not consider the annotation gaps, which may occur since the appearances of mitosis cells are very similar before and after the annotated frame. In this paper, we propose a novel mitosis detection method that can detect multiple mitosis events in a candidate sequence and mitigate the human annotation gap via estimating a spatiotemporal likelihood map by 3DCNN. In this training, the loss gradually decreases with the gap size between ground truth and estimation. This mitigates the annotation gaps. Our method outperformed the compared methods in terms of F1- score using a challenging dataset that contains the data under four different conditions.
Wireless VR/Haptic Open Platform for Multimodal Teleoperation
Venue: arXiv cs.NI 2020
Year: 2020

With emerging trends in the fifth generation and robotics, the Internet of Skills will enable us to deliver skills or expertise anywhere over the Internet. In this paper, we propose a wireless connected virtual reality and haptic communication open platform to show the proof of concept for multimodal teleoperation systems in real-time. We focus on a practical implementation with commercial products to facilitate the access and modification of the system. The performance of the system is measured in terms of system latency and user-centric metrics.
How to Return to Normalcy: Fast and Comprehensive Contact Tracing of COVID-19 through Proximity Sensing Using Mobile Devices
Venue: arXiv cs.CY 2020
Year: 2020

We outline a contact-tracing strategy based on proximity sensing using mobile devices. We discuss what an ideal system should look like and what it can do. We show that, when adopted sufficiently broadly, such a contact-tracing strategy can bring COVID-19 under complete control, end the need of social distancing, and return the society to full normalcy. We also review some of the challenges faced by the current generation of proximity-sensing technologies, including Bluetooth Low Energy used by phones, and consider both interim and longer-term solutions. Our main contribution is that we reason through why such a contact-tracing strategy is likely to achieve the stated goal of returning to full normalcy. Using probabilistic models, we show that universal adoption is not necessary to achieve the stated goal, thus there is some room for exceptions; however, the adoption rate needs to be very high, e.g., above $95\%$ depending on the disease parameters. With more vigilance in disease surveillance to detect mild cases earlier, the number may be brought down to about $90\%$. The results call for deployment effort to be led by public authorities at the state or federal level so that the required adoption rate can be reached and the tracing coverage is wide enough to be relevant for disease control.
VTGNet: A Vision-based Trajectory Generation Network for Autonomous Vehicles in Urban Environments
Venue: arXiv cs.CV 2020
Year: 2020

Traditional methods for autonomous driving are implemented with many building blocks from perception, planning and control, making them difficult to generalize to varied scenarios due to complex assumptions and interdependencies. Recently, the end-to-end driving method has emerged, which performs well and generalizes to new environments by directly learning from export-provided data. However, many existing methods on this topic neglect to check the confidence of the driving actions and the ability to recover from driving mistakes. In this paper, we develop an uncertainty-aware end-to-end trajectory generation method based on imitation learning. It can extract spatiotemporal features from the front-view camera images for scene understanding, and then generate collision-free trajectories several seconds into the future. The experimental results suggest that under various weather and lighting conditions, our network can reliably generate trajectories in different urban environments, such as turning at intersections and slowing down for collision avoidance. Furthermore, closed-loop driving tests suggest that the proposed method achieves better cross-scene/platform driving results than the state-of-the-art (SOTA) end-to-end control method, where our model can recover from off-center and off-orientation errors and capture 80% of dangerous cases with high uncertainty estimations.
CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents
Venue: arXiv cs.CV 2020
Year: 2020

An automatic table recognition method for interpretation of tabular data in document images majorly involves solving two problems of table detection and table structure recognition. The prior work involved solving both problems independently using two separate approaches. More recent works signify the use of deep learning-based solutions while also attempting to design an end to end solution. In this paper, we present an improved deep learning-based end to end approach for solving both problems of table detection and structure recognition using a single Convolution Neural Network (CNN) model. We propose CascadeTabNet: a Cascade mask Region-based CNN High-Resolution Network (Cascade mask R-CNN HRNet) based model that detects the regions of tables and recognizes the structural body cells from the detected tables at the same time. We evaluate our results on ICDAR 2013, ICDAR 2019 and TableBank public datasets. We achieved 3rd rank in ICDAR 2019 post-competition results for table detection while attaining the best accuracy results for the ICDAR 2013 and TableBank dataset. We also attain the highest accuracy results on the ICDAR 2019 table structure recognition dataset. Additionally, we demonstrate effective transfer learning and image augmentation techniques that enable CNNs to achieve very accurate table detection results. Code and dataset has been made available at: https://github.com/DevashishPrasad/CascadeTabNet
Influence of Hand Tracking as a way of Interaction in Virtual Reality on User Experience
Venue: arXiv cs.HC 2020
Year: 2020

With the rising interest in Virtual Reality and the fast development and improvement of available devices, new features of interactions are becoming available. One of them that is becoming very popular is hand tracking, as the idea to replace controllers for interactions in virtual worlds. This experiment aims to compare different interaction types in VR using either controllers or hand tracking. Participants had to play two simple VR games with various types of tasks in those games - grabbing objects or typing numbers. While playing, they were using interactions with different visualizations of hands and controllers. The focus of this study was to investigate user experience of varying interactions (controller vs. hand tracking) for those two simple tasks. Results show that different interaction types statistically significantly influence reported emotions with Self-Assessment Manikin (SAM), where for hand tracking participants were feeling higher valence, but lower arousal and dominance. Additionally, task type of grabbing was reported to be more realistic, and participants experienced a higher presence. Surprisingly, participants rated the interaction type with controllers where both where hands and controllers were visualized as statistically most preferred. Finally, hand tracking for both tasks was rated with the System Usability Scale (SUS) scale, and hand tracking for the task typing was rated as statistically significantly more usable. These results can drive further research and, in the long term, contribute to help selecting the most matching interaction modality for a task.
Diversity in Action: General-Sum Multi-Agent Continuous Inverse Optimal Control
Venue: arXiv cs.LG 2020
Year: 2020

Traffic scenarios are inherently interactive. Multiple decision-makers predict the actions of others and choose strategies that maximize their rewards. We view these interactions from the perspective of game theory which introduces various challenges. Humans are not entirely rational, their rewards need to be inferred from real-world data, and any prediction algorithm needs to be real-time capable so that we can use it in an autonomous vehicle (AV). In this work, we present a game-theoretic method that addresses all of the points above. Compared to many existing methods used for AVs, our approach does 1) not require perfect communication, and 2) allows for individual rewards per agent. Our experiments demonstrate that these more realistic assumptions lead to qualitatively and quantitatively different reward inference and prediction of future actions that match better with expected real-world behaviour.
A Conflict Detection Framework for IoT Services in Multi-resident Smart Homes
Venue: arXiv cs.CY 2020
Year: 2020

We propose a novel framework to detect conflicts among IoT services in a multi-resident smart home. A novel IoT conflict model is proposed considering the functional and non-functional properties of IoT services. We design a conflict ontology that formally represents different types of conflicts. A hybrid conflict detection algorithm is proposed by combining both knowledge-driven and data-driven approaches. Experimental results on real-world datasets show the efficiency of the proposed approach.
On indirect assessment of heart rate in video
Venue: arXiv cs.CV 2020
Year: 2020

Problem of indirect assessment of heart rate in video is addressed. Several methods of indirect evaluations (adaptive baselines) were examined on Remote Physiological Signal Sensing challenge. Particularly, regression models of dependency of heart rate on estimated age and motion intensity were obtained on challenge's train set. Accounting both motion and age in regression model led to top-quarter position in the leaderboard. Practical value of such adaptive baseline approaches is discussed. Although such approaches are considered as non-applicable in medicine, they are valuable as baseline for the photoplethysmography problem.
Unsupervised Domain Adaptation with Multiple Domain Discriminators and Adaptive Self-Training
Venue: arXiv cs.CV 2020
Year: 2020

Unsupervised Domain Adaptation (UDA) aims at improving the generalization capability of a model trained on a source domain to perform well on a target domain for which no labeled data is available. In this paper, we consider the semantic segmentation of urban scenes and we propose an approach to adapt a deep neural network trained on synthetic data to real scenes addressing the domain shift between the two different data distributions. We introduce a novel UDA framework where a standard supervised loss on labeled synthetic data is supported by an adversarial module and a self-training strategy aiming at aligning the two domain distributions. The adversarial module is driven by a couple of fully convolutional discriminators dealing with different domains: the first discriminates between ground truth and generated maps, while the second between segmentation maps coming from synthetic or real world data. The self-training module exploits the confidence estimated by the discriminators on unlabeled data to select the regions used to reinforce the learning process. Furthermore, the confidence is thresholded with an adaptive mechanism based on the per-class overall confidence. Experimental results prove the effectiveness of the proposed strategy in adapting a segmentation network trained on synthetic datasets like GTA5 and SYNTHIA, to real world datasets like Cityscapes and Mapillary.
Lifelong Learning Process: Self-Memory Supervising and Dynamically Growing Networks
Venue: arXiv cs.LG 2020
Year: 2020

From childhood to youth, human gradually come to know the world. But for neural networks, this growing process seems difficult. Trapped in catastrophic forgetting, current researchers feed data of all categories to a neural network which remains the same structure in the whole training process. We compare this training process with human learing patterns, and find two major conflicts. In this paper, we study how to solve these conflicts on generative models based on the conditional variational autoencoder(CVAE) model. To solve the uncontinuous conflict, we apply memory playback strategy to maintain the model's recognizing and generating ability on invisible used categories. And we extend the traditional one-way CVAE to a circulatory mode to better accomplish memory playback strategy. To solve the `dead' structure conflict, we rewrite the CVAE formula then are able to make a novel interpretation about the funtions of different parts in CVAE models. Based on the new understanding, we find ways to dynamically extend the network structure when training on new categories. We verify the effectiveness of our methods on MNIST and Fashion MNIST and display some very insteresting results.
A Skip-connected Multi-column Network for Isolated Handwritten Bangla Character and Digit recognition
Venue: arXiv cs.CV 2020
Year: 2020

Finding local invariant patterns in handwrit-ten characters and/or digits for optical character recognition is a difficult task. Variations in writing styles from one person to another make this task challenging. We have proposed a non-explicit feature extraction method using a multi-scale multi-column skip convolutional neural network in this work. Local and global features extracted from different layers of the proposed architecture are combined to derive the final feature descriptor encoding a character or digit image. Our method is evaluated on four publicly available datasets of isolated handwritten Bangla characters and digits. Exhaustive comparative analysis against contemporary methods establishes the efficacy of our proposed approach.
EventWarden: A Decentralized Event-driven Proxy Service for Outsourcing Arbitrary Transactions in Ethereum-like Blockchains
Venue: arXiv cs.CR 2020
Year: 2020

Transactions represent a fundamental component in blockchains as they are the primary means for users to change the blockchain state. Current blockchain systems such as Bitcoin and Ethereum require users to constantly observe the state changes of interest or the events taking place in a blockchain and requires the user to explicitly release the required transactions to respond to the observed events in the blockchain. This paper proposes EventWarden, a decentralized event-driven proxy service for users to outsource transactions in Ethereum-like blockchains. EventWarden employs a novel combination of smart contracts and blockchain logs. EventWarden allows a user to create a proxy smart contract that specifies an interested event and also reserves an arbitrary transaction to release. Upon observing the occurrence of the prescribed event, anyone in the Blockchain network can call the proxy contract to earn the service fee reserved in the contract by proving to the contract that the event has been recorded into blockchain logs, which then automatically triggers the proxy contract to release the reserved transaction. We show that the reserved transaction can only get released from the proxy contract when the prescribed event has taken place. We also demonstrate that as long as a single member in the Blockchain network is incentivized by the service fee to call the proxy contract after the prescribed event has taken place, the reserved transaction is guaranteed to get released. We implement EventWarden over the Ethereum official test network. The results demonstrate that EventWarden is effective and is ready-to-use in practice.
Why should we add early exits to neural networks?
Venue: arXiv cs.NE 2020
Year: 2020

Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including: (i) significant reductions of the inference time, (ii) reduced tendency to overfitting and vanishing gradients, and (iii) capability of being distributed over multi-tier computation platforms. In addition, they connect to the wider themes of biological plausibility and layered cognitive reasoning. In this paper, we provide a comprehensive introduction to this family of neural networks, by describing in a unified fashion the way these architectures can be designed, trained, and actually deployed in time-constrained scenarios. We also describe in-depth their application scenarios in 5G and Fog computing environments, as long as some of the open research questions connected to them.
LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
Venue: arXiv cs.CL 2020
Year: 2020

While pre-training and fine-tuning, e.g., BERT~\citep{devlin2018bert}, GPT-2~\citep{radford2019language}, have achieved great success in language understanding and generation tasks, the pre-trained models are usually too big for online deployment in terms of both memory cost and inference speed, which hinders them from practical online usage. In this paper, we propose LightPAFF, a Lightweight Pre-training And Fine-tuning Framework that leverages two-stage knowledge distillation to transfer knowledge from a big teacher model to a lightweight student model in both pre-training and fine-tuning stages. In this way the lightweight model can achieve similar accuracy as the big teacher model, but with much fewer parameters and thus faster online inference speed. LightPAFF can support different pre-training methods (such as BERT, GPT-2 and MASS~\citep{song2019mass}) and be applied to many downstream tasks. Experiments on three language understanding tasks, three language modeling tasks and three sequence to sequence generation tasks demonstrate that while achieving similar accuracy with the big BERT, GPT-2 and MASS models, LightPAFF reduces the model size by nearly 5x and improves online inference speed by 5x-7x.
The Streaming k-Mismatch Problem: Tradeoffs between Space and Total Time
Venue: arXiv cs.DS 2020
Year: 2020

We revisit the $k$-mismatch problem in the streaming model on a pattern of length $m$ and a streaming text of length $n$, both over a size-$σ$ alphabet. The current state-of-the-art algorithm for the streaming $k$-mismatch problem, by Clifford et al. [SODA 2019], uses $\tilde O(k)$ space and $\tilde O\big(\sqrt k\big)$ worst-case time per character. The space complexity is known to be (unconditionally) optimal, and the worst-case time per character matches a conditional lower bound. However, there is a gap between the total time cost of the algorithm, which is $\tilde O(n\sqrt k)$, and the fastest known offline algorithm, which costs $\tilde O\big(n + \min\big(\frac{nk}{\sqrt m},σn\big)\big)$ time. Moreover, it is not known whether improvements over the $\tilde O(n\sqrt k)$ total time are possible when using more than $O(k)$ space. We address these gaps by designing a randomized streaming algorithm for the $k$-mismatch problem that, given an integer parameter $k\le s \le m$, uses $\tilde O(s)$ space and costs $\tilde O\big(n+\min\big(\frac {nk^2}m,\frac{nk}{\sqrt s},\frac{σnm}s\big)\big)$ total time. For $s=m$, the total runtime becomes $\tilde O\big(n + \min\big(\frac{nk}{\sqrt m},σn\big)\big)$, which matches the time cost of the fastest offline algorithm. Moreover, the worst-case time cost per character is still $\tilde O\big(\sqrt k\big)$.