Reviewerly/exHarmony huggingface dataset:
Spatial-Aware Graph Relation Network for Large-Scale Object Detection

How to proper encode high-order object relation in the detection system without any external knowledge? How to leverage the information between co-occurrence and locations of objects for better reasoning? These questions are key challenges towards large-scale object detection system that aims to recognize thousands of objects entangled with complex spatial and semantic relationships nowadays. Distilling key relations that may affect object recognition is crucially important since treating each region separately leads to a big performance drop when facing heavy long-tail data distributions and plenty of confusing categories. Recent works try to encode relation by constructing graphs, e.g. using handcraft linguistic knowledge between classes or implicitly learning a fully-connected graph between regions. However, the handcraft linguistic knowledge cannot be individualized for each image due to the semantic gap between linguistic and visual context while the fully-connected graph is inefficient and noisy by incorporating redundant and distracted relations/edges from irrelevant objects and backgrounds. In this work, we introduce a Spatial-aware Graph Relation Network (SGRN) to adaptive discover and incorporate key semantic and spatial relationships for reasoning over each object. Our method considers the relative location layouts and interactions among which can be easily injected into any detection pipelines to boost the performance. Specifically, our SGRN integrates a graph learner module for learning a interpatable sparse graph structure to encode relevant contextual regions and a spatial graph reasoning module with learnable spatial Gaussian kernels to perform graph inference with spatial awareness. Extensive experiments verify the effectiveness of our method, e.g. achieving around 32% improvement on VG(3000 classes) and 28% on ADE in terms of mAP.
Two-Dimensional Sine Chaotification System With Hardware Implementation

Chaotic systems are widely employed in many practical applications for their significant properties. Existing chaotic systems may suffer from the drawbacks of discontinuous chaotic ranges and frail chaotic behaviors. To solve this issue, this paper proposes a two-dimensional (2D) sine chaotification system (2D-SCS). 2D-SCS can not only significantly enhance the complexity of 2D chaotic maps, but also greatly extend their chaotic ranges. As examples, this paper applies 2D-SCS to two existing 2D chaotic maps to obtain two enhanced chaotic maps. Performance evaluations show that these two enhanced chaotic maps have robust chaotic behaviors in much larger chaotic ranges than existing 2D chaotic maps. A microcontroller-based experiment platform is also designed to implement these enhanced chaotic maps in hardware devices. Furthermore, to investigate the application of 2D-SCS, these two enhanced chaotic maps are applied to design a pseudorandom number generator. Experiment results show that these enhanced chaotic maps can produce better random sequences than the existing 2D and several state-of-the-art one-dimensional (1D) chaotic maps.
Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation

In this paper, we present an extension to LaserNet, an efficient and state-of-the-art LiDAR based 3D object detector. We propose a method for fusing image data with the LiDAR data and show that this sensor fusion method improves the detection performance of the model especially at long ranges. The addition of image data is straightforward and does not require image labels. Furthermore, we expand the capabilities of the model to perform 3D semantic segmentation in addition to 3D object detection. On a large benchmark dataset, we demonstrate our approach achieves state-of-the-art performance on both object detection and semantic segmentation while maintaining a low runtime.
Neural Network-based Detection of Self-Admitted Technical Debt

Technical debt is a metaphor to reflect the tradeoff software engineers make between short-term benefits and long-term stability. Self-admitted technical debt (SATD), a variant of technical debt, has been proposed to identify debt that is intentionally introduced during software development, e.g., temporary fixes and workarounds. Previous studies have leveraged human-summarized patterns (which represent n-gram phrases that can be used to identify SATD) or text-mining techniques to detect SATD in source code comments. However, several characteristics of SATD features in code comments, such as vocabulary diversity, project uniqueness, length, and semantic variations, pose a big challenge to the accuracy of pattern or traditional text-mining-based SATD detection, especially for cross-project deployment. Furthermore, although traditional text-mining-based method outperforms pattern-based method in prediction accuracy, the text features it uses are less intuitive than human-summarized patterns, which makes the prediction results hard to explain. To improve the accuracy of SATD prediction, especially for cross-project prediction, we propose a Convolutional Neural Network-- (CNN) based approach for classifying code comments as SATD or non-SATD. To improve the explainability of our model’s prediction results, we exploit the computational structure of CNNs to identify key phrases and patterns in code comments that are most relevant to SATD. We have conducted an extensive set of experiments with 62,566 code comments from 10 open-source projects and a user study with 150 comments of another three projects. Our evaluation confirms the effectiveness of different aspects of our approach and its superior performance, generalizability, adaptability, and explainability over current state-of-the-art traditional text-mining-based methods for SATD classification.
Deep Reinforcement Learning for Sequence-to-Sequence Models

In recent times, sequence-to-sequence (seq2seq) models have gained a lot of popularity and provide state-of-the-art performance in a wide variety of tasks, such as machine translation, headline generation, text summarization, speech-to-text conversion, and image caption generation. The underlying framework for all these models is usually a deep neural network comprising an encoder and a decoder. Although simple encoder-decoder models produce competitive results, many researchers have proposed additional improvements over these seq2seq models, e.g., using an attention-based model over the input, pointer-generation models, and self-attention models. However, such seq2seq models suffer from two common problems: 1) exposure bias and 2) inconsistency between train/test measurement. Recently, a completely novel point of view has emerged in addressing these two problems in seq2seq models, leveraging methods from reinforcement learning (RL). In this survey, we consider seq2seq problems from the RL point of view and provide a formulation combining the power of RL methods in decision-making with seq2seq models that enable remembering long-term memories. We present some of the most recent frameworks that combine the concepts from RL and deep neural networks. Our work aims to provide insights into some of the problems that inherently arise with current approaches and how we can address them with better RL models. We also provide the source code for implementing most of the RL models discussed in this paper to support the complex task of abstractive text summarization and provide some targeted experiments for these RL models, both in terms of performance and training time.
A new direction in social network analysis: Online social network analysis problems and applications

The use of online social networks has made significant progress in recent years as the use of the Internet has become widespread worldwide as the technological infrastructure and the use of technological products evolve. It has become more suitable to reach online social networking sites such as Facebook, Twitter, Instagram and LinkedIn via the internet and web 3.0 technologies. Thus, people have shared their views on many different topics and their emotions with other users more widely on these platforms. This means that a huge amount of data is created on platforms where millions of people connect with each other through social networks. Nevertheless, the development of computational paradigms at high speed and complexity with technological possibilities allows analysis of valuable data by means of social network analysis methods. Our goal for this paper is to present a review of novel and popular online social network analysis problems with related applications and a reference work for researchers interested in analyzing online social network data and social network problems. Unlike other individual studies we have gathered 21 online social network problems and defined them with related studies. Thus, this study is original by presenting an important source of research by explaining the problems of online social network and the studies performed in this area.
HoloGAN: Unsupervised Learning of 3D Representations From Natural Images

We propose a novel generative adversarial network (GAN) for the task of unsupervised learning of 3D representations from natural images. Most generative models rely on 2D kernels to generate images and make few assumptions about the 3D world. These models therefore tend to create blurry images or artefacts in tasks that require a strong 3D understanding, such as novel-view synthesis. HoloGAN instead learns a 3D representation of the world, and to render this representation in a realistic manner. Unlike other GANs, HoloGAN provides explicit control over the pose of generated objects through rigid-body transformations of the learnt 3D features. Our experiments show that using explicit 3D features enables HoloGAN to disentangle 3D pose and identity, which is further decomposed into shape and appearance, while still being able to generate images with similar or higher visual quality than other generative models. HoloGAN can be trained end-to-end from unlabelled 2D images only. Particularly, we do not require pose labels, 3D shapes, or multiple views of the same objects. This shows that HoloGAN is the first generative model that learns 3D representations from natural images in an entirely unsupervised manner.
BreakHis based breast cancer automatic diagnosis using deep learning: Taxonomy, survey and insights

There are several breast cancer datasets for building Computer Aided Diagnosis systems (CADs) using either deep learning or traditional models. However, most of these datasets impose various trade-offs on practitioners related to their availability or inner clinical value. Recently, a public dataset called BreakHis has been released to overcome these limitations. BreakHis is organized into four magnification levels, each image is labeled according to its main category (Benign/Malignant) and its subcategory (A/F/PT/TA/PC/DC/LC/MC). This organization allows practitioners to address this problem either as a binary or a multi-category classification task with either a magnification dependent or independent training approach. In this work, we define a taxonomy that categorize this problem into four different reformulations: Magnification-Specific Binary (MSB), Magnification-Independent Binary (MIB), Magnification-Specific Multi-category (MSM) and Magnification-Independent Multi-category (MIM) classifications. We provide a comprehensive survey of all related works. We identify the best reformulation from clinical and practical standpoints. Finally, we explore for the first time the MIM approach using deep learning and draw the learnt lessons.
PHiSeg: Capturing Uncertainty in Medical Image Segmentation

Segmentation of anatomical structures and pathologies is inherently ambiguous. For instance, structure borders may not be clearly visible or different experts may have different styles of annotating. The majority of current state-of-the-art methods do not account for such ambiguities but rather learn a single mapping from image to segmentation. In this work, we propose a novel method to model the conditional probability distribution of the segmentations given an input image. We derive a hierarchical probabilistic model, in which separate latent variables are responsible for modelling the segmentation at different resolutions. Inference in this model can be efficiently performed using the variational autoencoder framework. We show that our proposed method can be used to generate significantly more realistic and diverse segmentation samples compared to recent related work, both, when trained with annotations from a single or multiple annotators. The code for this paper is freely available at https://github.com/baumgach/PHiSeg-code .
An improved image encryption algorithm with finite computing precision

When a digital chaotic system is realized on hardware of finite computing precision, it will lead to short period orbits. Although many existing image encryption algorithms declared that the average cycle lengths of their digital chaotic systems are larger than the required cycle lengths in image encryption, there are still many period orbits whose lengths are far smaller than the average cycle length. To further improve security with finite precision, an improved cryptosystem is proposed based on a new two-dimensional chaotic map derived from the Sine map, the Chebyshev map and a linear function (2D-SCL). Performance estimation demonstrates that the 2D-SCL has good ergodicity, hyperchaotic behavior, large cycle lengths under low computing precisions, and its complexity is larger and more stable than that of several newly developed 2D chaotic maps. Thus we design an improved cryptosystem based on the 2D-SCL map. In the scheme, we combine the confusion and diffusion processes in one stage to improve the running speed. Based on the SHA-1 hash values of plain image and the chaotic sequence, a pseudorandom sequence is designed and then an anti-degradation method is introduced to improve the dynamic degradation of the 2D-SCL map under finite computing precision. Meanwhile, this algorithm also updates the initial values of the 2D-SCL map in real-time, thus enhancing the ability to resist known-plaintext and chosen-plaintext attacks. The largest precision is set at 2−16, and simulation results show that this algorithm has high security, low time complexity, and the ability to withstand common attacks.
SROBB: Targeted Perceptual Loss for Single Image Super-Resolution

By benefiting from perceptual losses, recent studies have improved significantly the performance of the superresolution task, where a high-resolution image is resolved from its low-resolution counterpart. Although such objective functions generate near-photorealistic results, their capability is limited, since they estimate the reconstruction error for an entire image in the same way, without considering any semantic information. In this paper, we propose a novel method to benefit from perceptual loss in a more objective way. We optimize a deep network-based decoder with a targeted objective function that penalizes images at different semantic levels using the corresponding terms. In particular, the proposed method leverages our proposed OBB (Object, Background and Boundary) labels, generated from segmentation labels, to estimate a suitable perceptual loss for boundaries, while considering texture similarity for backgrounds. We show that our proposed approach results in more realistic textures and sharper edges, and outperforms other state-of-the-art algorithms in terms of both qualitative results on standard benchmarks and results of extensive user studies.
BiLSTM with Multi-Polarity Orthogonal Attention for Implicit Sentiment Analysis

Sentiment analysis has been a popular field in natural language processing. Sentiments can be expressed explicitly or implicitly. Most current studies on sentiment analysis focus on the identification of explicit sentiments. However, implicit sentiment analysis has become one of the most difficult tasks in sentiment analysis due to the absence of explicit sentiment words. In this article, we propose a BiLSTM model with multi-polarity orthogonal attention for implicit sentiment analysis. Compared to the traditional single attention model, the difference between the words and the sentiment orientation can be identified by using multi-polarity attention. This difference can be regarded as a significant feature for implicit sentiment analysis. Moreover, an orthogonal restriction mechanism is adopted to ensure that the discriminatory performance can be maintained during optimization. The experimental results on the SMP2019 implicit sentiment analysis dataset and two explicit sentiment analysis datasets demonstrate that our model more accurately captures the characteristic differences among sentiment polarities.
Self-adaptive resource allocation for cloud-based software services based on iterative QoS prediction model

Emerging cloud-based software services have proposed request for self-adaptive resource allocation that provides to dynamically adjust resources on demand. Traditional self-adaptive resource allocation methods are rule-driven that they combine expert knowledge with a separate set of management rules for each software, leading to high implementation complexity and administrative cost. Recent researches on self-adaptive resource allocation mainly focus on machine learning and control theory. However, machine learning techniques require plenty training data, which in many cases is insufficient and leads to low accuracy of QoS prediction model as well as the ineffectiveness of resource allocation, while resource allocation based on control theory needs a large number of iterations, resulting in a high overhead such as frequent virtual machine switches. This paper proposes a self-adaptive resource allocation method that is actually a framework composed of feedback loops, each of which goes through our designed iterative QoS prediction model and PSO-based runtime decision algorithm. In contrast to previous QoS prediction models which predict a QoS value once and for all, ours improves the predicted QoS value in iterations towards the best one. In the prediction, we first use the workload, the allocated resource, the real QoS value and an operation of resource allocation to produce a QoS value. Then we employ PSO-based runtime decision algorithm together with the predicted QoS value to determine future resource allocation operations. The loops iterate until the PSO-based algorithm suggests no further improvement over the current resource allocation. We evaluate our approach on RUBiS benchmark, illustrating that based on the same historical data, our method can achieve a better QoS prediction accuracy that is 15% higher than the current state of the art. Moreover, it is also shown the effectiveness of cloud application resource allocation is improved by roughly 5%–6%.
Sarcasm Detection Using Multi-Head Attention Based Bidirectional LSTM

Sarcasm is often used to express a negative opinion using positive or intensified positive words in social media. This intentional ambiguity makes sarcasm detection, an important task of sentiment analysis. Sarcasm detection is considered a binary classification problem wherein both feature-rich traditional models and deep learning models have been successfully built to predict sarcastic comments. In previous research works, models have been built using lexical, semantic and pragmatic features. We extract the most significant features and build a feature-rich SVM that outperforms these models. In this paper, we introduce a multi-head attention-based bidirectional long-short memory (MHA-BiLSTM) network to detect sarcastic comments in a given corpus. The experiment results reveal that a multi-head attention mechanism enhances the performance of BiLSTM, and it performs better than feature-rich SVM models.
Progress in Neural NLP: Modeling, Learning, and Reasoning

Natural language processing (NLP) is a subfield of artificial intelligence that focuses on enabling computers to understand and process human languages. In the last five years, we have witnessed the rapid development of NLP in tasks such as machine translation, question-answering, and machine reading comprehension based on deep learning and an enormous volume of annotated and unannotated data. In this paper, we will review the latest progress in the neural network-based NLP framework (neural NLP) from three perspectives: modeling, learning, and reasoning. In the modeling section, we will describe several fundamental neural network-based modeling paradigms, such as word embedding, sentence embedding, and sequence-to-sequence modeling, which are widely used in modern NLP engines. In the learning section, we will introduce widely used learning methods for NLP models, including supervised, semi-supervised, and unsupervised learning; multitask learning; transfer learning; and active learning. We view reasoning as a new and exciting direction for neural NLP, but it has yet to be well addressed. In the reasoning section, we will review reasoning mechanisms, including the knowledge, existing non-neural inference methods, and new neural inference methods. We emphasize the importance of reasoning in this paper because it is important for building interpretable and knowledge-driven neural NLP models to handle complex tasks. At the end of this paper, we will briefly outline our thoughts on the future directions of neural NLP.
FIESTA: Fast Incremental Euclidean Distance Fields for Online Motion Planning of Aerial Robots

Euclidean Signed Distance Field (ESDF) is useful for online motion planning of aerial robots since it can easily query the distance and gradient information against obstacles. Fast incrementally built ESDF map is the bottleneck for conducting real-time motion planning. In this paper, we investigate this problem and propose a mapping system called FIESTA to build global ESDF map incrementally. By introducing two independent updating queues for inserting and deleting obstacles separately, and using Indexing Data Structures and Doubly Linked Lists for map maintenance, our algorithm updates as few as possible nodes using a BFS framework. Our ESDF map has high computational performance and produces near-optimal results. We show our method outperforms other up-to-date methods in term of performance and accuracy by both theory and experiments. We integrate FIESTA into a completed quadrotor system and validate it by both simulation and onboard experiments. We release our method as open-source software for the community.