OpenRaiser/Intern-Atlas huggingface dataset:
Symbiotic Hybrid Neural Network Watchdog For Outlier Detection
Venue: arXiv cs.LG 2021
Year: 2021

Neural networks are largely black boxes. A neural network trained to classify fruit may classify a picture of a giraffe as a banana. A neural network watchdog's job is to identify such inputs, allowing a classifier to disregard such data. We investigate whether the watchdog should be separate from the neural network or symbiotically attached. We present empirical evidence that the symbiotic watchdog performs better than when the neural networks are disjoint.
A Kinematic Bottleneck Approach For Pose Regression of Flexible Surgical Instruments directly from Images
Venue: arXiv cs.RO 2021
Year: 2021

3-D pose estimation of instruments is a crucial step towards automatic scene understanding in robotic minimally invasive surgery. Although robotic systems can potentially directly provide joint values, this information is not commonly exploited inside the operating room, due to its possible unreliability, limited access and the time-consuming calibration required, especially for continuum robots. For this reason, standard approaches for 3-D pose estimation involve the use of external tracking systems. Recently, image-based methods have emerged as promising, non-invasive alternatives. While many image-based approaches in the literature have shown accurate results, they generally require either a complex iterative optimization for each processed image, making them unsuitable for real-time applications, or a large number of manually-annotated images for efficient learning. In this paper we propose a self-supervised image-based method, exploiting, at training time only, the imprecise kinematic information provided by the robot. In order to avoid introducing time-consuming manual annotations, the problem is formulated as an auto-encoder, smartly bottlenecked by the presence of a physical model of the robotic instruments and surgical camera, forcing a separation between image background and kinematic content. Validation of the method was performed on semi-synthetic, phantom and in-vivo datasets, obtained using a flexible robotized endoscope, showing promising results for real-time image-based 3-D pose estimation of surgical instruments.
Detection of Malicious Android Applications: Classical Machine Learning vs. Deep Neural Network Integrated with Clustering
Venue: arXiv cs.CR 2021
Year: 2021

Today anti-malware community is facing challenges due to the ever-increasing sophistication and volume of malware attacks developed by adversaries. Traditional malware detection mechanisms are not able to cope-up with next-generation malware attacks. Therefore in this paper, we propose effective and efficient Android malware detection models based on machine learning and deep learning integrated with clustering. We performed a comprehensive study of different feature reduction, classification and clustering algorithms over various performance metrics to construct the Android malware detection models. Our experimental results show that malware detection models developed using Random Forest eclipsed deep neural network and other classifiers on the majority of performance metrics. The baseline Random Forest model without any feature reduction achieved the highest AUC of 99.4%. Also, the segregating of vector space using clustering integrated with Random Forest further boosted the AUC to 99.6% in one cluster and direct detection of Android malware in another cluster, thus reducing the curse of dimensionality. Additionally, we found that feature reduction in detection models does improve the model efficiency (training and testing time) many folds without much penalty on the effectiveness of the detection model.
Accelerating Recommendation System Training by Leveraging Popular Choices
Venue: arXiv cs.IR 2021
Year: 2021

Recommender models are commonly used to suggest relevant items to a user for e-commerce and online advertisement-based applications. These models use massive embedding tables to store numerical representation of items' and users' categorical variables (memory intensive) and employ neural networks (compute intensive) to generate final recommendations. Training these large-scale recommendation models is evolving to require increasing data and compute resources. The highly parallel neural networks portion of these models can benefit from GPU acceleration however, large embedding tables often cannot fit in the limited-capacity GPU device memory. Hence, this paper deep dives into the semantics of training data and obtains insights about the feature access, transfer, and usage patterns of these models. We observe that, due to the popularity of certain inputs, the accesses to the embeddings are highly skewed with a few embedding entries being accessed up to 10000x more. This paper leverages this asymmetrical access pattern to offer a framework, called FAE, and proposes a hot-embedding aware data layout for training recommender models. This layout utilizes the scarce GPU memory for storing the highly accessed embeddings, thus reduces the data transfers from CPU to GPU. At the same time, FAE engages the GPU to accelerate the executions of these hot embedding entries. Experiments on production-scale recommendation models with real datasets show that FAE reduces the overall training time by 2.3x and 1.52x in comparison to XDL CPU-only and XDL CPU-GPU execution while maintaining baseline accuracy
Active Reconfigurable Intelligent Surface Aided Wireless Communications
Venue: arXiv cs.IT 2021
Year: 2021

Reconfigurable Intelligent Surface (RIS) is a promising solution to reconfigure the wireless environment in a controllable way. To compensate for the double-fading attenuation in the RIS-aided link, a large number of passive reflecting elements (REs) are conventionally deployed at the RIS, resulting in large surface size and considerable circuit power consumption. In this paper, we propose a new type of RIS, called active RIS, where each RE is assisted by active loads (negative resistance), that reflect and amplify the incident signal instead of only reflecting it with the adjustable phase shift as in the case of a passive RIS. Therefore, for a given power budget at the RIS, a strengthened RIS-aided link can be achieved by increasing the number of active REs as well as amplifying the incident signal. We consider the use of an active RIS to a single input multiple output (SIMO) system. {However, it would unintentionally amplify the RIS-correlated noise, and thus the proposed system has to balance the conflict between the received signal power maximization and the RIS-correlated noise minimization at the receiver. To achieve this goal, it has to optimize the reflecting coefficient matrix at the RIS and the receive beamforming at the receiver.} An alternating optimization algorithm is proposed to solve the problem. Specifically, the receive beamforming is obtained with a closed-form solution based on linear minimum-mean-square-error (MMSE) criterion, while the reflecting coefficient matrix is obtained by solving a series of sequential convex approximation (SCA) problems. Simulation results show that the proposed active RIS-aided system could achieve better performance over the conventional passive RIS-aided system with the same power budget.
Towards Enhancing Database Education: Natural Language Generation Meets Query Execution Plans
Venue: arXiv cs.DB 2021
Year: 2021

The database systems course is offered as part of an undergraduate computer science degree program in many major universities. A key learning goal of learners taking such a course is to understand how SQL queries are processed in a RDBMS in practice. Since a query execution plan (QEP) describes the execution steps of a query, learners can acquire the understanding by perusing the QEPs generated by a RDBMS. Unfortunately, in practice, it is often daunting for a learner to comprehend these QEPs containing vendor-specific implementation details, hindering her learning process. In this paper, we present a novel, end-to-end, generic system called lantern that generates a natural language description of a qep to facilitate understanding of the query execution steps. It takes as input an SQL query and its QEP, and generates a natural language description of the execution strategy deployed by the underlying RDBMS. Specifically, it deploys a declarative framework called pool that enables subject matter experts to efficiently create and maintain natural language descriptions of physical operators used in QEPs. A rule-based framework called RULE-LANTERN is proposed that exploits pool to generate natural language descriptions of QEPs. Despite the high accuracy of RULE-LANTERN, our engagement with learners reveal that, consistent with existing psychology theories, perusing such rule-based descriptions lead to boredom due to repetitive statements across different QEPs. To address this issue, we present a novel deep learning-based language generation framework called NEURAL-LANTERN that infuses language variability in the generated description by exploiting a set of paraphrasing tools and word embedding. Our experimental study with real learners shows the effectiveness of lantern in facilitating comprehension of QEPs.
Long Document Summarization in a Low Resource Setting using Pretrained Language Models
Venue: arXiv cs.CL 2021
Year: 2021

Abstractive summarization is the task of compressing a long document into a coherent short document while retaining salient information. Modern abstractive summarization methods are based on deep neural networks which often require large training datasets. Since collecting summarization datasets is an expensive and time-consuming task, practical industrial settings are usually low-resource. In this paper, we study a challenging low-resource setting of summarizing long legal briefs with an average source document length of 4268 words and only 120 available (document, summary) pairs. To account for data scarcity, we used a modern pretrained abstractive summarizer BART (Lewis et al., 2020), which only achieves 17.9 ROUGE-L as it struggles with long documents. We thus attempt to compress these long documents by identifying salient sentences in the source which best ground the summary, using a novel algorithm based on GPT-2 (Radford et al., 2019) language model perplexity scores, that operates within the low resource regime. On feeding the compressed documents to BART, we observe a 6.0 ROUGE-L improvement. Our method also beats several competitive salience detection baselines. Furthermore, the identified salient sentences tend to agree with an independent human labeling by domain experts.
Mitigating Edge Machine Learning Inference Bottlenecks: An Empirical Study on Accelerating Google Edge Models
Venue: arXiv cs.AR 2021
Year: 2021

As the need for edge computing grows, many modern consumer devices now contain edge machine learning (ML) accelerators that can compute a wide range of neural network (NN) models while still fitting within tight resource constraints. We analyze a commercial Edge TPU using 24 Google edge NN models (including CNNs, LSTMs, transducers, and RCNNs), and find that the accelerator suffers from three shortcomings, in terms of computational throughput, energy efficiency, and memory access handling. We comprehensively study the characteristics of each NN layer in all of the Google edge models, and find that these shortcomings arise from the one-size-fits-all approach of the accelerator, as there is a high amount of heterogeneity in key layer characteristics both across different models and across different layers in the same model. We propose a new acceleration framework called Mensa. Mensa incorporates multiple heterogeneous ML edge accelerators (including both on-chip and near-data accelerators), each of which caters to the characteristics of a particular subset of models. At runtime, Mensa schedules each layer to run on the best-suited accelerator, accounting for both efficiency and inter-layer dependencies. As we analyze the Google edge NN models, we discover that all of the layers naturally group into a small number of clusters, which allows us to design an efficient implementation of Mensa for these models with only three specialized accelerators. Averaged across all 24 Google edge models, Mensa improves energy efficiency and throughput by 3.0x and 3.1x over the Edge TPU, and by 2.4x and 4.3x over Eyeriss v2, a state-of-the-art accelerator.
Self-Supervised Multi-View Learning via Auto-Encoding 3D Transformations
Venue: arXiv cs.CV 2021
Year: 2021

3D object representation learning is a fundamental challenge in computer vision to infer about the 3D world. Recent advances in deep learning have shown their efficiency in 3D object recognition, among which view-based methods have performed best so far. However, feature learning of multiple views in existing methods is mostly performed in a supervised fashion, which often requires a large amount of data labels with high costs. In contrast, self-supervised learning aims to learn multi-view feature representations without involving labeled data. To this end, we propose a novel self-supervised paradigm to learn Multi-View Transformation Equivariant Representations (MV-TER), exploring the equivariant transformations of a 3D object and its projected multiple views. Specifically, we perform a 3D transformation on a 3D object, and obtain multiple views before and after the transformation via projection. Then, we self-train a representation to capture the intrinsic 3D object representation by decoding 3D transformation parameters from the fused feature representations of multiple views before and after the transformation. Experimental results demonstrate that the proposed MV-TER significantly outperforms the state-of-the-art view-based approaches in 3D object classification and retrieval tasks, and show the generalization to real-world datasets.
Embedded Knowledge Distillation in Depth-Level Dynamic Neural Network
Venue: arXiv cs.CV 2021
Year: 2021

In real applications, different computation-resource devices need different-depth networks (e.g., ResNet-18/34/50) with high-accuracy. Usually, existing methods either design multiple networks and train them independently, or construct depth-level/width-level dynamic neural networks which is hard to prove the accuracy of each sub-net. In this article, we propose an elegant Depth-Level Dynamic Neural Network (DDNN) integrated different-depth sub-nets of similar architectures. To improve the generalization of sub-nets, we design the Embedded-Knowledge-Distillation (EKD) training mechanism for the DDNN to implement knowledge transfer from the teacher (full-net) to multiple students (sub-nets). Specifically, the Kullback-Leibler (KL) divergence is introduced to constrain the posterior class probability consistency between full-net and sub-nets, and self-attention distillation on the same resolution feature of different depth is addressed to drive more abundant feature representations of sub-nets. Thus, we can obtain multiple high-accuracy sub-nets simultaneously in a DDNN via the online knowledge distillation in each training iteration without extra computation cost. Extensive experiments on CIFAR-10/100, and ImageNet datasets demonstrate that sub-nets in DDNN with EKD training achieve better performance than individually training networks while preserving the original performance of full-nets.
Emotion pattern detection on facial videos using functional statistics
Venue: arXiv cs.CV 2021
Year: 2021

There is an increasing scientific interest in automatically analysing and understanding human behavior, with particular reference to the evolution of facial expressions and the recognition of the corresponding emotions. In this paper we propose a technique based on Functional ANOVA to extract significant patterns of face muscles movements, in order to identify the emotions expressed by actors in recorded videos. We determine if there are time-related differences on expressions among emotional groups by using a functional F-test. Such results are the first step towards the construction of a reliable automatic emotion recognition system
A Brief Summary of Interactions Between Meta-Learning and Self-Supervised Learning
Venue: arXiv cs.LG 2021
Year: 2021

This paper briefly reviews the connections between meta-learning and self-supervised learning. Meta-learning can be applied to improve model generalization capability and to construct general AI algorithms. Self-supervised learning utilizes self-supervision from original data and extracts higher-level generalizable features through unsupervised pre-training or optimization of contrastive loss objectives. In self-supervised learning, data augmentation techniques are widely applied and data labels are not required since pseudo labels can be estimated from trained models on similar tasks. Meta-learning aims to adapt trained deep models to solve diverse tasks and to develop general AI algorithms. We review the associations of meta-learning with both generative and contrastive self-supervised learning models. Unlabeled data from multiple sources can be jointly considered even when data sources are vastly different. We show that an integration of meta-learning and self-supervised learning models can best contribute to the improvement of model generalization capability. Self-supervised learning guided by meta-learner and general meta-learning algorithms under self-supervision are both examples of possible combinations.
FineNet: Frame Interpolation and Enhancement for Face Video Deblurring
Venue: arXiv cs.CV 2021
Year: 2021

The objective of this work is to deblur face videos. We propose a method that tackles this problem from two directions: (1) enhancing the blurry frames, and (2) treating the blurry frames as missing values and estimate them by interpolation. These approaches are complementary to each other, and their combination outperforms individual ones. We also introduce a novel module that leverages the structure of faces for finding positional offsets between video frames. This module can be integrated into the processing pipelines of both approaches, improving the quality of the final outcome. Experiments on three real and synthetically generated blurry video datasets show that our method outperforms the previous state-of-the-art methods by a large margin in terms of both quantitative and qualitative results.
Understanding Emotions of Developer Community Towards Software Documentation
Venue: arXiv cs.SE 2021
Year: 2021

The availability of open-source projects facilitates developers to contribute and collaborate on a wide range of projects. As a result, the developer community contributing to such open-source projects is also increasing. Many of the projects involve frequent updates and extensive reuses. A well-updated documentation helps in a better understanding of the software project and also facilitates efficient contribution and reuse. Though software documentation plays an important role in the development and maintenance of software, it also suffers from various issues that include insufficiency, inconsistency, ill-maintainability, and so on. Exploring the perception of developers towards documentation could help in understanding the reasons behind prevalent issues in software documentation. It could further aid in deciding on training that could be given to the developer community towards building more sustainable projects for society. Analyzing sentiments of contributors to a project could provide insights on understanding developer perceptions. Hence, as the first step towards this direction, we analyze sentiments of commit messages specific to the documentation of a software project. To this end, we considered the commit history of 998 GitHub projects from the GHTorrent dataset and identified 10,996 commits that correspond to the documentation of repositories. Further, we apply sentiment analysis techniques to obtain insights on the type of sentiment being expressed in commit messages of the selected commits. We observe that around 45% of the identified commit messages express trust emotion.
An open-source framework for ExpFinder integrating $N$-gram Vector Space Model and $μ$CO-HITS
Venue: arXiv cs.IR 2021
Year: 2021

Finding experts drives successful collaborations and high-quality product development in academic and research domains. To contribute to the expert finding research community, we have developed ExpFinder which is a novel ensemble model for expert finding by integrating an $N$-gram vector space model ($n$VSM) and a graph-based model ($μ$CO-HITS). This paper provides descriptions of ExpFinder's architecture, key components, functionalities, and illustrative examples. ExpFinder is an effective and competitive model for expert finding, significantly outperforming a number of expert finding models.
Directing Words of Fuzzy Finite Automata
Venue: arXiv cs.FL 2021
Year: 2021

A deterministic finite automaton is directable if it has a directing word which takes the automaton from every state to the same state. These notions have been extended also to other kinds of automata. Thus, B.~Imreh and M.~Steinby (1999) identified three natural types of directing words, called D1-, D2- and D3-directing words, for nondeterministic finite automata (NFAs). Here we adapt these notions for fuzzy finite automata (FFAs). The D3-directing words obtained this way are precisely the directing words introduced by V.~Karthikeyan and M. Rajasekar (2015). With any FFA F we associate an NFA Fnd which has the same Di-directing words as F. Thus, if these definitions are used, the theory of directable FFAs reduces to that of NFAs. We also introduce three new kinds of directing words of fuzzy automata that we call DD1-, DD2- and DD3-directing words, respectively which depend more on the fuzzy transition degrees between states. We establish some basic properties of the sets $DDi(F)$ of DDi-directing words of any given FFA F. In particular, it is shown that these languages are regular, and that DDi-directability is decidable. For so-called normal FFAs the languages DDi(F) are shown to have some special properties. Several relationships between the families of the corresponding sets DDi(F) of DDi-directing words are presented. We also determine the complete meet-semilattice of the various classes of DDi-directable FFAs and normal FFAs and their intersections.
Automatic Stockpile Volume Monitoring using Multi-view Stereo from SkySat Imagery
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes a system for automatic surface volume monitoring from time series of SkySat pushframe imagery. A specific challenge of building and comparing large 3D models from SkySat data is to correct inconsistencies between the camera models associated to the multiple views that are necessary to cover the area at a given time, where these camera models are represented as Rational Polynomial Cameras (RPCs). We address the problem by proposing a date-wise RPC refinement, able to handle dynamic areas covered by sets of partially overlapping views. The cameras are refined by means of a rotation that compensates for errors due to inaccurate knowledge of the satellite attitude. The refined RPCs are then used to reconstruct multiple consistent Digital Surface Models (DSMs) from different stereo pairs at each date. RPC refinement strengthens the consistency between the DSMs of each date, which is extremely beneficial to accurately measure volumes in the 3D surface models. The system is tested in a real case scenario, to monitor large coal stockpiles. Our volume estimates are validated with measurements collected on site in the same period of time.