OpenRaiser/Intern-Atlas huggingface dataset:
An Empirical Study of the Characteristics of Popular Minecraft Mods
Venue: arXiv cs.SE 2021
Year: 2021

It is becoming increasingly difficult for game developers to manage the cost of developing a game, while meeting the high expectations of gamers. One way to balance the increasing gamer expectation and development stress is to build an active modding community around the game. There exist several examples of games with an extremely active and successful modding community, with the Minecraft game being one of the most notable ones. This paper reports on an empirical study of 1,114 popular and 1,114 unpopular Minecraft mods from the CurseForge mod distribution platform, one of the largest distribution platforms for Minecraft mods. We analyzed the relationship between 33 features across 5 dimensions of mod characteristics and the popularity of mods (i.e., mod category, mod documentation, environmental context of the mod, remuneration for the mod, and community contribution for the mod), to understand the characteristics of popular Minecraft mods. We firstly verify that the studied dimensions have significant explanatory power in distinguishing the popularity of the studied mods. Then we evaluated the contribution of each of the 33 features across the 5 dimensions. We observed that popular mods tend to have a high quality description and promote community contribution.
Visual Explanations from Spiking Neural Networks using Interspike Intervals
Venue: arXiv cs.CV 2021
Year: 2021

Spiking Neural Networks (SNNs) compute and communicate with asynchronous binary temporal events that can lead to significant energy savings with neuromorphic hardware. Recent algorithmic efforts on training SNNs have shown competitive performance on a variety of classification tasks. However, a visualization tool for analysing and explaining the internal spike behavior of such temporal deep SNNs has not been explored. In this paper, we propose a new concept of bio-plausible visualization for SNNs, called Spike Activation Map (SAM). The proposed SAM circumvents the non-differentiable characteristic of spiking neurons by eliminating the need for calculating gradients to obtain visual explanations. Instead, SAM calculates a temporal visualization map by forward propagating input spikes over different time-steps. SAM yields an attention map corresponding to each time-step of input data by highlighting neurons with short inter-spike interval activity. Interestingly, without both the backpropagation process and the class label, SAM highlights the discriminative region of the image while capturing fine-grained details. With SAM, for the first time, we provide a comprehensive analysis on how internal spikes work in various SNN training configurations depending on optimization types, leak behavior, as well as when faced with adversarial examples.
Incorporating Connections Beyond Knowledge Embeddings: A Plug-and-Play Module to Enhance Commonsense Reasoning in Machine Reading Comprehension
Venue: arXiv cs.CL 2021
Year: 2021

Conventional Machine Reading Comprehension (MRC) has been well-addressed by pattern matching, but the ability of commonsense reasoning remains a gap between humans and machines. Previous methods tackle this problem by enriching word representations via pre-trained Knowledge Graph Embeddings (KGE). However, they make limited use of a large number of connections between nodes in Knowledge Graphs (KG), which could be pivotal cues to build the commonsense reasoning chains. In this paper, we propose a Plug-and-play module to IncorporatE Connection information for commonsEnse Reasoning (PIECER). Beyond enriching word representations with knowledge embeddings, PIECER constructs a joint query-passage graph to explicitly guide commonsense reasoning by the knowledge-oriented connections between words. Further, PIECER has high generalizability since it can be plugged into suitable positions in any MRC model. Experimental results on ReCoRD, a large-scale public MRC dataset requiring commonsense reasoning, show that PIECER introduces stable performance improvements for four representative base MRC models, especially in low-resource settings.
RCT: Resource Constrained Training for Edge AI
Venue: arXiv cs.LG 2021
Year: 2021

Neural networks training on edge terminals is essential for edge AI computing, which needs to be adaptive to evolving environment. Quantised models can efficiently run on edge devices, but existing training methods for these compact models are designed to run on powerful servers with abundant memory and energy budget. For example, quantisation-aware training (QAT) method involves two copies of model parameters, which is usually beyond the capacity of on-chip memory in edge devices. Data movement between off-chip and on-chip memory is energy demanding as well. The resource requirements are trivial for powerful servers, but critical for edge devices. To mitigate these issues, We propose Resource Constrained Training (RCT). RCT only keeps a quantised model throughout the training, so that the memory requirements for model parameters in training is reduced. It adjusts per-layer bitwidth dynamically in order to save energy when a model can learn effectively with lower precision. We carry out experiments with representative models and tasks in image application and natural language processing. Experiments show that RCT saves more than 86\% energy for General Matrix Multiply (GEMM) and saves more than 46\% memory for model parameters, with limited accuracy loss. Comparing with QAT-based method, RCT saves about half of energy on moving model parameters.
An Overview of Collision Avoidance Approaches and Network Architecture of Unmanned Aerial Vehicles (UAVs)
Venue: arXiv cs.NI 2021
Year: 2021

As an autonomous vehicles, Unmanned Aerial Vehicles (UAVs) are subjected to several challenges. One of the challenges is for UAV to be able to avoid collision. Many collision avoidance methods have been proposed to address this issue. Furthermore, in a multi-UAV system, it is also important to address communication issue among UAVs for cooperation and collaboration. This issue can be addressed by setting up an ad-hoc network among UAVs. There is also a need to consider the challenges in the deployment of UAVs, as well as, in the development of collision avoidance methods and the establishment of communication for cooperation and collaboration in a multi-UAV system. In this paper, we present general challenges in the deployment of UAV and comparison of UAV communication services based on its operating frequency. We also present major collision avoidance approaches, and specifically discuss collision avoidance approaches that are suitable for indoor applications. We also present the Flying Ad-hoc Networks (FANET) network architecture, communication and routing protocols for each Open System Interconnection (OSI) communication layers.
GeoSP: A parallel method for a cortical surface parcellation based on geodesic distance
Venue: arXiv cs.CV 2021
Year: 2021

We present GeoSP, a parallel method that creates a parcellation of the cortical mesh based on a geodesic distance, in order to consider gyri and sulci topology. The method represents the mesh with a graph and performs a K-means clustering in parallel. It has two modes of use, by default, it performs the geodesic cortical parcellation based on the boundaries of the anatomical parcels provided by the Desikan-Killiany atlas. The other mode performs the complete parcellation of the cortex. Results for both modes and with different values for the total number of sub-parcels show homogeneous sub-parcels. Furthermore, the execution time is 82 s for the whole cortex mode and 18 s for the Desikan-Killiany atlas subdivision, for a parcellation into 350 sub-parcels. The proposed method will be available to the community to perform the evaluation of data-driven cortical parcellations. As an example, we compared GeoSP parcellation with Desikan-Killiany and Destrieux atlases in 50 subjects, obtaining more homogeneous parcels for GeoSP and minor differences in structural connectivity reproducibility across subjects.
YouTubing at Home: Media Sharing Behavior Change as Proxy for MobilityAround COVID-19 Lockdowns
Venue: arXiv cs.SI 2021
Year: 2021

Compliance with public health measures, such as restrictions on movement and socialization, is paramount in limiting the spread of diseases such as the severe acute respiratory syndrome coronavirus 2 (also referred to as COVID-19). Although large population datasets, such as phone-based mobility data, may provide some glimpse into such compliance, it is often proprietary, and may not be available for all locales. In this work, we examine the usefulness of video sharing on social media as a proxy of the amount of time Internet users spend at home. In particular, we focus on the number of people sharing YouTube videos on Twitter before and during COVID-19 lockdown measures were imposed by 109 countries. We find that the media sharing behavior differs widely between countries, in some having immediate response to the lockdown decrees - mostly by increasing the sharing volume dramatically - while in others having a substantial lag. We confirm that these insights correlate strongly with mobility, as measured using phone data. Finally, we illustrate that both media sharing and mobility behaviors change more drastically around mandated lockdowns, and less so around more lax recommendations. We make the media sharing volume data available to the research community for continued monitoring of behavior change around public health measures.
SKID RAW: Skill Discovery from Raw Trajectories
Venue: arXiv cs.LG 2021
Year: 2021

Integrating robots in complex everyday environments requires a multitude of problems to be solved. One crucial feature among those is to equip robots with a mechanism for teaching them a new task in an easy and natural way. When teaching tasks that involve sequences of different skills, with varying order and number of these skills, it is desirable to only demonstrate full task executions instead of all individual skills. For this purpose, we propose a novel approach that simultaneously learns to segment trajectories into reoccurring patterns and the skills to reconstruct these patterns from unlabelled demonstrations without further supervision. Moreover, the approach learns a skill conditioning that can be used to understand possible sequences of skills, a practical mechanism to be used in, for example, human-robot-interactions for a more intelligent and adaptive robot behaviour. The Bayesian and variational inference based approach is evaluated on synthetic and real human demonstrations with varying complexities and dimensionality, showing the successful learning of segmentations and skill libraries from unlabelled data.
Dodrio: Exploring Transformer Models with Interactive Visualization
Venue: arXiv cs.CL 2021
Year: 2021

Why do large pre-trained transformer-based models perform so well across a wide variety of NLP tasks? Recent research suggests the key may lie in multi-headed attention mechanism's ability to learn and represent linguistic information. Understanding how these models represent both syntactic and semantic knowledge is vital to investigate why they succeed and fail, what they have learned, and how they can improve. We present Dodrio, an open-source interactive visualization tool to help NLP researchers and practitioners analyze attention mechanisms in transformer-based models with linguistic knowledge. Dodrio tightly integrates an overview that summarizes the roles of different attention heads, and detailed views that help users compare attention weights with the syntactic structure and semantic information in the input text. To facilitate the visual comparison of attention weights and linguistic knowledge, Dodrio applies different graph visualization techniques to represent attention weights scalable to longer input text. Case studies highlight how Dodrio provides insights into understanding the attention mechanism in transformer-based models. Dodrio is available at https://poloclub.github.io/dodrio/.
One Algorithm to Evaluate Them All: Unified Linear Algebra Based Approach to Evaluate Both Regular and Context-Free Path Queries
Venue: arXiv cs.DB 2021
Year: 2021

The Kronecker product-based algorithm for context-free path querying (CFPQ) was proposed by Orachev et al. (2020). We reduce this algorithm to operations over Boolean matrices and extend it with the mechanism to extract all paths of interest. We also prove $O(n^3/\log{n})$ time complexity of the proposed algorithm, where n is a number of vertices of the input graph. Thus, we provide the alternative way to construct a slightly subcubic algorithm for CFPQ which is based on linear algebra and incremental transitive closure (a classic graph-theoretic problem), as opposed to the algorithm with the same complexity proposed by Chaudhuri (2008). Our evaluation shows that our algorithm is a good candidate to be the universal algorithm for both regular and context-free path querying.
Exploiting Adam-like Optimization Algorithms to Improve the Performance of Convolutional Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Stochastic gradient descent (SGD) is the main approach for training deep networks: it moves towards the optimum of the cost function by iteratively updating the parameters of a model in the direction of the gradient of the loss evaluated on a minibatch. Several variants of SGD have been proposed to make adaptive step sizes for each parameter (adaptive gradient) and take into account the previous updates (momentum). Among several alternative of SGD the most popular are AdaGrad, AdaDelta, RMSProp and Adam which scale coordinates of the gradient by square roots of some form of averaging of the squared coordinates in the past gradients and automatically adjust the learning rate on a parameter basis. In this work, we compare Adam based variants based on the difference between the present and the past gradients, the step size is adjusted for each parameter. We run several tests benchmarking proposed methods using medical image data. The experiments are performed using ResNet50 architecture neural network. Moreover, we have tested ensemble of networks and the fusion with ResNet50 trained with stochastic gradient descent. To combine the set of ResNet50 the simple sum rule has been applied. Proposed ensemble obtains very high performance, it obtains accuracy comparable or better than actual state of the art. To improve reproducibility and research efficiency the MATLAB source code used for this research is available at GitHub: https://github.com/LorisNanni.
Focused LRP: Explainable AI for Face Morphing Attack Detection
Venue: arXiv cs.CV 2021
Year: 2021

The task of detecting morphed face images has become highly relevant in recent years to ensure the security of automatic verification systems based on facial images, e.g. automated border control gates. Detection methods based on Deep Neural Networks (DNN) have been shown to be very suitable to this end. However, they do not provide transparency in the decision making and it is not clear how they distinguish between genuine and morphed face images. This is particularly relevant for systems intended to assist a human operator, who should be able to understand the reasoning. In this paper, we tackle this problem and present Focused Layer-wise Relevance Propagation (FLRP). This framework explains to a human inspector on a precise pixel level, which image regions are used by a Deep Neural Network to distinguish between a genuine and a morphed face image. Additionally, we propose another framework to objectively analyze the quality of our method and compare FLRP to other DNN interpretability methods. This evaluation framework is based on removing detected artifacts and analyzing the influence of these changes on the decision of the DNN. Especially, if the DNN is uncertain in its decision or even incorrect, FLRP performs much better in highlighting visible artifacts compared to other methods.
Cyclic Defense GAN Against Speech Adversarial Attacks
Venue: arXiv cs.SD 2021
Year: 2021

This paper proposes a new defense approach for counteracting state-of-the-art white and black-box adversarial attack algorithms. Our approach fits into the implicit reactive defense algorithm category since it does not directly manipulate the potentially malicious input signals. Instead, it reconstructs a similar signal with a synthesized spectrogram using a cyclic generative adversarial network. This cyclic framework helps to yield a stable generative model. Finally, we feed the reconstructed signal into the speech-to-text model for transcription. The conducted experiments on targeted and non-targeted adversarial attacks developed for attacking DeepSpeech, Kaldi, and Lingvo models demonstrate the proposed defense's effectiveness in adverse scenarios.
An empirical study into the relationship between class features and test smells
Venue: arXiv cs.SE 2021
Year: 2021

While a substantial body of prior research has investigated the form and nature of production code, comparatively little attention has examined characteristics of test code, and, in particular, test smells in that code. In this paper, we explore the relationship between production code properties (at the class level) and a set of test smells, in five open source systems. Specifically, we examine whether complexity properties of a production class can be used as predictors of the presence of test smells in the associated unit test. Our results, derived from the analysis of 975 production class-unit test pairs, show that the Cyclomatic Complexity (CC) and Weighted Methods per Class (WMC) of production classes are strong indicators of the presence of smells in their associated unit tests. The Lack of Cohesion of Methods in a production class (LCOM) also appears to be a good indicator of the presence of test smells. Perhaps more importantly, all three metrics appear to be good indicators of particular test smells, especially Eager Test and Duplicated Code. The Depth of the Inheritance Tree (DIT), on the other hand, was not found to be significantly related to the incidence of test smells. The results have important implications for large-scale software development, particularly in a context where organizations are increasingly using, adopting or adapting open source code as part of their development strategy and need to ensure that classes and methods are kept as simple as possible.
A Comprehensive Review of the Video-to-Text Problem
Venue: arXiv cs.CV 2021
Year: 2021

Research in the Vision and Language area encompasses challenging topics that seek to connect visual and textual information. When the visual information is related to videos, this takes us into Video-Text Research, which includes several challenging tasks such as video question answering, video summarization with natural language, and video-to-text and text-to-video conversion. This paper reviews the video-to-text problem, in which the goal is to associate an input video with its textual description. This association can be mainly made by retrieving the most relevant descriptions from a corpus or generating a new one given a context video. These two ways represent essential tasks for Computer Vision and Natural Language Processing communities, called text retrieval from video task and video captioning/description task. These two tasks are substantially more complex than predicting or retrieving a single sentence from an image. The spatiotemporal information present in videos introduces diversity and complexity regarding the visual content and the structure of associated language descriptions. This review categorizes and describes the state-of-the-art techniques for the video-to-text problem. It covers the main video-to-text methods and the ways to evaluate their performance. We analyze twenty-six benchmark datasets, showing their drawbacks and strengths for the problem requirements. We also show the progress that researchers have made on each dataset, we cover the challenges in the field, and we discuss future research directions.
Gender differences in research performance within and between countries: Italy vs Norway
Venue: arXiv cs.DL 2021
Year: 2021

In this study, the scientific performance of Italian and Norwegian university professors is analysed using bibliometric indicators. The study is based on over 36,000 individuals and their publication output during the period 2011-2015. Applying a multidimensional indicator in which several aspects of the research performance are captured, we find large differences in the performance of men and women. These gender differences are evident across all analysed levels, such as country, field, and academic position. However, most of the gender differences can be explained by the tails of the distributions-in particular, there is a much higher proportion of men among the top 10% performing scientists. For the remaining 90% of the population, the gender differences are practically non-existent. The results of the two countries, which differ in terms of the societal role of women, are contrasting. Further, we discuss possible biases that are intrinsic in quantitative performance indicators, which might disfavour female researchers.
A comparison of two approaches for measuring interdisciplinary research output: the disciplinary diversity of authors vs the disciplinary diversity of the reference list
Venue: arXiv cs.DL 2021
Year: 2021

This study investigates the convergence of two bibliometric approaches to the measurement of interdisciplinary research: one based on analyzing disciplinary diversity in the reference list of publications, the other based on the disciplinary diversity of authors of publications. In particular we measure the variety, balance, disparity and integrated diversity index of, respectively, single-author, multi-author single-field, and multi-author multi-field publications. We find that, in general, the diversity of the reference list grows with the number of fields reflected in a paper's authors' list and, to a lesser extent, with the number of authors being equal the number of fields. Further, we find that when fields belonging to different disciplines are reflected in the authors' list, the disparity in the reference list is higher than in the case of fields belonging to the same discipline. However, this general tendency varies across disciplines, and noticeable exceptions are found at individual paper level.
An Efficiently Coupled Shape and Appearance Prior for Active Contour Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes a novel training model based on shape and appearance features for object segmentation in images and videos. Whereas most such models rely on two-dimensional appearance templates or a finite set of descriptors, our appearance-based feature is a one-dimensional function, which is efficiently coupled with the object's shape by integrating intensities along the object's iso-contours. Joint PCA training on these shape and appearance features further exploits shape-appearance correlations and the resulting training model is incorporated in an active-contour-type energy functional for recognition-segmentation tasks. Experiments on synthetic and infrared images demonstrate how this shape and appearance training model improves accuracy compared to methods based on the Chan-Vese energy.