OpenRaiser/Intern-Atlas huggingface dataset:
schuBERT: Optimizing Elements of BERT
Venue: arXiv cs.CL 2020
Year: 2020

Transformers \citep{vaswani2017attention} have gradually become a key component for many state-of-the-art natural language representation models. A recent Transformer based model- BERT \citep{devlin2018bert} achieved state-of-the-art results on various natural language processing tasks, including GLUE, SQuAD v1.1, and SQuAD v2.0. This model however is computationally prohibitive and has a huge number of parameters. In this work we revisit the architecture choices of BERT in efforts to obtain a lighter model. We focus on reducing the number of parameters yet our methods can be applied towards other objectives such FLOPs or latency. We show that much efficient light BERT models can be obtained by reducing algorithmically chosen correct architecture design dimensions rather than reducing the number of Transformer encoder layers. In particular, our schuBERT gives $6.6\%$ higher average accuracy on GLUE and SQuAD datasets as compared to BERT with three encoder layers while having the same number of parameters.
When Wireless Communication Faces COVID-19: Combating the Pandemic and Saving the Economy
Venue: arXiv cs.CY 2020
Year: 2020

The year 2020 is experiencing a global health and economic crisis due to the COVID-19 pandemic. Countries across the world are using digital technologies to fight this global crisis. These digital technologies, in one way or another, strongly rely on the availability of wireless communication technologies. In this paper, we present the role of wireless communications in the COVID-19 pandemic from different perspectives. First, we show how these technologies are helping to combat this pandemic, including monitoring of the virus spread, enabling healthcare automation, and allowing virtual education and conferencing. Also, we show the importance of digital inclusiveness in the pandemic and possible solutions to connect the unconnected. Next, we discuss the challenges faced by wireless technologies, including privacy, security, and misinformation. Then, we present the importance of wireless communication technologies in the survival of the global economy, such as automation of industries and supply chain, e-commerce, and supporting occupations that are at risk. Finally, we reveal that how the technologies developed during the pandemic can be helpful in the post-pandemic era.
Systematic Ensemble Model Selection Approach for Educational Data Mining
Venue: arXiv cs.CY 2020
Year: 2020

A plethora of research has been done in the past focusing on predicting student's performance in order to support their development. Many institutions are focused on improving the performance and the education quality; and this can be achieved by utilizing data mining techniques to analyze and predict students' performance and to determine possible factors that may affect their final marks. To address this issue, this work starts by thoroughly exploring and analyzing two different datasets at two separate stages of course delivery (20 percent and 50 percent respectively) using multiple graphical, statistical, and quantitative techniques. The feature analysis provides insights into the nature of the different features considered and helps in the choice of the machine learning algorithms and their parameters. Furthermore, this work proposes a systematic approach based on Gini index and p-value to select a suitable ensemble learner from a combination of six potential machine learning algorithms. Experimental results show that the proposed ensemble models achieve high accuracy and low false positive rate at all stages for both datasets.
The Information & Mutual Information Ratio for Counting Image Features and Their Matches
Venue: arXiv cs.CV 2020
Year: 2020

Feature extraction and description is an important topic of computer vision, as it is the starting point of a number of tasks such as image reconstruction, stitching, registration, and recognition among many others. In this paper, two new image features are proposed: the Information Ratio (IR) and the Mutual Information Ratio (MIR). The IR is a feature of a single image, while the MIR describes features common across two or more images.We begin by introducing the IR and the MIR and motivate these features in an information theoretical context as the ratio of the self-information of an intensity level over the information contained over the pixels of the same intensity. Notably, the relationship of the IR and MIR with the image entropy and mutual information, classic information measures, are discussed. Finally, the effectiveness of these features is tested through feature extraction over INRIA Copydays datasets and feature matching over the Oxfords Affine Covariant Regions. These numerical evaluations validate the relevance of the IR and MIR in practical computer vision tasks
How Reversibility Can Solve Traditional Questions: The Example of Hereditary History-Preserving Bisimulation
Venue: arXiv cs.DC 2020
Year: 2020

Reversible computation opens up the possibility of overcoming some of the hardware's current physical limitations. It also offers theoretical insights, as it enriches multiple paradigms and models of computation, and sometimes retrospectively enlightens them. Concurrent reversible computation, for instance, offered interesting extensions to the Calculus of Communicating Systems, but was still lacking a natural and pertinent bisimulation to study processes equivalences. Our paper formulates an equivalence exploiting the two aspects of reversibility: backward moves and memory mechanisms. This bisimulation captures classical equivalences relations for denotational models of concurrency (History-and hereditary history-preserving bisimulation, (H)HPB), that were up to now only partially characterized by process algebras. This result gives an insight on the expressiveness of reversibility, as both backward moves and a memory mechanism-providing 'backward determinism'-are needed to capture HHPB.
Street Marketing: How Proximity and Context drive Coupon Redemption
Venue: arXiv cs.CY 2020
Year: 2020

Purpose: In 2009, US coupons set a new record of 367 billion coupons distributed. Yet, while coupon distribution is on the rise, redemption rates remain below 1 percent. This paper aims to show how recognizing context variables, such as proximity, weather, part of town and financial incentives interplay to determine a coupon campaigns success. Design/methodology/approach. The paper reports an empirical study conducted in co-operation with a restaurant chain 9.880 Subway coupons were distributed under different experimental context conditions. Redemption behavior was analyzed with the help of logistic regressions. Findings: It was found that even though proximity drives coupon redemption, city center campaigns seem to be much more sensitive to distance than suburban areas. The further away the distribution place from the restaurant, the less does the amount of monetary incentive determine the motivation to redeem. Practical implications. When designing a coupon campaign for a company, coupon distribution should not follow a -- one is good for all strategy -- even for one marketer within one product category. Instead each coupon strategy should carefully consider contextual influence. Originality/value. This paper is the first to the authors knowledge that systematically investigates the impact of context variables on coupon redemption. It focuses on context variables that electronic marketing channels will be able to easily incorporate into personalized mobile marketing campaigns.
Quantifying the Impact of Making and Breaking Interface Habits
Venue: arXiv cs.HC 2020
Year: 2020

The frequency with which people interact with technology means that users may develop interface habits, i.e. fast, automatic responses to stable interface cues. Design guidelines often assume that interface habits are beneficial. However, we lack quantitative evidence of how the development of habits actually affect user performance and an understanding of how changes in the interface design may affect habit development. Our work quantifies the effect of habit formation and disruption on user performance in interaction. Through a forced choice lab study task (n=19) and in the wild deployment (n=18) of a notificationdialog experiment on smartphones, we show that people become more accurate and faster at option selection as they develop an interface habit. Crucially this performance gain is entirely eliminated once the habit is disrupted. We discuss reasons for this performance shift and analyse some disadvantages of interface habits, outlining general design patterns on how to both support and disrupt them.Keywords: Interface habits, user behaviour, breaking habit, interaction science, quantitative research.
Solve Traveling Salesman Problem by Monte Carlo Tree Search and Deep Neural Network
Venue: arXiv cs.LG 2020
Year: 2020

We present a self-learning approach that combines deep reinforcement learning and Monte Carlo tree search to solve the traveling salesman problem. The proposed approach has two advantages. First, it adopts deep reinforcement learning to compute the value functions for decision, which removes the need of hand-crafted features and labelled data. Second, it uses Monte Carlo tree search to select the best policy by comparing different value functions, which increases its generalization ability. Experimental results show that the proposed method performs favorably against other methods in small-to-medium problem settings. And it shows comparable performance as state-of-the-art in large problem setting.
Identifying Bugs in Make and JVM-Oriented Builds
Venue: arXiv cs.SE 2020
Year: 2020

Incremental and parallel builds are crucial features of modern build systems. Parallelism enables fast builds by running independent tasks simultaneously, while incrementality saves time and computing resources by processing the build operations that were affected by a particular code change. Writing build definitions that lead to error-free incremental and parallel builds is a challenging task. This is mainly because developers are often unable to predict the effects of build operations on the file system and how different build operations interact with each other. Faulty build scripts may seriously degrade the reliability of automated builds, as they cause build failures, and non-deterministic and incorrect build results. To reason about arbitrary build executions, we present buildfs, a generally-applicable model that takes into account the specification (as declared in build scripts) and the actual behavior (low-level file system operation) of build operations. We then formally define different types of faults related to incremental and parallel builds in terms of the conditions under which a file system operation violates the specification of a build operation. Our testing approach, which relies on the proposed model, analyzes the execution of single full build, translates it into buildfs, and uncovers faults by checking for corresponding violations. We evaluate the effectiveness, efficiency, and applicability of our approach by examining hundreds of Make and Gradle projects. Notably, our method is the first to handle Java-oriented build systems. The results indicate that our approach is (1) able to uncover several important issues (245 issues found in 45 open-source projects have been confirmed and fixed by the upstream developers), and (2) orders of magnitude faster than a state-of-the-art tool for Make builds.
ZynqNet: An FPGA-Accelerated Embedded Convolutional Neural Network
Venue: arXiv cs.CV 2020
Year: 2020

Image Understanding is becoming a vital feature in ever more applications ranging from medical diagnostics to autonomous vehicles. Many applications demand for embedded solutions that integrate into existing systems with tight real-time and power constraints. Convolutional Neural Networks (CNNs) presently achieve record-breaking accuracies in all image understanding benchmarks, but have a very high computational complexity. Embedded CNNs thus call for small and efficient, yet very powerful computing platforms. This master thesis explores the potential of FPGA-based CNN acceleration and demonstrates a fully functional proof-of-concept CNN implementation on a Zynq System-on-Chip. The ZynqNet Embedded CNN is designed for image classification on ImageNet and consists of ZynqNet CNN, an optimized and customized CNN topology, and the ZynqNet FPGA Accelerator, an FPGA-based architecture for its evaluation. ZynqNet CNN is a highly efficient CNN topology. Detailed analysis and optimization of prior topologies using the custom-designed Netscope CNN Analyzer have enabled a CNN with 84.5% top-5 accuracy at a computational complexity of only 530 million multiplyaccumulate operations. The topology is highly regular and consists exclusively of convolutional layers, ReLU nonlinearities and one global pooling layer. The CNN fits ideally onto the FPGA accelerator. The ZynqNet FPGA Accelerator allows an efficient evaluation of ZynqNet CNN. It accelerates the full network based on a nested-loop algorithm which minimizes the number of arithmetic operations and memory accesses. The FPGA accelerator has been synthesized using High-Level Synthesis for the Xilinx Zynq XC-7Z045, and reaches a clock frequency of 200MHz with a device utilization of 80% to 90 %.
Service mining for Internet of Things
Venue: arXiv cs.NI 2020
Year: 2020

A service mining framework is proposed that enables discovering interesting relationships in Internet of Things services bottom-up. The service relationships are modeled based on spatial-temporal aspects, environment, people, and operation. An ontology-based service model is proposed to describe services. We present a set of metrics to evaluate the interestingness of discovered service relationships. Analytical and simulation results are presented to show the effectiveness of the proposed evaluation measures.
Mitigating Gender Bias in Machine Learning Data Sets
Venue: arXiv cs.CL 2020
Year: 2020

Artificial Intelligence has the capacity to amplify and perpetuate societal biases and presents profound ethical implications for society. Gender bias has been identified in the context of employment advertising and recruitment tools, due to their reliance on underlying language processing and recommendation algorithms. Attempts to address such issues have involved testing learned associations, integrating concepts of fairness to machine learning and performing more rigorous analysis of training data. Mitigating bias when algorithms are trained on textual data is particularly challenging given the complex way gender ideology is embedded in language. This paper proposes a framework for the identification of gender bias in training data for machine learning.The work draws upon gender theory and sociolinguistics to systematically indicate levels of bias in textual training data and associated neural word embedding models, thus highlighting pathways for both removing bias from training data and critically assessing its impact.
Protecting the integrity of the training procedure of neural networks
Venue: arXiv cs.CR 2020
Year: 2020

Due to significant improvements in performance in recent years, neural networks are currently used for an ever-increasing number of applications. However, neural networks have the drawback that their decisions are not readily interpretable and traceable for a human. This creates several problems, for instance in terms of safety and IT security for high-risk applications, where assuring these properties is crucial. One of the most striking IT security problems aggravated by the opacity of neural networks is the possibility of so-called poisoning attacks during the training phase, where an attacker inserts specially crafted data to manipulate the resulting model. We propose an approach to this problem which allows provably verifying the integrity of the training procedure by making use of standard cryptographic mechanisms.
Multi-Component V2X Applications Placement in Edge Computing Environment
Venue: arXiv cs.NI 2020
Year: 2020

Vehicle-to-everything (V2X) services are attracting a lot of attention in the research and industry communities due to their applicability in the landscape of connected and autonomous vehicles. Such applications have stringent performance requirements in terms of complex data processing and low latency communications which are utilized to ensure road safety and improve road conditions. To address these challenges, the placement of V2X applications through leveraging of edge computing paradigm, that distributes the computing capabilities to access points in proximity to the vehicles, presents itself as a viable solution. However, the realistic implementation of the edge enabled V2X applications is hindered by the limited computational power provided at the edge and the nature of V2X applications that are composed of multiple independent V2X basic services. To address these challenges, this work targets the efficient placement of V2X basic services in a highway scenario subject to the delay constraints of V2X applications using them and the limited computational resources at the edge. To that end, this work formulates a binary integer linear programming model that minimizes the delay of V2X applications while satisfying the resource requirements of V2X basic services. To demonstrate the soundness of the approach, simulations with varying vehicle densities were conducted, and the results reported show that it can satisfy the delay requirements of V2X applications.
Big Computing: Where are we heading?
Venue: arXiv cs.DC 2020
Year: 2020

This paper presents the overview of the current trends of Big data against the computing scenario from different aspects. Some of the important aspect includes the Exascale, the computing power and the kind of applications which offer the Big data. This starts with the current computing hardware constraint against the need of the rising Big data applications. We highlight the issues and challenges of energy requirement, software complexity, hardware failure, fault tolerant computing, and communication. As the complexity of computation is going to rise in the future. The paper also highlights the future direction of Big computing systems for Bioinformatics, social media, hardware and software requirements, data intensive computation and then towards GPU era.
A network-based transfer learning approach to improve sales forecasting of new products
Venue: arXiv cs.LG 2020
Year: 2020

Data-driven methods -- such as machine learning and time series forecasting -- are widely used for sales forecasting in the food retail domain. However, for newly introduced products insufficient training data is available to train accurate models. In this case, human expert systems are implemented to improve prediction performance. Human experts rely on their implicit and explicit domain knowledge and transfer knowledge about historical sales of similar products to forecast new product sales. By applying the concept of Transfer Learning, we propose an analytical approach to transfer knowledge between listed stock products and new products. A network-based Transfer Learning approach for deep neural networks is designed to investigate the efficiency of Transfer Learning in the domain of food sales forecasting. Furthermore, we examine how knowledge can be shared across different products and how to identify the products most suitable for transfer. To test the proposed approach, we conduct a comprehensive case study for a newly introduced product, based on data of an Austrian food retailing company. The experimental results show, that the prediction accuracy of deep neural networks for food sales forecasting can be effectively increased using the proposed approach.
Comment on "Improved mutual information measure for clustering, classification, and community detection"
Venue: arXiv cs.SI 2020
Year: 2020

A recent article proposed reduced mutual information for evaluation of clustering, classification and community detection. The motivation is that the standard normalized mutual information (NMI) may give counter-intuitive answers under certain conditions and particularly when the number of clusters differs between the two divisions under consideration. The motivation makes sense. However, the examples given in the article are not accurate, and this comment discusses why. In addition, this comment also empirically demonstrates that the reduced mutual information cannot handle the difficulties of NMI and even brings more. The necessity of Kappa is also empirically validated in this comment.
Converting Anyone's Emotion: Towards Speaker-Independent Emotional Voice Conversion
Venue: arXiv cs.SD 2020
Year: 2020

Emotional voice conversion aims to convert the emotion of speech from one state to another while preserving the linguistic content and speaker identity. The prior studies on emotional voice conversion are mostly carried out under the assumption that emotion is speaker-dependent. We consider that there is a common code between speakers for emotional expression in a spoken language, therefore, a speaker-independent mapping between emotional states is possible. In this paper, we propose a speaker-independent emotional voice conversion framework, that can convert anyone's emotion without the need for parallel data. We propose a VAW-GAN based encoder-decoder structure to learn the spectrum and prosody mapping. We perform prosody conversion by using continuous wavelet transform (CWT) to model the temporal dependencies. We also investigate the use of F0 as an additional input to the decoder to improve emotion conversion performance. Experiments show that the proposed speaker-independent framework achieves competitive results for both seen and unseen speakers.
Anomaly Detection And Classification In Time Series With Kervolutional Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Recently, with the development of deep learning, end-to-end neural network architectures have been increasingly applied to condition monitoring signals. They have demonstrated superior performance for fault detection and classification, in particular using convolutional neural networks. Even more recently, an extension of the concept of convolution to the concept of kervolution has been proposed with some promising results in image classification tasks. In this paper, we explore the potential of kervolutional neural networks applied to time series data. We demonstrate that using a mixture of convolutional and kervolutional layers improves the model performance. The mixed model is first applied to a classification task in time series, as a benchmark dataset. Subsequently, the proposed mixed architecture is used to detect anomalies in time series data recorded by accelerometers on helicopters. We propose a residual-based anomaly detection approach using a temporal auto-encoder. We demonstrate that mixing kervolutional with convolutional layers in the encoder is more sensitive to variations in the input data and is able to detect anomalous time series in a better way.