Reviewerly/exHarmony huggingface dataset:
MOXA: A Deep Learning Based Unmanned Approach For Real-Time Monitoring of People Wearing Medical Masks

With 6.93M confirmed cases of COVID-19 worldwide, making individuals aware of their sanitary health and ongoing pandemic remains the only way to prevent the spread of this virus. Wearing masks is an important step in this prevention. Hence, there is a need for monitoring if people are wearing masks or not. Closed circuit television (CCTV) cameras endowed with computer vision function by embedded systems, have become popular in a wide range of applications, and can be used in this case for real time monitoring of people wearing masks or not. In this paper, we propose to model this task of monitoring as a special case of object detection. However, real-time scene parsing through object detection running on edge devices is very challenging, due to limited memory and computing power of embedded devices. To deal with these challenges, we used a few popular object detection algorithms such as YOLOv3, YOLOv3Tiny, SSD and Faster R-CNN and evaluated them on Moxa3K benchmark dataset. The results obtained from these evaluations help us to determine methods that are more efficient, faster, and thus are more suitable for real-time object detection specialized for this task.
Bitcoin’s energy consumption is underestimated: A market dynamics approach

As the resource intensity of running Bitcoin has increased over recent years, it has become a serious concern for its potential impact on health and climate. Within this context, there exists a growing need for accurate information. Various organizations need this for multiple purposes like properly assessing the urgency of the problem, implementing the right policy response in the right locations and for setting up mitigation programs. We propose a market dynamics approach to evaluate the current methods for obtaining information on Bitcoin’s energy demand. This allows us to establish that, while historically the Bitcoin mining industry has been growing most of the time, this growth allows market participants to pursue strategies that don’t necessarily involve the best devices, device settings, or locations. The bigger the profitability of mining, the more it allows market participants to make decisions that result in suboptimal power efficiency of the Bitcoin network. Specifically, while the profitability of mining peaked during 2019, we find that market participants primarily used older generations of devices with better availability and lower acquisition costs. Common estimation approaches don’t only fail to capture this behavior, but also fail to properly capture the market circumstances, like seasonal and geographic variation in electricity prices, that help enable participants to do so in the first place. This combination leaves common approaches prone to providing optimistic estimates during growth cycles. We conservatively estimate the Bitcoin network to consume 87.1 TWh of electrical energy annually per September 30, 2019 (equaling a country like Belgium).
Color Image Encryption with Cipher Feedback and Coupling Chaotic Map

Based on the classic quadratic map (CQM) with abundant bifurcations and periodic windows, a new 3D improved coupling quadratic map (3D-ICQM) is constructed, and its phase diagram, Lyapunov exponent (LE) and randomness testing by TestU01 demonstrated that it has better ergodicity, more complex nonlinear behavior, larger chaotic range and better randomness. To investigate its application in cryptography, a 3D-ICQM based color image encryption algorithm with round key expansion is designed. To further enhance the randomness of key stream, the hash value of intermediate cipher of the previous round is converted into new initial values and fed back into 3D-ICQM, to generate the key stream for the current round. The security and statistical analyses demonstrated that the encryption scheme has better security performance than several existing algorithms.
Impact of anthropomorphic features of artificially intelligent service robots on consumer acceptance: moderating role of sense of humor

Purpose This study aims to investigate the impact of both physical and personality-related anthropomorphic features of an artificial intelligence service robot on the cognitive and affective appraisals and acceptance of consumers during service delivery. Design/methodology/approach The proposed hypotheses that investigate the effects of service robots’ physical appearance on the emphasis consumers place on each evaluation criteria they use in determining their willingness to accept the use of service robots in service delivery and the moderating role of sense of humor are tested by conducting two studies using scenario-based experiments. Findings The results show that humanlike appearance leads to higher performance expectancy, mascot-like appearance generates higher positive emotions and machine-like appearance results in higher effort expectancy. The effects of humanlike and mascot-like appearances on consumer acceptance are moderated by the sense of humor of service robots. However, the sense of humor effect is attenuated with a machine-like appearance owing to the lack of anthropomorphism. Practical implications This study provides crucial insights for hospitality managers who plan to use service robots in service delivery. The findings highlight the key roles of appearance type and sense of humor of service robots in influencing the appraisals and acceptance of consumers regarding the use of service robots in service delivery. Originality/value This study focuses on comparing the effects of traditional and mascot-like appearances of service robots on consumer appraisals and identifies sense of humor as a cute anthropomorphized personality trait of service robots.
Investigation of Enhanced Tacotron Text-to-speech Synthesis Systems with Self-attention for Pitch Accent Language

End-to-end speech synthesis is a promising approach that directly converts raw text to speech.Although it was shown that Tacotron2 outperforms classical pipeline systems with regards to naturalness in English, its applicability to other languages is still unknown.Japanese could be one of the most difficult languages for which to achieve end-to-end speech synthesis, largely due to its character diversity and pitch accents.Therefore, state-of-theart systems are still based on a traditional pipeline framework that requires a separate text analyzer and duration model.Towards endto-end Japanese speech synthesis, we extend Tacotron to systems with self-attention to capture long-term dependencies related to pitch accents and compare their audio quality with classical pipeline systems under various conditions to show their pros and cons.In a large-scale listening test, we investigated the impacts of the presence of accentual-type labels, the use of force or predicted alignments, and acoustic features used as local condition parameters of the Wavenet vocoder.Our results reveal that although the proposed systems still do not match the quality of a top-line pipeline system for Japanese, we show important stepping stones towards end-to-end Japanese speech synthesis.
Learning Optimized Structure of Neural Networks by Hidden Node Pruning With $L_{1}$ Regularization

We propose three different methods to determine the optimal number of hidden nodes based on L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularization for a multilayer perceptron network. The first two methods, respectively, use a set of multiplier functions and multipliers for the hidden-layer nodes and implement the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularization on those, while the third method equipped with the same multipliers uses a smoothing approximation of the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularization. Each of these methods begins with a given number of hidden nodes, then the network is trained to obtain an optimal architecture discarding redundant hidden nodes using the multiplier functions or multipliers. A simple and generic method, namely, the matrix-based convergence proving method (MCPM), is introduced to prove the weak and strong convergence of the presented smoothing algorithms. The performance of the three pruning methods has been tested on 11 different classification datasets. The results demonstrate the efficient pruning abilities and competitive generalization by the proposed methods. The theoretical results are also validated by the results.
On the Integration of Knowledge Graphs into Deep Learning Models for a More Comprehensible AI—Three Challenges for Future Research

Deep learning models contributed to reaching unprecedented results in prediction and classification tasks of Artificial Intelligence (AI) systems. However, alongside this notable progress, they do not provide human-understandable insights on how a specific result was achieved. In contexts where the impact of AI on human life is relevant (e.g., recruitment tools, medical diagnoses, etc.), explainability is not only a desirable property, but it is -or, in some cases, it will be soon-a legal requirement. Most of the available approaches to implement eXplainable Artificial Intelligence (XAI) focus on technical solutions usable only by experts able to manipulate the recursive mathematical functions in deep learning algorithms. A complementary approach is represented by symbolic AI, where symbols are elements of a lingua franca between humans and deep learning. In this context, Knowledge Graphs (KGs) and their underlying semantic technologies are the modern implementation of symbolic AI—while being less flexible and robust to noise compared to deep learning models, KGs are natively developed to be explainable. In this paper, we review the main XAI approaches existing in the literature, underlying their strengths and limitations, and we propose neural-symbolic integration as a cornerstone to design an AI which is closer to non-insiders comprehension. Within such a general direction, we identify three specific challenges for future research—knowledge matching, cross-disciplinary explanations and interactive explanations.
Automated screening of sickle cells using a smartphone-based microscope and deep learning

Sickle cell disease (SCD) is a major public health priority throughout much of the world, affecting millions of people. In many regions, particularly those in resource-limited settings, SCD is not consistently diagnosed. In Africa, where the majority of SCD patients reside, more than 50% of the 0.2-0.3 million children born with SCD each year will die from it; many of these deaths are in fact preventable with correct diagnosis and treatment. Here, we present a deep learning framework which can perform automatic screening of sickle cells in blood smears using a smartphone microscope. This framework uses two distinct, complementary deep neural networks. The first neural network enhances and standardizes the blood smear images captured by the smartphone microscope, spatially and spectrally matching the image quality of a laboratory-grade benchtop microscope. The second network acts on the output of the first image enhancement neural network and is used to perform the semantic segmentation between healthy and sickle cells within a blood smear. These segmented images are then used to rapidly determine the SCD diagnosis per patient. We blindly tested this mobile sickle cell detection method using blood smears from 96 unique patients (including 32 SCD patients) that were imaged by our smartphone microscope, and achieved ~98% accuracy, with an area-under-the-curve of 0.998. With its high accuracy, this mobile and cost-effective method has the potential to be used as a screening tool for SCD and other blood cell disorders in resource-limited settings.
Real-time Image Smoothing via Iterative Least Squares

Edge-preserving image smoothing is a fundamental procedure for many computer vision and graphic applications. There is a tradeoff between the smoothing quality and the processing speed: the high smoothing quality usually requires a high computational cost, which leads to the low processing speed. In this article, we propose a new global optimization based method, named iterative least squares (ILS), for efficient edge-preserving image smoothing. Our approach can produce high-quality results but at a much lower computational cost. Comprehensive experiments demonstrate that the proposed method can produce results with little visible artifacts. Moreover, the computation of ILS can be highly parallel, which can be easily accelerated through either multi-thread computing or the GPU hardware. With the acceleration of a GTX 1080 GPU, it is able to process images of 1080p resolution (1920 × 1080) at the rate of 20fps for color images and 47fps for gray images. In addition, the ILS is flexible and can be modified to handle more applications that require different smoothing properties. Experimental results of several applications show the effectiveness and efficiency of the proposed method. The code is available at https://github.com/wliusjtu/Real-time-Image-Smoothing-via-Iterative-Least-Squares.
An Overview and Comparison of Technical Debt Measurement Tools

Different tools adopt different terms, metrics, and ways to identify and measure technical debt. We attempt to clarify the situation by comparing the features and popularity of technical debt measurement tools and analyzing the existing empirical evidence on their validity.
SMIL: Multimodal Learning with Severely Missing Modality

A common assumption in multimodal learning is the completeness of training data, i.e., full modalities are available in all training examples. Although there exists research endeavor in developing novel methods to tackle the incompleteness of testing data, e.g., modalities are partially missing in testing examples, few of them can handle incomplete training modalities. The problem becomes even more challenging if considering the case of severely missing, e.g., ninety percent of training examples may have incomplete modalities. For the first time in the literature, this paper formally studies multimodal learning with missing modality in terms of flexibility (missing modalities in training, testing, or both) and efficiency (most training data have incomplete modality). Technically, we propose a new method named SMIL that leverages Bayesian meta-learning in uniformly achieving both objectives. To validate our idea, we conduct a series of experiments on three popular benchmarks: MM-IMDb, CMU-MOSI, and avMNIST. The results prove the state-of-the-art performance of SMIL over existing methods and generative baselines including autoencoders and generative adversarial networks.
Deep-Learning-Based Neural Network Training for State Estimation Enhancement: Application to Attitude Estimation

Achieving precise state estimation is needed for the unmanned aerial vehicle to perform a successful flight with a high degree of stability. Nonetheless, obtaining accurate state estimation is considered challenging due to the inaccuracies associated with the measurements of the onboard commercial-off-the-shelf inertial measurement unit. The immense vibration of the vehicle's rotors makes these measurements suffer from issues like large drifts, biases, and immense unpredictable noise sequences. These issues cannot be significantly tackled using classical estimators, and an accurate sensor fusion technique needs to be developed. In this paper, a deep learning (DL) framework is developed to enhance the performance of the state estimator. A deep neural network (DNN) is trained using a deep-learning-based technique to identify the associated measurement noise models and filter them out. The dropout technique is adopted for training DNN to avoid overfitting and reduce the complexity of nets computations. Compared to the classical estimation results, the proposed DL technique demonstrates capabilities in identifying the measurement's noise characteristics. As an example, an enhancement in estimating the attitude states at near hover is proven using this approach. Furthermore, an actual hover flight was performed to validate the proposed estimation enhancement method.
Augmenting Neural Networks with First-order Logic

Today, the dominant paradigm for training neural networks involves minimizing task loss on a large dataset. Using world knowledge to inform a model, and yet retain the ability to perform end-to-end training remains an open question. In this paper, we present a novel framework for introducing declarative knowledge to neural network architectures in order to guide training and prediction. Our framework systematically compiles logical statements into computation graphs that augment a neural network without extra learnable parameters or manual redesign. We evaluate our modeling strategy on three tasks: machine comprehension, natural language inference, and text chunking. Our experiments show that knowledge-augmented networks can strongly improve over baselines, especially in low-data regimes.
Learning Transferable Self-Attentive Representations for Action Recognition in Untrimmed Videos with Weak Supervision

Action recognition in videos has attracted a lot of attention in the past decade. In order to learn robust models, previous methods usually assume videos are trimmed as short sequences and require ground-truth annotations of each video frame/sequence, which is quite costly and time-consuming. In this paper, given only video-level annotations, we propose a novel weakly supervised framework to simultaneously locate action frames as well as recognize actions in untrimmed videos. Our proposed framework consists of two major components. First, for action frame localization, we take advantage of the self-attention mechanism to weight each frame, such that the influence of background frames can be effectively eliminated. Second, considering that there are trimmed videos publicly available and also they contain useful information to leverage, we present an additional module to transfer the knowledge from trimmed videos for improving the classification performance in untrimmed ones. Extensive experiments are conducted on two benchmark datasets (i.e., THUMOS14 and ActivityNet1.3), and experimental results clearly corroborate the efficacy of our method.
Semantic Product Search

We study the problem of semantic matching in product search, that is, given a customer query, retrieve all semantically related products from the catalog. Pure lexical matching via an inverted index falls short in this respect due to several factors: a) lack of understanding of hypernyms, synonyms, and antonyms, b) fragility to morphological variants (e.g. "woman" vs. "women"), and c) sensitivity to spelling errors. To address these issues, we train a deep learning model for semantic matching using customer behavior data. Much of the recent work on large-scale semantic search using deep learning focuses on ranking for web search. In contrast, semantic matching for product search presents several novel challenges, which we elucidate in this paper. We address these challenges by a) developing a new loss function that has an inbuilt threshold to differentiate between random negative examples, impressed but not purchased examples, and positive examples (purchased items), b) using average pooling in conjunction with n-grams to capture short-range linguistic patterns, c) using hashing to handle out of vocabulary tokens, and d) using a model parallel training architecture to scale across 8 GPUs. We present compelling offline results that demonstrate at least 4.7% improvement in [email protected] and 14.5% improvement in mean average precision (MAP) over baseline state-of-the-art semantic search methods using the same tokenization method. Moreover, we present results and discuss learnings from online A/B tests which demonstrate the efficacy of our method.
Adaptive Feature Recombination and Recalibration for Semantic Segmentation With Fully Convolutional Networks

Fully convolutional networks have been achieving remarkable results in image semantic segmentation, while being efficient. Such efficiency results from the capability of segmenting several voxels in a single forward pass. So, there is a direct spatial correspondence between a unit in a feature map and the voxel in the same location. In a convolutional layer, the kernel spans over all channels and extracts information from them. We observe that linear recombination of feature maps by increasing the number of channels followed by compression may enhance their discriminative power. Moreover, not all feature maps have the same relevance for the classes being predicted. In order to learn the inter-channel relationships and recalibrate the channels to suppress the less relevant ones, squeeze and excitation blocks were proposed in the context of image classification with convolutional neural networks. However, this is not well adapted for segmentation with fully convolutional networks since they segment several objects simultaneously, hence a feature map may contain relevant information only in some locations. In this paper, we propose recombination of features and a spatially adaptive recalibration block that is adapted for semantic segmentation with fully convolutional networks- the SegSE block. Feature maps are recalibrated by considering the cross-channel information together with spatial relevance. The experimental results indicate that recombination and recalibration improve the results of a competitive baseline, and generalize across three different problems: brain tumor segmentation, stroke penumbra estimation, and ischemic stroke lesion outcome prediction. The obtained results are competitive or outperform the state of the art in the three applications.
Flow2Vec: value-flow-based precise code embedding

Code embedding, as an emerging paradigm for source code analysis, has attracted much attention over the past few years. It aims to represent code semantics through distributed vector representations, which can be used to support a variety of program analysis tasks (e.g., code summarization and semantic labeling). However, existing code embedding approaches are intraprocedural, alias-unaware and ignoring the asymmetric transitivity of directed graphs abstracted from source code, thus they are still ineffective in preserving the structural information of code. This paper presents Flow2Vec, a new code embedding approach that precisely preserves interprocedural program dependence (a.k.a value-flows). By approximating the high-order proximity, i.e., the asymmetric transitivity of value-flows, Flow2Vec embeds control-flows and alias-aware data-flows of a program in a low-dimensional vector space. Our value-flow embedding is formulated as matrix multiplication to preserve context-sensitive transitivity through CFL reachability by filtering out infeasible value-flow paths. We have evaluated Flow2Vec using 32 popular open-source projects. Results from our experiments show that Flow2Vec successfully boosts the performance of two recent code embedding approaches codevec and codeseq for two client applications, i.e., code classification and code summarization. For code classification, Flow2Vec improves codevec with an average increase of 21.2%, 20.1% and 20.7% in precision, recall and F1, respectively. For code summarization, Flow2Vec outperforms codeseq by an average of 13.2%, 18.8% and 16.0% in precision, recall and F1, respectively.