Reviewerly/exHarmony huggingface dataset:
Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems

Training an end-to-end (E2E) neural network speech-to-intent (S2I) system that directly extracts intents from speech requires large amounts of intent-labeled speech data, which is time consuming and expensive to collect. Initializing the S2I model with an ASR model trained on copious speech data can alleviate data sparsity. In this paper, we attempt to leverage NLU text resources. We implemented a CTC-based S2I system that matches the performance of a state-of-the-art, traditional cascaded SLU system. We performed controlled experiments with varying amounts of speech and text training data. When only a tenth of the original data is available, intent classification accuracy degrades by 7.6% absolute. Assuming we have additional text-to-intent data (without speech) available, we investigated two techniques to improve the S2I system: (1) transfer learning, in which acoustic embeddings for intent classification are tied to fine-tuned BERT text embeddings; and (2) data augmentation, in which the text-to-intent data is converted into speech-to-intent data using a multi-speaker text-to-speech system. The proposed approaches recover 80% of performance lost due to using limited intent-labeled speech.
Wearable Sensors for Activity Analysis using SMO-based Random Forest over Smart home and Sports Datasets

Human activity recognition using MotionNode sensors is getting prominence effect in our daily life logs. Providing accurate information on human's activities and behaviors is one of the most challenging tasks in ubiquitous computing and human-Computer interaction. In this paper, we proposed an efficient model for having statistical features along SMO-based random forest. Initially, we processed a 1-D Hadamard transform wavelet and 1-D LBP based extraction algorithm to extract valuable features. For activity classification, we used sequential minimal optimization along with Random Forest over two benchmarks USC-HAD dataset and IMSB datasets. Experimental results show that our proposed model can compete with other state-of-the-art methods and can be effectively used to recognize robust human activities in terms of efficiency and accuracy.
Teacher-Student Training For Robust Tacotron-Based TTS

While neural end-to-end text-to-speech (TTS) is superior to conventional statistical methods in many ways, the exposure bias problem in the autoregressive models remains an issue to be resolved. The exposure bias problem arises from the mismatch between the training and inference process, that results in unpredictable performance for out-of-domain test data at run-time. To overcome this, we propose a teacher-student training scheme for Tacotron-based TTS by introducing a distillation loss function in addition to the feature loss function. We first train a Tacotron2-based TTS model by always providing natural speech frames to the decoder, that serves as a teacher model. We then train another Tacotron2-based model as a student model, of which the decoder takes the predicted speech frames as input, similar to how the decoder works during run-time inference. With the distillation loss, the student model learns the output probabilities from the teacher model, that is called knowledge distillation. Experiments show that our proposed training scheme consistently improves the voice quality for out-of-domain test data both in Chinese and English systems.
Cross-Database Micro-Expression Recognition: A Benchmark

Cross-database micro-expression recognition (CDMER) is one of recently emerging and interesting problem in micro-expression analysis. CDMER is more challenging than the conventional micro-expression recognition (MER), because the training and testing samples in CDMER come from different micro-expression databases, resulting in inconsistency of the feature distributions between the training and testing sets. In this paper, we contribute to this topic from three aspects. First, we establish a CDMER experimental evaluation protocol aiming to allow the researchers to conveniently work on this topic and evaluate their proposed methods under the same standard. Second, we conduct benchmark experiments by using NINE state-of-the-art domain adaptation (DA) methods and SIX popular spatiotemporal descriptors for investigating CDMER problem from two different perspectives. Third, we propose a novel DA method called region selective transfer regression (RSTR) to deal with the CDMER task. The overall superior performance of RSTR over the state-of-the-art DA methods demonstrates that taking into consideration the facial local region information used in RSTR contributes to developing effective DA methods for dealing with CDMER problem.
Trustworthiness of Self-Driving Vehicles for Intelligent Transportation Systems in Industry Applications

To enhance industrial production and automation, rapid and faster transportation of raw materials and finished products to and from distributed factories, warehouses and outlets are essential. To reduce cost with increased efficiency, this will increasingly see the use of connected and self-driving commercial vehicles fitted with industrial grade sensors on roads, shared with normal and self-driving passenger vehicles. For its wide adoption, the trustworthiness of self-driving vehicles in the intelligent transportation system (ITS) is pivotal. In this article, we introduce a novel model to measure the overall trustworthiness of a self-driving vehicle considering on-Board unit (OBU) components, GPS data and safety messages. In calculating the trustworthiness of individual OBU components, CertainLogic and beta distribution function (BDF) are used. Those trust values are fused using both the dempster-Shafer Theory (DST) and a logical operator of CertainLogic. Results of our simulation show that our proposed method can effectively determine the trust of self-driving vehicles.
Blockchain Ethereum Clients Performance Analysis Considering E-Voting Application

Blockchain technology is evolving and revolutionizing the IT industry with better security, efficiency, and resilience. Blockchain technology is being used in many applications majorly in cryptocurrencies and bitcoin applications. Verified transactions which make a block and group of such transactions or blocks are immutable making the blockchain more secured and reliable. Blockchain achieves decentralization of power, trust, and secured of being hacked, which solves major problems or issues with the current systems. Ethereum, the most widely used blockchain platform because of its unlimited block size. Many complex problems with smart contracts can be implemented with Ethereum and the eradication of third party organizations interfering in transactions helps solving the issues of financial crisis and it is easy to implement compared to other blockchain technologies. There are certain limitations/issues in processing large number of transactions due to lack of speed in processing the transactions. Ethereum Blockchain code will be executed by different clients with varying speed and the performance level will be different. The goal of this paper is to understand Ethereum transactions and perform the comparative analysis of Geth and Parity ethereum clients on the private blockchain. In this paper, a private blockchain network is setup where the nodes will share the data among peer nodes or blocks within the network. Using this network setup a democracy voting application is developed which makes use of the blockchain to store and process the data, smart contracts are deployed to execute the transactions. Performance analysis of the two most popular ethereum clients Geth and Parity is carried out considering time, consistency and scalability parameters. Results interpret that the overall transactions are 91% on average faster in parity client as compared to Geth client.
Source Code Assessment and Classification Based on Estimated Error Probability Using Attentive LSTM Language Model and Its Application in Programming Education

The rate of software development has increased dramatically. Conventional compilers cannot assess and detect all source code errors. Software may thus contain errors, negatively affecting end-users. It is also difficult to assess and detect source code logic errors using traditional compilers, resulting in software that contains errors. A method that utilizes artificial intelligence for assessing and detecting errors and classifying source code as correct (error-free) or incorrect is thus required. Here, we propose a sequential language model that uses an attention-mechanism-based long short-term memory (LSTM) neural network to assess and classify source code based on the estimated error probability. The attentive mechanism enhances the accuracy of the proposed language model for error assessment and classification. We trained the proposed model using correct source code and then evaluated its performance. The experimental results show that the proposed model has logic and syntax error detection accuracies of 92.2% and 94.8%, respectively, outperforming state-of-the-art models. We also applied the proposed model to the classification of source code with logic and syntax errors. The average precision, recall, and F-measure values for such classification are much better than those of benchmark models. To strengthen the proposed model, we combined the attention mechanism with LSTM to enhance the results of error assessment and detection as well as source code classification. Finally, our proposed model can be effective in programming education and software engineering by improving code writing, debugging, error-correction, and reasoning.
HEAWS: An Accelerator for Homomorphic Encryption on the Amazon AWS FPGA

Homomorphic Encryption makes privacy preserving computing possible in a third party owned cloud by enabling computation on the encrypted data of users. However, software implementations of homomorphic encryption are very slow on general purpose processors. With the emergence of `FPGAs as a service', hardware-acceleration of computationally heavy workloads in the cloud are getting popular. In this article we propose HEAWS, a domain-specific coprocessor architecture for accelerating homomorphic function evaluation on the encrypted data using high-performance FPGAs available in the Amazon AWS cloud. To the best of our knowledge, we are the first to report hardware acceleration of homomorphic encryption using Amazon AWS FPGAs. Utilizing the massive size of the AWS FPGAs, we design a high-performance and parallel coprocessor architecture for the FV homomorphic encryption scheme which has become popular for computing exact arithmetic on the encrypted data. We design parallel building blocks and apply pipeline processing at different levels of the implementation hierarchy, and on top of such optimizations we instantiate multiple parallel coprocessors in the FPGA to execute several homomorphic computations simultaneously. While the absolute computation time can be reduced by deploying more computational resources, efficiency of the HW/SW communication interface plays an important role in homomorphic encryption as it is computation as well as data intensive. Our implementation utilizes state of the art 512-bit XDMA feature of high bandwidth communication available in the AWS Shell to reduce the overhead of HW/SW data transfer. Moreover, we explore the design-space to identify optimal off-chip data transfer strategy for feeding the parallel coprocessors in a time-shared manner. As a result of these optimizations, our AWS-based accelerator can perform 613 homomorphic multiplications per second for a parameter set that enables homomorphic computations of depth 4. Finally, we benchmark an artificial neural network for privacy-preserving forecasting of energy consumption in a Smart Grid application and observe five times speed up.
FOSNet: An End-to-End Trainable Deep Neural Network for Scene Recognition

Scene recognition is a kind of image recognition problems which is aimed at predicting the category of the place at which the image is taken.In this paper, a new scene recognition method using the convolutional neural network (CNN) is proposed.The proposed method is based on the fusion of the object and the scene information in the given image and the CNN framework is named as FOS (fusion of object and scene) Net.To combine the object and the scene information effectively, a new fusion framework named CCG (correlative context gating) is proposed.In addition, a new loss named scene coherence loss (SCL) is developed to train the FOSNet and to improve the scene recognition performance.The proposed SCL is based on the idea that the scene class does not change all over the image.The proposed FOSNet was experimented with three most popular scene recognition datasets, and their state-of-the-art performance is obtained in two sets: 60.14% on Places 2 and 90.30% on MIT indoor 67.The second highest performance of 77.28% is obtained on SUN 397.
Synthesis of covert actuator attackers for free

In this paper, we shall formulate and address a problem of covert actuator attacker synthesis for cyber-physical systems that are modeled by discrete-event systems. We assume the actuator attacker partially observes the execution of the closed-loop system and is able to modify each control command issued by the supervisor on a specified attackable subset of controllable events. We provide straightforward but in general exponential-time reductions, due to the use of subset construction procedure, from the covert actuator attacker synthesis problems to the Ramadge-Wonham supervisor synthesis problems. It then follows that it is possible to use the many techniques and tools already developed for solving the supervisor synthesis problem to solve the covert actuator attacker synthesis problem for free. In particular, we show that, if the attacker cannot attack unobservable events to the supervisor, then the reductions can be carried out in polynomial time. We also provide a brief discussion on some other conditions under which the exponential blowup in state size can be avoided. Finally, we show how the reduction based synthesis procedure can be extended for the synthesis of successful covert actuator attackers that also eavesdrop the control commands issued by the supervisor.
Cross-Modal Data Programming Enables Rapid Medical Machine Learning

A major bottleneck in developing clinically impactful machine learning models is a lack of labeled training data for model supervision. Thus, medical researchers increasingly turn to weaker, noisier sources of supervision, such as leveraging extractions from unstructured text reports to supervise image classification. A key challenge in weak supervision is combining sources of information that may differ in quality and have correlated errors. Recently, a statistical theory of weak supervision called data programming has shown promise in addressing this challenge. Data programming now underpins many deployed machine-learning systems in the technology industry, even for critical applications. We propose a new technique for applying data programming to the problem of cross-modal weak supervision in medicine, wherein weak labels derived from an auxiliary modality (e.g., text) are used to train models over a different target modality (e.g., images). We evaluate our approach on diverse clinical tasks via direct comparison to institution-scale, hand-labeled datasets. We find that our supervision technique increases model performance by up to 6 points area under the receiver operating characteristic curve (ROC-AUC) over baseline methods by improving both coverage and quality of the weak labels. Our approach yields models that on average perform within 1.75 points ROC-AUC of those supervised with physician-years of hand labeling and outperform those supervised with physician-months of hand labeling by 10.25 points ROC-AUC, while using only person-days of developer time and clinician work-a time saving of 96%. Our results suggest that modern weak supervision techniques such as data programming may enable more rapid development and deployment of clinically useful machine-learning models.
A complete empirical ensemble mode decomposition and support vector machine-based approach to predict Bitcoin prices

Bitcoin as an asset class has received phenomenal investor attention and is considered to have similar characteristics like gold. This study aims to analyze the price behavior of bitcoin and apply machine learning algorithm for its prediction. Understanding the nature of Bitcoin price series is a multi-scale problem, and it can be best examined by analyzing its compositional characteristics. This study uses complete empirical ensemble mode decomposition (CEEMD) to analyze the nature of Bitcoin price series. Daily Bitcoin prices from 2012 to 2018 are used to perform CEEMD to identify the short term, medium term, and long-term trend in the Bitcoin price series. The study uses support vector machine (SVM) learning algorithm to find whether it can predict Bitcoin prices and finds that SVM predicts five steps ahead Bitcoin prices for the short term, medium term, long term, and overall Bitcoin price level.
When to update systematic literature reviews in software engineering

[Context] Systematic Literature Reviews (SLRs) have been adopted by the Software Engineering (SE) community for approximately 15 years to provide meaningful summaries of evidence on several topics. Many of these SLRs are now potentially outdated, and there are no systematic proposals on when to update SLRs in SE. [Objective] The goal of this paper is to provide recommendations on when to update SLRs in SE. [Method] We evaluated, using a three-step approach, a third-party decision framework (3PDF) employed in other fields, to decide whether SLRs need updating. First, we conducted a literature review of SLR updates in SE and contacted the authors to obtain their feedback relating to the usefulness of the 3PDF within the context of SLR updates in SE. Second, we used these authors’ feedback to see whether the framework needed any adaptation; none was suggested. Third, we applied the 3PDF to the SLR updates identified in our literature review. [Results] The 3PDF showed that 14 of the 20 SLRs did not need updating. This supports the use of a decision support mechanism (such as the 3PDF) to help the SE community decide when to update SLRs. [Conclusions] We put forward that the 3PDF should be adopted by the SE community to keep relevant evidence up to date and to avoid wasting effort with unnecessary updates.
SensCapsNet: Deep Neural Network for Non-Obtrusive Sensing Based Human Activity Recognition

Recently, the recent advancement of deep learning with the capacity to perform automatic high-level feature extraction has achieved promising performance for sensor-based human activity recognition (HAR). Among different deep learning methods, Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) have been widely adopted. However, scalar outputs and pooling in CNN only allow to get the invariance but not the equivariance. The capsule networks (CapsNet) with the vector output and routing by agreement is able to capture the equivariance. In this paper, we propose a method for recognizing human activity from wearable sensors based on a capsule network named SensCapsNet. The architecture of SensCapsNet is designed to be suitable for spatial-temporal data coming from wearable sensors. Experimental results show that the proposed network outperforms CNN and LSTM methods. The performance of the proposed CapsNet architecture is assessed by altering dynamic routing between capsule layers. The proposed SensCapsNet yields improved accuracy values of 77.7% and 70.5% for 1 routing on two testing datasets in comparison with the baseline methods based on CNN and LSTM that yields the F1-score of 67.7% and 69.2% for the first dataset and 65.3% and 67.6% for the second dataset respectively. Moreover, even several human activity datasets are available, privacy invasion and obtrusive concerns have not been carefully taken in to consideration in dataset building. Toward to build a non-obstructive sensing based human activity recognition method, in this paper, a dataset named 19NonSens is designed and collected from twelve subjects wearing e-Shoes and a smart watch to perform 19 activities under multiple contexts. This dataset will be made publicity available. Finally, thanks to the promising results obtained by the proposed method, we develop a life logging application which achieves a real-time computation and the accuracy rate greater than 80% for 5 common upper body activities.
LogoSENSE: A companion HOG based logo detection scheme for phishing web page and E-mail brand recognition

With the advent of Internet and opportunities in e-commerce, a visual perception oriented cyber-attack so-called phishing has become one of the tremendous problems of the cyber world since it aims to access user credentials in order to gain illegal financial profit and steal sensitive personal data. In order to fight with this security threat, various studies using a different source of information such as URL, text content, DOM trees or visual features belonging to web pages have been utilized. Apart from other works, we propose a companion scheme to recognize brands of “zero hour” phishing web pages by localizing and classifying the target brand logos involved in page screenshots by solely use of computer vision methods in object detection manner. For this purpose, the features of Histogram of Oriented Gradients (HOG) have been employed to obtain visual representations of target brand logos in scale invariant fashion. In addition, throughout the classification, a max-margin loss equipped SVM classifier has been used in order to work with a low number of training images and to decrease the number of false positives. Moreover, we prepared a publicly available dataset having a total of 3060 training and 1979 unique phishing and legitimate web page/e-mail snapshots along with their bounding box annotations for evaluation and further academic usage. Detailed experiments show that, at the best configuration, our schema named “LogoSENSE” is able to achieve 93.50% precision and of 77.94% recall score along with obtaining F1 score of 85.02%. The experiments show that the proposed approach outperforms SIFT based detection and presents comparative results against a state-of-art deep learning based object detection method. As a result, LogoSENSE serves promising results in terms of detection accuracy and run-time efficiency, yielding a companion tool that can be used as a brand recognition mechanism for phishing web pages and emails.