Reviewerly/exHarmony huggingface dataset:
Robust Image Hashing Scheme using Laplacian Pyramids

Due to tremendous growth in multimedia applications and services, people can easily create, distribute, broadcast and store information. The fact that multimedia content can be easily copied and tampered has motivated a large number of researchers to devise image verification techniques using perceptual image hashing. In this paper, a perceptual image hashing technique is proposed by using difference of Laplacian pyramids. A Laplacian pyramid is a multi-scale representation of an image and can be used to extract stable features. In the proposed scheme, two different pyramids are generated by using filters of different diameters. The difference of Laplacian is calculated to get a unique and robust hash. A number of experiments have been carried out to gauge the effectiveness of the proposed scheme. The results reveal that the proposed technique is robust against non-malicious manipulations and can detect minute level tampering.
Fully convolutional online tracking

Online learning has turned out to be effective for improving tracking performance. However, it could be simply applied for classification branch, but still remains challenging to adapt to regression branch due to its complex design and intrinsic requirement for high-quality online samples. To tackle this issue, we present the fully convolutional online tracking framework, coined as FCOT, and focus on enabling online learning for both classification and regression branches by using a target filter based tracking paradigm. Our key contribution is to introduce an online regression model generator (RMG) for initializing weights of the target filter in the regression branch with online samples, and then optimizing this target filter weights based on the ground-truth samples at the first frame. Specifically, we devise a simple fully online tracker, composed of a feature extraction backbone, an up-sampling decoder, a multi-scale classification branch, and an anchor-free regression branch. Thanks to the unique design of RMG, our FCOT can not only handle the target variation along temporal dimension, but also overcome the issue of error accumulation during the tracking procedure. In addition, due to its simplicity in design, our FCOT could be trained and deployed in a fully convolutional manner with a real-time running speed. Our FCOT achieves promising performance on seven benchmarks, including VOT2018, LaSOT, TrackingNet, GOT-10k, OTB100, UAV123, and NFS. Code and models of our FCOT are available at: https://github.com/MCG-NJU/FCOT.
A New Workload Prediction Model Using Extreme Learning Machine and Enhanced Tug of War optimization

Time series data is widely accessible in many life areas like economy, weather, stock price, retail sales, distributed system workloads. While many studies have focused on improving existing prediction techniques on accuracy aspect, less efforts is paid towards simple but efficient forecasting models in order to keep the balance between computation cost and prediction accuracy. In this work, we propose a novel time series prediction model, which aims to both model simplicity and accuracy. The core of the model is built based on extreme learning machine. Due to the random determination process for input weights and hidden biases, extreme learning machine requires a large number of hidden neurons to achieve good results and this increases the model complexity. To overcome this drawback, we propose a new opposition-based tug of war optimization to select optimally input weights, and hidden biases then apply to extreme learning machine. Two real public traffic monitoring datasets from Internet service providers were employed to evaluate our design. The achieved outcomes demonstrate that our proposed solution works effectively with satisfied performance in comparison with existing models for distributed system workload prediction.
A non-cooperative meta-modeling game for automated third-party calibrating, validating and falsifying constitutive laws with parallelized adversarial attacks

The evaluation of constitutive models, especially for high-risk and high-regret engineering applications, requires efficient and rigorous third-party calibration, validation and falsification. While there are numerous efforts to develop paradigms and standard procedures to validate models, difficulties may arise due to the sequential, manual, and often biased nature of the commonly adopted calibration and validation processes, thus slowing down data collections, hampering the progress towards discovering new physics, increasing expenses and possibly leading to misinterpretations of the credibility and application ranges of proposed models. This work attempts to introduce concepts from game theory and machine learning techniques to overcome many of these existing difficulties. We introduce an automated meta-modeling game where two competing AI agents systematically generate experimental data to calibrate a given constitutive model and to explore its weakness such that the experiment design and model robustness can be improved through competitions. The two agents automatically search for the Nash equilibrium of the meta-modeling game in an adversarial reinforcement learning framework without human intervention. In particular, a protagonist agent seeks to find the more effective ways to generate data for model calibrations, while an adversary agent tries to find the most devastating test scenarios that expose the weaknesses of the constitutive model calibrated by the protagonist. By capturing all possible design options of the laboratory experiments into a single decision tree, we recast the design of experiments as a game of combinatorial moves that can be resolved through deep reinforcement learning by the two competing players. Our adversarial framework emulates idealized scientific collaborations and competitions among researchers to achieve a better understanding of the application range of the learned material laws and prevent misinterpretations caused by conventional AI-based third-party validation. Numerical examples are given to demonstrate the wide applicability of the proposed meta-modeling game with adversarial attacks on both human-crafted constitutive models and machine learning models.
Using Challenges to Enhance a Learning Game for Pronunciation Training of English as a Second Language

Learning games have a remarkable potential for education. They provide an emergent form of social participation that deserves the assessment of their usefulness and efficiency in learning processes. This study describes a novel learning game for foreign pronunciation training in which players can challenge each other. Native Spanish speakers performed several pronunciation activities during a one-month competition using a mobile application, designed under a minimal pairs approach, to improve their pronunciation of English as a foreign language. This game took place in a competitive scenario in which students had to challenge other participants in order to get high scores and climb up a leaderboard. Results show intense practice supported by a significant number of activities and playing regularity, so the most active and motivated players in the competition achieved significant pronunciation improvement results. The integration of automatic speech recognition (ASR) and text-to-speech (TTS) technology allowed users to improve their pronunciation while being immersed in a highly motivational game.
Prior Knowledge Regularized Multiview Self-Representation and its Applications

To learn the self-representation matrices/tensor that encodes the intrinsic structure of the data, existing multiview self-representation models consider only the multiview features and, thus, impose equal membership preference across samples. However, this is inappropriate in real scenarios since the prior knowledge, e.g., explicit labels, semantic similarities, and weak-domain cues, can provide useful insights into the underlying relationship of samples. Based on this observation, this article proposes a prior knowledge regularized multiview self-representation (P-MVSR) model, in which the prior knowledge, multiview features, and high-order cross-view correlation are jointly considered to obtain an accurate self-representation tensor. The general concept of “prior knowledge” is defined as the complement of multiview features, and the core of P-MVSR is to take advantage of the membership preference, which is derived from the prior knowledge, to purify and refine the discovered membership of the data. Moreover, P-MVSR adopts the same optimization procedure to handle different prior knowledge and, thus, provides a unified framework for weakly supervised clustering and semisupervised classification. Extensive experiments on real-world databases demonstrate the effectiveness of the proposed P-MVSR model.
Shadow Detection and Removal for Illumination Consistency on the Road

Shadow detection and removal is an important task for on-road visual perception. However, effectively detecting and removing the shadows on the road to maintain illumination consistency remain challenging. Road shadows blur and change road features, greatly damaging the image quality and making further detection and tracking more difficult to implement. For example, detection could be mistaken due to incorrect identification of shadow boundaries. The adaptability and accuracy can be affected as well. To tackle the problems, this paper employs Support Vector Machine (SVM) based on color saliency space and gradient field to detect shadow. Nonlinear SVM classifier analyzes its color saliency space and gradient information, then reconstructs road shadow descriptor to distinguish shadowed regions. To remove the shadow, adaptive variable scale regional compensation operator is adopted. Following experiments verify the detection and removal method are feasible in the real world, and robust to many types of road conditions.
Secure and Privacy Focused Electronic Health Record Management System using Permissioned Blockchain

Blockchain has tremendous potential for use in various industries, including health care. It provides an immutable, shared and transparent history of all the transactions to build the applications with trust and accountability. It helps in simplifying pharmaceutical supply chains, aiding researchers in drug development by making patients' data more secure and accessible. Health care researchers today struggle with delayed communications, scattered data and delayed medical workflows caused by conflicting eHealth care systems which makes it difficult to provide personalized care. The main problem is a lack of trust between these independent eHealth care systems which makes it difficult to establish an end-to-end accessible network. Blockchain, due to its properties, can be a potential solution to provide this link. We propose a model which focuses on providing healthcare data to researchers for statistical analysis and providing privacy at the same time. The model exhibits high data security by aggregating the customized access control protocol and asymmetric cryptography. It uses proxy re-encryption technique for sensitive medical information sharing.
End-to-End Speech Translation With Transcoding by Multi-Task Learning for Distant Language Pairs

Directly translating spoken utterances from a source language to a target language is challenging because it requires a fundamental transformation in both linguistic and para/non-linguistic features. Traditional speech-to-speech translation approaches concatenate automatic speech recognition (ASR), text-to-text machine translation (MT), and text-to-speech synthesizer (TTS) by text information. The current state-of-the-art models for ASR, MT, and TTS have mainly been built using deep neural networks, in particular, an attention-based encoder-decoder neural network with an attention mechanism. Recently, several works have constructed end-to-end direct speech-to-text translation by combining ASR and MT into a single model. However, the usefulness of these models has only been investigated on language pairs of similar syntax and word order (e.g., English-French or English-Spanish). For syntactically distant language pairs (e.g., English-Japanese), speech translation requires distant word reordering. Furthermore, parallel texts with corresponding speech utterances that are suitable for training end-to-end speech translation are generally unavailable. Collecting such corpora is usually time-consuming and expensive. This article proposes the first attempt to build an end-to-end direct speech-to-text translation system on syntactically distant language pairs that suffer from long-distance reordering. We train the model on English (subject-verb-object (SVO) word order) and Japanese (SOV word order) language pairs. To guide the attention-based encoder-decoder model on this difficult problem, we construct end-to-end speech translation with transcoding and utilize curriculum learning (CL) strategies that gradually train the network for end-to-end speech translation tasks by adapting the decoder or encoder parts. We use TTS for data augmentation to generate corresponding speech utterances from the existing parallel text data. Our experiment results show that the proposed approach provides significant improvements compared with conventional cascade models and the direct speech translation approach that uses a single model without transcoding and CL strategies.
A Stride-Based Convolution Decomposition Method to Stretch CNN Acceleration Algorithms for Efficient and Flexible Hardware Implementation

To reduce multiplication operations in convolution of convolutional neural networks (CNNs), there are three widely used convolutional acceleration algorithms, i.e., Winograd, FFT and FFA. However, current accelerators based on these convolutional acceleration algorithms have issues on flexibility and efficiency. Firstly, some accelerators utilized a combination of these acceleration algorithms and employed multiple types of computational units to achieve their respective advantages. As a result, some computational units are left unused when the best-performing unit is working, which causes much area inefficiency. Secondly, current accelerators tend to choose small parameters of these convolutional acceleration algorithms to avoid unacceptable precision loss, as a result, they are hardly to support large kernel sizes and lack of flexibility. Thirdly, these acceleration algorithms are typically presented for 1-stride convolutions, consequently, few implementation considers the acceleration of large-stride convolutions, which is a major restriction to hardware flexibility. This paper proposed a stride-based convolution decomposition method (SCDM) to reform different convolution shapes (i.e., kernel sizes & strides) to an identical pattern. With the aid of SCDM, a Winograd-stretched and hardware-efficient design (WHD) is presented to utilize one uniform computational unit for the acceleration of different convolution shapes, which combines complementary performance advantages on both Winograd F(4,3)andF(4,2) units. Compared to current FFT-based or FFA-based works, WHD can stretch the use range of Winograd and simplify implementation, thereby achieving hardware flexibility and efficiency. Evaluation results show that 34.08%~55.41% operation reduction were achieved on six CNN models, while incurring a slight hardware overhead.
COVID-19 and the secret virtual assistants: the social weapons for a state of emergency

<ns3:p>Technologies are ubiquitous in modern Britain, gradually infiltrating many areas of our working and personal lives. But what role can technology play in the current COVID-19 pandemic? At a time when our usual face to face social interactions are temporarily suspended, many of us have reached out to technology (e.g. Skype, WhatsApp, Facebook, Zoom) to help maintain a sense of closeness and connection to friends, family and vital services. </ns3:p><ns3:p> One largely unsung technology is the virtual assistant (VA), a cost-efficient technology enabling users to access the Internet of Things using little more than voice. Deploying an ecological framework, in the context of smart age-friendly cities, this paper explores how VA technology can function as an emergency response system, providing citizens with systems to connect with friends, family, vital services and offering assistance in the diagnosis of COVID-19.</ns3:p><ns3:p> We provide an illustration of the potentials and challenges VAs present, concluding stricter regulation and controls should be implemented before VAs can be safely integrated into smart age-friendly cities across the globe.</ns3:p>
3D ResNet with Ranking Loss Function for Abnormal Activity Detection in Videos

Abnormal activity detection is one of the most challenging tasks in the field of computer vision. This study is motivated by the recent state-of-art work of abnormal activity detection, which utilizes both abnormal and normal videos in learning abnormalities with the help of multiple instance learning by providing the data with video-level information. In the absence of temporal-annotations, such a model is prone to give a false alarm while detecting the abnormalities. For this reason, in this paper, we focus on the task of minimizing the false alarm rate while performing an abnormal activity detection task. The mitigation of these false alarms and recent advancement of 3D deep neural network in video action recognition task collectively give us motivation to exploit the 3D ResNet in our proposed method, which helps to extract spatial-temporal features from the videos. Afterwards, using these features and deep multiple instance learning along with the proposed ranking loss, our model learns to predict the abnormality score at the video segment level. Therefore, our proposed method 3D deep Multiple Instance Learning with ResNet (MILR) along with the new proposed ranking loss function achieves the best performance on the UCF-Crime benchmark dataset, as compared to other state-of-art methods. The effectiveness of our proposed method is demonstrated on the UCF-Crime dataset.
CodeGRU: Context-aware deep learning with gated recurrent unit for source code modeling

Recently deep learning based Natural Language Processing (NLP) models have shown great potential in the modeling of source code. However, a major limitation of these approaches is that they take source code as simple tokens of text and ignore its contextual, syntactical and structural dependencies. In this work, we present CodeGRU, a gated recurrent unit based source code language model that is capable of capturing source code's contextual, syntactical and structural dependencies. We introduce a novel approach which can capture the source code context by leveraging the source code token types. Further, we adopt a novel approach which can learn variable size context by taking into account source code's syntax, and structural information. We evaluate CodeGRU with real-world data set and it shows that CodeGRU outperforms the state-of-the-art language models and help reduce the vocabulary size up to 24.93\%. Unlike previous works, we tested CodeGRU with an independent test set which suggests that our methodology does not requisite the source code comes from the same domain as training data while providing suggestions. We further evaluate CodeGRU with two software engineering applications: source code suggestion, and source code completion. Our experiment confirms that the source code's contextual information can be vital and can help improve the software language models. The extensive evaluation of CodeGRU shows that it outperforms the state-of-the-art models. The results further suggest that the proposed approach can help reduce the vocabulary size and is of practical use for software developers.
Numerical calculations of the finite key rate for general quantum key distribution protocols

Finite key analysis of quantum key distribution (QKD) is an important tool for any QKD implementation. While much work has been done on the framework of finite key analysis, the application to individual protocols often relies on the the specific protocol being simple or highly symmetric as well as represented in small finite-dimensional Hilbert spaces. In this work, we extend our pre-existing reliable, efficient, tight, and generic numerical method for calculating the asymptotic key rate of device-dependent QKD protocols in finite-dimensional Hilbert spaces to the finite key regime using the security analysis framework of Renner. We explain how this extension preserves the reliability, efficiency, and tightness of the asymptotic method. We then explore examples which illustrate both the generality of our method as well as the importance of parameter estimation and data processing within the framework.
Online RBM: Growing Restricted Boltzmann Machine on the fly for unsupervised representation

In this work, we endeavor to investigate and propose a novel unsupervised online learning algorithm, namely the Online Restricted Boltzmann Machine (O-RBM). The O-RBM is able to construct and adapt the architecture of a Restricted Boltzmann Machine (RBM) artificial neural network, according to the statistics of the streaming input data. Specifically, for a training data that is not fully available at the onset of training, the proposed O-RBM begins with a single neuron in the hidden layer of the RBM, progressively adds and suitably adapts the network to account for the variations in streaming data distributions. Such an unsupervised learning helps to effectively model the probability distribution of the entire data stream, and generates robust features. We will demonstrate that such unsupervised representations can be used for discriminative classifications on a set of multi-category and binary classification problems for unstructured image and structured signal data sets, having varying degrees of class-imbalance. We first demonstrate the O-RBM algorithm and characterize the network evolution using the simple and conventional multi-class MNIST image dataset, aimed at recognizing hand-written digit. We then benchmark O-RBM performance to other machine learning, neural network and Class RBM techniques using a number of public non-stationary datasets. Finally, we study the performance of the O-RBM on a real-world problem involving predictive maintenance of an aircraft component using time series data. In all these studies, it is observed that the O-RBM converges to a stable, concise network architecture, wherein individual neurons are inherently discriminative to the class labels despite unsupervised training. It can be observed from the performance results that on an average O-RBM improves accuracy by 2.5%–3% over conventional offline batch learning techniques while requiring at least 24%–70% fewer neurons.
Steganographic universal adversarial perturbations

We propose a steganography based technique to generate adversarial perturbations to fool deep models on any image. The proposed perturbations are computed in a transform domain where a single secret image embedded in any target image makes any deep model misclassify the target image with high probability. The attack resulting from our perturbation is ideal for black-box setting, as it does not require any information about the target model. Moreover, being a non-iterative technique, our perturbation estimation remains computationally efficient. The computed perturbations are also imperceptible to humans while they achieve high fooling ratios for the models trained on large-scale ImageNet dataset. We demonstrate successful fooling of ResNet-50, VGG-16, Inception-V3 and MobileNet-V2, achieving up to 89% fooling of these popular classification models.