Reviewerly/exHarmony huggingface dataset:
Evaluating Question Answering Evaluation

As the complexity of question answering (QA) datasets evolve, moving away from restricted formats like span extraction and multiplechoice (MC) to free-form answer generation, it is imperative to understand how well current metrics perform in evaluating QA.This is especially important as existing metrics (BLEU, ROUGE, METEOR, and F1) are computed using n-gram similarity and have a number of well-known drawbacks.In this work, we study the suitability of existing metrics in QA.For generative QA, we show that while current metrics do well on existing datasets, converting multiple-choice datasets into free-response datasets is challenging for current metrics.We also look at span-based QA, where F 1 is a reasonable metric.We show that F 1 may not be suitable for all extractive QA tasks depending on the answer types.Our study suggests that while current metrics may be suitable for existing QA datasets, they limit the complexity of QA datasets that can be created.This is especially true in the context of free-form QA, where we would like our models to be able to generate more complex and abstractive answers, thus necessitating new metrics that go beyond n-gram based matching.As a step towards a better QA metric, we explore using BERTScore, a recently proposed metric for evaluating translation, for QA.We find that although it fails to provide stronger correlation with human judgements, future work focused on tailoring a BERT-based metric to QA evaluation may prove fruitful.
Recycling a Pre-trained BERT Encoder for Neural Machine Translation

In this paper, a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model is applied to Transformerbased neural machine translation (NMT).In contrast to monolingual tasks, the number of unlearned model parameters in an NMT decoder is as huge as the number of learned parameters in the BERT model.To train all the models appropriately, we employ twostage optimization, which first trains only the unlearned parameters by freezing the BERT model, and then fine-tunes all the sub-models.In our experiments, stable two-stage optimization was achieved, in contrast the BLEU scores of direct fine-tuning were extremely low.Consequently, the BLEU scores of the proposed method were better than those of the Transformer base model and the same model without pre-training.Additionally, we confirmed that NMT with the BERT encoder is more effective in low-resource settings.
Interpreting and Improving Adversarial Robustness of Deep Neural Networks With Neuron Sensitivity

Deep neural networks (DNNs) are vulnerable to adversarial examples where inputs with imperceptible perturbations mislead DNNs to incorrect results. Despite the potential risk they bring, adversarial examples are also valuable for providing insights into the weakness and blind-spots of DNNs. Thus, the interpretability of a DNN in the adversarial setting aims to explain the rationale behind its decision-making process and makes deeper understanding which results in better practical applications. To address this issue, we try to explain adversarial robustness for deep models from a new perspective of neuron sensitivity which is measured by neuron behavior variation intensity against benign and adversarial examples. In this paper, we first draw the close connection between adversarial robustness and neuron sensitivities, as sensitive neurons make the most non-trivial contributions to model predictions in the adversarial setting. Based on that, we further propose to improve adversarial robustness by stabilizing the behaviors of sensitive neurons. Moreover, we demonstrate that state-of-the-art adversarial training methods improve model robustness by reducing neuron sensitivities, which in turn confirms the strong connections between adversarial robustness and neuron sensitivity. Extensive experiments on various datasets demonstrate that our algorithm effectively achieves excellent results. To the best of our knowledge, we are the first to study adversarial robustness using neuron sensitivities.
Multi-Source Domain Adaptation for Visual Sentiment Classification

Existing domain adaptation methods on visual sentiment classification typically are investigated under the single-source scenario, where the knowledge learned from a source domain of sufficient labeled data is transferred to the target domain of loosely labeled or unlabeled data. However, in practice, data from a single source domain usually have a limited volume and can hardly cover the characteristics of the target domain. In this paper, we propose a novel multi-source domain adaptation (MDA) method, termed Multi-source Sentiment Generative Adversarial Network (MSGAN), for visual sentiment classification. To handle data from multiple source domains, it learns to find a unified sentiment latent space where data from both the source and target domains share a similar distribution. This is achieved via cycle consistent adversarial learning in an end-to-end manner. Extensive experiments conducted on four benchmark datasets demonstrate that MSGAN significantly outperforms the state-of-the-art MDA approaches for visual sentiment classification.
Symmetric Metric Learning with Adaptive Margin for Recommendation

Metric learning based methods have attracted extensive interests in recommender systems. Current methods take the user-centric way in metric space to ensure the distance between user and negative item to be larger than that between the current user and positive item by a fixed margin. While they ignore the relations among positive item and negative item. As a result, these two items might be positioned closely, leading to incorrect results. Meanwhile, different users usually have different preferences, the fixed margin used in those methods can not be adaptive to various user biases, and thus decreases the performance as well. To address these two problems, a novel Symmetic Metric Learning with adaptive margin (SML) is proposed. In addition to the current user-centric metric, it symmetically introduces a positive item-centric metric which maintains closer distance from positive items to user, and push the negative items away from the positive items at the same time. Moreover, the dynamically adaptive margins are well trained to mitigate the impact of bias. Experimental results on three public recommendation datasets demonstrate that SML produces a competitive performance compared with several state-of-the-art methods.
When AWGN-Based Denoiser Meets Real Noises

Discriminative learning based image denoisers have achieved promising performance on synthetic noises such as Additive White Gaussian Noise (AWGN). The synthetic noises adopted in most previous work are pixel-independent, but real noises are mostly spatially/channel-correlated and spatially/channel-variant. This domain gap yields unsatisfied performance on images with real noises if the model is only trained with AWGN. In this paper, we propose a novel approach to boost the performance of a real image denoiser which is trained only with synthetic pixel-independent noise data dominated by AWGN. First, we train a deep model that consists of a noise estimator and a denoiser with mixed AWGN and Random Value Impulse Noise (RVIN). We then investigate Pixel-shuffle Down-sampling (PD) strategy to adapt the trained model to real noises. Extensive experiments demonstrate the effectiveness and generalization of the proposed approach. Notably, our method achieves state-of-the-art performance on real sRGB images in the DND benchmark among models trained with synthetic noises. Codes are available at https://github.com/yzhouas/PD-Denoising-pytorch.
Comprehensive Survey on Big Data Privacy Protection

In recent years, the ever-mounting problem of Internet phishing has been threatening the secure propagation of sensitive data over the web, thereby resulting in either outright decline of data distribution or inaccurate data distribution from several data providers. Therefore, user privacy has evolved into a critical issue in various data mining operations. User privacy has turned out to be a foremost criterion for allowing the transfer of confidential information. The intense surge in storing the personal data of customers (i.e., big data) has resulted in a new research area, which is referred to as privacy-preserving data mining (PPDM). A key issue of PPDM is how to manipulate data using a specific approach to enable the development of a good data mining model on modified data, thereby meeting a specified privacy need with minimum loss of information for the intended data analysis task. The current review study aims to utilize the tasks of data mining operations without risking the security of individuals' sensitive information, particularly at the record level. To this end, PPDM techniques are reviewed and classified using various approaches for data modification. Furthermore, a critical comparative analysis is performed for the advantages and drawbacks of PPDM techniques. This review study also elaborates on the existing challenges and unresolved issues in PPDM.
On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning

Compressed communication, in the form of sparsification or quantization of stochastic gradients, is employed to reduce communication costs in distributed data-parallel training of deep neural networks. However, there exists a discrepancy between theory and practice: while theoretical analysis of most existing compression methods assumes compression is applied to the gradients of the entire model, many practical implementations operate individually on the gradients of each layer of the model.In this paper, we prove that layer-wise compression is, in theory, better, because the convergence rate is upper bounded by that of entire-model compression for a wide range of biased and unbiased compression methods. However, despite the theoretical bound, our experimental study of six well-known methods shows that convergence, in practice, may or may not be better, depending on the actual trained model and compression ratio. Our findings suggest that it would be advantageous for deep learning frameworks to include support for both layer-wise and entire-model compression.
Leukemia Image Segmentation Using a Hybrid Histogram-Based Soft Covering Rough K-Means Clustering Algorithm

Segmenting an image of a nucleus is one of the most essential tasks in a leukemia diagnostic system. Accurate and rapid segmentation methods help the physicians identify the diseases and provide better treatment at the appropriate time. Recently, hybrid clustering algorithms have started being widely used for image segmentation in medical image processing. In this article, a novel hybrid histogram-based soft covering rough k-means clustering (HSCRKM) algorithm for leukemia nucleus image segmentation is discussed. This algorithm combines the strengths of a soft covering rough set and rough k-means clustering. The histogram method was utilized to identify the number of clusters to avoid random initialization. Different types of features such as gray level co-occurrence matrix (GLCM), color, and shape-based features were extracted from the segmented image of the nucleus. Machine learning prediction algorithms were applied to classify the cancerous and non-cancerous cells. The proposed strategy is compared with an existing clustering algorithm, and the efficiency is evaluated based on the prediction metrics. The experimental results show that the HSCRKM method efficiently segments the nucleus, and it is also inferred that logistic regression and neural network perform better than other prediction algorithms.
LightRec: A Memory and Search-Efficient Recommender System

Deep recommender systems have achieved remarkable improvements in recent years. Despite its superior ranking precision, the running efficiency and memory consumption turn out to be severe bottlenecks in reality. To overcome both limitations, we propose LightRec, a lightweight recommender system which enjoys fast online inference and economic memory consumption. The backbone of LightRec is a total of B codebooks, each of which is composed of W latent vectors, known as codewords. On top of such a structure, LightRec will have an item represented as additive composition of B codewords, which are optimally selected from each of the codebooks. To effectively learn the codebooks from data, we devise an end-to-end learning workflow, where challenges on the inherent differentiability and diversity are conquered by the proposed techniques. In addition, to further improve the representation quality, several distillation strategies are employed, which better preserves user-item relevance scores and relative ranking orders. LightRec is extensively evaluated with four real-world datasets, which gives rise to two empirical findings: 1) compared with those the state-of-the-art lightweight baselines, LightRec achieves over 11% relative improvements in terms of recall performance; 2) compared to conventional recommendation algorithms, LightRec merely incurs negligible accuracy degradation while leads to more than 27x speedup in top-k recommendation.
Designing Near-Optimal Steganographic Codes in Practice Based on Polar Codes

Steganography is an information hiding technique for covert communication. So far Syndrome-Trellis Codes (STC), a convolutional codes-based method, is the only near-optimal coding method, i.e., it can approach the rate-distortion bound of content-adaptive steganography in practice. However, as a secure communication application, steganography needs the diversity of coding methods. This paper proposes another and a better near-optimal steganographic coding method based on polar codes, using Successive Cancellation List (SCL) decoding algorithm to minimize additive distortion in steganography. Considering a steganographic channel as a binary symmetric channel, the proposed Steganographic Polar Codes (SPC) chooses parity-check matrix by setting embedding payload as the initial value of Arikan's heuristic and computes decoding channel metric from the optimal modification probability of minimal distortion model. To overcome the inherent defect of polar codes only suiting for code length of a power of 2, we introduce three strategies to generalize SPC for arbitrary length. Experimental results validate the versatility of SPC to minimize arbitrary distortion. When compared with STC, the overall coding performance of SPC is more superior with low embedding complexity. This work verifies the availability of polar codes for the practical construction of steganographic codes and provides a methodology for designing better steganographic codes based on any advance of polar coding/decoding.
A Novel Method for Performance Improvement of Chaos-Based Substitution Boxes

Symmetry plays an important role in nonlinear system theory. In particular, it offers several methods by which to understand and model the chaotic behavior of mathematical, physical and biological systems. This study examines chaotic behavior in the field of information security. A novel method is proposed to improve the performance of chaos-based substitution box structures. Substitution box structures have a special role in block cipher algorithms, since they are the only nonlinear components in substitution permutation network architectures. However, the substitution box structures used in modern block encryption algorithms contain various vulnerabilities to side-channel attacks. Recent studies have shown that chaos-based designs can offer a variety of opportunities to prevent side-channel attacks. However, the problem of chaos-based designs is that substitution box performance criteria are worse than designs based on mathematical transformation. In this study, a postprocessing algorithm is proposed to improve the performance of chaos-based designs. The analysis results show that the proposed method can improve the performance criteria. The importance of these results is that chaos-based designs may offer opportunities for other practical applications in addition to the prevention of side-channel attacks.
Overview and Impact of Blockchain on Auditing

Blockchain has become very popular as the underlying technology powering Bitcoin.However, the benefits behind this technology further surpass just supporting cryptocurrencies.Blockchain can be defined as a digital ledger that allows to capture transactions conducted among several parties on real-time and serves as a decentralized database where each participant keeps an identical copy of the ledger.The appeal behind blockchain resides on its peer-to-peer network infrastructure along cryptographic capabilities.This combination enables users to conduct transactions without a trusted third-party intermediary.Benefits in accounting are even more promising as blockchain will provide a triple entry accounting system where all transactions are immutable and have been time stamped, recorded on real-time and encrypted (Alarcon & Ng, 2018).The purpose of this paper is to review extant research on this technology and assess the impact of blockchain in the audit profession, including new risks, change in procedures and additional opportunities.
Improving the Accuracy of Convolutional Neural Networks by Identifying and Removing Outlier Images in Datasets Using t-SNE

In the field of supervised machine learning, the quality of a classifier model is directly correlated with the quality of the data that is used to train the model. The presence of unwanted outliers in the data could significantly reduce the accuracy of a model or, even worse, result in a biased model leading to an inaccurate classification. Identifying the presence of outliers and eliminating them is, therefore, crucial for building good quality training datasets. Pre-processing procedures for dealing with missing and outlier data, commonly known as feature engineering, are standard practice in machine learning problems. They help to make better assumptions about the data and also prepare datasets in a way that best expose the underlying problem to the machine learning algorithms. In this work, we propose a multistage method for detecting and removing outliers in high-dimensional data. Our proposed method is based on utilising a technique called t-distributed stochastic neighbour embedding (t-SNE) to reduce high-dimensional map of features into a lower, two-dimensional, probability density distribution and then use a simple descriptive statistical method called interquartile range (IQR) to identifying any outlier values from the density distribution of the features. t-SNE is a machine learning algorithm and a nonlinear dimensionality reduction technique well-suited for embedding high-dimensional data for visualisation in a low-dimensional space of two or three dimensions. We applied this method on a dataset containing images for training a convolutional neural network model (ConvNet) for an image classification problem. The dataset contains four different classes of images: three classes contain defects in construction (mould, stain, and paint deterioration) and a no-defect class (normal). We used the transfer learning technique to modify a pre-trained VGG-16 model. We used this model as a feature extractor and as a benchmark to evaluate our method. We have shown that, when using this method, we can identify and remove the outlier images in the dataset. After removing the outlier images from the dataset and re-training the VGG-16 model, the results have also shown that the accuracy of the classification has significantly improved and the number of misclassified cases has also dropped. While many feature engineering techniques for handling missing and outlier data are common in predictive machine learning problems involving numerical or categorical data, there is little work on developing techniques for handling outliers in high-dimensional data which can be used to improve the quality of machine learning problems involving images such as ConvNet models for image classification and object detection problems.
Application of Event-Triggered Cubature Kalman Filter for Remote Nonlinear State Estimation in Wireless Sensor Network

This article proposes a novel filter, which is based on the event-triggered schedule and the cubature Kalman filter (CKF), to perform the nonlinear remote state estimation in the wireless sensor network with communication constraints. For the purpose of relieving the communication burden between sensors and the remote filter, the event-triggered schedule is utilized to determine whether the current observations should be sent. To deal with the non-Gaussian property due to the nonlinear transformation, the third-degree spherical-radial cubature rule is adopted to provide the accurate estimation. Moreover, the stochastic stability of the designed event-triggered CKF (ETCKF) is analyzed deriving the relationships among the estimation performance, the communication rate and the design parameter. Therefore, the proposed ETCKF can effectively balance the communication burden and the estimation accuracy. Finally, the numerical simulation on IEEE-39 bus system and unmanned aerial vehicles remote attitude monitoring system are performed to verify the practicability of ETCKF.
Efficient distribution of high-dimensional entanglement through 11 km fiber

High-dimensional entanglement enriches our understanding of entanglement and shows many advantages in quantum communications tasks, so its long-distance distribution is a key technology. Here, we use multicore fiber to distribute four-dimensional (4D) entanglement over 11 km. The observed fidelity is <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mi>F</mml:mi> <mml:mo>=</mml:mo> <mml:mn>0.921</mml:mn> <mml:mo>±<!-- ± --></mml:mo> <mml:mn>0.001</mml:mn> </mml:math> , which is feasible for high-dimensional quantum networks. We observed the violation of high-dimensional Bell inequality ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:msub> <mml:mi>I</mml:mi> <mml:mn>4</mml:mn> </mml:msub> </mml:mrow> <mml:mo>=</mml:mo> <mml:mn>2.635</mml:mn> <mml:mo>±<!-- ± --></mml:mo> <mml:mn>0.014</mml:mn> </mml:math> ) and steering inequality ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:msub> <mml:mi>S</mml:mi> <mml:mn>4</mml:mn> </mml:msub> </mml:mrow> <mml:mo>=</mml:mo> <mml:mn>1.900</mml:mn> <mml:mo>±<!-- ± --></mml:mo> <mml:mn>0.003</mml:mn> </mml:math> ) for a 4D system over a distance of 11 km and finally demonstrated 4D quantum key distribution based on entanglement in proof of principle. Our work provides an experimental platform for high-dimensional quantum technologies and enables the intracity high-dimensional quantum network.
Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension

Natural Questions is a new challenging machine reading comprehension benchmark with two-grained answers, which are a long answer (typically a paragraph) and a short answer (one or more entities inside the long answer).Despite the effectiveness of existing methods on this benchmark, they treat these two sub-tasks individually during training while ignoring their dependencies.To address this issue, we present a novel multi-grained machine reading comprehension framework that focuses on modeling documents at their hierarchical nature, which are different levels of granularity: documents, paragraphs, sentences, and tokens.We utilize graph attention networks to obtain different levels of representations so that they can be learned simultaneously.The long and short answers can be extracted from paragraphlevel representation and token-level representation, respectively.In this way, we can model the dependencies between the two-grained answers to provide evidence for each other.We jointly train the two sub-tasks, and our experiments show that our approach significantly outperforms previous systems at both long and short answer criteria.