Reviewerly/exHarmony huggingface dataset:
Deep User Match Network for Click-Through Rate Prediction

Click-through rate (CTR) prediction is a crucial task in many applications (e.g. recommender systems). Recently deep learning based models have been proposed and successfully applied for CTR prediction by focusing on feature interaction or user interest based on the item-to-item relevance between user behaviors and candidate item. However, these existing models neglect the user-to-user relevance between the target user and those who like the candidate item, which can reflect the preference of target user. To this end, in this paper, we propose a novel Deep User Match Network (DUMN) which measures the user-to-user relevance for CTR prediction. Specifically, in DUMN, we design a User Representation Layer to learn a unified user representation which contains user latent interest based on user behaviors. Then, User Match Layer is designed to measure the user-to-user relevance by matching the target user and those who have interacted with candidate item and modeling their similarities in user representation space. Extensive experimental results on three public real-world datasets validate the effectiveness of DUMN compared with state-of-the-art methods.
Semisupervised Feature Selection With Sparse Discriminative Least Squares Regression

In big data time, selecting informative features has become an urgent need. However, due to the huge cost of obtaining enough labeled data for supervised tasks, researchers have turned their attention to semisupervised learning, which exploits both labeled and unlabeled data. In this article, we propose a sparse discriminative semisupervised feature selection (SDSSFS) method. In this method, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -dragging technique for the supervised task is extended to the semisupervised task, which is used to enlarge the distance between classes in order to obtain a discriminative solution. The flexible <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{2,p}$ </tex-math></inline-formula> norm is implicitly used as regularization in the new model. Therefore, we can obtain a more sparse solution by setting smaller <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> . An iterative method is proposed to simultaneously learn the regression coefficients and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -dragging matrix and predicting the unknown class labels. Experimental results on ten real-world datasets show the superiority of our proposed method.
Is Private Learning Possible with Instance Encoding

A private machine learning algorithm hides as much as possible about its training data while still preserving accuracy. In this work, we study whether a non-private learning algorithm can be made private by relying on an instance-encoding mechanism that modifies the training inputs before feeding them to a normal learner. We formalize both the notion of instance encoding and its privacy by providing two attack models. We first prove impossibility results for achieving a (stronger) model. Next, we demonstrate practical attacks in the second (weaker) attack model on InstaHide, a recent proposal by Huang, Song, Li and Arora [ICML'20] that aims to use instance encoding for privacy.
Knowing what You Know: valid and validated confidence sets in multiclass and multilabel prediction

We develop conformal prediction methods for constructing valid predictive confidence sets in multiclass and multilabel problems without assumptions on the data generating distribution. A challenge here is that typical conformal prediction methods---which give marginal validity (coverage) guarantees---provide uneven coverage, in that they address easy examples at the expense of essentially ignoring difficult examples. By leveraging ideas from quantile regression, we build methods that always guarantee correct coverage but additionally provide (asymptotically optimal) conditional coverage for both multiclass and multilabel prediction problems. To address the potential challenge of exponentially large confidence sets in multilabel prediction, we build tree-structured classifiers that efficiently account for interactions between labels. Our methods can be bolted on top of any classification model---neural network, random forest, boosted tree---to guarantee its validity. We also provide an empirical evaluation, simultaneously providing new validation methods, that suggests the more robust coverage of our confidence sets.
Data-Free Network Pruning for Model Compression

Convolutional neural networks(CNNs) are often over-parameterized and cannot apply to existing resource-limited artificial intelligence(AI) devices. Some methods are proposed to model compress the CNNs, but these methods are data-driven and often unable when lacking data. To solve this problem, in this paper, we propose a data-free model compression and acceleration method based on generative adversarial networks and network pruning(named DFNP), which can train a compact neural network only needs a pre-trained neural network. The DFNP consists of the source network, generator, and target network. First, the generator will generate the pseudo data under the supervise of the source network. Then the target network will get by pruning the source network and use these generated data for training. And the source network will transfer knowledge to the target network to promote the target network to achieve a similar performance of the source network. When the VGGNet- 19 is select as the source network, the target network trained by DFNP contains only 25% parameters and 65% calculations of the source network. Still, it retains 99.4% accuracy on the CIFAR-10 dataset without any real data.
Qubit-efficient encoding schemes for binary optimisation problems

We propose and analyze a set of variational quantum algorithms for solving quadratic unconstrained binary optimization problems where a problem consisting of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msub><mml:mi>n</mml:mi><mml:mi>c</mml:mi></mml:msub></mml:math> classical variables can be implemented on <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mi class="MJX-tex-caligraphic" mathvariant="script">O</mml:mi></mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mi>log</mml:mi><mml:mo>⁡</mml:mo><mml:msub><mml:mi>n</mml:mi><mml:mi>c</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math> number of qubits. The underlying encoding scheme allows for a systematic increase in correlations among the classical variables captured by a variational quantum state by progressively increasing the number of qubits involved. We first examine the simplest limit where all correlations are neglected, i.e. when the quantum state can only describe statistically independent classical variables. We apply this minimal encoding to find approximate solutions of a general problem instance comprised of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mn>64</mml:mn></mml:math> classical variables using <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mn>7</mml:mn></mml:math> qubits. Next, we show how two-body correlations between the classical variables can be incorporated in the variational quantum state and how it can improve the quality of the approximate solutions. We give an example by solving a <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mn>42</mml:mn></mml:math>-variable Max-Cut problem using only <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mn>8</mml:mn></mml:math> qubits where we exploit the specific topology of the problem. We analyze whether these cases can be optimized efficiently given the limited resources available in state-of-the-art quantum platforms. Lastly, we present the general framework for extending the expressibility of the probability distribution to any multi-body correlations.
Design of a Facial Landmark Detection System Using a Dynamic Optical Flow Approach

Many facial landmark methods based on convolutional neural networks (CNN) have been proposed to achieve favorable detection results. However, the instability landmarks that occur in video frames due to CNNs are extremely sensitive to input image noise. To solve this problem of landmark shaking, this study proposes a simple and effective facial landmark detection method comprising a lightweight U-Net model and a dynamic optical flow (DOF). The DOF uses the fast optical flow to obtain the optical flow vector of the landmark and uses dynamic routing to improve landmark stabilization. A lightweight U-Net model is designed to predict facial landmarks with a smaller model size and less computational complexity. The predicted facial landmarks are further fed to the DOF approach to deal with the unstable shaking. Finally, a comparison of several common methods and the proposed detection method is made on several benchmark datasets. Experimental evaluations and analyses show that not only can the lightweight U-Net model achieve favorable landmark prediction but also the DOF stabilizing method can improve the robustness of landmark prediction in both static images and video frames. It should be emphasized that the proposed detection system exhibits better performance than others without requiring heavy computational loadings.
Complex-Valued Vs. Real-Valued Neural Networks for Classification Perspectives: An Example on Non-Circular Data

This paper shows the benefits of using Complex-Valued Neural Network (CVNN) on classification tasks for non-circular complex-valued datasets. Motivated by radar and especially Synthetic Aperture Radar (SAR) applications, we propose a statistical analysis of fully connected feed-forward neural networks performance in the cases where real and imaginary parts of the data are correlated through the non-circular property. In this context, comparisons between CVNNs and their real-valued equivalent models are conducted, showing that CVNNs provide better performance for multiple types of non-circularity. Notably, CVNNs statistically perform less overfitting, higher accuracy and provide shorter confidence intervals than its equivalent Real-Valued Neural Network (RVNN).
Belief propagation with quantum messages for quantum-enhanced classical communications

Abstract For space-based laser communications, when the mean photon number per received optical pulse is much smaller than one, there is a large gap between communications capacity achievable with a receiver that performs individual pulse-by-pulse detection, and the quantum-optimal “joint-detection receiver” that acts collectively on long codeword-blocks of modulated pulses; an effect often termed “superadditive capacity”. In this paper, we consider the simplest scenario where a large superadditive capacity is known: a pure-loss channel with a coherent-state binary phase-shift keyed (BPSK) modulation. The two BPSK states can be mapped conceptually to two non-orthogonal states of a qubit, described by an inner product that is a function of the mean photon number per pulse. Using this map, we derive an explicit construction of the quantum circuit of a joint-detection receiver based on a recent idea of “belief-propagation with quantum messages” (BPQM). We quantify its performance improvement over the Dolinar receiver that performs optimal pulse-by-pulse detection, which represents the best “classical” approach. We analyze the scheme rigorously and show that it achieves the quantum limit of minimum average error probability in discriminating 8 (BPSK) codewords of a length-5 binary linear code with a tree factor graph. Our result suggests that a BPQM receiver might attain the Holevo capacity of this BPSK-modulated pure-loss channel. Moreover, our receiver circuit provides an alternative proposal for a quantum supremacy experiment, targeted at a specific application that can potentially be implemented on a small, special-purpose, photonic quantum computer capable of performing cat-basis universal qubit logic.
Generating Synthetic Training Data for Supervised De-Identification of Electronic Health Records

A major hurdle in the development of natural language processing (NLP) methods for Electronic Health Records (EHRs) is the lack of large, annotated datasets. Privacy concerns prevent the distribution of EHRs, and the annotation of data is known to be costly and cumbersome. Synthetic data presents a promising solution to the privacy concern, if synthetic data has comparable utility to real data and if it preserves the privacy of patients. However, the generation of synthetic text alone is not useful for NLP because of the lack of annotations. In this work, we propose the use of neural language models (LSTM and GPT-2) for generating artificial EHR text jointly with annotations for named-entity recognition. Our experiments show that artificial documents can be used to train a supervised named-entity recognition model for de-identification, which outperforms a state-of-the-art rule-based baseline. Moreover, we show that combining real data with synthetic data improves the recall of the method, without manual annotation effort. We conduct a user study to gain insights on the privacy of artificial text. We highlight privacy risks associated with language models to inform future research on privacy-preserving automated text generation and metrics for evaluating privacy-preservation during text generation.
A Comparison of Synchronous and Asynchronous Remote Usability Testing Methods

This paper presents the results of a study that compared two types of remote usability testing methods: the synchronous and asynchronous methods. The two methods were compared through an evaluation of hospital websites and involved three points of comparison: number and nature of usability problems discovered, overall task performance, and participants' testing experiences. The results show significant differences between the two methods. The asynchronous method led to the detection of a greater number of usability problems relating to content and navigation. The participants also found the asynchronous method to be more comfortable and convenient for them to use. They also felt it was easier to concentrate on tasks. However, no differences were found between the methods in terms of tasks performance metrics. The study concludes that the asynchronous method seems to outperform the synchronous one for testing hospital websites, as it is equally useful in revealing high-severity problems, and more profitable for detecting considerable number of usability issues, as well as ensuring that the usability testing experience is as natural as possible for participants. Moreover, it can be conducted with a larger sample size and without the need for the presence of any evaluators during the testing sessions, thus reducing financial and temporal costs.
Learning Student Networks in the Wild

Data-free learning for student networks is a new paradigm for solving users’ anxiety caused by the privacy problem of using original training data. Since the architectures of modern convolutional neural networks (CNNs) are compact and sophisticated, the alternative images or meta-data generated from the teacher network are often broken. Thus, the student network cannot achieve the comparable performance to that of the pre-trained teacher network especially on the large-scale image dataset. Different to previous works, we present to maximally utilize the massive available unlabeled data in the wild. Specifically, we first thoroughly analyze the output differences between teacher and student network on the original data and develop a data collection method. Then, a noisy knowledge distillation algorithm is proposed for achieving the performance of the student network. In practice, an adaptation matrix is learned with the student network for correcting the label noise produced by the teacher network on the collected unlabeled images. The effectiveness of our DFND (Data-Free Noisy Distillation) method is then verified on several benchmarks to demonstrate its superiority over state-of-the-art data-free distillation methods. Experiments on various datasets demonstrate that the student networks learned by the proposed method can achieve comparable performance with those using the original dataset. Code is available at https://github.com/huawei-noah/Data-Efficient-Model-Compression
Is It Possible to Forecast the Price of Bitcoin?

This paper focuses on forecasting the price of Bitcoin, motivated by its market growth and the recent interest of market participants and academics. We deploy six machine learning algorithms (e.g., Artificial Neural Network, Support Vector Machine, Random Forest, k-Nearest Neighbours, AdaBoost, Ridge regression), without deciding a priori which one is the ‘best’ model. The main contribution is to use these data analytics techniques with great caution in the parameterization, instead of classical parametric modelings (AR), to disentangle the non-stationary behavior of the data. As soon as Bitcoin is also used for diversification in portfolios, we need to investigate its interactions with stocks, bonds, foreign exchange, and commodities. We identify that other cryptocurrencies convey enough information to explain the daily variation of Bitcoin’s spot and futures prices. Forecasting results point to the segmentation of Bitcoin concerning alternative assets. Finally, trading strategies are implemented.
On Information Plane Analyses of Neural Network Classifiers—A Review

We review the current literature concerned with information plane (IP) analyses of neural network (NN) classifiers. While the underlying information bottleneck theory and the claim that information-theoretic compression is causally linked to generalization are plausible, empirical evidence was found to be both supporting and conflicting. We review this evidence together with a detailed analysis of how the respective information quantities were estimated. Our survey suggests that compression visualized in IPs is not necessarily information-theoretic but is rather often compatible with geometric compression of the latent representations. This insight gives the IP a renewed justification. Aside from this, we shed light on the problem of estimating mutual information in deterministic NNs and its consequences. Specifically, we argue that, even in feedforward NNs, the data processing inequality needs not to hold for estimates of mutual information. Similarly, while a fitting phase, in which the mutual information is between the latent representation and the target increases, is necessary (but not sufficient) for good classification performance, depending on the specifics of mutual information estimation, such a fitting phase needs to not be visible in the IP.
BERT & Family Eat Word Salad: Experiments with Text Understanding

In this paper, we study the response of large models from the BERT family to incoherent inputs that should confuse any model that claims to understand natural language. We define simple heuristics to construct such examples. Our experiments show that state-of-the-art models consistently fail to recognize them as ill-formed, and instead produce high confidence predictions on them. As a consequence of this phenomenon, models trained on sentences with randomly permuted word order perform close to state-of-the-art models. To alleviate these issues, we show that if models are explicitly trained to recognize invalid inputs, they can be robust to such attacks without a drop in performance.
A Content-Based Chinese Spam Detection Method Using a Capsule Network With Long-Short Attention

Most existing Chinese spam detection models suffer such problems as inaccurate representation, unsatisfactory detection effect and poor practicality. To address these problems, a capsule network model combining the long-short attention mechanism is proposed here to achieve efficient Chinese spam detection. For text representation, the proposed model uses a multi-channel structure based on the long-short attention mechanism, which can capture complex text features in spam and generate contextual word vectors with more semantic information. For feature mining and classification, the model improves the structure of the traditional capsule network without compromising the classification performance and optimizes the dynamic routing algorithm, so that the model has a high accuracy without reducing the running speed. Experimental results show that the model outperformed the current mainstream methods such as TextCNN, LSTM and even BERT in characterization and detection; and it achieved an accuracy as high as 98.72% on an unbalanced dataset and 99.30% on a balanced dataset.
Accelerating quantum computer developments

Abstract Product development Given the recent breakthroughs in quantum technology development in R&amp; D labs all over the world, the perspective of high-tech companies has changed. Product development is initiated next to the existing research and technology development activities. Quantum computer product roadmap Considering the quantum computer as a product requires standardization and integration of all its building blocks and a mature supply chain that can provide high-quality components and can ensure security of supply. The product development approach puts focus on functionality and performance requirements of the product and uses state-of-the-art technology to build the product. Based on the expected requirements of future products it is possible to outline a product development roadmap. It is expected that a fully functional quantum computer will be available within a decade from now, and will be used by the High Performance Computing (HPC) market, where it will replace (part of) the supercomputers that are currently used for complex calculations and data management. In the short term, a partly functional quantum computer will be available and of interest to the R&amp;D market, which has a need for such a product to expedite their quantum technology developments. ImpaQT project In this paper, we present the product development approach and roadmap for quantum computers, based on superconducting circuits as an example. A group of companies in the Dutch quantum ecosystem ( Quantum Delta ) have joined forces and have started the ImpaQT project. The companies of the ImpaQT consortium form a local supply chain for key components of quantum computers. This paper shows that quantum community has reached the next level of maturity and that the quantum computer as a commercial product looks set to become a reality.