Reviewerly/exHarmony huggingface dataset:
ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning

We present ATOMIC, an atlas of everyday commonsense reasoning, organized through 877k textual descriptions of inferential knowledge. Compared to existing resources that center around taxonomic knowledge, ATOMIC focuses on inferential knowledge organized as typed if-then relations with variables (e.g., “if X pays Y a compliment, then Y will likely return the compliment”). We propose nine if-then relation types to distinguish causes vs. effects, agents vs. themes, voluntary vs. involuntary events, and actions vs. mental states. By generatively training on the rich inferential knowledge described in ATOMIC, we show that neural models can acquire simple commonsense capabilities and reason about previously unseen events. Experimental results demonstrate that multitask models that incorporate the hierarchical structure of if-then relation types lead to more accurate inference compared to models trained in isolation, as measured by both automatic and human evaluation.
Deep Learning for Encrypted Traffic Classification: An Overview

Traffic classification has been studied for two decades and applied to a wide range of applications from QoS provisioning and billing in ISPs to security-related applications in firewalls and intrusion detection systems. Port-based, data packet inspection, and classical machine learning methods have been used extensively in the past, but their accuracy has declined due to the dramatic changes in Internet traffic, particularly the increase in encrypted traffic. With the proliferation of deep learning methods, researchers have recently investigated these methods for traffic classification and reported high accuracy. In this article, we introduce a general framework for deep-learning-based traffic classification. We present commonly used deep learning methods and their application in traffic classification tasks. Then we discuss open problems, challenges, and opportunities for traffic classification.
Blockchain based searchable encryption for electronic health record sharing

Data leakage in electronic health records (EHRs) could result in the compromise of patient privacy (e.g. medical conditions). Generally most data in EHRs remain unchanged once they are uploaded to the system; thus, blockchain can be potentially used to facilitate the sharing of such data. Different participating medical organizations and individuals (e.g. medical practitioners, hospitals, medical labs and insurance companies) can then access EHRs stored on the blockchain with a higher level of confidence. In this paper, a blockchain based searchable encryption scheme for EHRs is proposed. The index for EHRs is constructed through complex logic expressions and stored in the blockchain, so that a data user can utilize the expressions to search the index. As only the index is migrated to the blockchain to facilitate propagation, the data owners have full control over who can see their EHRs data. The use of blockchain technology ensures the integrity, anti-tampering, and traceability of EHRs’ index. Finally, the performance of the proposed scheme is evaluated from two aspects, namely in terms of the overhead for extracting the document IDs from EHRs and the overhead associated with conducting transactions on smart contract in Ethereum.
Expressibility and Entangling Capability of Parameterized Quantum Circuits for Hybrid Quantum‐Classical Algorithms

Abstract Parameterized quantum circuits (PQCs) play an essential role in the performance of many variational quantum algorithms. One challenge in implementing such algorithms is choosing an effective circuit that well represents the solution space while maintaining a low circuit depth and parameter count. To characterize and identify expressible, yet compact, circuits, several descriptors are proposed, including expressibility and entangling capability, that are statistically estimated from classical simulations. These descriptors are computed for different circuit structures, varying the qubit connectivity and selection of gates. From these simulations, circuit fragments that perform well with respect to the descriptors are identified. In particular, a substantial improvement in performance of two‐qubit gates in a ring or all‐to‐all connected arrangement, compared to that of those on a line, is observed. Furthermore, improvement in both descriptors is achieved by sequences of controlled X‐rotation gates compared to sequences of controlled Z‐rotation gates. In addition, it is investigated how expressibility “saturates” with increased circuit depth, finding that the rate and saturated value appear to be distinguishing features of a PQC. While the correlation between each descriptor and algorithm performance remains to be investigated, methods and results from this study can be useful for algorithm development and design of experiments.
A Hybrid BlockChain-Based Identity Authentication Scheme for Multi-WSN

Internet of Things (IoT) equipment is usually in a harsh environment, and its security has always been a widely concerned issue. Node identity authentication is an important means to ensure its security. Traditional IoT identity authentication protocols usually rely on trusted third parties. However, many IoT environments do not allow such conditions, and are prone to single point failure. Blockchain technology with decentralization features provides a new solution for distributed IoT system. In this paper, a blockchain based multi-WSN authentication scheme for IoT is proposed. The nodes of IoT are divided into base stations, cluster head nodes and ordinary nodes according to their capability differences, which are formed to a hierarchical network. A blockchain network is constructed among different types of nodes to form a hybrid blockchain model, including local chain and public chain. In this hybrid model, nodes identity mutual authentication in various communication scenarios is realized, ordinary node identity authentication operation is accomplished by local blockchain, and cluster head node identity authentication are realized in public blockchain. The analysis of security and performance shows that the scheme has comprehensive security and better performance.
A critique of pure learning and what artificial neural networks can learn from animal brains

Abstract Artificial neural networks (ANNs) have undergone a revolution, catalyzed by better supervised learning algorithms. However, in stark contrast to young animals (including humans), training such networks requires enormous numbers of labeled examples, leading to the belief that animals must rely instead mainly on unsupervised learning. Here we argue that most animal behavior is not the result of clever learning algorithms—supervised or unsupervised—but is encoded in the genome. Specifically, animals are born with highly structured brain connectivity, which enables them to learn very rapidly. Because the wiring diagram is far too complex to be specified explicitly in the genome, it must be compressed through a “genomic bottleneck”. The genomic bottleneck suggests a path toward ANNs capable of rapid learning.
Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning

In distributed training of deep neural networks, parallel minibatch SGD is widely used to speed up the training process by using multiple workers. It uses multiple workers to sample local stochastic gradients in parallel, aggregates all gradients in a single server to obtain the average, and updates each worker’s local model using a SGD update with the averaged gradient. Ideally, parallel mini-batch SGD can achieve a linear speed-up of the training time (with respect to the number of workers) compared with SGD over a single worker. However, such linear scalability in practice is significantly limited by the growing demand for gradient communication as more workers are involved. Model averaging, which periodically averages individual models trained over parallel workers, is another common practice used for distributed training of deep neural networks since (Zinkevich et al. 2010) (McDonald, Hall, and Mann 2010). Compared with parallel mini-batch SGD, the communication overhead of model averaging is significantly reduced. Impressively, tremendous experimental works have verified that model averaging can still achieve a good speed-up of the training time as long as the averaging interval is carefully controlled. However, it remains a mystery in theory why such a simple heuristic works so well. This paper provides a thorough and rigorous theoretical study on why model averaging can work as well as parallel mini-batch SGD with significantly less communication overhead.
End-to-End Structure-Aware Convolutional Networks for Knowledge Base Completion

Knowledge graph embedding has been an active research topic for knowledge base completion, with progressive improvement from the initial TransE, TransH, DistMult et al to the current state-of-the-art ConvE. ConvE uses 2D convolution over embeddings and multiple layers of nonlinear features to model knowledge graphs. The model can be efficiently trained and scalable to large knowledge graphs. However, there is no structure enforcement in the embedding space of ConvE. The recent graph convolutional network (GCN) provides another way of learning graph node embedding by successfully utilizing graph connectivity structure. In this work, we propose a novel end-to-end Structure-Aware Convolutional Network (SACN) that takes the benefit of GCN and ConvE together. SACN consists of an encoder of a weighted graph convolutional network (WGCN), and a decoder of a convolutional network called Conv-TransE. WGCN utilizes knowledge graph node structure, node attributes and edge relation types. It has learnable weights that adapt the amount of information from neighbors used in local aggregation, leading to more accurate embeddings of graph nodes. Node attributes in the graph are represented as additional nodes in the WGCN. The decoder Conv-TransE enables the state-of-the-art ConvE to be translational between entities and relations while keeps the same link prediction performance as ConvE. We demonstrate the effectiveness of the proposed SACN on standard FB15k-237 and WN18RR datasets, and it gives about 10% relative improvement over the state-of-the-art ConvE in terms of HITS@1, HITS@3 and HITS@10.
Boson Sampling with 20 Input Photons and a 60-Mode Interferometer in a<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mn>1</mml:mn><mml:msup><mml:mn>0</mml:mn><mml:mn>14</mml:mn></mml:msup></mml:math>-Dimensional Hilbert Space

Quantum computing experiments are moving into a new realm of increasing size and complexity, with the short-term goal of demonstrating an advantage over classical computers. Boson sampling is a promising platform for such a goal; however, the number of detected single photons is up to five so far, limiting these small-scale implementations to a proof-of-principle stage. Here, we develop solid-state sources of highly efficient, pure, and indistinguishable single photons and 3D integration of ultralow-loss optical circuits. We perform experiments with 20 pure single photons fed into a 60-mode interferometer. In the output, we detect up to 14 photons and sample over Hilbert spaces with a size up to 3.7×10^{14}, over 10 orders of magnitude larger than all previous experiments, which for the first time enters into a genuine sampling regime where it becomes impossible to exhaust all possible output combinations. The results are validated against distinguishable samplers and uniform samplers with a confidence level of 99.9%.
Contrastive Representation Learning: A Framework and Review

Contrastive Learning has recently received interest due to its success in self-supervised representation learning in the computer vision domain. However, the origins of Contrastive Learning date as far back as the 1990s and its development has spanned across many fields and domains including Metric Learning and natural language processing. In this paper, we provide a comprehensive literature review and we propose a general Contrastive Representation Learning framework that simplifies and unifies many different contrastive learning methods. We also provide a taxonomy for each of the components of contrastive learning in order to summarise it and distinguish it from other forms of machine learning. We then discuss the inductive biases which are present in any contrastive learning system and we analyse our framework under different views from various sub-fields of Machine Learning. Examples of how contrastive learning has been applied in computer vision, natural language processing, audio processing, and others, as well as in Reinforcement Learning are also presented. Finally, we discuss the challenges and some of the most promising future research directions ahead.
A Machine Learning Approach to Twitter User Classification

This paper addresses the task of user classification in social media, with an application to Twitter. We automatically infer the values of user attributes such as political orientation or ethnicity by leveraging observable information such as the user behavior, network structure and the linguistic content of the user’s Twitter feed. We employ a machine learning approach which relies on a comprehensive set of features derived from such user information. We report encouraging experimental results on 3 tasks with different characteristics: political affiliation detection, ethnicity identification and detecting affinity for a particular business. Finally, our analysis shows that rich linguistic features prove consistently valuable across the 3 tasks and show great promise for additional user classification needs.
Realization of a multinode quantum network of remote solid-state qubits

A three-node quantum network Future quantum networks will provide the means to develop truly secure communication channels and will have applications in many other quantum-based technologies. Pompili et al. present a three-node remote quantum network based on solid-state spin qubits (nitrogen-vacancy centers in diamond) coupled by photons. The implementation of two quantum protocols on the network. entanglement distribution and entanglement swapping, illustrates a key platform for exploring, testing, and developing multinode quantum networks and quantum protocols. Science , this issue p. 259
A Secure Federated Transfer Learning Framework

Machine learning relies on the availability of a vast amount of data for training. However, in reality, most data are scattered across different organizations and cannot be easily integrated under many legal and practical constraints. In this paper, we introduce a new technique and framework, known as federated transfer learning (FTL), to improve statistical models under a data federation. The federation allows knowledge to be shared without compromising user privacy, and enables complimentary knowledge to be transferred in the network. As a result, a target-domain party can build more flexible and powerful models by leveraging rich labels from a source-domain party. A secure transfer cross validation approach is also proposed to guard the FTL performance under the federation. The framework requires minimal modifications to the existing model structure and provides the same level of accuracy as the non-privacy-preserving approach. This framework is very flexible and can be effectively adapted to various secure multi-party machine learning tasks.
Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection

In this work, we propose a novel depth-induced multi-scale recurrent attention network for saliency detection. It achieves dramatic performance especially in complex scenarios. There are three main contributions of our network that are experimentally demonstrated to have significant practical merits. First, we design an effective depth refinement block using residual connections to fully extract and fuse multi-level paired complementary cues from RGB and depth streams. Second, depth cues with abundant spatial information are innovatively combined with multi-scale context features for accurately locating salient objects. Third, we boost our model's performance by a novel recurrent attention module inspired by Internal Generative Mechanism of human brain. This module can generate more accurate saliency results via comprehensively learning the internal semantic relation of the fused feature and progressively optimizing local details with memory-oriented scene understanding. In addition, we create a large scale RGB-D dataset containing more complex scenarios, which can contribute to comprehensively evaluating saliency models. Extensive experiments on six public datasets and ours demonstrate that our method can accurately identify salient objects and achieve consistently superior performance over 16 state-of-the-art RGB and RGB-D approaches.
Breast cancer detection using deep convolutional neural networks and support vector machines

It is important to detect breast cancer as early as possible. In this manuscript, a new methodology for classifying breast cancer using deep learning and some segmentation techniques are introduced. A new computer aided detection (CAD) system is proposed for classifying benign and malignant mass tumors in breast mammography images. In this CAD system, two segmentation approaches are used. The first approach involves determining the region of interest (ROI) manually, while the second approach uses the technique of threshold and region based. The deep convolutional neural network (DCNN) is used for feature extraction. A well-known DCNN architecture named AlexNet is used and is fine-tuned to classify two classes instead of 1,000 classes. The last fully connected (fc) layer is connected to the support vector machine (SVM) classifier to obtain better accuracy. The results are obtained using the following publicly available datasets (1) the digital database for screening mammography (DDSM); and (2) the Curated Breast Imaging Subset of DDSM (CBIS-DDSM). Training on a large number of data gives high accuracy rate. Nevertheless, the biomedical datasets contain a relatively small number of samples due to limited patient volume. Accordingly, data augmentation is a method for increasing the size of the input data by generating new data from the original input data. There are many forms for the data augmentation; the one used here is the rotation. The accuracy of the new-trained DCNN architecture is 71.01% when cropping the ROI manually from the mammogram. The highest area under the curve (AUC) achieved was 0.88 (88%) for the samples obtained from both segmentation techniques. Moreover, when using the samples obtained from the CBIS-DDSM, the accuracy of the DCNN is increased to 73.6%. Consequently, the SVM accuracy becomes 87.2% with an AUC equaling to 0.94 (94%). This is the highest AUC value compared to previous work using the same conditions.
TabNet: Attentive Interpretable Tabular Learning

We propose a novel high-performance and interpretable canonical deep tabular data learning architecture, TabNet. TabNet uses sequential attention to choose which features to reason from at each decision step, enabling interpretability and more efficient learning as the learning capacity is used for the most salient features. We demonstrate that TabNet outperforms other variants on a wide range of non-performance-saturated tabular datasets and yields interpretable feature attributions plus insights into its global behavior. Finally, we demonstrate self-supervised learning for tabular data, significantly improving performance when unlabeled data is abundant.
The Medical Segmentation Decathlon

International challenges have become the de facto standard for comparative assessment of image analysis algorithms given a specific task. Segmentation is so far the most widely investigated medical image processing task, but the various segmentation challenges have typically been organized in isolation, such that algorithm development was driven by the need to tackle a single specific clinical problem. We hypothesized that a method capable of performing well on multiple tasks will generalize well to a previously unseen task and potentially outperform a custom-designed solution. To investigate the hypothesis, we organized the Medical Segmentation Decathlon (MSD) - a biomedical image analysis challenge, in which algorithms compete in a multitude of both tasks and modalities. The underlying data set was designed to explore the axis of difficulties typically encountered when dealing with medical images, such as small data sets, unbalanced labels, multi-site data and small objects. The MSD challenge confirmed that algorithms with a consistent good performance on a set of tasks preserved their good average performance on a different set of previously unseen tasks. Moreover, by monitoring the MSD winner for two years, we found that this algorithm continued generalizing well to a wide range of other clinical problems, further confirming our hypothesis. Three main conclusions can be drawn from this study: (1) state-of-the-art image segmentation algorithms are mature, accurate, and generalize well when retrained on unseen tasks; (2) consistent algorithmic performance across multiple tasks is a strong surrogate of algorithmic generalizability; (3) the training of accurate AI segmentation models is now commoditized to non AI experts.
Applications of Blockchain Technology in Medicine and Healthcare: Challenges and Future Perspectives

Blockchain technology has gained considerable attention, with an escalating interest in a plethora of numerous applications, ranging from data management, financial services, cyber security, IoT, and food science to healthcare industry and brain research. There has been a remarkable interest witnessed in utilizing applications of blockchain for the delivery of safe and secure healthcare data management. Also, blockchain is reforming the traditional healthcare practices to a more reliable means, in terms of effective diagnosis and treatment through safe and secure data sharing. In the future, blockchain could be a technology that may potentially help in personalized, authentic, and secure healthcare by merging the entire real-time clinical data of a patient’s health and presenting it in an up-to-date secure healthcare setup. In this paper, we review both the existing and latest developments in the field of healthcare by implementing blockchain as a model. We also discuss the applications of blockchain, along with the challenges faced and future perspectives.
How Much Knowledge Can You Pack Into the Parameters of a Language Model?

It has recently been observed that neural language models trained on unstructured text can implicitly store and retrieve knowledge using natural language queries. In this short paper, we measure the practical utility of this approach by fine-tuning pre-trained models to answer questions without access to any external context or knowledge. We show that this approach scales with model size and performs competitively with open-domain systems that explicitly retrieve answers from an external knowledge source when answering questions. To facilitate reproducibility and future work, we release our code and trained models.