Reviewerly/exHarmony huggingface dataset:
LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification

Extreme multi-label text classification(XMC) is a task for finding the most relevant labels from a large label set. Nowadays deep learning-based methods have shown significant success in XMC. However, the existing methods (e.g., AttentionXML and X-Transformer etc) still suffer from 1) combining several models to train and predict for one dataset, and 2) sampling negative labels statically during the process of training label ranking model, which will harm the performance and accuracy of model. To address the above problems, we propose LightXML, which adopts end-to-end training and dynamical negative labels sampling. In LightXML, we use GAN like networks to recall and rank labels. The label recalling part will generate negative and positive labels, and the label ranking part will distinguish positive labels from these labels. Based on these networks, negative labels are sampled dynamically during label ranking part training. With feeding both label recalling and ranking parts with the same text representation, LightXML can reach high performance. Extensive experiments show that LightXML outperforms state-of-the-art methods in five extreme multi-label datasets with much smaller model size and lower computational complexity. In particular, on the Amazon dataset with 670K labels, LightXML can reduce the model size up to 72% compared to AttentionXML. Our code is available at http://github.com/kongds/LightXML.
Non-Local Graph Neural Networks

Modern graph neural networks (GNNs) learn node embeddings through multilayer local aggregation and achieve great success in applications on assortative graphs. However, tasks on disassortative graphs usually require non-local aggregation. In addition, we find that local aggregation is even harmful for some disassortative graphs. In this work, we propose a simple yet effective non-local aggregation framework with an efficient attention-guided sorting for GNNs. Based on it, we develop various non-local GNNs. We perform thorough experiments to analyze disassortative graph datasets and evaluate our non-local GNNs. Experimental results demonstrate that our non-local GNNs significantly outperform previous state-of-the-art methods on seven benchmark datasets of disassortative graphs, in terms of both model performance and efficiency.
Adaptive covariance inflation in the ensemble Kalman filter by Gaussian scale mixtures

This paper studies multiplicative inflation: the complementary scaling of the state covariance in the ensemble Kalman filter (EnKF). Firstly, error sources in the EnKF are catalogued and discussed in relation to inflation; nonlinearity is given particular attention as a source of sampling error. In response, the "finite-size" refinement known as the EnKF-N is re-derived via a Gaussian scale mixture, again demonstrating how it yields adaptive inflation. Existing methods for adaptive inflation estimation are reviewed, and several insights are gained from a comparative analysis. One such adaptive inflation method is selected to complement the EnKF-N to make a hybrid that is suitable for contexts where model error is present and imperfectly parameterized. Benchmarks are obtained from experiments with the two-scale Lorenz model and its slow-scale truncation. The proposed hybrid EnKF-N method of adaptive inflation is found to yield systematic accuracy improvements in comparison with the existing methods, albeit to a moderate degree.
2D and 3D Image Quality Assessment: A Survey of Metrics and Challenges

Image quality is important not only for the viewing experience, but also for the performance of image processing algorithms. Image quality assessment (IQA) has been a topic of intense research in the fields of image processing and computer vision. In this paper, we first analyze the factors that affect two-dimensional (2D) and three-dimensional (3D) image quality, and then provide an up-to-date overview on IQA for each main factor. The main factors that affect 2D image quality are fidelity and aesthetics. Another main factor that affects stereoscopic 3D image quality is visual comfort. We also describe the IQA databases and give the experimental results on representative IQA metrics. Finally, we discuss the challenges for IQA, including the influence of different factors on each other, the performance of IQA metrics in real applications, and the combination of quality assessment, restoration, and enhancement.
Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark

The GLUE benchmark (Wang et al., 2019b) is a suite of language understanding tasks which has seen dramatic progress in the past year, with average performance moving from 70.0 at launch to 83.9, state of the art at the time of writing (May 24, 2019). Here, we measure human performance on the benchmark, in order to learn whether significant headroom remains for further progress. We provide a conservative estimate of human performance on the benchmark through crowdsourcing: Our annotators are non-experts who must learn each task from a brief set of instructions and 20 examples. In spite of limited training, these annotators robustly outperform the state of the art on six of the nine GLUE tasks and achieve an average score of 87.1. Given the fast pace of progress however, the headroom we observe is quite limited. To reproduce the data-poor setting that our annotators must learn in, we also train the BERT model (Devlin et al., 2019) in limited-data regimes, and conclude that low-resource sentence classification remains a challenge for modern neural network approaches to text understanding.
Realization of Superadiabatic Two-Qubit Gates Using Parametric Modulation in Superconducting Circuits

Fast, robust two-qubit gate operation with low susceptibility to crosstalk are the key to scalable quantum-information processing. A parametrically driven gate is inherently insensitive to crosstalk, while superadiabatic control can speed up the gate without losing accuracy. We propose and experimentally implement superadiabatic two-qubit gates using parametric modulation on superconducting quantum circuits. Our results demonstrate the preservation of adiabaticity at a gate speed close to the quantum limit, in addition to robustness against control instability. We demonstrate a controlled-$Z$ gate with an error rate of $5.8\mathrm{%}$, mainly limited by qubit decoherence, promising future improvement and scalable implementation.
Context-Adaptive Neural Network-Based Prediction for Image Compression

This paper describes a set of neural network architectures, called Prediction Neural Networks Set (PNNS), based on both fully-connected and convolutional neural networks, for intra image prediction. The choice of neural network for predicting a given image block depends on the block size, hence does not need to be signalled to the decoder. It is shown that, while fully-connected neural networks give good performance for small block sizes, convolutional neural networks provide better predictions in large blocks with complex textures. Thanks to the use of masks of random sizes during training, the neural networks of PNNS well adapt to the available context that may vary, depending on the position of the image block to be predicted. When integrating PNNS into a H.265 codec, PSNR-rate performance gains going from 1.46% to 5.20% are obtained. These gains are on average 0.99% larger than those of prior neural network based methods. Unlike the H.265 intra prediction modes, which are each specialized in predicting a specific texture, the proposed PNNS can model a large set of complex textures.
Break the Ceiling: Stronger Multi-scale Deep Graph Convolutional Networks

Recently, neural network based approaches have achieved significant progress for solving large, complex, graph-structured problems. Nevertheless, the advantages of multi-scale information and deep architectures have not been sufficiently exploited. In this paper, we first analyze key factors constraining the expressive power of existing Graph Convolutional Networks (GCNs), including the activation function and shallow learning mechanisms. Then, we generalize spectral graph convolution and deep GCN in block Krylov subspace forms, upon which we devise two architectures, both scalable in depth however making use of multi-scale information differently. On several node classification tasks, the proposed architectures achieve state-of-the-art performance.
Reinforced Transformer for Medical Image Captioning

Computerized medical image report generation is of great significance in automating the workflow of medical diagnosis and treatment for reducing health disparities. However, this task presents several challenges, where the generated medical image report should be precise, coherent and contain heterogeneous information. Current deep learning based medical image captioning models rely on recurrent neural networks and only extract top-down visual features, which make them slow and prone to generate incoherent and hard to comprehend reports. To tackle this challenging problem, this paper proposes a hierarchical Transformer based medical imaging report generation model. Our proposed model consists of two parts: (1) An Image Encoder extracts heuristic visual features by a bottom-up attention mechanism; (2) a non-recurrent Captioning Decoder improves the computational efficiency by parallel computation. The former identifies regions of interest via a bottom-up attention module and extracts top-down visual features. Then the Transformer based captioning decoder generates a coherent paragraph of medical imaging report. The proposed model is trained by using a self-critical reinforcement learning method. We evaluate the proposed model on publicly available datasets of IU X-ray. The experiment results show that our proposed model has improved the performance in BLEU-1 by more than 50% compared with other state-of-the-art image captioning methods.
A Simple and Efficient Federated Recommender System

Federated Learning (FL) is recently explored as a machine learning paradigm to communally gain generalizable knowledge from the data available in a collection of edge devices without the requirement to transfer the data. FL gives rise to the opportunity to train models on edge devices while preserving user's privacy as the data never leaves user's premises. In this paper, we introduce a simple yet efficient extension of FL for recommender systems to improve on personalization and discuss closely-related meta-learning algorithms. Compared to state-of-the-art federated recommenders, our proposed algorithm is simpler and more robust in real-life scenarios. Through experiments on benchmark data, we evaluate our algorithm in root mean squared error (RMSE) of user's rating prediction.
Production of photonic universal quantum gates enhanced by machine learning

We introduce photonic architectures for universal quantum computation. The first step is to produce a resource state which is a superposition of the first four Fock states with a probability greater than or equal to ${10}^{\ensuremath{-}2}$, an increase by a factor of ${10}^{4}$ over standard sequential photon-subtraction techniques. The resource state is produced with near-perfect fidelity from a quantum gadget that uses displaced squeezed vacuum states, interferometers, and photon-number-resolving detectors. The parameters of this gadget are trained using machine learning algorithms for variational circuits. We discuss in detail various aspects of the non-Gaussian state preparation resulting from the numerical experiments. We then propose a notion of resource farms where these gadgets are stacked in parallel, to increase the success probability further. We find a trade-off between the success probability of the farm, the error tolerance, and the number of gadgets. Using the resource states in conventional gate teleportation techniques, we can then implement weak tunable cubic phase gates. The numerical tools that have been developed could potentially be useful for other applications in photonics as well.
A Novel Color Image Encryption Algorithm Based on Hyperchaotic Maps and Mitochondrial DNA Sequences

Multimedia encryption innovation is one of the primary ways of securely and privately guaranteeing the security of media transmission. There are many advantages when utilizing the attributes of chaos, for example, arbitrariness, consistency, ergodicity, and initial condition affectability, for any covert multimedia transmission. Additionally, many more benefits can be introduced with the exceptional space compliance, unique information, and processing capability of real mitochondrial deoxyribonucleic acid (mtDNA). In this article, color image encryption employs a confusion process based on a hybrid chaotic map, first to split each channel of color images into n-clusters; then to create global shuffling over the whole image; and finally, to apply intrapixel shuffling in each cluster, which results in very disordered pixels in the encrypted image. Then, it utilizes the rationale of human mitochondrial genome mtDNA to diffuse the previously confused pixel values. Hypothetical examination and trial results demonstrate that the anticipated scheme exhibits outstanding encryption, as well as successfully opposes chosen/known plain text, statistical, and differential attacks.
Quantum-optimal-control-inspired ansatz for variational quantum algorithms

A central component of variational quantum algorithms (VQA) is the state-preparation circuit, also known as ansatz or variational form. This circuit is most commonly designed to respect the symmetries of the problem Hamiltonian and, in this way, constrain the variational search to a subspace of interest. Here, we show that this approach is not always advantageous by introducing ans\"atze that incorporate symmetry-breaking unitaries. This class of ans\"atze, that we call Quantum-Optimal-Control-inspired Ans\"atze (QOCA), is inspired by the theory of quantum optimal control and leads to an improved convergence of VQAs for some important problems. Indeed, we benchmark QOCA against popular ans\"atze applied to the Fermi-Hubbard model at half-filling and show that our variational circuits can approximate the ground state of this model with significantly higher accuracy and for larger systems. We also show how QOCA can be used to find the ground state of the water molecule and compare the performance of our ansatz against other common choices used for chemistry problems. This work constitutes a first step towards the development of a more general class of symmetry-breaking ans\"atze with applications to physics and chemistry problems.
Neural networks and quantum field theory

We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many asymptotic neural networks are drawn from Gaussian processes, the analog of non-interacting field theories. Moving away from the asymptotic limit yields a non-Gaussian process and corresponds to turning on particle interactions, allowing for the computation of correlation functions of neural network outputs with Feynman diagrams. Minimal non-Gaussian process likelihoods are determined by the most relevant non-Gaussian terms, according to the flow in their coefficients induced by the Wilsonian renormalization group. This yields a direct connection between overparameterization and simplicity of neural network likelihoods. Whether the coefficients are constants or functions may be understood in terms of GP limit symmetries, as expected from 't Hooft's technical naturalness. General theoretical calculations are matched to neural network experiments in the simplest class of models allowing the correspondence. Our formalism is valid for any of the many architectures that becomes a GP in an asymptotic limit, a property preserved under certain types of training.
Immersive Virtual Reality Applications in Schizophrenia Spectrum Therapy: A Systematic Review

(1) Background: Virtual Reality (VR) is a fully immersive computer simulated experience consisting of a three-dimensional interactive virtual environment, through a head-mounted display (HMD) and controller. The use of virtual reality has recently been proposed for the treatment of various psychiatric conditions, including the spectrum of schizophrenia. Our review aims to investigate the current available evidence regarding the use of immersive virtual reality in the treatment of psychotic symptoms. (2) Methods: From April 2019 to June 2020, we conducted a systematic review aimed at identifying therapeutic applications in immersive virtual reality for the spectrum of schizophrenia, searching for relevant studies on Web of Science, EMBASE, PsycINFO and CINHAL. (3) Results: We identified a total of 2601 unique records. Of these, 64 full-text articles were assessed for eligibility, and six out of these met the inclusion criteria and were included in the final systematic review. (4) Conclusions: The available data on immersive virtual reality are currently limited due to the few studies carried out on the topic; however, it has demonstrated its effectiveness and versatility in successfully treating various psychotic symptoms including delusions, hallucinations, or cognitive and social skills. Existing literature agrees on safe, tolerable, and long-term persistence of the therapeutic effects obtained by immersive VR. No serious side effects have been reported. In some specific cases, VR therapy was found to be very effective compared to usual treatment, allowing effective drug free interventions, and therefore without side effects for patients, even in those resistant to normal drug therapies.
Do We Train on Test Data? Purging CIFAR of Near-Duplicates

The CIFAR-10 and CIFAR-100 datasets are two of the most heavily benchmarked datasets in computer vision and are often used to evaluate novel methods and model architectures in the field of deep learning. However, we find that 3.3% and 10% of the images from the test sets of these datasets have duplicates in the training set. These duplicates are easily recognizable by memorization and may, hence, bias the comparison of image recognition techniques regarding their generalization capability. To eliminate this bias, we provide the "fair CIFAR" (ciFAIR) dataset, where we replaced all duplicates in the test sets with new images sampled from the same domain. We then re-evaluate the classification performance of various popular state-of-the-art CNN architectures on these new test sets to investigate whether recent research has overfitted to memorizing data instead of learning abstract concepts. We find a significant drop in classification accuracy of between 9% and 14% relative to the original performance on the duplicate-free test set. The ciFAIR dataset and pre-trained models are available at https://cvjena.github.io/cifair/, where we also maintain a leaderboard.
A Multi-Level Approach to Waste Object Segmentation

We address the problem of localizing waste objects from a color image and an optional depth image, which is a key perception component for robotic interaction with such objects. Specifically, our method integrates the intensity and depth information at multiple levels of spatial granularity. Firstly, a scene-level deep network produces an initial coarse segmentation, based on which we select a few potential object regions to zoom in and perform fine segmentation. The results of the above steps are further integrated into a densely connected conditional random field that learns to respect the appearance, depth, and spatial affinities with pixel-level accuracy. In addition, we create a new RGBD waste object segmentation dataset, MJU-Waste, that is made public to facilitate future research in this area. The efficacy of our method is validated on both MJU-Waste and the Trash Annotation in Context (TACO) dataset.
Decentralized Federated Learning via Mutual Knowledge Transfer

In this article, we investigate the problem of decentralized federated learning (DFL) in Internet of Things (IoT) systems, where a number of IoT clients train models collectively for a common task without sharing their private training data in the absence of a central server. Most of the existing DFL schemes are composed of two alternating steps, i.e., model updating and model averaging. However, averaging model parameters directly to fuse different models at the local clients suffers from client-drift, especially when the training data are heterogeneous across different clients. This leads to slow convergence and degraded learning performance. As a possible solution, we propose the DFL via a mutual knowledge transfer (Def-KT) algorithm, where local clients fuse models by transferring their learned knowledge to each other. Our experiments on the MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 data sets reveal that the proposed Def-KT algorithm significantly outperforms the baseline DFL methods with model averaging, i.e., Combo and FullAvg, especially when the training data are not independent and identically distributed (non-IID) across different clients.
Criticality-Enhanced Quantum Sensing via Continuous Measurement

A new sensing framework is established, harnessing dissipative criticality of open quantum sensors and continuous measurement of the emitted radiation quanta, that promises Heisenberg-limited precision scaling.
Digitainability—Digital Competences Post-COVID-19 for a Sustainable Society

The digitalization of societies, accelerated by the COVID-19 pandemic, is an unstoppable process. This paper seeks to answer the question: what post-COVID-19 digital competences are needed for a sustainable society? It also aims to analyze the digitalization processes in education for shaping a sustainable digital society. A bibliographic search was performed on some of the most relevant international databases of scientific literature and the selected documents were analyzed through a content analysis. It is concluded that digital education has experienced a strong increase, reinforced by COVID-19, shaping the digital presence in all dimensions of life. However, it is not sufficient to assume that the new generations are naturally engaged in and can master digital social sustainability. The results demonstrate the importance of literacy and the unavoidable promotion of sustainability in a digital society. However, this digitalization of the educational process poses several challenges: it requires both software and hardware conditions, as well as digital literacy as a result of a complex of literacies. It also implies that teachers and students change their standpoints and practices with the attainment of new teaching and learning competences in order to fight the digital divide and to foster the widest possible social inclusion for the promotion of sustainable society—digitainability.