Reviewerly/exHarmony huggingface dataset:
Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition

Clinical named entity recognition aims to identify and classify clinical terms such as diseases, symptoms, treatments, exams, and body parts in electronic health records, which is a fundamental and crucial task for clinical and translational research. In recent years, deep neural networks have achieved significant success in named entity recognition and many other natural language processing tasks. Most of these algorithms are trained end to end, and can automatically learn features from large scale labeled datasets. However, these data-driven methods typically lack the capability of processing rare or unseen entities. Previous statistical methods and feature engineering practice have demonstrated that human knowledge can provide valuable information for handling rare and unseen cases. In this paper, we propose a new model which combines data-driven deep learning approaches and knowledge-driven dictionary approaches. Specifically, we incorporate dictionaries into deep neural networks. In addition, two different architectures that extend the bi-directional long short-term memory neural network and five different feature representation schemes are also proposed to handle the task. Computational results on the CCKS-2017 Task 2 benchmark dataset show that the proposed method achieves the highly competitive performance compared with the state-of-the-art deep learning methods.
Roadmap on quantum light spectroscopy

Conventional spectroscopy uses classical light to detect matter properties through the variation of its response with frequencies or time delays.Quantum light opens up new avenues for spectroscopy by utilizing parameters of the quantum state of light as novel control knobs and through the variation of photon statistics by coupling to matter.This Roadmap article focuses on using quantum light as a powerful sensing and spectroscopic tool to reveal novel information about complex molecules that is not accessible by classical light.It aims at bridging the quantum optics and spectroscopy communities which normally have opposite goals: manipulating complex light states with simple matter e.g.qubits vs. studying complex molecules with simple classical light, respectively.Articles cover advances in the generation and manipulation of state-of-the-art quantum light sources along with applications to sensing, spectroscopy, imaging and interferometry.
#Drugsforsale: An exploration of the use of social media and encrypted messaging apps to supply and access drugs

The use of new technology is frequently harnessed by drug suppliers to both increase profits and reduce risk. While a growing body of research has investigated drug sales through online pharmacies and cryptomarkets, despite growing media interest, no published research exists on how smartphone-enabled social media and messaging applications (‘apps’) are utilised in the drug economy. This study analyses the ways such apps (e.g. Snapchat, Instagram and WhatsApp) are utilised to supply and access drugs. Three data collection methods were employed: an international online survey of 358 drug users that had either used or considered using apps to access drugs; ‘rapid’ interviews (n = 20) with a similar population; and in-depth interviews (n = 27). Key issues explored were the perceived benefits and risks associated with sourcing drugs through apps, with specific attention paid to novel supply and purchasing practices. Apps appear to provide a quick, convenient method for connecting buyer and seller. They were often viewed as a valuable intermediary option between cryptomarkets and street dealing, providing ‘secure’ features and the opportunity to preview product without the requirement for technical expertise. Apps are used in a range of novel and diverse ways, including as social networking spaces in which drugs are advertised, and as encrypted messaging services for communicating with known sellers and arranging transactions. Key anxieties related to potential for exposure to law enforcement and legitimacy of substances. Though ‘social supply’ through friends is still typically preferred and there is a degree of wariness toward app-mediated supply, our data indicate that apps are fast becoming a viable option for accessing drugs. Apps can provide an easily accessible platform that connects buyers with commercial drug suppliers and substances that may otherwise remain elusive. Potential harms can be reduced through the provision of information which demystify common-sense assumptions that apps are secure and that this ‘visual’ drug economy promotes safer purchasing practices.
Multi-Agent Pathfinding: Definitions, Variants, and Benchmarks

The multi-agent pathfinding problem (MAPF) is the fundamental problem of planning paths for multiple agents, where the key constraint is that the agents will be able to follow these paths concurrently without colliding with each other. Applications of MAPF include automated warehouses, autonomous vehicles, and robotics. Research on MAPF has been flourishing in the past couple of years. Different MAPF research papers assume different sets of assumptions, e.g., whether agents can traverse the same road at the same time, and have different objective functions, e.g., minimize makespan or sum of agents' actions costs. These assumptions and objectives are sometimes implicitly assumed or described informally. This makes it difficult for establishing appropriate baselines for comparison in research papers, as well as making it difficult for practitioners to find the papers relevant to their concrete application. This paper aims to fill this gap and facilitate future research and practitioners by providing a unifying terminology for describing the common MAPF assumptions and objectives. In addition, we also provide pointers to two MAPF benchmarks. In particular, we introduce a new grid-based benchmark for MAPF, and demonstrate experimentally that it poses a challenge to contemporary MAPF algorithms.
Combining structured and unstructured data for predictive models: a deep learning approach

Abstract Background The broad adoption of electronic health records (EHRs) provides great opportunities to conduct health care research and solve various clinical problems in medicine. With recent advances and success, methods based on machine learning and deep learning have become increasingly popular in medical informatics. However, while many research studies utilize temporal structured data on predictive modeling, they typically neglect potentially valuable information in unstructured clinical notes. Integrating heterogeneous data types across EHRs through deep learning techniques may help improve the performance of prediction models. Methods In this research, we proposed 2 general-purpose multi-modal neural network architectures to enhance patient representation learning by combining sequential unstructured notes with structured data. The proposed fusion models leverage document embeddings for the representation of long clinical note documents and either convolutional neural network or long short-term memory networks to model the sequential clinical notes and temporal signals, and one-hot encoding for static information representation. The concatenated representation is the final patient representation which is used to make predictions. Results We evaluate the performance of proposed models on 3 risk prediction tasks (i.e. in-hospital mortality, 30-day hospital readmission, and long length of stay prediction) using derived data from the publicly available Medical Information Mart for Intensive Care III dataset. Our results show that by combining unstructured clinical notes with structured data, the proposed models outperform other models that utilize either unstructured notes or structured data only. Conclusions The proposed fusion models learn better patient representation by combining structured and unstructured data. Integrating heterogeneous data types across EHRs helps improve the performance of prediction models and reduce errors.
Americans’ perceptions of privacy and surveillance in the COVID-19 pandemic

Objective To study the U.S. public’s attitudes toward surveillance measures aimed at curbing the spread of COVID-19, particularly smartphone applications (apps) that supplement traditional contact tracing. Method We deployed a survey of approximately 2,000 American adults to measure support for nine COVID-19 surveillance measures. We assessed attitudes toward contact tracing apps by manipulating six different attributes of a hypothetical app through a conjoint analysis experiment. Results A smaller percentage of respondents support the government encouraging everyone to download and use contact tracing apps (42%) compared with other surveillance measures such as enforcing temperature checks (62%), expanding traditional contact tracing (57%), carrying out centralized quarantine (49%), deploying electronic device monitoring (44%), or implementing immunity passes (44%). Despite partisan differences on a range of surveillance measures, support for the government encouraging digital contact tracing is indistinguishable between Democrats (47%) and Republicans (46%), although more Republicans oppose the policy (39%) compared to Democrats (27%). Of the app features we tested in our conjoint analysis experiment, only one had statistically significant effects on the self-reported likelihood of downloading the app: decentralized data architecture increased the likelihood by 5.4 percentage points. Conclusion Support for public health surveillance policies to curb the spread of COVID-19 is relatively low in the U.S. Contact tracing apps that use decentralized data storage, compared with those that use centralized data storage, are more accepted by the public. While respondents’ support for expanding traditional contact tracing is greater than their support for the government encouraging the public to download and use contact tracing apps, there are smaller partisan differences in support for the latter policy.
Full-stack, real-system quantum computer studies

In recent years, Quantum Computing (QC) has progressed to the point where small working prototypes are available for use. Termed Noisy Intermediate-Scale Quantum (NISQ) computers, these prototypes are too small for large benchmarks or even for Quantum Error Correction (QEC), but they do have sufficient resources to run small benchmarks, particularly if compiled with optimizations to make use of scarce qubits and limited operation counts and coherence times. QC has not yet, however, settled on a particular preferred device implementation technology, and indeed different NISQ prototypes implement qubits with very different physical approaches and therefore widely-varying device and machine characteristics.
Star Graph Neural Networks for Session-based Recommendation

Session-based recommendation is a challenging task. Without access to a user's historical user-item interactions, the information available in an ongoing session may be very limited. Previous work on session-based recommendation has considered sequences of items that users have interacted with sequentially. Such item sequences may not fully capture complex transition relationship between items that go beyond inspection order. Thus graph neural network (GNN) based models have been proposed to capture the transition relationship between items. However, GNNs typically propagate information from adjacent items only, thus neglecting information from items without direct connections. Importantly, GNN-based approaches often face serious overfitting problems. We propose Star Graph Neural Networks with Highway Networks (SGNN-HN) for session-based recommendation. The proposed SGNN-HN applies a star graph neural network (SGNN) to model the complex transition relationship between items in an ongoing session. To avoid overfitting, we employ highway networks (HN) to adaptively select embeddings from item representations. Finally, we aggregate the item embeddings generated by the SGNN in an ongoing session to represent a user's final preference for item prediction. Experiments on two public benchmark datasets show that SGNN-HN can outperform state-of-the-art models in terms of [email protected] and [email protected] for session-based recommendation.
Implementation and relevance of FAIR data principles in biopharmaceutical R&amp;D

Biopharmaceutical industry R&D, and indeed other life sciences R&D such as biomedical, environmental, agricultural and food production, is becoming increasingly data-driven and can significantly improve its efficiency and effectiveness by implementing the FAIR (findable, accessible, interoperable, reusable) guiding principles for scientific data management and stewardship. By so doing, the plethora of new and powerful analytical tools such as artificial intelligence and machine learning will be able, automatically and at scale, to access the data from which they learn, and on which they thrive. FAIR is a fundamental enabler for digital transformation.
Lagging Inference Networks and Posterior Collapse in Variational Autoencoders

The variational autoencoder (VAE) is a popular combination of deep latent variable model and accompanying variational learning technique. By using a neural inference network to approximate the model's on latent variables, VAEs efficiently parameterize a lower bound on marginal data likelihood that can be optimized directly via gradient methods. In practice, however, VAE training often results in a degenerate local optimum known as posterior where the model learns to ignore the latent variable and the approximate mimics the prior. In this paper, we investigate collapse from the perspective of training dynamics. We find that during the initial stages of training the inference network fails to approximate the model's true posterior, which is a moving target. As a result, the model is encouraged to ignore the latent encoding and collapse occurs. Based on this observation, we propose an extremely simple modification to VAE training to reduce inference lag: depending on the model's current mutual information between latent variable and observation, we aggressively optimize the inference network before performing each model update. Despite introducing neither new model components nor significant complexity over basic VAE, our approach is able to avoid the problem of collapse that has plagued a large amount of previous work. Empirically, our approach outperforms strong autoregressive baselines on text and image benchmarks in terms of held-out likelihood, and is competitive with more complex techniques for avoiding collapse while being substantially faster.
A Comprehensive Survey of the Harmony Search Algorithm in Clustering Applications

The Harmony Search Algorithm (HSA) is a swarm intelligence optimization algorithm which has been successfully applied to a broad range of clustering applications, including data clustering, text clustering, fuzzy clustering, image processing, and wireless sensor networks. We provide a comprehensive survey of the literature on HSA and its variants, analyze its strengths and weaknesses, and suggest future research directions.
Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis

Cross-domain sentiment classification aims to address the lack of massive amounts of labeled data.It demands to predict sentiment polarity on a target domain utilizing a classifier learned from a source domain.In this paper, we investigate how to efficiently apply the pre-training language model BERT on the unsupervised domain adaptation.Due to the pre-training task and corpus, BERT is taskagnostic, which lacks domain awareness and can not distinguish the characteristic of source and target domain when transferring knowledge.To tackle these problems, we design a post-training procedure, which contains the target domain masked language model task and a novel domain-distinguish pre-training task.The post-training procedure will encourage BERT to be domain-aware and distill the domain-specific features in a self-supervised way.Based on this, we could then conduct the adversarial training to derive the enhanced domain-invariant features.Extensive experiments on Amazon dataset show that our model outperforms state-of-the-art methods by a large margin.The ablation study demonstrates that the remarkable improvement is not only from BERT but also from our method.
Attention is Not Only a Weight: Analyzing Transformers with Vector Norms

Attention is a key component of Transformers, which have recently achieved considerable success in natural language processing. Hence, attention is being extensively studied to investigate various linguistic capabilities of Transformers, focusing on analyzing the parallels between attention weights and specific linguistic phenomena. This paper shows that attention weights alone are only one of the two factors that determine the output of attention and proposes a norm-based analysis that incorporates the second factor, the norm of the transformed input vectors. The findings of our norm-based analyses of BERT and a Transformer-based neural machine translation system include the following: (i) contrary to previous studies, BERT pays poor attention to special tokens, and (ii) reasonable word alignment can be extracted from attention mechanisms of Transformer. These findings provide insights into the inner workings of Transformers.
Diffusion Models in Vision: A Survey

Denoising diffusion models represent a recent emerging topic in computer vision, demonstrating remarkable results in the area of generative modeling. A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage. In the forward diffusion stage, the input data is gradually perturbed over several steps by adding Gaussian noise. In the reverse stage, a model is tasked at recovering the original input data by learning to gradually reverse the diffusion process, step by step. Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens, i.e., low speeds due to the high number of steps involved during sampling. In this survey, we provide a comprehensive review of articles on denoising diffusion models applied in vision, comprising both theoretical and practical contributions in the field. First, we identify and present three generic diffusion modeling frameworks, which are based on denoising diffusion probabilistic models, noise conditioned score networks, and stochastic differential equations. We further discuss the relations between diffusion models and other deep generative models, including variational auto-encoders, generative adversarial networks, energy-based models, autoregressive models and normalizing flows. Then, we introduce a multi-perspective categorization of diffusion models applied in computer vision. Finally, we illustrate the current limitations of diffusion models and envision some interesting directions for future research.
Realization of an Error-Correcting Surface Code with Superconducting Qubits

Quantum error correction is a critical technique for transitioning from noisy intermediate-scale quantum devices to fully fledged quantum computers. The surface code, which has a high threshold error rate, is the leading quantum error correction code for two-dimensional grid architecture. So far, the repeated error correction capability of the surface code has not been realized experimentally. Here, we experimentally implement an error-correcting surface code, the distance-three surface code which consists of 17 qubits, on the Zuchongzhi 2.1 superconducting quantum processor. By executing several consecutive error correction cycles, the logical error can be significantly reduced after applying corrections, achieving the repeated error correction of surface code for the first time. This experiment represents a fully functional instance of an error-correcting surface code, providing a key step on the path towards scalable fault-tolerant quantum computing.
ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models

We collect a large real-world test set, ObjectNet, for object recognition with controls where object backgrounds, rotations, and imaging viewpoints are random. Most scientific experiments have controls, confounds which are removed from the data, to ensure that subjects cannot perform a task by exploiting trivial correlations in the data. Historically, large machine learning and computer vision datasets have lacked such controls. This has resulted in models that must be fine-tuned for new datasets and perform better on datasets than in real-world applications. When tested on ObjectNet, object detectors show a 40-45% drop in performance, with respect to their performance on other benchmarks, due to the controls for biases. Controls make ObjectNet robust to fine-tuning showing only small performance increases. We develop a highly automated platform that enables gathering datasets with controls by crowdsourcing image capturing and annotation. ObjectNet is the same size as the ImageNet test set (50,000 images), and by design does not come paired with a training set in order to encourage generalization. The dataset is both easier than ImageNet (objects are largely centered and unoccluded) and harder (due to the controls). Although we focus on object recognition here, data with controls can be gathered at scale using automated tools throughout machine learning to generate datasets that exercise models in new ways thus providing valuable feedback to researchers. This work opens up new avenues for research in generalizable, robust, and more human-like computer vision and in creating datasets where results are predictive of real-world performance.
Calibration and validation of accelerometer-based activity monitors: A systematic review of machine-learning approaches

Objective measures using accelerometer-based activity monitors have been extensively used in physical activity (PA) and sedentary behavior (SB) research. To measure PA and SB precisely, the field is shifting towards machine learning-based (ML) approaches for calibration and validation of accelerometer-based activity monitors. Nevertheless, various parameters regarding the use and development of ML-based models, including data type (raw acceleration data versus activity counts), sampling frequency, window size, input features, ML technique, accelerometer placement, and free-living settings, affect the predictive ability of ML-based models. The effects of these parameters on ML-based models have remained elusive, and will be systematically reviewed here. The open challenges were identified and recommendations are made for future studies and directions. We conducted a systematic search of PubMed and Scopus databases to identify studies published before July 2017 that used ML-based techniques for calibration and validation of accelerometer-based activity monitors. Additional articles were manually identified from references in the identified articles. Results: A total of 62 studies were eligible to be included in the review, comprising 48 studies that calibrated and validated ML-based models for predicting the type and intensity of activities, and 22 studies for predicting activity energy expenditure. It appears that various ML-based techniques together with raw acceleration data sampled at 20–30 Hz provide the opportunity of predicting the type and intensity of activities, as well as activity energy expenditure with comparable overall predictive accuracies regardless of accelerometer placement. However, the high predictive accuracy of laboratory-calibrated models is not reproducible in free-living settings, due to transitive and unseen activities together with differences in acceleration signals.