Reviewerly/exHarmony huggingface dataset:
Deep learning for electronic health records: A comparative review of multiple deep neural architectures

Despite the recent developments in deep learning models, their applications in clinical decision-support systems have been very limited. Recent digitalisation of health records, however, has provided a great platform for the assessment of the usability of such techniques in healthcare. As a result, the field is starting to see a growing number of research papers that employ deep learning on electronic health records (EHR) for personalised prediction of risks and health trajectories. While this can be a promising trend, vast paper-to-paper variability (from data sources and models they use to the clinical questions they attempt to answer) have hampered the field’s ability to simply compare and contrast such models for a given application of interest. Thus, in this paper, we aim to provide a comparative review of the key deep learning architectures that have been applied to EHR data. Furthermore, we also aim to: (1) introduce and use one of the world’s largest and most complex linked primary care EHR datasets (i.e., Clinical Practice Research Datalink, or CPRD) as a new asset for training such data-hungry models; (2) provide a guideline for working with EHR data for deep learning; (3) share some of the best practices for assessing the “goodness” of deep-learning models in clinical risk prediction; (4) and propose future research ideas for making deep learning models more suitable for the EHR data. Our results highlight the difficulties of working with highly imbalanced datasets, and show that sequential deep learning architectures such as RNN may be more suitable to deal with the temporal nature of EHR.
3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation

Accurate segmentation of infant brain images into different regions of interest is one of the most important fundamental steps in studying early brain development. In the isointense phase (approximately 6-8 months of age), white matter and gray matter exhibit similar levels of intensities in magnetic resonance (MR) images, due to the ongoing myelination and maturation. This results in extremely low tissue contrast and thus makes tissue segmentation very challenging. Existing methods for tissue segmentation in this isointense phase usually employ patch-based sparse labeling on single modality. To address the challenge, we propose a novel 3-D multimodal fully convolutional network (FCN) architecture for segmentation of isointense phase brain MR images. Specifically, we extend the conventional FCN architectures from 2-D to 3-D, and, rather than directly using FCN, we intuitively integrate coarse (naturally high-resolution) and dense (highly semantic) feature maps to better model tiny tissue regions, in addition, we further propose a transformation module to better connect the aggregating layers; we also propose a fusion module to better serve the fusion of feature maps. We compare the performance of our approach with several baseline and state-of-the-art methods on two sets of isointense phase brain images. The comparison results show that our proposed 3-D multimodal FCN model outperforms all previous methods by a large margin in terms of segmentation accuracy. In addition, the proposed framework also achieves faster segmentation results compared to all other methods. Our experiments further demonstrate that: 1) carefully integrating coarse and dense feature maps can considerably improve the segmentation performance; 2) batch normalization can speed up the convergence of the networks, especially when hierarchical feature aggregations occur; and 3) integrating multimodal information can further boost the segmentation performance.
A survey on evolutionary computation for complex continuous optimization

Abstract Complex continuous optimization problems widely exist nowadays due to the fast development of the economy and society. Moreover, the technologies like Internet of things, cloud computing, and big data also make optimization problems with more challenges including M any-dimensions, M any-changes, M any-optima, M any-constraints, and M any-costs. We term these as 5-M challenges that exist in large-scale optimization problems, dynamic optimization problems, multi-modal optimization problems, multi-objective optimization problems, many-objective optimization problems, constrained optimization problems, and expensive optimization problems in practical applications. The evolutionary computation (EC) algorithms are a kind of promising global optimization tools that have not only been widely applied for solving traditional optimization problems, but also have emerged booming research for solving the above-mentioned complex continuous optimization problems in recent years. In order to show how EC algorithms are promising and efficient in dealing with the 5-M complex challenges, this paper presents a comprehensive survey by proposing a novel taxonomy according to the function of the approaches, including reducing problem difficulty , increasing algorithm diversity , accelerating convergence speed , reducing running time , and extending application field . Moreover, some future research directions on using EC algorithms to solve complex continuous optimization problems are proposed and discussed. We believe that such a survey can draw attention, raise discussions, and inspire new ideas of EC research into complex continuous optimization problems and real-world applications.
Cloze-driven Pretraining of Self-attention Networks

We present a new approach for pretraining a bi-directional transformer model that provides significant performance gains across a variety of language understanding problems.Our model solves a cloze-style word reconstruction task, where each word is ablated and must be predicted given the rest of the text.Experiments demonstrate large performance gains on GLUE and new state of the art results on NER as well as constituency parsing benchmarks, consistent with BERT.We also present a detailed analysis of a number of factors that contribute to effective pretraining, including data domain and size, model capacity, and variations on the cloze objective.
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

Large-scale pre-trained language models such as BERT have brought significant improvements to NLP applications.However, they are also notorious for being slow in inference, which makes them difficult to deploy in realtime applications.We propose a simple but effective method, DeeBERT, to accelerate BERT inference.Our approach allows samples to exit earlier without passing through the entire model.Experiments show that DeeBERT is able to save up to ∼40% inference time with minimal degradation in model quality.Further analyses show different behaviors in the BERT transformer layers and also reveal their redundancy.Our work provides new ideas to efficiently apply deep transformer-based models to downstream tasks.
Beyond Low-frequency Information in Graph Convolutional Networks

Graph neural networks (GNNs) have been proven to be effective in various network-related tasks. Most existing GNNs usually exploit the low-frequency signals of node features, which gives rise to one fundamental question: is the low-frequency information all we need in the real world applications? In this paper, we first present an experimental investigation assessing the roles of low-frequency and high-frequency signals, where the results clearly show that exploring low-frequency signal only is distant from learning an effective node representation in different scenarios. How can we adaptively learn more information beyond low-frequency information in GNNs? A well-informed answer can help GNNs enhance the adaptability. We tackle this challenge and propose a novel Frequency Adaptation Graph Convolutional Networks (FAGCN) with a self-gating mechanism, which can adaptively integrate different signals in the process of message passing. For a deeper understanding, we theoretically analyze the roles of low-frequency signals and high-frequency signals on learning node representations, which further explains why FAGCN can perform well on different types of networks. Extensive experiments on six real-world networks validate that FAGCN not only alleviates the over-smoothing problem, but also has advantages over the state-of-the-arts.
Revisiting Milgram and Kishino's Reality-Virtuality Continuum

Since its introduction in 1994, Milgram and Kishino's reality-virtuality (RV) continuum has been used to frame virtual and augmented reality research and development. While originally, the RV continuum and the three dimensions of the supporting taxonomy (extent of world knowledge, reproduction fidelity, and extent of presence metaphor) were intended to characterize the capabilities of visual display technology, researchers have embraced the RV continuum while largely ignoring the taxonomy. Considering the leaps in technology made over the last 25 years, revisiting the RV continuum and taxonomy is timely. In reexamining Milgram and Kishino's ideas, we realized, first, that the RV continuum is actually discontinuous; perfect virtual reality cannot be reached. Secondly, mixed reality is broader than previously believed, and, in fact, encompasses conventional virtual reality experiences. Finally, our revised taxonomy adds coherence, accounting for the role of users, which is critical to assessing modern mixed reality experiences. The 3D space created by our taxonomy incorporates familiar constructs such as presence and immersion, and also proposes new constructs that may be important as mixed reality technology matures.
Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy

Abstract Objective To examine the accuracy of artificial intelligence (AI) for the detection of breast cancer in mammography screening practice. Design Systematic review of test accuracy studies. Data sources Medline, Embase, Web of Science, and Cochrane Database of Systematic Reviews from 1 January 2010 to 17 May 2021. Eligibility criteria Studies reporting test accuracy of AI algorithms, alone or in combination with radiologists, to detect cancer in women’s digital mammograms in screening practice, or in test sets. Reference standard was biopsy with histology or follow-up (for screen negative women). Outcomes included test accuracy and cancer type detected. Study selection and synthesis Two reviewers independently assessed articles for inclusion and assessed the methodological quality of included studies using the QUality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. A single reviewer extracted data, which were checked by a second reviewer. Narrative data synthesis was performed. Results Twelve studies totalling 131 822 screened women were included. No prospective studies measuring test accuracy of AI in screening practice were found. Studies were of poor methodological quality. Three retrospective studies compared AI systems with the clinical decisions of the original radiologist, including 79 910 women, of whom 1878 had screen detected cancer or interval cancer within 12 months of screening. Thirty four (94%) of 36 AI systems evaluated in these studies were less accurate than a single radiologist, and all were less accurate than consensus of two or more radiologists. Five smaller studies (1086 women, 520 cancers) at high risk of bias and low generalisability to the clinical context reported that all five evaluated AI systems (as standalone to replace radiologist or as a reader aid) were more accurate than a single radiologist reading a test set in the laboratory. In three studies, AI used for triage screened out 53%, 45%, and 50% of women at low risk but also 10%, 4%, and 0% of cancers detected by radiologists. Conclusions Current evidence for AI does not yet allow judgement of its accuracy in breast cancer screening programmes, and it is unclear where on the clinical pathway AI might be of most benefit. AI systems are not sufficiently specific to replace radiologist double reading in screening programmes. Promising results in smaller studies are not replicated in larger studies. Prospective studies are required to measure the effect of AI in clinical practice. Such studies will require clear stopping rules to ensure that AI does not reduce programme specificity. Study registration Protocol registered as PROSPERO CRD42020213590.
Evaluating Reputation Management Schemes of Internet of Vehicles Based on Evolutionary Game Theory

Conducting reputation management is very important for Internet of vehicles. However, most of the existing work evaluate the effectiveness of their schemes with settled attacking behaviors in their simulation, which cannot represent the real scenarios. In this paper, we propose to consider dynamical and diversity attacking strategies in the simulation of reputation management scheme evaluation. To that end, we apply evolutionary game theory to model the evolution process of malicious users’ attacking strategies, and discuss the methodology of the evaluation simulations. We further apply our evaluation method to a reputation management scheme with multiple utility functions, and discuss the evaluation results. The results indicate that our evaluation method is able to depict the evolving process of the dynamic attacking strategies in a vehicular network, and the final state of the simulation could be used to quantify the protection effectiveness of the reputation management scheme.
A Taxonomy of Blockchain Technologies: Principles of Identification and Classification

A comparative study across the most widely known blockchain technologies is conducted with a bottom-up approach. Blockchains are deconstructed into their building blocks. Each building block is then hierarchically classified into main and subcomponents. Then, varieties of the subcomponents are identified and compared. A taxonomy tree is used to summarise the study and provide a navigation tool across different blockchain architectural configurations.
Communication-Efficient Federated Learning and Permissioned Blockchain for Digital Twin Edge Networks

Emerging technologies, such as mobile-edge computing (MEC) and next-generation communications are crucial for enabling rapid development and deployment of the Internet of Things (IoT). With the increasing scale of IoT networks, how to optimize the network and allocate the limited resources to provide high-quality services remains a major concern. The existing work in this direction mainly relies on models that are of less practical value for resource-limited IoT networks, and can hardly simulate the dynamic systems in real time. In this article, we integrate digital twins with edge networks and propose the digital twin edge networks (DITENs) to fill the gap between physical edge networks and digital systems. Then, we propose a blockchain-empowered federated learning scheme to strengthen communication security and data privacy protection in DITEN. Furthermore, to improve the efficiency of the integrated scheme, we propose an asynchronous aggregation scheme and use digital twin empowered reinforcement learning to schedule relaying users and allocate spectrum resources. Theoretical analysis and numerical results confirm that the proposed scheme can considerably enhance both communication efficiency and data security for IoT applications.
Measuring the perceived benefits of implementing blockchain technology in the banking sector

This study aims to measure the perceived business benefits of blockchain technology implementation in the banking sector and establish factors to measure these benefits. Concerns regarding security, values, and standards are essential to banking operations. Data was collected from 291 respondents who are either blockchain consultants, blockchain marketing experts, or CEOs/business heads of banks that are in the process of advising, consulting, or implementing blockchain technology. Confirmatory factor analysis (CFA) was carried out to assess the reliability and validity of the proposed instrument. The results support the proposed instrument and its five constructs. The scale emerging from this study indicates a good degree of reliability, validity and unidimensionality in each of its constructs. Technologies like blockchain are in their initial stages, and recent advances in blockchain technology may impact our findings. The developed instrument could help give decision makers a foundational view to measure the benefits of implementing blockchain technology before they choose to integrate it in their existing system. The scientific and societal significance of the study based on its practical and theoretical applications is presented at the end.
DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations

Sentence embeddings are an important component of many natural language processing (NLP) systems.Like word embeddings, sentence embeddings are typically learned on large text corpora and then transferred to various downstream tasks, such as clustering and retrieval.Unlike word embeddings, the highest performing solutions for learning sentence embeddings require labelled data, limiting their usefulness to languages and domains where labelled data is abundant.In this paper, we present DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations.Inspired by recent advances in deep metric learning (DML), we carefully design a self-supervised objective for learning universal sentence embeddings that does not require labelled training data.When used to extend the pretraining of transformer-based language models, our approach closes the performance gap between unsupervised and supervised pretraining for universal sentence encoders.Importantly, our experiments suggest that the quality of the learned embeddings scale with both the number of trainable parameters and the amount of unlabelled training data.Our code and pretrained models are publicly available and can be easily adapted to new domains or used to embed unseen text. 1
Multiview Subspace Clustering via Tensorial t-Product Representation

The ubiquitous information from multiple-view data, as well as the complementary information among different views, is usually beneficial for various tasks, for example, clustering, classification, denoising, and so on. Multiview subspace clustering is based on the fact that multiview data are generated from a latent subspace. To recover the underlying subspace structure, a successful approach adopted recently has been sparse and/or low-rank subspace clustering. Despite the fact that existing subspace clustering approaches may numerically handle multiview data, by exploring all possible pairwise correlation within views, high-order statistics that can only be captured by simultaneously utilizing all views are often overlooked. As a consequence, the clustering performance of the multiview data is compromised. To address this issue, in this paper, a novel multiview clustering method is proposed by using t-product in the third-order tensor space. First, we propose a novel tensor construction method to organize multiview tensorial data, to which the tensor-tensor product can be applied. Second, based on the circular convolution operation, multiview data can be effectively represented by a t-linear combination with sparse and low-rank penalty using “self-expressiveness.” Our extensive experimental results on face, object, digital image, and text data demonstrate that the proposed method outperforms the state-of-the-art methods for a range of criteria.
Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs

Document-level relation extraction is a complex human process that requires logical inference to extract relationships between named entities in text.Existing approaches use graphbased neural models with words as nodes and edges as relations between them, to encode relations across sentences.These models are node-based, i.e., they form pair representations based solely on the two target node representations.However, entity relations can be better expressed through unique edge representations formed as paths between nodes.We thus propose an edge-oriented graph neural model for document-level relation extraction.The model utilises different types of nodes and edges to create a document-level graph.An inference mechanism on the graph edges enables to learn intra-and inter-sentence relations using multi-instance learning internally.Experiments on two document-level biomedical datasets for chemical-disease and genedisease associations show the usefulness of the proposed edge-oriented approach. 1
BAS-ADAM: an ADAM based approach to improve the performance of beetle antennae search optimizer

In this paper, we propose enhancements to Beetle Antennae search ( BAS ) algorithm, called BAS-ADAM, to smoothen the convergence behavior and avoid trapping in local-minima for a highly non-convex objective function. We achieve this by adaptively adjusting the step-size in each iteration using the adaptive moment estimation ( ADAM ) update rule. The proposed algorithm also increases the convergence rate in a narrow valley. A key feature of the ADAM update rule is the ability to adjust the step-size for each dimension separately instead of using the same step-size. Since ADAM is traditionally used with gradient-based optimization algorithms, therefore we first propose a gradient estimation model without the need to differentiate the objective function. Resultantly, it demonstrates excellent performance and fast convergence rate in searching for the optimum of non-convex functions. The efficiency of the proposed algorithm was tested on three different benchmark problems, including the training of a high-dimensional neural network. The performance is compared with particle swarm optimizer ( PSO ) and the original BAS algorithm.
2D Logistic-Modulated-Sine-Coupling-Logistic Chaotic Map for Image Encryption

Due to the significant properties of unpredictability, ergodicity, and initial state sensitivity, chaotic system is widely used as a useful tool in image encryption.In this paper, we propose a 2-dimensional logistic-modulated-sine-coupling-logistic chaotic map (LSMCL), where we use the logistic map to modulate Sine map and couple the result of modulation and Sine map together.In terms of chaotic trajectory, Lyapunov exponent, and Kolmogorov entropy, comparing with other existing chaotic maps, we can observe that LSMCL has better chaotic performance.Furthermore, we propose an LSMCL-based image encryption algorithm with two rounds of permutation and diffusion operation.First, we provide a secret key generation procedure to generate the initial values and do permutation operation with the chaotic matrix by LSMCL.Furthermore, in diffusion procedure, we use two different chaotic matrices generated by LSMCL to change the pixel values in row and column.Finally, we provide some theoretical analyses and simulations to confirm the security and the validity of the proposed algorithm.
Transfer Capsule Network for Aspect Level Sentiment Classification

Aspect-level sentiment classification aims to determine the sentiment polarity of a sentence towards an aspect.Due to the high cost in annotation, the lack of aspect-level labeled data becomes a major obstacle in this area.On the other hand, document-level labeled data like reviews are easily accessible from online websites.These reviews encode sentiment knowledge in abundant contexts.In this paper, we propose a Transfer Capsule Network (Tran-sCap) model for transferring document-level knowledge to aspect-level sentiment classification.To this end, we first develop an aspect routing approach to encapsulate the sentence-level semantic representations into semantic capsules from both aspect-level and document-level data.We then extend the dynamic routing approach to adaptively couple the semantic capsules with the class capsules under the transfer learning framework.Experiments on SemEval datasets demonstrate the effectiveness of TransCap.