Reviewerly/exHarmony huggingface dataset:
DeepNote-GNN

With the increasing availability of Electronic Health Records (EHRs) and advances in deep learning techniques, developing deep predictive models that use EHR data to solve healthcare problems has gained momentum in recent years. The majority of clinical predictive models benefit from structured data in EHR (e.g., lab measurements and medications). Still, learning clinical outcomes from all possible information sources is one of the main challenges when building predictive models. This work focuses on two sources of information that have been underused by researchers; unstructured data (e.g., clinical notes) and a patient network. We propose a novel hybrid deep learning model, DeepNote-GNN, that integrates clinical notes information and patient network topological structure to improve 30-day hospital readmission prediction. DeepNote-GNN is a robust deep learning framework consisting of two modules: DeepNote and patient network. DeepNote extracts deep representations of clinical notes using a feature aggregation unit on top of a state-of-the-art Natural Language Processing (NLP) technique - BERT. By exploiting these deep representations, a patient network is built, and Graph Neural Network (GNN) is used to train the network for hospital readmission predictions. Performance evaluation on the MIMIC-III dataset demonstrates that DeepNote-GNN achieves superior results compared to the state-of-the-art baselines on the 30-day hospital readmission task. We extensively analyze the DeepNote-GNN model to illustrate the effectiveness and contribution of each component of it. The model analysis shows that patient network has a significant contribution to the overall performance, and DeepNote-GNN is robust and can consistently perform well on the 30-day readmission prediction task.
Visualizing Multimodal Deep Learning for Lesion Prediction

A U-Net is a type of convolutional neural network that has been shown to output impressive results in medical imaging segmentation tasks. Still, neural networks in general form a black box that is hard to interpret, especially by noncomputer scientists. This work provides a visual system that allows users to examine U-Nets that were trained to predict brain lesions caused by stroke using multimodal imaging. We provide several visualization views that allow users to load trained U-Nets, run them on different patient data, and examine the results while visually following the computation of the U-Net. With these visualizations, we can provide useful information for our medical collaborators showing how the training database can be improved and which features are best learned by the neural network.
Design, alignment, and calibration of a focused laser differential interferometer

Methods are presented for systematic selection of optical components and dimensions for the design of both single- and double-focused laser differential interferometers (FLDIs). Step-by-step instructions for the assembly and alignment of each FLDI component are given, including detailed figures of the interferometer fringe behavior, as the required infinite-fringe configuration is approached. Calibration and data post-processing techniques are provided in order to obtain quantitative signals from the FLDI.
Privacy-Preserving Microservices in Industrial Internet-of-Things-Driven Smart Applications

Machine learning (ML) algorithms can effectively perform analytics and inferences for building smart applications, such as early detection of diseases in the Industrial Internet of Things (IIoT) and smart healthcare systems. The main components of ML, including training and testing phases, can be decomposed into microservices to improve service quality, along with fast implementation and integration with the edge and cloud services. However, the execution of ML in an edge-cloud environment introduces privacy risks to data owners (e.g., patients). In this article, we present a privacy-preserving ML framework by leveraging microservice technology for safeguarding healthcare IIoT systems. More specifically, we develop a microservice-based distributed privacy-preserving technique using differential privacy (DP) and a radial basis function network (RBFN) to balance between privacy protection and model performance in edge networks. We conduct extensive experiments to evaluate the performance of the proposed technique. The results revealed that DP has a significant influence on the model’s performance and achieves more than 90% accuracy with an epsilon value over 0.4, enhancing data protection and analytics through the implementation of microservices.
A Hierarchical BERT-Based Transfer Learning Approach for Multi-Dimensional Essay Scoring

The task of automated essay scoring (AES) continues to attract interdisciplinary attention due to its commercial and educational importance as well as related research challenges. Traditional AES approaches rely on handcrafted features, which are time-consuming and labor-intensive. Neural network approaches have recently given fantastic results in AES without feature engineering, but they usually require extensive annotated data. Moreover, most of the existing AES models only report a single holistic score without providing diagnostic information about various dimensions of writing quality. Focusing on these issues, we develop a novel approach using multi-task learning (MTL) with fine-tuning Bidirectional Encoder Representations from Transformers (BERT) for multi-dimensional AES tasks. As a state-of-the-art pre-trained language model, a BERT-based approach can improve AES tasks with limited training data. Meanwhile, we deal with long texts by proposing a hierarchical method and using the attention mechanism to automatically determine the contribution of different fractions of the input essay to the final score. For the multi-topic essay scoring tasks on the ASAP dataset, results reveal that our approach outperforms the average quadratic weighted Kappa (QWK) score by 4.5% compared with the strong baseline. We propose a self-collected dataset of Chinese EFL Learners’ Argumentation (CELA) to provide valuable information about writing quality from multiple rating dimensions, including holistic and five analytic scales. For the multi-rating dimensional essay scoring tasks on the CELA dataset, experimental results demonstrate that our model increases the average QWK score by 8.1% compared with the strong baseline.
Learning to Balance the Learning Rates Between Various Modalities via Adaptive Tracking Factor

Multimodal networks with richer information contents should always outperform the unimodal counterparts. In our experiment, however, we observe that this is not always the case. Prior efforts on multimodal tasks mainly tend to design a uniform optimization algorithm for all modalities, and yet only obtain a sub-optimal multimodal representation with the fusion of under-optimized unimodal representations, which are still challenged by performance drop on multimodal networks caused by heterogeneity among modalities. In this work, to remove the slowdowns in performance on multimodal tasks, we decouple the learning procedures of unimodal and multimodal networks by dynamically balancing the learning rates for various modalities, so that the modality-specific optimization algorithm for each modality can be obtained. Specifically, the adaptive tracking factor (ATF) is introduced to adjust the learning rate for each modality on a real-time basis. Furthermore, adaptive convergent equalization (ACE) and bilevel directional optimization (BDO) are proposed to equalize and update the ATF, avoiding sub-optimal unimodal representations due to overfitting or underfitting. Extensive experiments on multimodal sentiment analysis demonstrate that our method achieves superior performance.
Overcoming the uplink limit of satellite-based quantum communication with deterministic quantum teleportation

Satellite-based quantum communication is essential for both foundational quantum physics tests and scalable quantum networks. However, the practical implementation of such application is limited due to the turbulent atmosphere, which is especially challenging for the ground-to-satellite uplink scenario. Here we consider the configuration of the continuous-variable quantum teleportation protocol to overcome both the uplink limit and background-radiation constraints with an unconditional and deterministic transfer, where the propagation of quantum light is affected by the favorable satellite-to-ground downlink (including the effects of absorption, scattering, turbulence, pointing errors, and background light). In particular, we derive the turbulent-induced fidelity that is achievable by the protocol for both nighttime and daytime operations, showing the effectiveness of this approach for various squeezing levels and orbit conditions (satellite altitudes and zenith angles). Finally, several approaches for fidelity enhancement are suggested, in terms of optimizing the transmitted Gaussian entanglement in the free-space scenario.
A graph-based approach for positive and unlabeled learning

Positive and Unlabeled Learning (PUL) uses unlabeled documents and a few positive documents for retrieving a set of "interest" documents from a text collection. Usually, PUL approaches are based on the vector space model. However, when dealing with semi-supervised learning for text classification or information retrieval, graph-based approaches have been proved to outperform vector space model-based approaches. So, in this article, a graph-based approach for PUL is proposed: Label Propagation for Positive and Unlabeled Learning (LP-PUL). The proposed framework consists of three steps: (i) building a similarity graph, (ii) identifying reliable negative documents, and (iii) performing label propagation to classify the remaining unlabeled documents as positive or negative. We carried out experiments to measure the impact of the different choices in each step of the proposed framework. We also demonstrated that the proposal surpasses the classification performance of other PUL (RC-SVM, PU-LP, and PE-PUC) or one-class learning (k-NN-based, k-Means-based, and Dense Autoencoder) algorithms in terms of F1. Considering the best results of any algorithm used in the experimental evaluation, PU-PUL can improve the classification performance from 2%, when using only 1 labeled document, to 28%, when 30 labeled documents are employed.
A variational Bayesian based robust cubature Kalman filter under dynamic model mismatch and outliers interference

This paper proposes a variational Bayesian-based cubature Kalman filter to solve the state estimation problem of nonlinear discrete-time systems under dynamic model mismatch and outliers interference, which occurs in the maneuvering target tracking system. In the proposed filter, the measurement noise is divided into the normal measurement noise and the outlier noise, and two inverse Wishart distributions with different degree of freedom parameters are used to model the unknown normal measurement noise covariance and the unknown outlier noise covariance, respectively. To overcome outliers interference, the judgment factor which follows the Bernoulli-beta distribution is designed to identify the type of measurement noise. Meanwhile, the input vector which follows the Gaussian distribution is introduced to realize the dynamic model revision. The state, the judgment factor, the normal measurement noise covariance, the outlier noise covariance, and the input vector are jointly estimated under the variational Bayesian framework. The numerical simulation is illustrated to verify the performance of the proposed filter and the corresponding results show that the proposed filter has good state estimation performance and robustness.
Achieving Expert-Level Interpretation of Serum Protein Electrophoresis through Deep Learning Driven by Human Reasoning

Serum protein electrophoresis (SPE) is a common clinical laboratory test, mainly indicated for the diagnosis and follow-up of monoclonal gammopathies. A time-consuming and potentially subjective human expertise is required for SPE analysis to detect possible pitfalls and to provide a clinically relevant interpretation.An expert-annotated SPE dataset of 159 969 entries was used to develop SPECTR (serum protein electrophoresis computer-assisted recognition), a deep learning-based artificial intelligence, which analyzes and interprets raw SPE curves produced by an analytical system into text comments that can be used by practitioners. It was designed following academic recommendations for SPE interpretation, using a transparent architecture avoiding the "black box" effect. SPECTR was validated on an external, independent cohort of 70 362 SPEs and challenged by a panel of 9 independent experts from other hospital centers.SPECTR was able to identify accurately both quantitative abnormalities (r ≥ 0.98 for fractions quantification) and qualitative abnormalities [receiver operating characteristic-area under curve (ROC-AUC) ≥ 0.90 for M-spikes, restricted heterogeneity of immunoglobulins, and beta-gamma bridging]. Furthermore, it showed highly accurate at both detecting (ROC-AUC ≥ 0.99) and quantifying (r = 0.99) M-spikes. It proved highly reproducible and resilient to minor variations and its agreement with human experts was higher (κ = 0.632) than experts between each other (κ = 0.624).SPECTR is an algorithm based on artificial intelligence suitable to high-throughput SPEs analyses and interpretation. It aims at improving SPE reproducibility and reliability. It is freely available in open access through an online tool providing fully editable validation assistance for SPE.
Time Series Prediction Using Deep Learning Methods in Healthcare

Traditional machine learning methods face unique challenges when applied to healthcare predictive analytics. The high-dimensional nature of healthcare data necessitates labor-intensive and time-consuming processes when selecting an appropriate set of features for each new task. Furthermore, machine learning methods depend heavily on feature engineering to capture the sequential nature of patient data, oftentimes failing to adequately leverage the temporal patterns of medical events and their dependencies. In contrast, recent deep learning (DL) methods have shown promising performance for various healthcare prediction tasks by specifically addressing the high-dimensional and temporal challenges of medical data. DL techniques excel at learning useful representations of medical concepts and patient clinical data as well as their nonlinear interactions from high-dimensional raw or minimally processed healthcare data. In this article, we systematically reviewed research works that focused on advancing deep neural networks to leverage patient structured time series data for healthcare prediction tasks. To identify relevant studies, we searched MEDLINE, IEEE, Scopus, and ACM Digital Library for relevant publications through November 4, 2021. Overall, we found that researchers have contributed to deep time series prediction literature in 10 identifiable research streams: DL models, missing value handling, addressing temporal irregularity, patient representation, static data inclusion, attention mechanisms, interpretation, incorporation of medical ontologies, learning strategies, and scalability. This study summarizes research insights from these literature streams, identifies several critical research gaps, and suggests future research opportunities for DL applications using patient time series data.
A Novel Approach for Linguistic Steganography Evaluation Based on Artificial Neural Networks

Increasing prevalence and simplicity of using Artificial Intelligence (AI) techniques, Steganography is shifting from conventional model building to AI model building. AI enables computers to learn from their mistakes, adapt to emerging inputs, and carry out human-like activities. Traditional Linguistic Steganographic approaches lack automation, analysis of Cover text and hidden text volume and accuracy. A formal methodology is used in only a few Steganographic approaches. In the vast majority of situations, traditional approaches fail to survive third-party vulnerability. This study looks at evaluation of an AI-based statistical language model for text Steganography. Since the advent of Natural Language Processing (NLP) into the research field, linguistic Steganography has superseded other types of Steganography. This paper proposes the positive aspects of NLP-based Markov chain model for an auto-generative cover text. The embedding rate, volume, and other attributes of Recurrent Neural Networks (RNN) Steganographic schemes are contrasted in this article between RNN-Stega and RNN-generated Lyrics, two RNN methods. Here the RNN model follows Long Short Term Memory (LSTM) neural network. The paper also includes a case study on Artificial Intelligence and Information Security, which discusses history, applications, AI challenges, and how AI can help with security threats and vulnerabilities. The final portion is dedicated to the study’s shortcomings, which may be the subject of future research.
More Robust Dense Retrieval with Contrastive Dual Learning

Dense retrieval conducts text retrieval in the embedding space and has shown many advantages compared to sparse retrieval. Existing dense retrievers optimize representations of queries and documents with contrastive training and map them to the embedding space. The embedding space is optimized by aligning the matched query-document pairs and pushing the negative documents away from the query. However, in such training paradigm, the queries are only optimized to align to the documents and are coarsely positioned, leading to an anisotropic query embedding space. In this paper, we analyze the embedding space distributions and propose an effective training paradigm, Contrastive Dual Learning for Approximate Nearest Neighbor (DANCE) to learn fine-grained query representations for dense retrieval. DANCE incorporates an additional dual training object of query retrieval, inspired by the classic information retrieval training axiom, query likelihood. With contrastive learning, the dual training object of DANCE learns more tailored representations for queries and documents to keep the embedding space smooth and uniform, thriving on the ranking performance of DANCE on the MS MARCO document retrieval task. Different from ANCE that only optimized with the document retrieval task, DANCE concentrates the query embeddings closer to document representations while making the document distribution more discriminative. Such concentrated query embedding distribution assigns more uniform negative sampling probabilities to queries and helps to sufficiently optimize query representations in the query retrieval task. Our codes are released at https://github.com/thunlp/DANCE.
View Generalization for Single Image Textured 3D Models

Humans can easily infer the underlying 3D geometry and texture of an object only from a single 2D image. Current computer vision methods can do this, too, but suffer from view generalization problems – the models inferred tend to make poor predictions of appearance in novel views. As for generalization problems in machine learning, the difficulty is balancing single-view accuracy (cf. training error; bias) with novel view accuracy (cf. test error; variance). We describe a class of models whose geometric rigidity is easily controlled to manage this tradeoff. We describe a cycle consistency loss that improves view generalization (roughly, a model from a generated view should predict the original view well). View generalization of textures requires that models share texture information, so a car seen from the back still has headlights because other cars have headlights. We describe a cycle consistency loss that encourages model textures to be aligned, so as to encourage sharing. We compare our method against the state-of-the-art method and show both qualitative and quantitative improvements.
Seminar Learning for Click-Level Weakly Supervised Semantic Segmentation

Annotation burden has become one of the biggest barriers to semantic segmentation. Approaches based on click-level annotations have therefore attracted increasing attention due to their superior trade-off between supervision and annotation cost. In this paper, we propose seminar learning, a new learning paradigm for semantic segmentation with click-level supervision. The fundamental rationale of seminar learning is to leverage the knowledge from different networks to compensate for insufficient information provided in click-level annotations. Mimicking a seminar, our seminar learning involves a teacher-student and a student-student module, where a student can learn from both skillful teachers and other students. The teacher-student module uses a teacher network based on the exponential moving average to guide the training of the student network. In the student-student module, heterogeneous pseudo-labels are proposed to bridge the transfer of knowledge among students to enhance each other’s performance. Experimental results demonstrate the effectiveness of seminar learning, which achieves the new state-of-the-art performance of 72.51% (mIOU), surpassing previous methods by a large margin of up to 16.88% on the Pascal VOC 2012 dataset.
Blockchain Technology in China's Digital Economy

Compared with the traditional industrial economy, the Chinese digital economy uses brand-new production factors and production organization methods to bring changes to human society and promote the transformation of the economy. This chapter aims to explore the practical problems of adopting blockchain technology in China's digital economy and study how different cities (managed by various local governments) enhance their unique financial technology ecosystem's economic performance and promote RegTech policy in order to improve the digital economy under the central government's institutional setting. This chapter in turn analyzes the recent cases of blockchain in China's financial industry, compares the application and development of the latest financial technology related policies in major cities, and demonstrates how these regulations can promote the development of blockchain technology in the transformation of China's digital economy.
The Potential of Machine Learning for a More Responsible Sourcing of Critical Raw Materials

The digitization and automation of the raw material sector is required to attain the targets set by the Paris Agreements and support the sustainable development goals defined by the United Nations. While many aspects of the industry will be affected, most of the technological innovations will require smart imaging sensors. In this review, we assess the relevant recent developments of Machine Learning for the processing of imaging sensor data. We first describe the main imagers and the acquired data types as well as the platforms on which they can be installed. We briefly describe radiometric and geometric corrections as these procedures have been already described extensively in previous works. We focus on the description of innovative processing workflows and illustrate the most prominent approaches with examples. We also provide a list of available resources, codes, and libraries for researchers at different levels, from students to senior researchers, willing to explore novel methodologies on the challenging topics of raw material extraction, classification, and process automatization.