Reviewerly/exHarmony huggingface dataset:
DPVis: Visual Analytics With Hidden Markov Models for Disease Progression Pathways

Clinical researchers use disease progression models to understand patient status and characterize progression patterns from longitudinal health records. One approach for disease progression modeling is to describe patient status using a small number of states that represent distinctive distributions over a set of observed measures. Hidden Markov models (HMMs) and its variants are a class of models that both discover these states and make inferences of health states for patients. Despite the advantages of using the algorithms for discovering interesting patterns, it still remains challenging for medical experts to interpret model outputs, understand complex modeling parameters, and clinically make sense of the patterns. To tackle these problems, we conducted a design study with clinical scientists, statisticians, and visualization experts, with the goal to investigate disease progression pathways of chronic diseases, namely type 1 diabetes (T1D), Huntington's disease, Parkinson's disease, and chronic obstructive pulmonary disease (COPD). As a result, we introduce DPVis which seamlessly integrates model parameters and outcomes of HMMs into interpretable and interactive visualizations. In this study, we demonstrate that DPVis is successful in evaluating disease progression models, visually summarizing disease states, interactively exploring disease progression patterns, and building, analyzing, and comparing clinically relevant patient subgroups.
Remote sensing image captioning via Variational Autoencoder and Reinforcement Learning

Image captioning, i.e., generating the natural semantic descriptions of given image, is an essential task for machines to understand the content of the image. Remote sensing image captioning is a part of the field. Most of the current remote sensing image captioning models suffered the overfitting problem and failed to utilize the semantic information in images. To this end, we propose a Variational Autoencoder and Reinforcement Learning based Two-stage Multi-task Learning Model (VRTMM) for the remote sensing image captioning task. In the first stage, we finetune the CNN jointly with the Variational Autoencoder. In the second stage, the Transformer generates the text description using both spatial and semantic features. Reinforcement Learning is then applied to enhance the quality of the generated sentences. Our model surpasses the previous state of the art records by a large margin on all seven scores on Remote Sensing Image Caption Dataset. The experiment result indicates our model is effective on remote sensing image captioning and achieves the new state-of-the-art result.
Thermal infrared colorization via conditional generative adversarial network

Transforming a thermal infrared image into a realistic RGB image is a challenging task. In this paper we propose a deep learning method to bridge this gap. We propose learning the transformation mapping using a coarse-to-fine generator that preserves the details. Since the standard mean squared loss cannot penalize the distance between colorized and ground truth images well, we propose a composite loss function that combines content, adversarial, perceptual and total variation losses. The content loss is used to recover global image information while the latter three losses are used to synthesize local realistic textures. Quantitative and qualitative experiments demonstrate that our approach significantly outperforms existing approaches on the KAIST multispectral pedestrian dataset, achieving more plausible RGB images. Our code is available online2.
Inferring multimodal latent topics from electronic health records

Abstract Electronic health records (EHR) are rich heterogeneous collections of patient health information, whose broad adoption provides clinicians and researchers unprecedented opportunities for health informatics, disease-risk prediction, actionable clinical recommendations, and precision medicine. However, EHRs present several modeling challenges, including highly sparse data matrices, noisy irregular clinical notes, arbitrary biases in billing code assignment, diagnosis-driven lab tests, and heterogeneous data types. To address these challenges, we present MixEHR, a multi-view Bayesian topic model. We demonstrate MixEHR on MIMIC-III, Mayo Clinic Bipolar Disorder, and Quebec Congenital Heart Disease EHR datasets. Qualitatively, MixEHR disease topics reveal meaningful combinations of clinical features across heterogeneous data types. Quantitatively, we observe superior prediction accuracy of diagnostic codes and lab test imputations compared to the state-of-art methods. We leverage the inferred patient topic mixtures to classify target diseases and predict mortality of patients in critical conditions. In all comparison, MixEHR confers competitive performance and reveals meaningful disease-related topics.
A GPU-based residual network for medical image classification in smart medicine

The recent advance of high-performance computing techniques like graphics processing unit (GPU) enables large-scale deep learning models for medical image analytics in smart medicine. Smart medicine has made great progress by applying convolutional neural networks (CNNs) like ResNet and VGG-16 to medical image classification. However, various CNN models achieve very limited accuracy in some cases where multiple diseases are revealed in an X-ray image. This paper presents a variant ResNet model by replacing the global average pooling with the adaptive dropout for medical image classification. In order for the presented model to recognize multiple diseases (i.e., multi-label classification), we convert the multi-label classification to N binary classification by training the parameters of the presented model for N times. Finally, experiments are conducted on a GPU Cluster to evaluate the presented model on three datasets, namely Montgomery County chest X-ray set, Shenzhen X-ray set, and NIH chest X-ray set. The results show the presented model achieves a great performance improvement for medical image classification without a significant efficiency reduction compared to the traditional architecture and VGG-16.
Generating Counter Narratives against Online Hate Speech: Data and Strategies

Recently research has started focusing on avoiding undesired effects that come with content moderation, such as censorship and overblocking, when dealing with hatred online.The core idea is to directly intervene in the discussion with textual responses that are meant to counter the hate content and prevent it from further spreading.Accordingly, automation strategies, such as natural language generation, are beginning to be investigated.Still, they suffer from the lack of sufficient amount of quality data and tend to produce generic/repetitive responses.Being aware of the aforementioned limitations, we present a study on how to collect responses to hate effectively, employing large scale unsupervised language models such as GPT-2 for the generation of silver data, and the best annotation strategies/neural architectures that can be used for data filtering before expert validation/postediting.
CookGAN: Causality Based Text-to-Image Synthesis

This paper addresses the problem of text-to-image synthesis from a new perspective, i.e., the cause-and-effect chain in image generation. Causality is a common phenomenon in cooking. The dish appearance changes depending on the cooking actions and ingredients. The challenge of synthesis is that a generated image should depict the visual result of action-on-object. This paper presents a new network architecture, CookGAN, that mimics visual effect in causality chain, preserves fine-grained details and progressively upsamples image. Particularly, a cooking simulator sub-network is proposed to incrementally make changes to food images based on the interaction between ingredients and cooking methods over a series of steps. Experiments on Recipe1M verify that CookGAN manages to generate food images with reasonably impressive inception score. Furthermore, the images are semantically interpretable and manipulable.
The Challenge of Initial Training for Early Childhood Teachers. A Cross Sectional Study of Their Digital Competences

The scarce presence of technologies at the early childhood education level today is a cause for worry. This study aims to provide information on the training of future early childhood education teachers and their relationship with technologies. The work is based on a pretest–posttest methodology through a cross sectional descriptive study. The sample is made up of 535 4th year students of the Degree in Early Childhood Education at the University of Seville. Descriptive and contrast analyses were performed as well as contrast statistics and effect size. The results show that the training received by the students was a key element to improve self-perception of digital competence. There were statistically significant changes between before and after receiving the training. The changes produced always meant an improvement in the students’ self-perception. In the study of their profiles, relevant changes were also identified. Whereas before training subjects were grouped into newcomer and explorer categories, after training they were grouped into the highest profiles: integrator, expert, and pioneer. For this reason, it is necessary to manage training plans to allow future teachers to position themselves at an expert level.
Knowledge Hypergraphs: Prediction Beyond Binary Relations

Knowledge graphs store facts using relations between two entities. In this work, we address the question of link prediction in knowledge hypergraphs where relations are defined on any number of entities. While techniques exist (such as reification) that convert non-binary relations into binary ones, we show that current embedding-based methods for knowledge graph completion do not work well out of the box for knowledge graphs obtained through these techniques. To overcome this, we introduce HSimplE and HypE, two embedding-based methods that work directly with knowledge hypergraphs. In both models, the prediction is a function of the relation embedding, the entity embeddings and their corresponding positions in the relation. We also develop public datasets, benchmarks and baselines for hypergraph prediction and show experimentally that the proposed models are more effective than the baselines.
Bijective S-Boxes Method Using Improved Chaotic Map-Based Heuristic Search and Algebraic Group Structures

This paper presents a hybrid method for the construction of cryptographically strong bijective substitution-boxes by utilizing the merits of chaotic map and algebraic groups. The hybrid method first generates the key-dependent dynamic S-boxes using chaotic heuristic search strategy and then the S-boxes are evolved with the help of potent proposed algebraic group structures. This paper proposes a new improved combination chaotic map to operate initial search strategy. To augment the strength of generated S-boxes, the algebraic group structures are discovered which have the power to improve their cryptographic strength. The performance assessments using standard criterions are rendered to quantify the strengths of proposed bijective S-boxes. The experimental results and comparisons with recent S-box research findings justify the effectiveness and competence of the proposed bijective S-boxes and anticipated hybrid generation method.
Improving Knowledge Tracing via Pre-training Question Embeddings

Knowledge tracing (KT) defines the task of predicting whether students can correctly answer questions based on their historical response. Although much research has been devoted to exploiting the question information, plentiful advanced information among questions and skills hasn't been well extracted, making it challenging for previous work to perform adequately. In this paper, we demonstrate that large gains on KT can be realized by pre-training embeddings for each question on abundant side information, followed by training deep KT models on the obtained embeddings. To be specific, the side information includes question difficulty and three kinds of relations contained in a bipartite graph between questions and skills. To pre-train the question embeddings, we propose to use product-based neural networks to recover the side information. As a result, adopting the pre-trained embeddings in existing deep KT models significantly outperforms state-of-the-art baselines on three common KT datasets.
Image-Chat: Engaging Grounded Conversations

To achieve the long-term goal of machines being able to engage humans in conversation, our models should captivate the interest of their speaking partners.Communication grounded in images, whereby a dialogue is conducted based on a given photo, is a setup naturally appealing to humans (Hu et al., 2014).In this work we study large-scale architectures and datasets for this goal.We test a set of neural architectures using state-of-the-art image and text representations, considering various ways to fuse the components.To test such models, we collect a dataset of grounded human-human conversations, where speakers are asked to play roles given a provided emotional mood or style, as the use of such traits is also a key factor in engagingness (Guo et al., 2019).Our dataset, Image-Chat, consists of 202k dialogues over 202k images using 215 possible style traits.Automatic metrics and human evaluations of engagingness show the efficacy of our approach; in particular, we obtain state-of-the-art performance on the existing IGC task, and our best performing model is almost on par with humans on the Image-Chat test set (preferred 47.7% of the time).
Training Noise-Robust Deep Neural Networks via Meta-Learning

Label noise may significantly degrade the performance of Deep Neural Networks (DNNs). To train noise-robust DNNs, Loss correction (LC) approaches have been introduced. LC approaches assume the noisy labels are corrupted from clean (ground-truth) labels by an unknown noise transition matrix T. The backbone DNNs and T can be trained separately, where T is approximated with prior knowledge. For example, T is constructed by stacking the maximum or mean predictions of the samples from each class. In this work, we pro- pose a new loss correction approach, named as Meta Loss Correction (MLC), to directly learn T from data via the meta-learning framework. The MLC is model-agnostic and learns T from data rather than heuristically approximates it using prior knowledge. Extensive evaluations are conducted on computer vision (MNIST, CIFAR-10, CIFAR-100, Clothing1M) and natural language processing (Twitter) datasets. The experimental results show that MLC achieves very competitive performance against state-of-the-art approaches.
Generative Adversarial Networks for Spatio-temporal Data: A Survey

Generative Adversarial Networks (GANs) have shown remarkable success in producing realistic-looking images in the computer vision area. Recently, GAN-based techniques are shown to be promising for spatio-temporal-based applications such as trajectory prediction, events generation, and time-series data imputation. While several reviews for GANs in computer vision have been presented, no one has considered addressing the practical applications and challenges relevant to spatio-temporal data. In this article, we have conducted a comprehensive review of the recent developments of GANs for spatio-temporal data. We summarise the application of popular GAN architectures for spatio-temporal data and the common practices for evaluating the performance of spatio-temporal applications with GANs. Finally, we point out future research directions to benefit researchers in this area.
Continuous-variable gate teleportation and bosonic-code error correction

We examine continuous-variable gate teleportation using entangled states made from pure product states sent through a beam splitter. We show that such states are Choi states for a (typically) nonunitary gate, and we derive the associated Kraus operator for teleportation, which can be used to realize non-Gaussian, nonunitary quantum operations on an input state. With this result, we show how gate teleportation is used to perform error correction on bosonic qubits encoded using the Gottesman-Kitaev-Preskill (GKP) code. This result is presented in the context of deterministically produced macronode cluster states, generated by constant-depth linear optical networks, supplemented with a probabilistic supply of GKP states. The upshot of our technique is that state injection for both gate teleportation and error correction can be achieved without active squeezing operations---an experimental bottleneck for quantum optical implementations.
Rodeo Algorithm for Quantum Computing

We present a stochastic quantum computing algorithm that can prepare any eigenvector of a quantum Hamiltonian within a selected energy interval $[E\ensuremath{-}\ensuremath{\epsilon},E+\ensuremath{\epsilon}]$. In order to reduce the spectral weight of all other eigenvectors by a suppression factor $\ensuremath{\delta}$, the required computational effort scales as $O[|\mathrm{log}\ensuremath{\delta}|/(p\ensuremath{\epsilon})]$, where $p$ is the squared overlap of the initial state with the target eigenvector. The method, which we call the rodeo algorithm, uses auxiliary qubits to control the time evolution of the Hamiltonian minus some tunable parameter $E$. With each auxiliary qubit measurement, the amplitudes of the eigenvectors are multiplied by a stochastic factor that depends on the proximity of their energy to $E$. In this manner, we converge to the target eigenvector with exponential accuracy in the number of measurements. In addition to preparing eigenvectors, the method can also compute the full spectrum of the Hamiltonian. We illustrate the performance with several examples. For energy eigenvalue determination with error $\ensuremath{\epsilon}$, the computational scaling is $O[(\mathrm{log}\ensuremath{\epsilon}{)}^{2}/(p\ensuremath{\epsilon})]$. For eigenstate preparation, the computational scaling is $O(\mathrm{log}\mathrm{\ensuremath{\Delta}}/p)$, where $\mathrm{\ensuremath{\Delta}}$ is the magnitude of the orthogonal component of the residual vector. The speed for eigenstate preparation is exponentially faster than that for phase estimation or adiabatic evolution.
Deep Learning Enabled Design of Complex Transmission Matrices for Universal Optical Components

Recent breakthroughs in photonics-based quantum, neuromorphic and analogue processing have pointed out the need for new schemes for fully programmable nanophotonic devices. Universal optical elements based on interferometer meshes are underpinning many of these new technologies, however this is achieved at the cost of an overall footprint that is very large compared to the limited chip real estate, restricting the scalability of this approach. Here, we consider an ultracompact platform for low-loss programmable elements using the complex transmission matrix of a multi-port multimode waveguide. We propose a deep learning inverse network approach to design arbitrary transmission matrices using patterns of weakly scattering perturbations. The demonstrated technique allows control over both the intensity and phase in a multiport device at a four orders reduced device footprint compared to conventional technologies, thus opening the door for large-scale integrated universal networks.
Illuminating the Black Box: Interpreting Deep Neural Network Models for Psychiatric Research

Psychiatric research is often confronted with complex abstractions and dynamics that are not readily accessible or well-defined to our perception and measurements, making data-driven methods an appealing approach. Deep neural networks (DNNs) are capable of automatically learning abstractions in the data that can be entirely novel and have demonstrated superior performance over classical machine learning models across a range of tasks and, therefore, serve as a promising tool for making new discoveries in psychiatry. A key concern for the wider application of DNNs is their reputation as a "black box" approach-i.e., they are said to lack transparency or interpretability of how input data are transformed to model outputs. In fact, several existing and emerging tools are providing improvements in interpretability. However, most reviews of interpretability for DNNs focus on theoretical and/or engineering perspectives. This article reviews approaches to DNN interpretability issues that may be relevant to their application in psychiatric research and practice. It describes a framework for understanding these methods, reviews the conceptual basis of specific methods and their potential limitations, and discusses prospects for their implementation and future directions.
Lightweight Feature Fusion Network for Single Image Super-Resolution

Single image super-resolution (SISR) has witnessed great progress as convolutional neural network (CNN) gets deeper and wider. However, enormous parameters hinder its application to real world problems. In this letter, We propose a lightweight feature fusion network (LFFN) that can fully explore multi-scale contextual information and greatly reduce network parameters while maximizing SISR results. LFFN is built on spindle blocks and a softmax feature fusion module (SFFM). Specifically, a spindle block is composed of a dimension extension unit, a feature exploration unit. and a feature refinement unit. The dimension extension layer expands low dimension to high dimension and implicitly learns the feature maps which are suitable for the next unit. The feature exploration unit performs linear and nonlinear feature exploration aimed at different feature maps. The feature refinement layer is used to fuse and refine features. SFFM fuses the features from different modules in a self-adaptive learning manner with softmax function, making full use of hierarchical information with a small amount of parameter cost. Both qualitative and quantitative experiments on benchmark datasets show that LFFN achieves favorable performance against state-of-the-art methods with similar parameters.
IoT Device Identification Using Deep Learning

The growing use of IoT devices in organizations has increased the number of attack vectors available to attackers due to the less secure nature of the devices. The widely adopted bring your own device (BYOD) policy which allows an employee to bring any IoT device into the workplace and attach it to an organization's network also increases the risk of attacks. In order to address this threat, organizations often implement security policies in which only the connection of white-listed IoT devices is permitted. To monitor adherence to such policies and protect their networks, organizations must be able to identify the IoT devices connected to their networks and, more specifically, to identify connected IoT devices that are not on the white-list (unknown devices). In this study, we applied deep learning on network traffic to automatically identify IoT devices connected to the network. In contrast to previous work, our approach does not require that complex feature engineering be applied on the network traffic, since we represent the communication behavior of IoT devices using small images built from the IoT devices network traffic payloads. In our experiments, we trained a multiclass classifier on a publicly available dataset, successfully identifying 10 different IoT devices and the traffic of smartphones and computers, with over 99% accuracy. We also trained multiclass classifiers to detect unauthorized IoT devices connected to the network, achieving over 99% overall average detection accuracy.