Reviewerly/exHarmony huggingface dataset:
Identifying transcriptomic correlates of histology using deep learning

Linking phenotypes to specific gene expression profiles is an extremely important problem in biology, which has been approached mainly by correlation methods or, more fundamentally, by studying the effects of gene perturbations. However, genome-wide perturbations involve extensive experimental efforts, which may be prohibitive for certain organisms. On the other hand, the characterization of the various phenotypes frequently requires an expert’s subjective interpretation, such as a histopathologist’s description of tissue slide images in terms of complex visual features (e.g. ‘acinar structures’). In this paper, we use Deep Learning to eliminate the inherent subjective nature of these visual histological features and link them to genomic data, thus establishing a more precisely quantifiable correlation between transcriptomes and phenotypes. Using a dataset of whole slide images with matching gene expression data from 39 normal tissue types, we first developed a Deep Learning tissue classifier with an accuracy of 94%. Then we searched for genes whose expression correlates with features inferred by the classifier and demonstrate that Deep Learning can automatically derive visual (phenotypical) features that are well correlated with the transcriptome and therefore biologically interpretable . As we are particularly concerned with interpretability and explainability of the inferred histological models, we also develop visualizations of the inferred features and compare them with gene expression patterns determined by immunohistochemistry. This can be viewed as a first step toward bridging the gap between the level of genes and the cellular organization of tissues.
Anomaly Detection in Electroencephalography Signal Using Deep Learning Model

Biosignals such as Electroencephalogram (EEG), Electrocardiogram (ECG), Electromyogram (EMG) represent the electrical activities of various parts of human body. Various low cost non-invasive bio-sensors measures bio-signals and assist medical practitioner to monitor physiological conditions of a human health and identify associated risk. The volume bio-signals is a big data and can not be analyzed and identify anomaly manually, therefore intelligent algorithms have been proposed to detect personalized anomaly in real time data. This paper presents a review on Deep Learning (DL) based anomaly detection techniques in EEG. The convolutional neural network, recurrent neural network and autoencoder based DL algorithms are considered. Here EEG signal acquisition, feature extracting techniques and key-anomaly features and corresponding performance of the various techniques found in the literature are also discussed. The challenges and open research questions are outlined at the end of the article.
Object Detection in Aerial Images Using a Multiscale Keypoint Detection Network

Automatic object detection in aerial imagery is being increasingly adopted in many applications, such as traffic monitoring, smart cities, and disaster assistance. In keypoint-based detectors, the prediction modules are usually generated from a fixed feature map scale. This configuration significantly limits the ability to detect multiscale objects in aerial scenes. The corner selection module in these detectors often ignores that a category in an aerial image is relatively unitary. In this article, a novel network, called the multiscale keypoint detection network (MKD-Net), is proposed to address these challenges. MKD-Net fuses multiscale layers to generate multiple feature maps for objects of different sizes. During the inference phase, both feature maps can be exploited for predicting corners. Moreover, a category attention module is designed to reduce the channel noise for a single-category scene. Experiments on benchmarks PASCAL VOC and DOTA show promising performance of MKD-Net compared with the baseline network. The code is available on https://github.com/jason-su/MKD-NET.
Federated Learning for Spoken Language Understanding

Recently, spoken language understanding (SLU) has attracted extensive research interests, and various SLU datasets have been proposed to promote the development. However, most of the existing methods focus on a single individual dataset, the efforts to improve the robustness of models and obtain better performance by combining the merits of various datasets are not well studied. In this paper, we argue that if these SLU datasets are considered together, different knowledge from different datasets could be learned jointly, and there are high chances to promote the performance of each dataset. At the same time, we further attempt to prevent data leakage when unifying multiple datasets which, arguably, is more useful in an industry setting. To this end, we propose a federated learning framework, which could unify various types of datasets as well as tasks to learn and fuse various types of knowledge, i.e., text representations, from different datasets and tasks, without the sharing of downstream task data. The fused text representations merge useful features from different SLU datasets and tasks and are thus much more powerful than the original text representations alone in individual tasks. At last, in order to provide multi-granularity text representations for our framework, we propose a novel Multi-view Encoder (MV-Encoder) as the backbone of our federated learning framework. Experiments on two SLU benchmark datasets, including two tasks (intention detection and slot filling) and federated learning settings (horizontal federated learning, vertical federated learning and federated transfer learning), demonstrate the effectiveness and universality of our approach. Specifically, we are able to get 1.53% improvement on the intent detection metric accuracy. And we could also boost the performance of a strong baseline by up to 5.29% on the slot filling metric F1. Furthermore, by leveraging BERT as an additional encoder, we establish new state-of-the-art results on SNIPS and ATIS datasets, where we get 99.33% and 98.28% in terms of accuracy on intent detection task as well as 97.20% and 96.41% in terms of F1 score on slot filling task, respectively.
An Evaluation of Communication Protocol Languages for Engineering Multiagent Systems

Communication protocols are central to engineering decentralized multiagent systems. Modern protocol languages are typically formal and address aspects of decentralization, such as asynchrony. However, modern languages differ in important ways in their basic abstractions and operational assumptions. This diversity makes a comparative evaluation of protocol languages a challenging task.&#x0D; We contribute a rich evaluation of diverse and modern protocol languages. Among the selected languages, Scribble is based on session types; Trace-C and Trace-F on trace expressions; HAPN on hierarchical state machines, and BSPL on information causality. Our contribution is four-fold. One, we contribute important criteria for evaluating protocol languages. Two, for each criterion, we compare the languages on the basis of whether they are able to specify elementary protocols that go to the heart of the criterion. Three, for each language, we map our findings to a canonical architecture style for multiagent systems, highlighting where the languages depart from the architecture. Four, we identify design principles for protocol languages as guidance for future research.
Solving Ontology Meta-Matching Problem Through an Evolutionary Algorithm With Approximate Evaluation Indicators and Adaptive Selection Pressure

Ontology applies commonly to solve the problem of heterogeneity of data in the Semantic Web, but the heterogeneity problem between two ontologies seriously affects their communication. As an effective method, ontology matching can address the problem above, whose core technique is the similarity measure. A single similarity measure calculates the similarity value about a feature between two concepts, but none of the similarity measures can ensure their effectiveness in all context due to the diverse heterogeneous features between two ontologies. Therefore, multiple similarity measures are usually aggregated to improve the result's confidence. The problem that how to determine the optimal aggregating weights for the different similarity measures to obtain a high-quality alignment is called the meta-matching problem of ontology, which is modeled as a nonlinear problem with many local optimal solutions. Evolutionary Algorithm (EA) can represent an efficient methodology to address the ontology meta-matching problem, but EA-based ontology matching techniques suffer from the premature convergence and the requirement of a reference alignment to evaluate the solutions. To overcome the defects mentioned above, in this work, an improved EA-based matching approach is proposed, where two approximate evaluation indicators, i.e. pseudo-recall and pseudo-precision, are presented to evaluate the solution's quality, and an adaptive selection pressure is utilized to overcome the premature convergence. The experiment utilizes the Ontology Alignment Evaluation Initiative (OAEI)'s benchmark, and the experimental results will prove the effectiveness of our proposed method.
A New User Revocable Ciphertext-Policy Attribute-Based Encryption with Ciphertext Update

The revocable ciphertext-policy attribute-based encryption (R-CP-ABE) is an extension of ciphertext-policy attribute-based encryption (CP-ABE), which can realize user direct revocation and maintain a short revocation list. However, the revoked users can still decrypt the previously authorized encrypted data with their old key. The R-CP-ABE scheme should provide a mechanism to protect the encrypted data confidentiality by disqualifying the revoked users from accessing the previously encrypted data. Motivated by practical needs, we propose a new user R-CP-ABE scheme that simultaneously supports user direct revocation, short revocation list, and ciphertext update by incorporating the identity-based and time-based revocable technique. The scheme provides a strongly selective security proof under the modified decisional <math xmlns="http://www.w3.org/1998/Math/MathML" id="M1"> <mi>q</mi> </math> -parallel bilinear Diffie–Hellman Exponent problem, where “strongly” means that the adversary can query the secret key of a user whose attribute set satisfies the challenge ciphertext access structure and whose identity is in the revocation list.
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes

Adaptive gradient methods such as AdaGrad and its variants update the stepsize in stochastic gradient descent on the fly according to the gradients received along the way; such methods have gained widespread use in large-scale optimization for their ability to converge robustly, without the need to fine-tune the stepsize schedule. Yet, the theoretical guarantees to date for AdaGrad are for online and convex optimization. We bridge this gap by providing theoretical guarantees for the convergence of AdaGrad for smooth, nonconvex functions. We show that the norm version of AdaGrad (AdaGrad-Norm) converges to a stationary point at the $\mathcal{O}(\log(N)/\sqrt{N})$ rate in the stochastic setting, and at the optimal $\mathcal{O}(1/N)$ rate in the batch (non-stochastic) setting -- in this sense, our convergence guarantees are 'sharp'. In particular, the convergence of AdaGrad-Norm is robust to the choice of all hyper-parameters of the algorithm, in contrast to stochastic gradient descent whose convergence depends crucially on tuning the step-size to the (generally unknown) Lipschitz smoothness constant and level of stochastic noise on the gradient. Extensive numerical experiments are provided to corroborate our theory; moreover, the experiments suggest that the robustness of AdaGrad-Norm extends to state-of-the-art models in deep learning, without sacrificing generalization.
A History of Instructional Media, Instructional Design, and Theories

This paper discusses the history of the instructional design and technology field in four major time periods: (1) 1900s – 1930s, (2) World War II – 1970s, (3) 1980s – 1990s, and (4) 21st century. Since the 20th century has been discussed in detail in earlier works, this paper puts more focus on the 21st century section, which includes discussions of social media, online and blended learning, mobile learning, the open educational resource (OER) movement, massive open online courses (MOOCs), virtual reality (VR) and augmented reality (AR), digital game-based learning (DGBL), gamification, the maker movement, new instructional design process models, and instructional-design theory. While previous discussions of the history of the field focused on the history of instructional media and instructional design and the evolution of learning theories, this paper includes the discussion of the evolution of instructional-design theories and models, which have received relatively less attention.
Language Model Transformers as Evaluators for Open-domain Dialogues

Computer-based systems for communication with humans are a cornerstone of AI research since the 1950s.So far, the most effective way to assess the quality of the dialogues produced by these systems is to use resource-intensive manual labor instead of automated means.In this work, we investigate whether language models (LM) based on transformer neural networks can indicate the quality of a conversation.In a general sense, language models are methods that learn to predict one or more words based on an already given context.Due to their unsupervised nature, they are candidates for efficient, automatic indication of dialogue quality.We demonstrate that human evaluators have a positive correlation between the output of the language models and scores.We also provide some insights into their behavior and inner-working in a conversational context.
AGRNet: Adaptive Graph Representation Learning and Reasoning for Face Parsing

Face parsing infers a pixel-wise label to each facial component, which has drawn much attention recently. Previous methods have shown their success in face parsing, which however overlook the correlation among facial components. As a matter of fact, the component-wise relationship is a critical clue in discriminating ambiguous pixels in facial area. To address this issue, we propose adaptive graph representation learning and reasoning over facial components, aiming to learn representative vertices that describe each component, exploit the component-wise relationship and thereby produce accurate parsing results against ambiguity. In particular, we devise an adaptive and differentiable graph abstraction method to represent the components on a graph via pixel-to-vertex projection under the initial condition of a predicted parsing map, where pixel features within a certain facial region are aggregated onto a vertex. Further, we explicitly incorporate the image edge as a prior in the model, which helps to discriminate edge and non-edge pixels during the projection, thus leading to refined parsing results along the edges. Then, our model learns and reasons over the relations among components by propagating information across vertices on the graph. Finally, the refined vertex features are projected back to pixel grids for the prediction of the final parsing map. To train our model, we propose a discriminative loss to penalize small distances between vertices in the feature space, which leads to distinct vertices with strong semantics. Experimental results show the superior performance of the proposed model on multiple face parsing datasets, along with the validation on the human parsing task to demonstrate the generalizability of our model.
Matching Large Baseline Oblique Stereo Images Using an End-to-End Convolutional Neural Network

The available stereo matching algorithms produce large number of false positive matches or only produce a few true-positives across oblique stereo images with large baseline. This undesired result happens due to the complex perspective deformation and radiometric distortion across the images. To address this problem, we propose a novel affine invariant feature matching algorithm with subpixel accuracy based on an end-to-end convolutional neural network (CNN). In our method, we adopt and modify a Hessian affine network, which we refer to as IHesAffNet, to obtain affine invariant Hessian regions using deep learning framework. To improve the correlation between corresponding features, we introduce an empirical weighted loss function (EWLF) based on the negative samples using K nearest neighbors, and then generate deep learning-based descriptors with high discrimination that is realized with our multiple hard network structure (MTHardNets). Following this step, the conjugate features are produced by using the Euclidean distance ratio as the matching metric, and the accuracy of matches are optimized through the deep learning transform based least square matching (DLT-LSM). Finally, experiments on Large baseline oblique stereo images acquired by ground close-range and unmanned aerial vehicle (UAV) verify the effectiveness of the proposed approach, and comprehensive comparisons demonstrate that our matching algorithm outperforms the state-of-art methods in terms of accuracy, distribution and correct ratio. The main contributions of this article are: (i) our proposed MTHardNets can generate high quality descriptors; and (ii) the IHesAffNet can produce substantial affine invariant corresponding features with reliable transform parameters.
Gone full circle: A radial approach to visualize event-based networks in digital humanities

In the application domain of digital humanities network visualization is increasingly being used to conduct research as the main interests of the domain experts lie in exploring and analyzing relationships between entities and their changes over time. Visualizing the dynamics and different perspectives of such data is a non-trivial task but it enables researchers to explore connections between disparate entities and investigate historical narratives that emerge. In this paper we present Circular, an interactive exploration environment to visualize event-based networks and support research in digital humanities through visualization of historical subjects in space and time. Our radial design is the result of iterative collaboration with domain experts, and we discuss the process of collaborative development and exploration of public music festivities in Vienna as an example of immersive development methodology. We validate our approach by means of both domain and visualization expert interviews and show the potential of this approach in supporting the visual exploration of historical subjects. We discuss our design rationales, visual encodings, and interactions as to allow the reproducibility of this approach within a framework of transdisciplinary collaboration with digital humanities.
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch

Sparsity in Deep Neural Networks (DNNs) has been widely studied to compress and accelerate the models on resource-constrained environments. It can be generally categorized into unstructured fine-grained sparsity that zeroes out multiple individual weights distributed across the neural network, and structured coarse-grained sparsity which prunes blocks of sub-networks of a neural network. Fine-grained sparsity can achieve a high compression ratio but is not hardware friendly and hence receives limited speed gains. On the other hand, coarse-grained sparsity cannot simultaneously achieve both apparent acceleration on modern GPUs and decent performance. In this paper, we are the first to study training from scratch an N:M fine-grained structured sparse network, which can maintain the advantages of both unstructured fine-grained sparsity and structured coarse-grained sparsity simultaneously on specifically designed GPUs. Specifically, a 2 : 4 sparse network could achieve 2× speed-up without performance drop on Nvidia A100 GPUs. Furthermore, we propose a novel and effective ingredient, sparse-refined straight-through estimator (SR-STE), to alleviate the negative influence of the approximated gradients computed by vanilla STE during optimization. We also define a metric, Sparse Architecture Divergence (SAD), to measure the sparse network’s topology change during the training process. Finally, We justify SR-STE’s advantages with SAD and demonstrate the effectiveness of SR-STE by performing comprehensive experiments on various tasks. Anonymous code and model will be at available at https://github.com/anonymous-NM-sparsity/NM-sparsity.
Language Chatbot–The Design and Implementation of English Language Transfer Learning Agent Apps

Language Chatbot has widely used in customer service and personal assistant for task orientated, interactive chats in special domains, knowledge base for question-answer systems, in general, chatbot including automatic speech recognition (ASR), natural language understanding (NLU), dialogue management (DM), natural language generation (NLG), speech synthesis (SS). In our research, we proposed a transfer learning-based English Language learning chatbot with THREE levels learning system in real-world application, which integrate recognition service from Google and GPT-2 from Open AI with dialogue tasks in NLU and NLG at miniprogram of WeChat.
A blockchain-based platform architecture for multimedia data management

Massive amounts of multimedia data (i.e., text, audio, video, graphics and animation) are being generated everyday. Conventionally, multimedia data are managed by the platforms maintained by multimedia service providers, which are generally designed using centralised architecture. However, such centralised architecture may lead to a single point of failure and disputes over royalties or other rights. It is hard to ensure the data integrity and track fulfilment of obligations listed on the copyright agreement. To tackle these issues, in this paper, we present a blockchain-based platform architecture for multimedia data management. We adopt self-sovereign identity for identity management and design a multi-level capability-based mechanism for access control. We implement a proof-of-concept prototype using the proposed approach and evaluate it using a use case. The results show that the proposed approach is feasible and has scalable performance.
SCEP—A New Image Dimensional Emotion Recognition Model Based on Spatial and Channel-Wise Attention Mechanisms

Images are an important carrier for emotional expression. Human can understand emotions in image easily and quickly, whereas it is a very challenging task for machines to extract accurate emotions. In this study, we propose a novel spatial and channel-wise attention-based emotion prediction model, SCEP, to assist computers in recognizing the emotions of images more accurately. SCEP integrates both spatial attention and channel-wise weight mechanisms into a classical convolutional neural network (CNN) layer structure to predict image emotions, on the grounds that the spatial attention mechanism can enhance the contrast between salient regions and potentially irrelevant regions, and that the channel-wise weight mechanism can emphasize informative features while suppressing less useful features. The SCEP model outputs emotion values in a continuous 2-D valence and arousal space, so that more emotions can be expressed than by simply discretely classifying emotions. To validate the effectiveness of our model, we use an existing image dataset with a widespread emotion distribution for testing. Extensive experiments show that when compared to base models (i.e. VGG and ResNet) without spatial attention or channel-wise mechanisms, SCEP can improve the accuracy of emotion prediction (evaluated by concordance correlation coefficient) by ~ 3%-5% in the arousal domain, and by ~ 3-6% in the valence domain. Therefore, we conclude that using SCEP can bring higher accuracy in emotion prediction.