Reviewerly/exHarmony huggingface dataset:
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision

Vision-and-Language Pre-training (VLP) has improved performance on various joint vision-and-language downstream tasks. Current approaches to VLP heavily rely on image feature extraction processes, most of which involve region supervision (e.g., object detection) and the convolutional architecture (e.g., ResNet). Although disregarded in the literature, we find it problematic in terms of both (1) efficiency/speed, that simply extracting input features requires much more computation than the multimodal interaction steps; and (2) expressive power, as it is upper bounded to the expressive power of the visual embedder and its predefined visual vocabulary. In this paper, we present a minimal VLP model, Vision-and-Language Transformer (ViLT), monolithic in the sense that the processing of visual inputs is drastically simplified to just the same convolution-free manner that we process textual inputs. We show that ViLT is up to tens of times faster than previous VLP models, yet with competitive or better downstream task performance. Our code and pre-trained weights are available at https://github.com/dandelin/vilt.
Obtaining EHR-derived datasets for COVID-19 research within a short time: a flexible methodology based on Detailed Clinical Models

COVID-19 ranks as the single largest health incident worldwide in decades. In such a scenario, electronic health records (EHRs) should provide a timely response to healthcare needs and to data uses that go beyond direct medical care and are known as secondary uses, which include biomedical research. However, it is usual for each data analysis initiative to define its own information model in line with its requirements. These specifications share clinical concepts, but differ in format and recording criteria, something that creates data entry redundancy in multiple electronic data capture systems (EDCs) with the consequent investment of effort and time by the organization. This study sought to design and implement a flexible methodology based on detailed clinical models (DCM), which would enable EHRs generated in a tertiary hospital to be effectively reused without loss of meaning and within a short time. The proposed methodology comprises four stages: (1) specification of an initial set of relevant variables for COVID-19; (2) modeling and formalization of clinical concepts using ISO 13606 standard and SNOMED CT and LOINC terminologies; (3) definition of transformation rules to generate secondary use models from standardized EHRs and development of them using R language; and (4) implementation and validation of the methodology through the generation of the International Severe Acute Respiratory and emerging Infection Consortium (ISARIC-WHO) COVID-19 case report form. This process has been implemented into a 1300-bed tertiary Hospital for a cohort of 4489 patients hospitalized from 25 February 2020 to 10 September 2020. An initial and expandable set of relevant concepts for COVID-19 was identified, modeled and formalized using ISO-13606 standard and SNOMED CT and LOINC terminologies. Similarly, an algorithm was designed and implemented with R and then applied to process EHRs in accordance with standardized concepts, transforming them into secondary use models. Lastly, these resources were applied to obtain a data extract conforming to the ISARIC-WHO COVID-19 case report form, without requiring manual data collection. The methodology allowed obtaining the observation domain of this model with a coverage of over 85% of patients in the majority of concepts. This study has furnished a solution to the difficulty of rapidly and efficiently obtaining EHR-derived data for secondary use in COVID-19, capable of adapting to changes in data specifications and applicable to other organizations and other health conditions. The conclusion to be drawn from this initial validation is that this DCM-based methodology allows the effective reuse of EHRs generated in a tertiary Hospital during COVID-19 pandemic, with no additional effort or time for the organization and with a greater data scope than that yielded by conventional manual data collection process in ad-hoc EDCs.
Experimental quantum teleportation of propagating microwaves

The field of quantum communication promises to provide efficient and unconditionally secure ways to exchange information, particularly, in the form of quantum states. Meanwhile, recent breakthroughs in quantum computation with superconducting circuits trigger a demand for quantum communication channels between spatially separated superconducting processors operating at microwave frequencies. In pursuit of this goal, we demonstrate the unconditional quantum teleportation of propagating coherent microwave states by exploiting two-mode squeezing and analog feedforward over a macroscopic distance of d = 0.42 m. We achieve a teleportation fidelity of F = 0.689 ± 0.004, exceeding the asymptotic no-cloning threshold. Thus, the quantum nature of the teleported states is preserved, opening the avenue toward unconditional security in microwave quantum communication.
Path planning for coal mine robot via improved ant colony optimization algorithm

This paper is concerned with the path planning of the coal mine robot. A new workspace model is presented to describe the complex coal mine environment. Thus, the cost of a path is composed of not only the distance of the path but also some hybrid costs that can be linked to the criteria of path optimization. To overcome the drawbacks of conventional ant colony optimization (ACO) algorithm, an improved ACO algorithm is developed to tackle the issues of path planning of coal mine robot based on the new workspace model. Some simulation experiments are carried out on the path planning of coal mine robot, and the validity and superiority of the new approach can be confirmed by the simulation results.
Neural network attribution methods for problems in geoscience: A novel synthetic benchmark dataset

Abstract Despite the increasingly successful application of neural networks to many problems in the geosciences, their complex and nonlinear structure makes the interpretation of their predictions difficult, which limits model trust and does not allow scientists to gain physical insights about the problem at hand. Many different methods have been introduced in the emerging field of eXplainable Artificial Intelligence (XAI), which aims at attributing the network’s prediction to specific features in the input domain. XAI methods are usually assessed by using benchmark datasets (such as MNIST or ImageNet for image classification). However, an objective, theoretically derived ground truth for the attribution is lacking for most of these datasets, making the assessment of XAI in many cases subjective. Also, benchmark datasets specifically designed for problems in geosciences are rare. Here, we provide a framework, based on the use of additively separable functions, to generate attribution benchmark datasets for regression problems for which the ground truth of the attribution is known a priori. We generate a large benchmark dataset and train a fully connected network to learn the underlying function that was used for simulation. We then compare estimated heatmaps from different XAI methods to the ground truth in order to identify examples where specific XAI methods perform well or poorly. We believe that attribution benchmarks as the ones introduced herein are of great importance for further application of neural networks in the geosciences, and for more objective assessment and accurate implementation of XAI methods, which will increase model trust and assist in discovering new science.
A Traditional Chinese Medicine Traceability System Based on Lightweight Blockchain

Recently, the problem of traditional Chinese medicine (TCM) safety has attracted attention worldwide. To prevent the spread of counterfeit drugs, it is necessary to establish a drug traceability system. A traditional drug traceability system can record the whole circulation process of drugs, from planting, production, processing, and warehousing to use by hospitals and patients. Once counterfeit drugs are found, they can be traced back to the source. However, traditional drug traceability systems have some drawbacks, such as failure to prevent tampering and facilitation of sensitive disclosure. Blockchain (including Bitcoin and Ethernet Square) is an effective technology to address the problems of traditional drug traceability systems. However, some risks impact the reliability of blockchain, such as information explosion, sensitive information leakage, and poor scalability.To avoid the risks associated with the application of blockchain, we propose a lightweight block chain framework.In this framework, both horizontal and vertical segmentations are performed when designing the blocks, and effective strategies are provided for both segmentations. For horizontal segmentation operations, the header and body of the blockchain are separated and stored in the blockchain, and the body is stored in the InterPlanetary File System. For vertical segmentation operations, the blockchain is cut off according to time or size. For the addition of new blocks, miners only need to copy the latest part of the blockchain and append the tail and vertical segmentation of the block through the consensus mechanism.Our framework could greatly reduce the size of the blockchain and improve the verification efficiency.Experimental results have shown that the efficiency improves compared with ethernet when a new block is added to the blockchain and a search is conducted.
Deep Open Intent Classification with Adaptive Decision Boundary

Open intent classification is a challenging task in dialogue systems. On the one hand, it should ensure the quality of known intent identification. On the other hand, it needs to detect the open (unknown) intent without prior knowledge. Current models are limited in finding the appropriate decision boundary to balance the performances of both known intents and the open intent. In this paper, we propose a post-processing method to learn the adaptive decision boundary (ADB) for open intent classification. We first utilize the labeled known intent samples to pre-train the model. Then, we automatically learn the adaptive spherical decision boundary for each known class with the aid of well-trained features. Specifically, we propose a new loss function to balance both the empirical risk and the open space risk. Our method does not need open intent samples and is free from modifying the model architecture. Moreover, our approach is surprisingly insensitive with less labeled data and fewer known intents. Extensive experiments on three benchmark datasets show that our method yields significant improvements compared with the state-of-the-art methods. The codes are released at https://github.com/thuiar/Adaptive-Decision-Boundary.
Unsupervised Model Adaptation for Continual Semantic Segmentation

We develop an algorithm for adapting a semantic segmentation model that is trained using a labeled source domain to generalize well in an unlabeled target domain. A similar problem has been studied extensively in the unsupervised domain adaptation (UDA) literature, but existing UDA algorithms require access to both the source domain labeled data and the target domain unlabeled data for training a domain agnostic semantic segmentation model. Relaxing this constraint enables a user to adapt pretrained models to generalize in a target domain, without requiring access to source data. To this end, we learn a prototypical distribution for the source domain in an intermediate embedding space. This distribution encodes the abstract knowledge that is learned from the source domain. We then use this distribution for aligning the target domain distribution with the source domain distribution in the embedding space. We provide theoretical analysis and explain conditions under which our algorithm is effective. Experiments on benchmark adaptation tasks demonstrate our method achieves competitive performance even compared with joint UDA approaches.
The Challenges of Learning English Skills and the Integration of Social Media and Video Conferencing Tools to Help ESL Learners Coping with the Challenges during COVID-19 Pandemic: A Literature Review

During the COVID-19 pandemic, the abrupt closures of all learning institutions have caused English as second language (ESL) learners to have significant difficulties learning the English language especially for speaking skills.Speaking is one of the most important skills to learn because it is the primary mode of communication around the world.However, the majority of ESL learners are still trying to improve their English speaking abilities.Since the implementation of the COVID-19 lockdowns, e-learning has been a well-known solution all over the world.Learners and educators need more time to adjust to online teaching and learning because most of them are exploring new technical innovations and strategies to be used in the classroom.As a result, this study includes a literature review on the challenges ESL learners encounter in learning speaking skills, as well as the use of social media and video conferencing tools to teach speaking skills.Lack of motivation and self-confidence, fear, hesitation, and limited vocabulary are some of the challenges faced by ESL learners.Hence, it is vital to determine the technology intervention used in teaching and developing speaking skills among these learners based on previous study.During the COVID-19 pandemic, some of the interventions in teaching speaking skills included utilizing social media and video conferencing applications like Facebook, WhatsApp, and others for online teaching and learning.Educators of English as a Second Language (ESL) might then pick the ones that would work best in their particular classroom.
Federated Transfer Learning for IIoT Devices With Low Computing Power Based on Blockchain and Edge Computing

With the development of artificial intelligence and Internet of Things (IoT), the era of industry 4.0 has come.According to the prediction of IBM, with the continuous popularization of 5G technology, the IoT technology will be more widely used in factories.In recent years, federated learning has become a hot topic for Industrial Internet of Things (IIoT) researchers.However, many devices in the IIoT currently have a problem of low computing power, so these devices cannot perform well facing the tasks of training and updating models in federated learning.In order to solve the above problems, we introduce edge computing into the IIot, so that the device can complete the federated learning operation.In order to ensure the security of data transmission, blockchain is introduced as the main algorithm of equipment authentication in the system.What's more, in order to increase the efficiency and versatility of training model in IIoT, we introduce transfer learning to improve the system performance.The experimental results show that our algorithm can achieve high security and high training accuracy.
Reinforcement learning-based dynamic obstacle avoidance and integration of path planning

Deep reinforcement learning has the advantage of being able to encode fairly complex behaviors by collecting and learning empirical information. In the current study, we have proposed a framework for reinforcement learning in decentralized collision avoidance where each agent independently makes its decision without communication with others. In an environment exposed to various kinds of dynamic obstacles with irregular movements, mobile robot agents could learn how to avoid obstacles and reach a target point efficiently. Moreover, a path planner was integrated with the reinforcement learning-based obstacle avoidance to solve the problem of not finding a path in a specific situation, thereby imposing path efficiency. The robots were trained about the policy of obstacle avoidance in environments where dynamic characteristics were considered with soft actor critic algorithm. The trained policy was implemented in the robot operating system (ROS), tested in virtual and real environments for the differential drive wheel robot to prove the effectiveness of the proposed method. Videos are available at https://youtu.be/xxzoh1XbAl0.
ViBERTgrid: A Jointly Trained Multi-modal 2D Document Representation for Key Information Extraction from Documents

Recent grid-based document representations like BERTgrid allow the simultaneous encoding of the textual and layout information of a document in a 2D feature map so that state-of-the-art image segmentation and/or object detection models can be straightforwardly leveraged to extract key information from documents. However, such methods have not achieved comparable performance to state-of-the-art sequence- and graph-based methods such as LayoutLM and PICK yet. In this paper, we propose a new multi-modal backbone network by concatenating a BERTgrid to an intermediate layer of a CNN model, where the input of CNN is a document image and the BERTgrid is a grid of word embeddings, to generate a more powerful grid-based document representation, named ViBERTgrid. Unlike BERTgrid, the parameters of BERT and CNN in our multi-modal backbone network are trained jointly. Our experimental results demonstrate that this joint training strategy improves significantly the representation ability of ViBERTgrid. Consequently, our ViBERTgrid-based key information extraction approach has achieved state-of-the-art performance on real-world datasets.
DisenKGAT

Knowledge graph completion (KGC) has become a focus of attention across deep learning community owing to its excellent contribution to numerous downstream tasks. Although recently have witnessed a surge of work on KGC, they are still insufficient to accurately capture complex relations, since they adopt the single and static representations. In this work, we propose a novel Disentangled Knowledge Graph Attention Network (DisenKGAT) for KGC, which leverages both micro-disentanglement and macro-disentanglement to exploit representations behind Knowledge graphs (KGs). To achieve micro-disentanglement, we put forward a novel relation-aware aggregation to learn diverse component representation. For macro-disentanglement, we leverage mutual information as a regularization to enhance independence. With the assistance of disentanglement, our model is able to generate adaptive representations in terms of the given scenario. Besides, our work has strong robustness and flexibility to adapt to various score functions. Extensive experiments on public benchmark datasets have been conducted to validate the superiority of DisenKGAT over existing methods in terms of both accuracy and explainability.
Deep Learning and Particle Swarm Optimisation-Based Techniques for Visually Impaired Humans' Text Recognition and Identification

Blind people can benefit greatly from a system capable of localising and reading comprehension text embedded in natural scenes and providing useful information that boosts their self-esteem and autonomy in everyday situations. Regardless of the fact that existing optical character recognition programmes seem to be quick and effective, the majority of them are not able to correctly recognise text embedded in usual panorama images. The methodology described in this paper is to localise textual image regions and pre-process them using the naïve Bayesian algorithm. A weighted reading technique is used to generate the correct text data from the complicated image regions. Usually, images hold some disturbance as a result of the fact that filtration is proposed during the early pre-processing step. To restore the image's quality, the input image is processed employing gradient and contrast image methods. Following that, the contrast of the source images would be enhanced using an adaptive image map. The stroke width transform, Gabor’s transform, and weighted naïve Bayesian classifier methodologies have been used in complicated degraded images to segment, feature extraction, and detect textual and non-textual elements. Finally, to identify categorised textual data, the confluence of deep neural networks and particle swarm optimisation is being used. The text in the image is transformed into an acoustic output after identification. The dataset IIIT5K is used for the development portion, and the performance of the suggested come up is evaluated using parameters such as accuracy, recall, precision, and F1-score.
A Survey of AI-Based Facial Emotion Recognition: Features, ML &amp; DL Techniques, Age-Wise Datasets and Future Directions

Facial expressions are mirrors of human thoughts and feelings. It provides a wealth of social cues to the viewer, including the focus of attention, intention, motivation, and emotion. It is regarded as a potent tool of silent communication. Analysis of these expressions gives a significantly more profound insight into human behavior. AI-based Facial Expression Recognition (FER) has become one of the crucial research topics in recent years, with applications in dynamic analysis, pattern recognition, interpersonal interaction, mental health monitoring, and many more. However, with the global push towards online platforms due to the Covid-19 pandemic, there has been a pressing need to innovate and offer a new FER analysis framework with the increasing visual data generated by videos and photographs.Furthermore, the emotion-wise facial expressions of kids, adults, and senior citizens vary, which must also be considered in the FER research. Lots of research work has been done in this area. However, it lacks a comprehensive overview of the literature that showcases the past work done and provides the aligned future directions. In this paper, the authors have provided a comprehensive evaluation of AI-based FER methodologies, including datasets, feature extraction techniques, algorithms, and the recent breakthroughs with their applications in facial expression identification. To the best of the author’s knowledge, this is the only review paper stating all aspects of FER for various age brackets and would significantly impact the research community in the coming years.
Artificial intelligence, machine learning, and deep learning for clinical outcome prediction

AI is a broad concept, grouping initiatives that use a computer to perform tasks that would usually require a human to complete. AI methods are well suited to predict clinical outcomes. In practice, AI methods can be thought of as functions that learn the outcomes accompanying standardized input data to produce accurate outcome predictions when trialed with new data. Current methods for cleaning, creating, accessing, extracting, augmenting, and representing data for training AI clinical prediction models are well defined. The use of AI to predict clinical outcomes is a dynamic and rapidly evolving arena, with new methods and applications emerging. Extraction or accession of electronic health care records and combining these with patient genetic data is an area of present attention, with tremendous potential for future growth. Machine learning approaches, including decision tree methods of Random Forest and XGBoost, and deep learning techniques including deep multi-layer and recurrent neural networks, afford unique capabilities to accurately create predictions from high dimensional, multimodal data. Furthermore, AI methods are increasing our ability to accurately predict clinical outcomes that previously were difficult to model, including time-dependent and multi-class outcomes. Barriers to robust AI-based clinical outcome model deployment include changing AI product development interfaces, the specificity of regulation requirements, and limitations in ensuring model interpretability, generalizability, and adaptability over time.