Reviewerly/exHarmony huggingface dataset:
Probing for Referential Information in Language Models

Language models keep track of complex linguistic information about the preceding context -including, e.g., syntactic relations in a sentence.We investigate whether they also capture information beneficial for resolving pronominal anaphora in English.We analyze two state of the art models with LSTM and Transformer architectures, respectively, using probe tasks on a coreference annotated corpus.Our hypothesis is that language models will capture grammatical properties of anaphora (such as agreement between a pronoun and its antecedent), but not semantico-referential information (the fact that pronoun and antecedent refer to the same entity).Instead, we find evidence that models capture referential aspects to some extent -though they are still much better at grammar.The Transformer outperforms the LSTM in all analyses, and exhibits in particular better semantico-referential abilities.
Morphogo: An Automatic Bone Marrow Cell Classification System on Digital Images Analyzed by Artificial Intelligence

The nucleated-cell differential count on the bone marrow aspirate smears is required for the clinical diagnosis of hematological malignancy. Manual bone marrow differential count is time consuming and lacks consistency. In this study, a novel artificial intelligence (AI)-based system was developed to perform cell automatic classification of bone marrow cells and determine its potential clinical applications.Bone marrow aspirate smears were collected from the Xinqiao Hospital of Army Medical University. First, an automated analysis system (Morphogo) scanned and generated whole digital images of bone marrow smears. Then, the nucleated marrow cells in the selected areas of the smears at a magnification of ×1,000 were analyzed by the software utilizing an AI-based platform. The cell classification results were further reviewed and confirmed independently by 2 experienced pathologists. The automatic cell classification performance of the system was evaluated using 3 categories: accuracy, sensitivity, and specificity. Correlation coefficients and linear regression equations between automatic cell classification by the AI-based system and concurrent manual differential count were calculated.In 230 cases, the classification accuracy was above 85.7% for hematopoietic lineage cells. Averages of sensitivity and specificity of the system were found to be 69.4 and 97.2%, respectively. The differential cell percentage of the automated count based on 200-500 cell counts was correlated with differential cell percentage provided by the pathologists for granulocytes, erythrocytes, and lymphocytes (r ≥ 0.762, p < 0.001).This pilot study confirmed that the Morphogo system is a reliable tool for automatic bone marrow cell differential count analysis and has potential for clinical applications. Current ongoing large-scale multicenter validation studies will provide more information to further confirm the clinical utility of the system.
Multi-stage deep learning perception system for mobile robots

This paper presents a novel multi-stage perception system for collision avoidance in mobile robots. In the here considered scenario, a mobile robot stands in a workspace with a set of potential targets to reach or interact with. When a human partner appears gesturing to the target, the robot must plan a collision-free trajectory to reach the goal. To solve this problem, a full-perception system composed of consecutive convolutional neural networks in parallel and processing stages is proposed for generating a collision-free trajectory according to the desired goal. This system is evaluated at each step in real environments and through several performance tests, proving to be a robust and fast system suitable for real-time applications.
Blockchain for Healthcare: Securing Patient Data and Enabling Trusted Artificial Intelligence

Advances in information technology are digitizing the healthcare domain with the aim of improved medical services, diagnostics, continuous monitoring using wearables, etc., at reduced costs.This digitization improves the ease of computation, storage and access of medical records which enables better treatment experiences for patients.However, it comes with a risk of cyber attacks and security and privacy concerns on this digital data.In this work, we propose a Blockchain based solution for healthcare records to address the security and privacy concerns which are currently not present in existing e-Health systems.This work also explores the potential of building trusted Artificial Intelligence models over Blockchain in e-Health, where a transparent platform for consent-based data sharing is designed.Provenance of the consent of individuals and traceability of data sources used for building and training the AI model is captured in an immutable distributed data store.The audit trail of the data access captured using Blockchain provides the data owner to understand the exposure of the data.It also helps the user to understand the revenue models that could be built on top of this framework for commercial data sharing to build trusted AI models.
What skills do IT companies look for in new developers? A study with Stack Overflow jobs

Context: There is a growing demand for information on how IT companies look for candidates to their open positions. Objective: This paper investigates which hard and soft skills are more required in IT companies by analyzing the description of 20,000 job opportunities. Method: We applied open card sorting to perform a high-level analysis on which types of hard skills are more requested. Further, we manually analyzed the most mentioned soft skills. Results: Programming languages are the most demanded hard skills. Communication, collaboration, and problem-solving are the most demanded soft skills. Conclusion: We recommend developers to organize their resumé according to the positions they are applying. We also highlight the importance of soft skills, as they appear in many job opportunities.
Design of a Reinforcement Learning-Based Lane Keeping Planning Agent for Automated Vehicles

Reinforcement learning-based approaches are widely studied in the literature for solving different control tasks for Connected and Autonomous Vehicles, from which this paper deals with the problem of lateral control of a dynamic nonlinear vehicle model, performing the task of lane-keeping. In this area, the appropriate formulation of the goals and environment information is crucial, for which the research outlines the importance of lookahead information, enabling to accomplish maneuvers with complex trajectories. Another critical part is the real-time manner of the problem. On the one hand, optimization or search based methods, such as the presented Monte Carlo Tree Search method, can solve the problem with the trade-off of high numerical complexity. On the other hand, single Reinforcement Learning agents struggle to learn these tasks with high performance, though they have the advantage that after the training process, they can operate in a real-time manner. Two planning agent structures are proposed in the paper to resolve this duality, where the machine learning agents aid the tree search algorithm. As a result, the combined solution provides high performance and low computational needs.
Towards Domain-independent Complex and Fine-grained Gesture Recognition with RFID

Gesture recognition plays a fundamental role in emerging Human-Computer Interaction (HCI) paradigms. Recent advances in wireless sensing show promise for device-free and pervasive gesture recognition. Among them, RFID has gained much attention given its low-cost, light-weight and pervasiveness, but pioneer studies on RFID sensing still suffer two major problems when it comes to gesture recognition. The first is they are only evaluated on simple whole-body activities, rather than complex and fine-grained hand gestures. The second is they can not effectively work without retraining in new domains, i.e. new users or environments. To tackle these problems, in this paper, we propose RFree-GR, a domain-independent RFID system for complex and fine-grained gesture recognition. First of all, we exploit signals from the multi-tag array to profile the sophisticated spatio-temporal changes of hand gestures. Then, we elaborate a Multimodal Convolutional Neural Network (MCNN) to aggregate information across signals and abstract complex spatio-temporal patterns. Furthermore, we introduce an adversarial model to our deep learning architecture to remove domain-specific information while retaining information relevant to gesture recognition. We extensively evaluate RFree-GR on 16 commonly used American Sign Language (ASL) words. The average accuracy for new users and environments (new setup and new position) are $89.03%$, $90.21%$ and $88.38%$, respectively, significantly outperforming existing RFID based solutions, which demonstrates the superior effectiveness and generalizability of RFree-GR.
RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling

In order to alleviate the shortage of multidomain data and to capture discourse phenomena for task-oriented dialogue modeling, we propose RiSAWOZ, a large-scale multidomain Chinese Wizard-of-Oz dataset with Rich Semantic Annotations.RiSAWOZ contains 11.2K human-to-human (H2H) multiturn semantically annotated dialogues, with more than 150K utterances spanning over 12 domains, which is larger than all previous annotated H2H conversational datasets.Both single-and multi-domain dialogues are constructed, accounting for 65% and 35%, respectively.Each dialogue is labeled with comprehensive dialogue annotations, including dialogue goal in the form of natural language description, domain, dialogue states and acts at both the user and system side.In addition to traditional dialogue annotations, we especially provide linguistic annotations on discourse phenomena, e.g., ellipsis and coreference, in dialogues, which are useful for dialogue coreference and ellipsis resolution tasks.Apart from the fully annotated dataset, we also present a detailed description of the data collection procedure, statistics and analysis of the dataset.A series of benchmark models and results are reported, including natural language understanding (intent detection & slot filling), dialogue state tracking and dialogue contextto-text generation, as well as coreference and ellipsis resolution, which facilitate the baseline comparison for future research on this corpus.
Contrastive learning of global and local features for medical image segmentation with limited annotations

A key requirement for the success of supervised deep learning is a large labeled dataset - a condition that is difficult to meet in medical image analysis. Self-supervised learning (SSL) can help in this regard by providing a strategy to pre-train a neural network with unlabeled data, followed by fine-tuning for a downstream task with limited annotations. Contrastive learning, a particular variant of SSL, is a powerful technique for learning image-level representations. In this work, we propose strategies for extending the contrastive learning framework for segmentation of volumetric medical images in the semi-supervised setting with limited annotations, by leveraging domain-specific and problem-specific cues. Specifically, we propose (1) novel contrasting strategies that leverage structural similarity across volumetric medical images (domain-specific cue) and (2) a local version of the contrastive loss to learn distinctive representations of local regions that are useful for per-pixel segmentation (problem-specific cue). We carry out an extensive evaluation on three Magnetic Resonance Imaging (MRI) datasets. In the limited annotation setting, the proposed method yields substantial improvements compared to other self-supervision and semi-supervised learning techniques. When combined with a simple data augmentation technique, the proposed method reaches within 8% of benchmark performance using only two labeled MRI volumes for training, corresponding to only 4% (for ACDC) of the training data used to train the benchmark. The code is made public at this https URL.
PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation

Pre-trained Transformers have enabled impressive breakthroughs in generating long and fluent text, yet their outputs are often "rambling" without coherently arranged content.In this work, we present a novel content-controlled text generation framework, PAIR, with planning and iterative refinement, which is built upon a large model, BART.We first adapt the BERT model to automatically construct the content plans, consisting of keyphrase assignments and their corresponding sentence-level positions.The BART model is employed for generation without modifying its structure.We then propose a refinement algorithm to gradually enhance the generation quality within the sequence-tosequence framework.Evaluation with automatic metrics shows that adding planning consistently improves the generation quality on three distinct domains, with an average of 20 BLEU points and 12 METEOR points improvements.In addition, human judges rate our system outputs to be more relevant and coherent than comparisons without planning.
No Gestures Left Behind: Learning Relationships between Spoken Language and Freeform Gestures

We study relationships between spoken language and co-speech gestures in context of two key challenges.First, distributions of text and gestures are inherently skewed making it important to model the long tail.Second, gesture predictions are made at a subword level, making it important to learn relationships between language and acoustic cues.We introduce Adversarial Importance Sampled Learning (or AISLe), which combines adversarial learning with importance sampling to strike a balance between precision and coverage.We propose the use of a multimodal multiscale attention block to perform subword alignment without the need of explicit alignment between language and acoustic cues.Finally, to empirically study the importance of language in this task, we extend the dataset proposed in Ahuja et al. ( 2020) with automatically extracted transcripts for audio signals.We substantiate the effectiveness of our approach through large-scale quantitative and user studies, which show that our proposed methodology significantly outperforms previous state-of-the-art approaches for gesture generation.Link to code, data and videos: https: //github.com/chahuja/aisle
Shallow-to-Deep Training for Neural Machine Translation

Deep encoders have been proven to be effective in improving neural machine translation (NMT) systems, but training an extremely deep encoder is time consuming.Moreover, why deep models help NMT is an open question.In this paper, we investigate the behavior of a well-tuned deep Transformer system.We find that stacking layers is helpful in improving the representation ability of N-MT models and adjacent layers perform similarly.This inspires us to develop a shallowto-deep training method that learns deep models by stacking shallow models.In this way, we successfully train a Transformer system with a 54-layer encoder.Experimental results on WMT'16 English-German and WMT'14 English-French translation tasks show that it is 1.4 × faster than training from scratch, and achieves a BLEU score of 30.33 and 43.29 on two tasks.The code is publicly available at
Classification by Attention: Scene Graph Classification with Prior Knowledge

A major challenge in scene graph classification is that the appearance of objects and relations can be significantly different from one image to another. Previous works have addressed this by relational reasoning over all objects in an image or incorporating prior knowledge into classification. Unlike previous works, we do not consider separate models for perception and prior knowledge. Instead, we take a multi-task learning approach by introducing schema representations and implementing the classification as an attention layer between image-based representations and the schemata. This allows for the prior knowledge to emerge and propagate within the perception model. By enforcing the model also to represent the prior, we achieve a strong inductive bias. We show that our model can accurately generate commonsense knowledge and that the iterative injection of this knowledge to scene representations, as a top-down mechanism, leads to significantly higher classification performance. Additionally, our model can be fine-tuned on external knowledge given as triples. When combined with self-supervised learning and with 1% of annotated images only, this gives more than 3% improvement in object classification, 26% in scene graph classification, and 36% in predicate prediction accuracy.
A Novel Deep Learning-Based Multilevel Parallel Attention Neural (MPAN) Model for Multidomain Arabic Sentiment Analysis

Over the past few years, much work has been done to develop machine learning models that perform Arabic sentiment analysis (ASA) tasks at various levels and in different domains. However, most of this work has been based on shallow machine learning, with little attention given to deep learning approaches. Furthermore, the deep learning models used for ASA have been based on noncontextualized embedding schemes that negatively impact model performances. This article proposes a novel deep learning-based multilevel parallel attention neural (MPAN) model that uses a simple positioning binary embedding scheme (PBES) to simultaneously compute contextualized embeddings at the character, word, and sentence levels. The MPAN model then computes multilevel attention vectors and concatenates them at the output level to produce competitive accuracies. Specifically, the MPAN model produces state-of-the-art results that outperform all established ASA baselines using 34 publicly available ASA datasets. The proposed model is further shown to produce new state-of-the-art accuracies for two multidomain collections: 95.61% for a binary classification collection and 94.25% for a tertiary classification collection. Finally, the performance of the MPAN model is further validated using the public IMDB movie review dataset, on which it produces an accuracy of 96.13%, placing it in second position on the global IMDB leaderboard.
A novel periocular biometrics solution for authentication during Covid-19 pandemic situation

The outbreak of novel coronavirus in 2019 has shaken the whole world and it quickly evolved as a global pandemic, placing everyone in a panic situation. Considering its long-term effects on day to day lives, the necessity of wearing face mask and social distancing brings in picture the requirement of a contact less biometric system for all future authentication systems. One of the solutions is to use periocular biometric as it does not need physical contact like fingerprint biometric and is able to identify even people wearing face masks. Since, the periocular region is a small area as compared to face, extraction of required number of features from that small region is the major concern to make the system highly robust. This research proposes a feature fusion approach which combines the handcrafted features HOG, non-handcrafted features extracted using pretrained CNN models and gender related features extracted using a five layer CNN model. The proposed feature fusion approach is evaluated using multiclass SVM classifier with three different benchmark databases, UBIPr, Color FERET and Ethnic Ocular as well as for three non-ideal scenarios i.e. the effect of eyeglasses, effect of eye occlusion and pose variations. The proposed approach shows remarkable improvement in performance over pre-existing approaches.
Learning to Solve Combinatorial Optimization Problems on Real-World Graphs in Linear Time

Combinatorial optimization algorithms for graph problems are usually designed afresh for each new problem with careful attention by an expert to the problem structure. In this work, we develop a new framework to solve any combinatorial optimization problem over graphs that can be formulated as a single player game defined by states, actions, and rewards, including minimum spanning tree, shortest paths, traveling salesman problem, and vehicle routing problem, without expert knowledge. Our method trains a graph neural network using reinforcement learning on an unlabeled training set of graphs. The trained network then outputs approximate solutions to new graph instances in linear running time. In contrast, previous approximation algorithms or heuristics tailored to NP-hard problems on graphs generally have at least quadratic running time. We demonstrate the applicability of our approach on both polynomial and NP-hard problems with optimality gaps close to 1, and show that our method is able to generalize well: (i) from training on small graphs to testing on large graphs; (ii) from training on random graphs of one type to testing on random graphs of another type; and (iii) from training on random graphs to running on real world graphs.
A novel autonomous learning framework to enhance sEMG-based hand gesture recognition using depth information

Hand gesture recognition using surface electromyography (sEMG) has been one of the most efficient motion analysis techniques in human–computer interaction in the last few decades. In particular, multichannel sEMG techniques have achieved stable performance in hand gesture recognition. However, the general solution of collecting and labeling large data manually leads to time-consuming implementation. A novel learning method is therefore needed to facilitate efficient data collection and preprocessing. In this paper, a novel autonomous learning framework is proposed to integrate the benefits of both depth vision and EMG signals, which automatically label the class of collected EMG data using depth information. It then utilizes a multiple layer neural network (MNN) classifier to achieve real-time recognition of the hand gestures using only the sEMG. The overall framework is demonstrated in an augmented reality application by the recognition of 10 hand gestures using the Myo armband and an HTC VIVE PRO. The results show prominent performance by introducing depth information for real-time data labeling.
Self-Supervised Learning Across Domains

Human adaptability relies crucially on learning and merging knowledge from both supervised and unsupervised tasks: the parents point out few important concepts, but then the children fill in the gaps on their own. This is particularly effective, because supervised learning can never be exhaustive and thus learning autonomously allows to discover invariances and regularities that help to generalize. In this paper we propose to apply a similar approach to the problem of object recognition across domains: our model learns the semantic labels in a supervised fashion, and broadens its understanding of the data by learning from self-supervised signals on the same images. This secondary task helps the network to focus on object shapes, learning concepts like spatial orientation and part correlation, while acting as a regularizer for the classification task over multiple visual domains. Extensive experiments confirm our intuition and show that our multi-task method, combining supervised and self-supervised knowledge, provides competitive results with respect to more complex domain generalization and adaptation solutions. It also proves its potential in the novel and challenging predictive and partial domain adaptation scenarios.