Reviewerly/exHarmony huggingface dataset:
RAAT: Relation-Augmented Attention Transformer for Relation Modeling in Document-Level Event Extraction

In document-level event extraction (DEE) task, event arguments always scatter across sentences (across-sentence issue) and multiple events may lie in one document (multi-event issue).In this paper, we argue that the relation information of event arguments is of great significance for addressing the above two issues, and propose a new DEE framework which can model the relation dependencies, called Relation-augmented Document-level Event Extraction (ReDEE).More specifically, this framework features a novel and tailored transformer, named as Relation-augmented Attention Transformer (RAAT).RAAT is scalable to capture multi-scale and multi-amount argument relations.To further leverage relation information, we introduce a separate event relation prediction task and adopt multi-task learning method to explicitly enhance event extraction performance.Extensive experiments demonstrate the effectiveness of the proposed method, which can achieve state-ofthe-art performance on two public datasets.Our code is available at https://github.com/TencentYoutuResearch/RAAT.
Detecting anomalous traffic behaviors with seasonal deep Kalman filter graph convolutional neural networks

Anomaly detection over traffic data is crucial for transportation management and abnormal behavior identification. An anomaly in real-world scenarios usually causes abnormal observations for multiple detectors in an extended period. However, existing anomaly detection methods overly leverage the single or isolated feature interdependent contextual information in anomalies, inevitably dropping the detection performance. In this paper, we propose S-DKFN (Seasonal Deep Kalman Filter Network), to identify abnormal patterns with a long duration and wide coverage. S-DKFN models traffic data with a graph and simultaneously investigates the spatial and temporal features to hunt abnormal behaviors. Specifically, a dilation temporal convolutional network (TCN) is used to merge the multi-granular seasonal features and a graph convolution network (GCN) to extract spatial features. The outputs of TCN and GCN are then fed to long-short term models (LSTM) and merged by Kalman filters for denoising. An encoder-decoder module is introduced to predict traffic attributes with seasonal features. The mean squared errors (MSE) of the predictions are considered the anomaly scores. Experimental results on two real-world datasets show that our proposed S-DKFN framework outperforms the state-of-the-art baseline methods in detecting anomalies with long-duration and wide-coverage, especially its ability to detect accidents.
Mutual match for semi-supervised online evolutive learning

Abstract Semi-supervised learning (SSL) can utilize a large amount of unlabeled data for self-training and continuous evolution with only a few annotations. This feature makes SSL a potential candidate for dealing with data from changing and real-time environments, where deep-learning models need to be adapting to evolving and nonstable (non-i.i.d.) data streams from the real world, i.e., online evolutive scenarios. However, state-of-the-art SSL methods often have complex model design mechanisms and may cause performance degradation in a generalized and open environment. In an edge computing setup, e.g., typical in modern Internet of Things (IoT) applications, a multi-agent SSL architecture can help resolve generalization problems by sharing knowledge between models. In this paper, we introduce Mutual Match (MM), an online-evolutive SSL algorithm that integrates mutual interactive learning and soft-supervision consistency regularization, as well as unsupervised sample mining. By leveraging extra knowledge in the training process and the interactive collaboration between models, MM surpasses multiple top SSL algorithms in accuracy and convergence efficiency under the same online-evolutive experiment setup. MM simplifies the complexity of model design and follows a unified and easy-to-expandable pipeline, which can be beneficial to tasks with insufficient labeled data and frequently changing data distribution.
Powering Finetuning in Few-Shot Learning: Domain-Agnostic Bias Reduction with Selected Sampling

In recent works, utilizing a deep network trained on meta-training set serves as a strong baseline in few-shot learning. In this paper, we move forward to refine novel-class features by finetuning a trained deep network. Finetuning is designed to focus on reducing biases in novel-class feature distributions, which we define as two aspects: class-agnostic and class-specific biases. Class-agnostic bias is defined as the distribution shifting introduced by domain difference, which we propose Distribution Calibration Module(DCM) to reduce. DCM owes good property of eliminating domain difference and fast feature adaptation during optimization. Class-specific bias is defined as the biased estimation using a few samples in novel classes, which we propose Selected Sampling(SS) to reduce. Without inferring the actual class distribution, SS is designed by running sampling using proposal distributions around support-set samples. By powering finetuning with DCM and SS, we achieve state-of-the-art results on Meta-Dataset with consistent performance boosts over ten datasets from different domains. We believe our simple yet effective method demonstrates its possibility to be applied on practical few-shot applications.
Learning from Students: Online Contrastive Distillation Network for General Continual Learning

The goal of General Continual Learning (GCL) is to preserve learned knowledge and learn new knowledge with constant memory from an infinite data stream where task boundaries are blurry. Distilling the model's response of reserved samples between the old and the new models is an effective way to achieve promise performance on GCL. However, it accumulates the inherent old model's response bias and is not robust to model changes. To this end, we propose an Online Contrastive Distillation Network (OCD-Net) to tackle these problems, which explores the merit of the student model in each time step to guide the training process of the student model. Concretely, the teacher model is devised to help the student model to consolidate the learned knowledge, which is trained online via integrating the model weights of the student model to accumulate the new knowledge. Moreover, our OCD-Net incorporates both relation and adaptive response to help the student model alleviate the catastrophic forgetting, which is also beneficial for the teacher model preserves the learned knowledge. Extensive experiments on six benchmark datasets demonstrate that our proposed OCD-Net significantly outperforms state-of-the-art approaches in 3.26%~8.71% with various buffer sizes. Our code is available at https://github.com/lijincm/OCD-Net.
A Speech-driven Sign Language Avatar Animation System for Hearing Impaired Applications

Sign language is the communication language used in hearing impaired community. Recently, the research of sign language production has made great progress but still need to cope with some critical challenges. In this paper, we propose a system-level scheme and push forward the implementation of sign language production for practical usage. We build a system capable of translating speech into sign language avatar. Different from previous approach only focusing on single technology, we systematically combine algorithms of language translation, body gesture animation and facial avatar generation. We also develop two applications: Sign Language Interpretation APP and Virtual Sign Language Anchor, to facilitate easy and clear communication for hearing impaired people.
Efficient and Highly Accurate Diagnosis of Malignant Hematological Diseases Based on Whole-Slide Images Using Deep Learning

Hematopoietic disorders are serious diseases that threaten human health, and the diagnosis of these diseases is essential for treatment. However, traditional diagnosis methods rely on manual operation, which is time consuming and laborious, and examining entire slide is challenging. In this study, we developed a weakly supervised deep learning method for diagnosing malignant hematological diseases requiring only slide-level labels. The method improves efficiency by converting whole-slide image (WSI) patches into low-dimensional feature representations. Then the patch-level features of each WSI are aggregated into slide-level representations by an attention-based network. The model provides final diagnostic predictions based on these slide-level representations. By applying the proposed model to our collection of bone marrow WSIs at different magnifications, we found that an area under the receiver operating characteristic curve of 0.966 on an independent test set can be obtained at 10× magnification. Moreover, the performance on microscopy images can achieve an average accuracy of 94.2% on two publicly available datasets. In conclusion, we have developed a novel method that can achieve fast and accurate diagnosis in different scenarios of hematological disorders.
IAUFD: A 100k images dataset for automatic football image/video analysis

Nowadays, analyzing football videos using computer vision techniques has attracted increasing attention. Significant events detection, football video summarization, football results predictions, statistics etc. are exciting applications in this area. On the other hand, the deep learning approaches are very successful methods for image and video analysis that need much data. Nevertheless, to the best of our knowledge, publicly available datasets in this area are small or individual, which are not enough for such deep learning-based approaches. A public dataset was collected, annotated, and prepared, namely IAUFD*, to meet this gap for researches in this direction. The IAUFD contains 100,000 real-world images from 33 football videos in 2,508 min, annotated in 10 event categories. These categories include the goal, center of the field, celebration, red card, yellow card, the ball, stadium, the referee, penalty-kick, and free-kick. It is believed that these moments are the basis and useful for any high-level action or event exploration. For a generalization of our dataset, we paid attention to various weather (e.g., sunny, rainy, cloudy etc.), season, time of day, and location. We also used two deep neural networks (VggNet-13 and ResNet-18) to evaluate our proposed dataset as the baseline for future studies and comparison.
Enhanced prediction performance of a time-delay reservoir computing system based on a VCSEL by dual-training method

In this paper, a new dual-training method for a time-delay reservoir computing (RC) system based on a single vertical-cavity surface-emitting laser (VCSEL) is proposed and demonstrated experimentally for the first time. The prediction performance of the RC system by using the dual-training method has been experimentally and numerically investigated. Here, the dual-training method is defined as performing a further RC based on the difference between the target value and the predicted value of the traditional single training. It is found that enhanced prediction performance of the RC system can be obtained by employing the dual-training method, compared to the traditional single training method. More specifically, the NMSE values of the RC system with the dual-training method applied can be improved to 760% compared with the single training method in experiments. Besides, the effects of injection power, bias currents, feedback strength, and frequency detuning are also considered. The proposed dual-training method is of great significance to the performance enhancement of the RC and has an important promotion effect on the application of the RC in the future.
Modeling Trust in COVID-19 Contact-Tracing Apps Using the Human-Computer Trust Scale: Online Survey Study

Background The COVID-19 pandemic has caused changes in technology use worldwide, both socially and economically. This pandemic crisis has brought additional measures such as contact-tracing apps (CTAs) to help fight against spread of the virus. Unfortunately, the low adoption rate of these apps affected their success. There could be many reasons for the low adoption, including concerns of security and privacy, along with reported issues of trust in CTAs. Some concerns are related with how CTAs could be used as surveillance tools or their potential threats to privacy as they involve health data. For example, in Estonia, the CTA named HOIA had approximately 250,000 downloads in the middle of January 2021. However, in 2021, only 4.7% of the population used HOIA as a COVID-19 CTA. The reasons for the low adoption include lack of competency, and privacy and security concerns. This lower adoption and the lack of trustworthiness persist despite efforts of the European Union in building ethics and trustworthy artificial intelligence (AI)-based apps. Objective The aim of this study was to understand how to measure trust in health technologies. Specifically, we assessed the usefulness of the Human-Computer Trust Scale (HCTS) to measure Estonians’ trust in the HOIA app and the causes for this lack of trust. Methods The main research question was: Can the HCTS be used to assess citizens’ perception of trust in health technologies? We established four hypotheses that were tested with a survey. We used a convenience sample for data collection, including sharing the questionnaire on social network sites and using the snowball method to reach all potential HOIA users in the Estonian population. Results Among the 78 respondents, 61 had downloaded the HOIA app with data on usage patterns. However, 20 of those who downloaded the app admitted that it was never opened despite most claiming to regularly use mobile apps. The main reasons included not understanding how it works, and privacy and security concerns. Significant correlations were found between participants’ trust in CTAs in general and their perceived trust in the HOIA app regarding three attributes: competency (P&lt;.001), risk perception (P&lt;.001), and reciprocity (P=.01). Conclusions This study shows that trust in the HOIA app among Estonian residents did affect their predisposition to use the app. Participants did not generally believe that HOIA could help to control the spread of the virus. The result of this work is limited to HOIA and health apps that use similar contact-tracing methods. However, the findings can contribute to gaining a broader understanding and awareness of the need for designing trustworthy technologies. Moreover, this work can help to provide design recommendations that ensure trustworthiness in CTAs, and the ability of AI to use highly sensitive data and serve society.
Static and Dynamic Path Planning Algorithms Design for a Wheeled Mobile Robot Based on a Hybrid Technique

During the previous years, the number of researchers who discussed the field of path planning algorithms has increased.In this paper, a proposed hybrid technique for path planning based on the benefits of sampling-based algorithms and heuristic path planning algorithms was enhanced.Rapidly-exploring Random Tree star (RRT*) was used to generate the candidate nodes that represent a collision-free path.These candidate nodes were applied to the Particle Swarm Optimization (PSO) algorithm to generate the shortest and the smoothest path planning (the optimal path planning) for the mobile robot.The RRT*PSO algorithm was applied in two scenarios (static and dynamic environments) with a workspace of [500×500] cm using MATLAB 2021a.In the static environment, a workspace full of obstacles was chosen to find the shortest collision-free path.In addition, a reference path equation was found to calculate the reference linear and angular velocities of the mobile robot as well as the linear and angular velocities of the right and left wheels.In this work, the suggested hybrid RRT*PSO algorithm improves path length 34.27% compared to the A* algorithm and the fuzzy analytic hierarchy process (A*-FAHP hybrid) algorithm and 0.35% compared to the Self-adaptive evolutionary game-based particle swarm optimization (SAEGBPSO) algorithm.In the dynamic environment, an algorithm named (contour path down and contour path up) was suggested to avoid collisions with dynamic obstacles.By using this suggested algorithm, the reference linear and angular velocities of the mobile robot as well as the linear and angular velocities of the right and left wheels were calculated to obtain a smooth path followed and free-navigation with dynamic obstacles in an environment.
Confidence Score Based Conformer Speaker Adaptation for Speech Recognition

A key challenge for automatic speech recognition (ASR) systems is to model the speaker level variability. In this paper, compact speaker dependent learning hidden unit contributions (LHUC) are used to facilitate both speaker adaptive training (SAT) and test time unsupervised speaker adaptation for state-of-the-art Conformer based end-to-end ASR systems. The sensitivity during adaptation to supervision error rate is reduced using confidence score based selection of the more "trustworthy" subset of speaker specific data. A confidence estimation module is used to smooth the over-confident Conformer decoder output probabilities before serving as confidence scores. The increased data sparsity due to speaker level data selection is addressed using Bayesian estimation of LHUC parameters. Experiments on the 300-hour Switchboard corpus suggest that the proposed LHUC-SAT Conformer with confidence score based test time unsupervised adaptation outperformed the baseline speaker independent and i-vector adapted Conformer systems by up to 1.0%, 1.0%, and 1.2% absolute (9.0%, 7.9%, and 8.9% relative) word error rate (WER) reductions on the NIST Hub5'00, RT02, and RT03 evaluation sets respectively. Consistent performance improvements were retained after external Transformer and LSTM language models were used for rescoring.
Extended U-Net for Speaker Verification in Noisy Environments

Background noise is a well-known factor that deteriorates the accuracy and reliability of speaker verification (SV) systems by blurring speech intelligibility. Various studies have used separate pretrained enhancement models as the front-end module of the SV system in noisy environments, and these methods effectively remove noises. However, the denoising process of independent enhancement models not tailored to the SV task can also distort the speaker information included in utterances. We argue that the enhancement network and speaker embedding extractor should be fully jointly trained for SV tasks under noisy conditions to alleviate this issue. Therefore, we proposed a U-Net-based integrated framework that simultaneously optimizes speaker identification and feature enhancement losses. Moreover, we analyzed the structural limitations of using U-Net directly for noise SV tasks and further proposed Extended U-Net to reduce these drawbacks. We evaluated the models on the noise-synthesized VoxCeleb1 test set and VOiCES development set recorded in various noisy scenarios. The experimental results demonstrate that the U-Net-based fully joint training framework is more effective than the baseline, and the extended U-Net exhibited state-of-the-art performance versus the recently proposed compensation systems.
DRCNNTLe: A deep recurrent convolutional neural network with transfer learning through pre-trained embeddings for automated ICD coding

The International Classification of Diseases (ICD), which is endorsed by the World Health Organization, is a diagnostic classification standard. ICD codes store, retrieve, and analyze health information to make clinical decisions. Currently, ICD coding has been adopted by more than 137 countries. However, in Pakistan, very few hospitals have implemented ICD coding and conducted different epidemiological studies. Moreover, none of them have reported the spectrum of liver disease burden based on ICD coding, nor implemented automated ICD coding. In this study, we annotated ICD codes for the database of the liver transplant unit of the Pir Abdul Qadir Shah Jeelani Institute of Medical Sciences. We named this database Medical Information Mart for Liver Transplantation (MIMLT). The results revealed that the database contains 34 ICD codes, of which V70.8 is the most frequent code. Furthermore, we determined the spectrum of liver disease burden in liver recipients based on ICD coding. We found that chronic hepatitis C (070.54) is the most frequent indication for liver transplantation. Additionally, we implemented automated ICD coding utilizing the MIMLT database and proposed a novel Deep Recurrent Convolutional Neural Network with Transfer Learning through pre-trained Embeddings (DRCNNTLe) model, which is an extended version of our DRCNN-HP model. DRCNNTLe extracts robust text representations from its pre-trained embedding layer, which is trained on a large domain-specific MIMIC III database corpus. The results indicate that utilizing pre-trained word embeddings, which are trained on large domain-specific corpora can significantly improve the performance of the DRCNNTLe model and provide state-of-the-art results when the target database is small.
Word Level Robustness Enhancement: Fight Perturbation with Perturbation

State-of-the-art deep NLP models have achieved impressive improvements on many tasks. However, they are found to be vulnerable to some perturbations. Before they are widely adopted, the fundamental issues of robustness need to be addressed. In this paper, we design a robustness enhancement method to defend against word substitution perturbation, whose basic idea is to fight perturbation with perturbation. We find that: although many well-trained deep models are not robust in the setting of the presence of adversarial samples, they satisfy weak robustness. That means they can handle most non-crafted perturbations well. Taking advantage of the weak robustness property of deep models, we utilize non-crafted perturbations to resist the adversarial perturbations crafted by attackers. Our method contains two main stages. The first stage is using randomized perturbation to conform the input to the data distribution. The second stage is using randomized perturbation to eliminate the instability of prediction results and enhance the robustness guarantee. Experimental results show that our method can significantly improve the ability of deep models to resist the state-of-the-art adversarial attacks while maintaining the prediction performance on the original clean data.
Adversarial Learning from Crowds

Learning from Crowds (LFC) seeks to induce a high-quality classifier from training instances, which are linked to a range of possible noisy annotations from crowdsourcing workers under their various levels of skills and their own preconditions. Recent studies on LFC focus on designing new methods to improve the performance of the classifier trained from crowdsourced labeled data. To this day, however, there remain under-explored security aspects of LFC systems. In this work, we seek to bridge this gap. We first show that LFC models are vulnerable to adversarial examples---small changes to input data can cause classifiers to make prediction mistakes. Second, we propose an approach, A-LFC for training a robust classifier from crowdsourced labeled data. Our empirical results on three real-world datasets show that the proposed approach can substantially improve the performance of the trained classifier even with the existence of adversarial examples. On average, A-LFC has 10.05% and 11.34% higher test robustness than the state-of-the-art in the white-box and black-box attack settings, respectively.