Reviewerly/exHarmony huggingface dataset:
Chinese clinical named entity recognition with radical-level feature and self-attention mechanism

Named entity recognition is a fundamental and crucial task in medical natural language processing problems. In medical fields, Chinese clinical named entity recognition identifies boundaries and types of medical entities from unstructured text such as electronic medical records. Recently, a composition model of bidirectional Long Short-term Memory Networks (BiLSTMs) and conditional random field (BiLSTM-CRF) based character-level semantics has achieved great success in Chinese clinical named entity recognition tasks. But this method can only capture contextual semantics between characters in sentences. However, Chinese characters are hieroglyphics, and deeper semantic information is hidden inside, the BiLSTM-CRF model failed to get this information. In addition, some of the entities in the sentence are dependent, but the Long Short-term Memory (LSTM) does not capture long-term dependencies perfectly between characters. So we propose a BiLSTM-CRF model based on the radical-level feature and self-attention mechanism to solve these problems. We use the convolutional neural network (CNN) to extract radical-level features, aims to capture the intrinsic and internal relevances of characters. In addition, we use self-attention mechanism to capture the dependency between characters regardless of their distance. Experiments show that our model achieves F1-score 93.00% and 86.34% on CCKS-2017 and TP_CNER dataset respectively.
Fine-Grained Named Entity Recognition in Legal Documents

This paper describes an approach at Named Entity Recognition (NER) in German language documents from the legal domain. For this purpose, a dataset consisting of German court decisions was developed. The source texts were manually annotated with 19 semantic classes: person, judge, lawyer, country, city, street, landscape, organization, company, institution, court, brand, law, ordinance, European legal norm, regulation, contract, court decision, and legal literature. The dataset consists of approx. 67,000 sentences and contains 54,000 annotated entities. The 19 fine-grained classes were automatically generalised to seven more coarse-grained classes (person, location, organization, legal norm, case-by-case regulation, court decision, and legal literature). Thus, the dataset includes two annotation variants, i.e., coarse- and fine-grained. For the task of NER, Conditional Random Fields (CRFs) and bidirectional Long-Short Term Memory Networks (BiLSTMs) were applied to the dataset as state of the art models. Three different models were developed for each of these two model families and tested with the coarse- and fine-grained annotations. The BiLSTM models achieve the best performance with an 95.46 F $$_1$$ score for the fine-grained classes and 95.95 for the coarse-grained ones. The CRF models reach a maximum of 93.23 for the fine-grained classes and 93.22 for the coarse-grained ones. The work presented in this paper was carried out under the umbrella of the European project LYNX that develops a semantic platform that enables the development of various document processing and analysis applications for the legal domain.
"You Know What to Do"

Video sharing platforms like YouTube are increasingly targeted by aggression and hate attacks. Prior work has shown how these attacks often take place as a result of "raids," i.e., organized efforts by ad-hoc mobs coordinating from third-party communities. Despite the increasing relevance of this phenomenon, however, online services often lack effective countermeasures to mitigate it. Unlike well-studied problems like spam and phishing, coordinated aggressive behavior both targets and is perpetrated by humans, making defense mechanisms that look for automated activity unsuitable. Therefore, the de-facto solution is to reactively rely on user reports and human moderation. In this paper, we propose an automated solution to identify YouTube videos that are likely to be targeted by coordinated harassers from fringe communities like 4chan. First, we characterize and model YouTube videos along several axes (metadata, audio transcripts, thumbnails) based on a ground truth dataset of videos that were targeted by raids. Then, we use an ensemble of classifiers to determine the likelihood that a video will be raided with very good results (AUC up to 94%). Overall, our work provides an important first step towards deploying proactive systems to detect and mitigate coordinated hate attacks on platforms like YouTube.
Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks

Recent work has revealed that overparameterized networks trained by gradient descent achieve arbitrarily low training error, and sometimes even low test error. The required width, however, is always polynomial in at least one of the sample size n, the (inverse) training error 1/epsilon, and the (inverse) failure probability 1/delta. This work shows that O(1/epsilon) iterations of gradient descent on two-layer networks of any width exceeding polylog(n, 1/epsilon, 1/delta) and Omega(1/epsilon^2) training examples suffices to achieve a test error of epsilon. The analysis further relies upon a margin property of the limiting kernel, which is guaranteed positive, and can distinguish between true labels and random labels.
The HSIC Bottleneck: Deep Learning without Back-Propagation

We introduce the HSIC (Hilbert-Schmidt independence criterion) bottleneck for training deep neural networks. The HSIC bottleneck is an alternative to the conventional cross-entropy loss and backpropagation that has a number of distinct advantages. It mitigates exploding and vanishing gradients, resulting in the ability to learn very deep networks without skip connections. There is no requirement for symmetric feedback or update locking. We find that the HSIC bottleneck provides performance on MNIST/FashionMNIST/CIFAR10 classification comparable to backpropagation with a cross-entropy target, even when the system is not encouraged to make the output resemble the classification labels. Appending a single layer trained with SGD (without backpropagation) to reformat the information further improves performance.
TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion

Inferring missing facts in temporal knowledge graphs (TKGs) is a fundamental and challenging task.Previous works have approached this problem by augmenting methods for static knowledge graphs to leverage time-dependent representations.However, these methods do not explicitly leverage multi-hop structural information and temporal facts from recent time steps to enhance their predictions.Additionally, prior work does not explicitly address the temporal sparsity and variability of entity distributions in TKGs.We propose the Temporal Message Passing (TeMP) framework to address these challenges by combining graph neural networks, temporal dynamics models, data imputation and frequency-based gating techniques.Experiments 1 on standard TKG tasks show that our approach provides substantial gains compared to the previous state of the art, achieving a 10.7% average relative improvement in Hits@10 across three standard benchmarks.Our analysis also reveals important sources of variability both within and across TKG datasets, and we introduce several simple but strong baselines that outperform the prior state of the art in certain settings.
Contact Tracing and the COVID-19 Response in Africa: Best Practices, Key Challenges, and Lessons Learned from Nigeria, Rwanda, South Africa, and Uganda

Most African countries have recorded relatively lower COVID-19 burdens than Western countries. This has been attributed to early and strong political commitment and robust implementation of public health measures, such as nationwide lockdowns, travel restrictions, face mask wearing, testing, contact tracing, and isolation, along with community education and engagement. Other factors include the younger population age strata and hypothesized but yet-to-be confirmed partially protective cross-immunity from parasitic diseases and/or other circulating coronaviruses. However, the true burden may also be underestimated due to operational and resource issues for COVID-19 case identification and reporting. In this perspective article, we discuss selected best practices and challenges with COVID-19 contact tracing in Nigeria, Rwanda, South Africa, and Uganda. Best practices from these country case studies include sustained, multi-platform public communications; leveraging of technology innovations; applied public health expertise; deployment of community health workers; and robust community engagement. Challenges include an overwhelming workload of contact tracing and case detection for healthcare workers, misinformation and stigma, and poorly sustained adherence to isolation and quarantine. Important lessons learned include the need for decentralization of contact tracing to the lowest geographic levels of surveillance, rigorous use of data and technology to improve decision-making, and sustainment of both community sensitization and political commitment. Further research is needed to understand the role and importance of contact tracing in controlling community transmission dynamics in African countries, including among children. Also, implementation science will be critically needed to evaluate innovative, accessible, and cost-effective digital solutions to accommodate the contact tracing workload.
A Rapid Hybrid Clustering Algorithm for Large Volumes of High Dimensional Data

Clustering large volumes of high-dimensional data is a challenging task. Many clustering algorithms have been developed to address either handling datasets with a very large sample size or with a very high number of dimensions, but they are often impractical when the data is large in both aspects. To simultaneously overcome both the `curse of dimensionality' problem due to high dimensions and scalability problems due to large sample size, we propose a new fast clustering algorithm called FensiVAT. FensiVAT is a hybrid, ensemble-based clustering algorithm which uses fast data-space reduction and an intelligent sampling strategy. In addition to clustering, FensiVAT also provides visual evidence that is used to estimate the number of clusters (cluster tendency assessment) in the data. In our experiments, we compare FensiVAT with nine state-of-the-art approaches which are popular for large sample size or high-dimensional data clustering. Experimental results suggest that FensiVAT, which can cluster large volumes of high-dimensional datasets in a few seconds, is the fastest and most accurate method of the ones tested.
Proof-of-Authentication for Scalable Blockchain in Resource-Constrained Distributed Systems

Resource -constrained distributed systems such as the Internet of Things (IoT), edge computing and fog computing are deployed for real-time monitoring and evaluation. Current security solutions are problematic when there is a centralized controlling entity. The blockchain provides decentralized security architectures using proof-of-work (PoW). Proof-of-work is an expensive process for IoT and edge computing due to the deployment of resource-constrained devices. This paper presents a novel consensus algorithm called Proof-of-Authentication (PoAh) to replace Proof-of-Work and introduce authentication in such environments to make the blockchain application-specific. This paper implemented the Proof-of-Authentication system to evaluate its sustainability and applicability for the IoT and edge computing. The evaluation process is conducted in both simulation and real-time testbeds to evaluate performance. Finally, the process of Proof-of-Authentication and its integration with blockchain in resource-constrained distributed systems is discussed. Our proposed PoAh, while running in limited computer resources (e.g. single-board computing devices like the Raspberry Pi) has a latency in the order of 3 secs.
Evaluation of CNN-Based Single-Image Depth Estimation Methods

While an increasing interest in deep models for single-image depth estimation (SIDE) can be observed, established schemes for their evaluation are still limited. We propose a set of novel quality criteria, allowing for a more detailed analysis by focusing on specific characteristics of depth maps. In particular, we address the preservation of edges and planar regions, depth consistency, and absolute distance accuracy. In order to employ these metrics to evaluate and compare state-of-the-art SIDE approaches, we provide a new high-quality RGB-D dataset. We used a digital single-lens reflex (DSLR) camera together with a laser scanner to acquire high-resolution images and highly accurate depth maps. Experimental results show the validity of our proposed evaluation protocol.
Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning

Uncertainty estimation and ensembling methods go hand-in-hand. Uncertainty estimation is one of the main benchmarks for assessment of ensembling performance. At the same time, deep learning ensembles have provided state-of-the-art results in uncertainty estimation. In this work, we focus on in-domain uncertainty for image classification. We explore the standards for its quantification and point out pitfalls of existing metrics. Avoiding these pitfalls, we perform a broad study of different ensembling techniques. To provide more insight in the broad comparison, we introduce the deep ensemble equivalent (DEE) and show that many sophisticated ensembling techniques are equivalent to an ensemble of very few independently trained networks in terms of the test log-likelihood.
Anomaly Detection in Videos Using Optical Flow and Convolutional Autoencoder

Today, public areas, such as airports, hospitals, city centers are monitored by surveillance systems.The widespread use of surveillance systems reduces security concerns while creating an amount of video data that cannot be examined by people in real-time.Therefore, the concept of automatic understanding of video activities has raised the standards of security camera systems.In this paper, we propose a framework (OF-ConvAE-LSTM) to detect anomalies using Convolutional Autoencoder and Convolutional Long Short-Term Memory in an unsupervised manner.Besides the deep learning model, the feature extraction stage based on dense optical flow is applied in the framework to obtain the velocity and direction information of foreground objects.The experiments were carried out on three popular public datasets consisting of Avenue, UCSD Ped1, and UCSD Peds2.The experimental results have shown that the proposed framework models the complex distribution of the pattern of regular motion changes with high accuracy.Besides, this method was observed to outperform state-of-the-art approaches based on unsupervised and semi-supervised deep learning models.
COVID-19 in South Korea: Lessons for developing countries

South Korea's response is considered by many as one the most effective models against COVID-19. The average number of new cases per day fell to 6.4 in the first week of May and currently 90% of all confirmed cases have fully recovered after a massive outbreak in February 29. First, South Korea has flattened the curve of COVID-19 by combining testing, early isolation, and free treatment of positive cases combined with digital technologies without taking to “lockdown” measures. Second, South Korea has been holding press briefings twice a day and disclosed all information on COVID-19 to the public in an open and transparent manner. Third, South Korea has worked to secure the civic participation and voluntary engagement of citizens and businesses. The South Korean approach to COVID-19 may be difficult to emulate even for countries like the U.S. and U.K., but provides important policy implications for developing countries and needs for strengthening three core competencies against the outbreak of an epidemic: digital technology, efficient health governance, and civic partnership. It is a great challenge for developing countries to fight alone against COVID-19. As such, the international community should work to strengthen these core competencies accordingly.
Covid-19 and the accelerating smart home

Home, digital technologies and data are intersecting in new ways as responses to the COVID-19 pandemic emerge. We consider the data practices associated with COVID-19 responses and their implications for housing and home through two overarching themes: the notion of home as a private space, and digital technology and surveillance in the home. We show that although home has never been private, the rapid adoption and acceptance of technologies in the home for quarantine, work and study, enabled by the pandemic, is rescripting privacy. The acceleration of technology adoption and surveillance in the home has implications for privacy and potential discrimination, and should be approached with a critical lens.
Reduction of False-Positive Markings on Mammograms: a Retrospective Comparison Study Using an Artificial Intelligence-Based CAD

The aim was to determine whether an artificial intelligence (AI)-based, computer-aided detection (CAD) software can be used to reduce false positive per image (FPPI) on mammograms as compared to an FDA-approved conventional CAD. A retrospective study was performed on a set of 250 full-field digital mammograms between January 1, 2013, and March 31, 2013, and the number of marked regions of interest of two different systems was compared for sensitivity and specificity in cancer detection. The count of false-positive marks per image (FPPI) of the two systems was also evaluated as well as the number of cases that were completely mark-free. All results showed statistically significant reductions in false marks with the use of AI-CAD vs CAD (confidence interval = 95%) with no reduction in sensitivity. There is an overall 69% reduction in FPPI using the AI-based CAD as compared to CAD, consisting of 83% reduction in FPPI for calcifications and 56% reduction for masses. Almost half (48%) of cases showed no AI-CAD markings while only 17% show no conventional CAD marks. There was a significant reduction in FPPI with AI-CAD as compared to CAD for both masses and calcifications at all tissue densities. A 69% decrease in FPPI could result in a 17% decrease in radiologist reading time per case based on prior literature of CAD reading times. Additionally, decreasing false-positive recalls in screening mammography has many direct social and economic benefits.
MPCE: A Maximum Probability Based Cross Entropy Loss Function for Neural Network Classification

In recent years, multi-classifier learning is of significant interest in industrial and economic fields. Moreover, neural network is a popular approach in multi-classifier learning. However, the accuracies of neural networks are often limited by their loss functions. For this reason, we design a novel cross entropy loss function, named MPCE, which based on the maximum probability in predictive results. In this paper, we first analyze the difference of gradients between MPCE and the cross entropy loss function. Then, we propose the gradient update algorithm based on MPCE. In the experimental part of this paper, we utilize four groups of experiments to verify the performance of the proposed algorithm on six public datasets. The first group of experimental results show that the proposed algorithm converge faster than the algorithms based on other loss functions. Moreover, the results of the second group show that the proposed algorithm obtains the highest training and test accuracy on the six datasets, and the proposed algorithm perform better than others when class number changing on the sensor dataset. Furthermore, we use the model of convolutional neural network to implement the compared methods on the mnist dataset in the fourth group of experiments. The results show that the proposed algorithm has the highest accuracy among all executed methods.
Artificial intelligence (AI) and management analytics

Artificial Intelligence (AI), defined as “a system’s ability to correctly interpret external data, to learn from such data and to use those learnings to achieve specific goals and tasks through fle...
Semisupervised Distributed Learning With Non-IID Data for AIoT Service Platform

Thanks to the advances in wireless communication and machine learning technologies, we can envision a novel AIoT (AI + IoT) service platform that collects video data from the individuals' edge devices. Then, it transforms the video data into useful information, providing services to IoT or smart city applications. However, collecting raw video data directly to the cloud server is merely possible due to network bandwidth limitations and data privacy concerns. One possible solution is to adopt federated learning, which enables edge devices to collaboratively train a shared model without sending the raw data to the cloud. Unfortunately, this scheme cannot directly be applied to the targeted scenario since it assumes labeled data for training, and only at the cloud, we have the human power and time to label the video data. Thus, to tackle those issues, we propose an edge learning system based on semisupervised learning and federated learning technologies. The system trains AI models at edge devices using an improved semisupervised learning scheme and periodically uploads the training results to the cloud server to form a single model by adapting the federated learning technology. Then, we observe that in the real world, the data on the end devices are nonindependent and identically distributed (non-IID) such that it may cause weight divergence during training and result in a considerable decrease in the model performance. Therefore, we propose a new operation called federated swapping (FedSwap) to replace partial federated learning operations based on a few shared data during federated training to alleviate the adverse impact of weight divergence. We evaluate our system on both image classification using the state-of-the-art benchmark data and object detection using real-world video data. The experimental results show that the proposed system can have up to 5.9% higher accuracy of object detection for the video analysis applications by fully utilizing unlabeled data, compared with the situation that only labeled data are used. Moreover, the proposed FedSwap can improve the accuracy of image classification by 3.8% and the object detection task by 1.1%.
SimulSpeech: End-to-End Simultaneous Speech to Text Translation

In this work, we develop SimulSpeech, an endto-end simultaneous speech to text translation system which translates speech in source language to text in target language concurrently.SimulSpeech consists of a speech encoder, a speech segmenter and a text decoder, where 1) the segmenter builds upon the encoder and leverages a connectionist temporal classification (CTC) loss to split the input streaming speech in real time, 2) the encoder-decoder attention adopts a wait-k strategy for simultaneous translation.SimulSpeech is more challenging than previous cascaded systems (with simultaneous automatic speech recognition (ASR) and simultaneous neural machine translation (NMT)).We introduce two novel knowledge distillation methods to ensure the performance: 1) Attention-level knowledge distillation transfers the knowledge from the multiplication of the attention matrices of simultaneous NMT and ASR models to help the training of the attention mechanism in SimulSpeech; 2) Data-level knowledge distillation transfers the knowledge from the full-sentence NMT model and also reduces the complexity of data distribution to help on the optimization of Simul-Speech.Experiments on MuST-C English-Spanish and English-German spoken language translation datasets show that SimulSpeech achieves reasonable BLEU scores and lower delay compared to full-sentence end-to-end speech to text translation (without simultaneous translation), and better performance than the two-stage cascaded simultaneous translation model in terms of BLEU scores and translation delay.