Reviewerly/exHarmony huggingface dataset:
Federated Learning of Unsegmented Chinese Text Recognition Model

Unsegmented text recognition is a crucial component in financial document processing systems. Financial text materials, such as receipts, transcripts, identification documents, etc. often involve critical personal information. In many circumstances, these data reside in protected servers of different institutions and must not be transferred beyond the institutional firewall. The emerging technology of Federated Learning (FL) provides a data-secure way of uniting isolated datasets in model training. Using the FL framework, text recognition models can be trained with larger collection of image samples. In previous works, federated text recognition models only deal with single-character images and alpha-numeric corpus. Such models are not competent in industrial applications, especially in Chinese text recognition problems. In this paper, we apply federated learning with a deep convolutional network to perform variable-length text string recognition with a large corpus. In our experiments, we compared two prevalent federated learning frameworks, namely, Tensorflow Federated and PySyft. Results show that federated text recognition models can achieve similar or even higher accuracy than models trained on deep learning framework. On a 5-client distributed dataset, the best character accuracy is achieved by TFF at 49.20%. Extensive experiments are also conducted to evaluate the effect of distributed data storage over the performance of trained models. TFF again achieved a maximum character precision of 54.33% with non-distributed dataset.
Smart Care

Cross-institutional sharing of medical data is essential to provide e.ective collaborative treatment and clinical decisions for patients. Medical data privacy involves ensuring only authorized parties may access the health records under the awareness and approval of patients in any circumstances. This is crucial to any healthcare system because the protection of patients' clinical data is not only an ethical responsibility but also a legal mandate. Despite the importance of medical data sharing, today's healthcare systems have not provided enough protection of patients' sensitive information to be utilized deliberately or unintentionally. Hence, there is an urgent demand for a clinical transaction mechanism that allows patients to access, trace and control their health records. In this paper, the authors focus on several limitations in the literature and propose appropriate improvement in healthcare systems by (i) addressing information security and privacy, (ii) solving the lack of trust between providers, and (iii) encouraging scalability of healthcare interoperability. Building upon these key insights, we introduce several components of a patient-centered healthcare system using smart contracts via blockchain technology. A complete code solution is publicized on the authors' GitHub repository to engage further reproducibility and improvement.
Research on Chinese medical named entity recognition based on collaborative cooperation of multiple neural network models

Medical named entity recognition (NER) in Chinese electronic medical records (CEMRs) has drawn much research attention, and plays a vital prerequisite role for extracting high-value medical information. In 2018, China Health Information Processing Conference (CHIP2018) organized a medical NER academic competition aiming to extract three types of malignant tumor entity from CEMRs. Since the three types of entity are highly domain-specific and interdependency, extraction of them cannot be achieved with a single neural network model. Based on comprehensive study of the three types of entity and the entity interdependencies, we propose a collaborative cooperation of multiple neural network models based approach, which consists of two BiLSTM-CRF models and a CNN model. In order to tackle the problem that target scene dataset is small and entity distributions are sparse, we introduce non-target scene datasets and propose sentence-level neural network model transfer learning. Based on 30,000 real-world CEMRs, we pre-train medical domain-specific Chinese character embeddings with word2vec, GloVe and ELMo, and apply them to our approach respectively to validate effects of pre-trained language models in Chinese medical NER. Also, as control experiments, we apply Gated Recurrent Unit to our approach. Finally, our approach achieves an overall F1-score of 87.60%, which is the state-of-the-art performance to the best of our knowledge. In addition, our approach has won the champion of the medical NER academic competition organized by 2019 China Conference on Knowledge Graph and Semantic Computing, which proves the outstanding generalization ability of our approach.
Optimus Prime: Accelerating Data Transformation in Servers

Modern online services are shifting away from monolithic applications to loosely-coupled microservices because of their improved scalability, reliability, programmability and development velocity. Microservices communicating over the datacenter network require data transformation (DT) to convert messages back and forth between their internal formats. This work identifies DT as a bottleneck due to reductions in latency of the surrounding system components, namely application runtimes, protocol stacks, and network hardware. We therefore propose Optimus Prime (OP), a programmable DT accelerator that uses a novel abstraction, an in-memory schema, to represent DT operations. The schema is compatible with today's DT frameworks and enables any compliant accelerator to perform the transformations comprising a request in parallel. Our evaluation shows that OP's DT throughput matches the line rate of today's NICs and has ~60x higher throughput compared to software, at a tiny fraction of the CPU's silicon area and power. We also evaluate a set of microservices running on Thrift, and show up to 30% reduction in service latency.
Single Image Dehazing via Lightweight Multi-scale Networks

Single image haze removal is a challenging ill-posed problem in computer vision. Instead of leveraging the traditional model or handcrafted image priors, an end-to-end multi-scale convolutional neural network is proposed for single image haze removal task by directly mapping the hazy image to its corresponding haze-free image. To better retain the coarse and fine information, a multi-scale block is elaborated and embedded into the proposed architecture. This block can extract the feature at varying scales with a model size that is as small as possible. The global skip connection is adopted to promote the model performance. Extensive experiment results demonstrate that the proposed network outperforms the state-of-the-art single image haze removal algorithms on both synthetical and real-world images. In addition, the size of the model in this paper dominates among the high performance methods based on convolutional neural networks.
The online anti-public sphere

In this article, I outline the online ‘anti-public sphere’ as an object for analysis, defined as that space of online socio-political interaction where discourse routinely and radically flouts the ethical and rational norms of democratic discourse. This is a formerly offline space made newly visible by digital networked media. It includes discursive spaces and forms such as White supremacist websites, anti-climate science forums, militant ‘men’s rights’ sites, anti-immigration Facebook pages, gay hate memes, misogynist trolling, anti-Semitic websites, alt-right websites and ‘truth’ (conspiracy) websites, to name a few, where discussion flouts norms of public debate, rules of argument and requirements for the rational consideration of evidence for its own ends. Building on earlier work on anti-publics by McKenzie Wark and Bart Cammaerts, and working from examples from several different domains of online anti-public discourse, I argue that despite its size and complexity, it is possible and necessary to theorise this heterogeneous discursive field, not least because while such discourse is often dismissed, the meanings developed in such domains increasingly intermingle with and inform everyday democratic discourse. While we tend to think of extreme and irrational online discourse as aberrant and alien to everyday democratic discourse, analysis suggests that such discourse in fact is a precise reflection of an everyday ‘post-normative’ democratic discourse that has itself become deeply inflected with reactionary and populist themes.
A location-based orientation-aware recommender system using IoT smart devices and Social Networks

The rapid development of IoT sensors and data provided by Social Networks has necessitated the fast development of recommender systems as they can be used as a tool to filter items that are more likely to be preferred by users. A major goal of recommender systems is to provide users with personalized recommendations after analyzing their preferences. IoT smart devices and Social Networks have opened windows of opportunities for user preferences to be dynamically recognized. Although analyzing user preferences helps to provide more personalized recommendations, considering location and orientation information as user contextual information results in more relevant recommendations being provided. Location information has been used by Social Networks, especially Location-Based Social Networks, in order to provide recommendations based on current user location. However, the importance of the user orientation context has been overlooked by almost all of the research done in this area. Developing a location-based orientation-aware recommender system can perfectly bridge this gap. For this study, a location-based orientation-aware recommender system is proposed as an innovative type of recommender system. The proposed recommender system is able to not only apply contemporary user contextual information to the recommender algorithm, but also makes progress towards preparing more personalized recommendations by taking user orientation context into account. For this study, user preferences are dynamically measured by IoT smart devices such as smartphones, Google Home, and smartwatches. Information provided by virtual communities extracted from Social Networks helps the recommender system in situations in which user preferences are not extracted from their IoT devices. In addition to user preferences, their smartphone pointing direction has also been applied as their orientation context for the recommender algorithm in outdoor environments. To evaluate the impact of the user pointing direction in our proposed methodology, an event recommender system based on the real data was implemented and examined in the city of Tehran in Iran. Because of the challenging nature of social events, a simulated experiment is also presented for the City of Calgary. Also, the system results are compared with the results of Collaborative Filtering and Content-based recommender algorithms to demonstrate the power of the recommendation engine. The evaluation indexes prove that our proposed recommender system outperforms its counterparts by providing more accurate and personalized recommendations.
Detail retaining convolutional neural network for image denoising

Compared with the traditional image denoising method, although the convolutional neural network (CNN) has better denoising performance, there is an important issue that has not been well resolved: the residual image obtained by learning the difference between noisy image and clean image pairs contains abundant image detail information, resulting in the serious loss of detail in the denoised image. In this paper, in order to relearn the lost image detail information, a mathematical model is deducted from a minimization problem and an end-to-end detail retaining CNN (DRCNN) is proposed. Unlike most denoising methods based on CNN, DRCNN is not only focus to image denoising, but also the integrity of high frequency image content. DRCNN needs less parameters and storage space, therefore it has better generalization ability. Moreover, DRCNN can also adapt to different image restoration tasks such as blind image denoising, single image superresolution (SISR), blind deburring and image inpainting. Extensive experiments show that DRCNN has a better effect than some classic and novel methods.
Controlling Emotion Strength with Relative Attribute for End-to-End Speech Synthesis

Recently, attention-based end-to-end speech synthesis has achieved superior performance compared to traditional speech synthesis models, and several approaches like global style tokens are proposed to explore the style controllability of the end-to-end model. Although the existing methods show good performance in style disentanglement and transfer, it is still unable to control the explicit emotion of generated speech. In this paper, we mainly focus on the subtle control of expressive speech synthesis, where the emotion category and strength can be easily controlled with a discrete emotional vector and a continuous simple scalar, respectively. The continuous strength controller is learned by a ranking function according to the relative attribute measured on an emotion dataset. Our method automatically learns the relationship between low-level acoustic features and high-level subtle emotion strength. Experiments show that our method can effectively improve the controllability for an expressive end-to-end model.
A Weighted Fidelity and Regularization-Based Method for Mixed or Unknown Noise Removal From Images on Graphs

Image denoising technologies in a Euclidean domain have achieved good results and are becoming mature. However, in recent years, many real-world applications encountered in computer vision and geometric modeling involve image data defined in irregular domains modeled by huge graphs, which results in the problem on how to solve image denoising problems defined on graphs. In this paper, we propose a novel model for removing mixed or unknown noise in images on graphs. The objective is to minimize the sum of a weighted fidelity term and a sparse regularization term that additionally utilizes wavelet frame transform on graphs to retain feature details of images defined on graphs. Specifically, the weighted fidelity term with ℓ1-norm and ℓ2-norm is designed based on a analysis of the distribution of mixed noise. The augmented Lagrangian and accelerated proximal gradient methods are employed to achieve the optimal solution to the problem. Finally, some supporting numerical results and comparative analyses with other denoising algorithms are provided. It is noted that we investigate image denoising with unknown noise or a wide range of mixed noise, especially the mixture of Poisson, Gaussian, and impulse noise. Experimental results reported for synthetic and real images on graphs demonstrate that the proposed method is effective and efficient, and exhibits better performance for the removal of mixed or unknown noise in images on graphs than other denoising algorithms in the literature. The method can effectively remove mixed or unknown noise and retain feature details of images on graphs. It delivers a new avenue for denoising images in irregular domains.
Single image-based head pose estimation with spherical parametrization and 3D morphing

Head pose estimation plays a vital role in various applications, e.g., driver-assistance systems, human-computer interaction, virtual reality technology, and so on. We propose a novel geometry-based method for accurately estimating the head pose from a single 2D face image at a very low computational cost. Specifically, the rectangular coordinates of only four non-coplanar feature points from a predefined 3D facial model as well as the corresponding ones automatically/manually extracted from a 2D face image are first normalized to exclude the effect of external factors (i.e., scale factor and translation parameters). Then, the four normalized 3D feature points are represented in spherical coordinates with reference to the uniquely determined sphere by themselves. Due to the spherical parametrization, the coordinates of feature points can then be morphed along all the three directions in the rectangular coordinates effectively. Finally, the rotation matrix indicating the head pose is obtained by minimizing the Euclidean distance between the normalized 2D feature points and the 2D re-projections of the morphed 3D feature points. Comprehensive experimental results over two popular datasets, i.e., Pointing’04 and Biwi Kinect, demonstrate that the proposed method can estimate head poses with higher accuracy and lower run time than state-of-the-art geometry-based methods. Even compared with start-of-the-art learning-based methods or geometry-based methods with additional depth information, our method still produces comparable performance.
Human Activity Recognition using PCA and BiLSTM Recurrent Neural Networks

This paper presents an original approach to Human Activity Recognition (HAR) tasks based on wearable sensors data. We have trained a Bidirectional Long-Short Term Memory (BLSTM) Recurrent Neural Network (RNN) model to predict the physical activity of subjects taken from the mHealth dataset. PCA has been used to reduce dimensionality of the dataset while keeping the total variation at no less than 85%. The resulting dataset consisted of 15 features instead of features at a reduction rate of 34.7% The mhealth has 12 different activities recorded out of 10 subjects. We present a comparison of the PCA-BLSTM approach to several classical machine learning models. Results indicated that the PCA-BLSTM model has registered the highest accuracy of 97.64%. The SVM algorithm has registered the least accuracy of 64.67. We recommend the application of this procedure to other HAR datasets which have higher dimensionality and complexity of human activities.
Adversarial Discriminative Attention for Robust Anomaly Detection

Existing methods for visual anomaly detection predominantly rely on global level pixel comparisons for anomaly score computation without emphasizing on unique local features. However, images from real-world applications are susceptible to unwanted noise and distractions, that might jeopardize the robustness of such anomaly score. To alleviate this problem, we propose a self-supervised masking method that specifically focuses on discriminative parts of images to enable robust anomaly detection. Our experiments reveal that discriminator's class activation map in adversarial training evolves in three stages and finally fixates on the foreground location in the images. Using this property of the activation map, we construct a mask that suppresses spurious signals from the background thus enabling robust anomaly detection by focusing on local discriminative attributes. Additionally, our method can further improve the accuracy by learning a semi-supervised discriminative classifier in cases where a few samples from anomaly classes are available during the training. Experimental evaluations on four different types of datasets demonstrate that our method outperforms previous state-of-the-art methods for each condition and in all domains.
Detection of Hate Speech and Offensive Language in Twitter Data Using LSTM Model

In today’s world, internet is an emerging technology with exponential user growth. A major concern with that is the increase of toxic online content by people of different backgrounds. With the expansion of deep learning, quite a lot of researches have inclined toward using their deep neural networks for abundant discipline. Even for natural language processing (NLP)-based tasks, deep networks, specifically recurrent neural network (RNN), and their types are lately being considered over the traditional shallow networks. This paper addresses the problem of hate speech hovering on social media. We propose an LTSM-based classification system that differentiates between hate speech and offensive language. This system describes a contemporary approach that employs word embeddings with LSTM and Bi-LSTM neural networks for the identification of hate speech on Twitter. The best performing LSTM network classifier achieved an accuracy of 86% with early stopping criterion based on loss function during training.
Latency and Performance Analyses of Real-World Wireless IoT-Blockchain Application

The Internet of Things (IoT) is increasingly being utilized, by both businesses and individuals, for many applications. This utilization means increases in the smart devices that are connected to the Internet of Things, which will significantly increase the challenges related to devices' interconnectivity and management, data and user privacy, and network, data, and device security. At the same time, blockchain approaches provide a decentralized, immutable, and peer-to-peer ledger technology that could be the right answer to these challenges. Significant challenges, however, accompany the integration of blockchain into the Internet of Things, since IoT smart devices may suffer from resource and power constraints and blockchain is associated with scalability and delay issues. In this paper, a practical incorporation of blockchain into the Internet of Things is demonstrated using Ethereum Proof of Authority (PoA). This provides performance analyses, which include measurement of the transaction arrival time, the system end-to-end latency for different network implementations over cellular and Wi-Fi, and the average power consumption. This includes the study of the effect of network bandwidth on the stability and synchronization of all nodes on the blockchain network.
Multi-Level Representation Learning for Deep Subspace Clustering

This paper proposes a novel deep subspace clustering approach which uses convolutional autoencoders to transform input images into new representations lying on a union of linear subspaces. The first contribution of our work is to insert multiple fully-connected linear layers between the encoder layers and their corresponding decoder layers to promote learning more favorable representations for subspace clustering. These connection layers facilitate the feature learning procedure by combining low-level and high-level information for generating multiple sets of self-expressive and informative representations at different levels of the encoder. Moreover, we introduce a novel loss minimization problem which leverages an initial clustering of the samples to effectively fuse the multi-level representations and recover the underlying subspaces more accurately. The loss function is then minimized through an iterative scheme which alternatively updates the network parameters and produces new clusterings of the samples. Experiments on four real-world datasets demonstrate that our approach exhibits superior performance compared to the state-of-the-art methods on most of the subspace clustering problems.