Reviewerly/exHarmony huggingface dataset:
Spelling Error Correction with Soft-Masked BERT

Spelling error correction is an important yet challenging task because a satisfactory solution of it essentially needs human-level language understanding ability.Without loss of generality we consider Chinese spelling error correction (CSC) in this paper.A state-ofthe-art method for the task selects a character from a list of candidates for correction (including non-correction) at each position of the sentence on the basis of BERT, the language representation model.The accuracy of the method can be sub-optimal, however, because BERT does not have sufficient capability to detect whether there is an error at each position, apparently due to the way of pre-training it using mask language modeling.In this work, we propose a novel neural architecture to address the aforementioned issue, which consists of a network for error detection and a network for error correction based on BERT, with the former being connected to the latter with what we call soft-masking technique.Our method of using 'Soft-Masked BERT' is general, and it may be employed in other language detectioncorrection problems.Experimental results on two datasets demonstrate that the performance of our proposed method is significantly better than the baselines including the one solely based on BERT.
Collaborative city digital twin for the COVID-19 pandemic: A federated learning solution

The novel coronavirus, COVID-19, has caused a crisis that affects all segments of the population.As the knowledge and understanding of COVID-19 evolve, an appropriate response plan for this pandemic is considered one of the most effective methods for controlling the spread of the virus.Recent studies indicate that a city Digital Twin (DT) is beneficial for tackling this health crisis, because it can construct a virtual replica to simulate factors, such as climate conditions, response policies, and people's trajectories, to help plan efficient and inclusive decisions.However, a city DTsystem relies on long-term and high-quality data collection to make appropriate decisions, limiting its advantages when facing urgent crises, such as the COVID-19 pandemic.Federated Learning (FL), in which all clients can learn a shared model while retaining all training data locally, emerges as a promising solution for accumulating the insights from multiple data sources efficiently.Furthermore, the enhanced privacy protection settings removing the privacy barriers lie in this collaboration.In this work, we propose a framework that fused city DT with FL to achieve a novel collaborative paradigm that allows multiple city DTs to share the local strategy and status quickly.In particular, an FL central server manages the local updates of multiple collaborators (city DTs), providing a global model that is trained in multiple iterations at different city DT systems until the model gains the correlations between various response plans and infection trends.This approach means a collaborative city DT paradigm fused with FL techniques can obtain knowledge and patterns from multiple DTs and eventually establish a "global view" of city crisis management.Meanwhile, it also helps improve each city's DT by consolidating other DT's data without violating privacy rules.In this paper, we use the COVID-19 pandemic as the use case of the proposed framework.The experimental results on a real dataset with various response plans validate our proposed solution and demonstrate its superior performance.
DRL360: 360-degree Video Streaming with Deep Reinforcement Learning

360-degree videos have gained more popularity in recent years, owing to the great advance of panoramic cameras and head-mounted devices. However, as 360-degree videos are usually in high resolution, transmitting the content requires extremely high bandwidth. To protect the Quality of Experience (QoE) of users, researchers have proposed tile-based 360-degree video streaming systems that allocate high/low bit rates to selected tiles of video frames for streaming over the limited bandwidth. It is challenging to determine which tiles should be allocated with a high/low rate, because (1) the video playbacks include too many features that dynamically change over time when making the rate allocation; (2) most of the state-of-the-art systems focus on a fixed set of heuristics to optimize a specific QoE objective, while users may have various QoE objectives that need to be optimized in different ways. This paper presents a Deep Reinforcement Learning (DRL) based framework for 360-degree video streaming, named DRL360. The DRL360 framework helps improve the system performance by jointly optimizing multiple QoE objectives across a broad set of dynamic features. The DRL-based model adaptively allocates rates for the tiles of the future video frames based on the observations collected by client video players. We compare the proposed DRL360 to the existing systems by trace-driven evaluations as well as conducting a realworld experiment over a wide variety of network conditions. Evaluation results reveal that DRL360 can adapt to all considered scenarios, and outperform the state-of-the-art approaches by 20%-30% on average given different QoE objectives.
Hyperspectral Image Denoising and Anomaly Detection Based on Low-Rank and Sparse Representations

Hyperspectral imaging measures the amount of electromagnetic energy across the instantaneous field of view at a very high resolution in hundreds or thousands of spectral channels. This enables objects to be detected and the identification of materials that have subtle differences between them. However, the increase in spectral resolution often means that there is a decrease in the number of photons received in each channel, which means that the noise linked to the image formation process is greater. This degradation limits the quality of the extracted information and its potential applications. Thus, denoising is a fundamental problem in hyperspectral image (HSI) processing. As images of natural scenes with highly correlated spectral channels, HSIs are characterized by a high level of self-similarity and can be well approximated by low-rank representations. These characteristics underlie the state-of-the-art methods used in HSI denoising. However, where there are rarely occurring pixel types, the denoising performance of these methods is not optimal, and the subsequent detection of these pixels may be compromised. To address these hurdles, in this article, we introduce RhyDe (Robust hyperspectral Denoising), a powerful HSI denoiser, which implements explicit low-rank representation, promotes self-similarity, and, by using a form of collaborative sparsity, preserves rare pixels. The denoising and detection effectiveness of the proposed robust HSI denoiser is illustrated using semireal and real data.
Progress and Challenges in Intelligent Remote Sensing Satellite Systems

Due to advances in remote sensing satellite imaging and image processing technologies and their wide applications, intelligent remote sensing satellites are facing an opportunity for rapid development. The key technologies, standards, and laws of intelligent remote sensing satellites are also experiencing a series of new challenges. Novel concepts and key technologies in the intelligent hyperspectral remote sensing satellite system have been proposed since 2011. The aim of these intelligent remote sensing satellites is to provide real-time, accurate, and personalized remote sensing information services. This article reviews the current developments in new-generation intelligent remote sensing satellite systems, with a focus on intelligent remote sensing satellite platforms, imaging payloads, onboard processing systems, and other key technological chains. The technological breakthroughs and current defects of intelligence-oriented designs are also analyzed. Intelligent remote sensing satellites collect personalized remote sensing data and information, with real-time data features and information interaction between remote sensing satellites or between satellites and the ground. Such developments will expand the use of remote sensing applications beyond government departments and industrial users to a massive number of individual users. However, this extension faces challenges regarding privacy protection, societal values, and laws regarding the sharing and distribution of data and information.
The Effects of Sharing Awareness Cues in Collaborative Mixed Reality

Augmented and Virtual Reality provide unique capabilities for Mixed Reality collaboration. This paper explores how different combinations of virtual awareness cues can provide users with valuable information about their collaborator’s attention and actions. In a user study (n=32, 16 pairs), we compared different combinations of three cues: Field-of-View (FoV) frustum, Eye-gaze ray, and Head-gaze ray against a baseline condition showing only virtual representations of each collaborator’s head and hands. Through a collaborative object finding and placing task, the results showed that awareness cues significantly improved user performance, usability, and subjective preferences, with the combination of the FoV frustum and the Head-gaze ray being best. This work establishes the feasibility of room-scale MR collaboration and the utility of providing virtual awareness cues.
Reinforcement Learning and Deep Learning Based Lateral Control for Autonomous Driving [Application Notes]

t his paper investigates the vision- based autonomous driving with deep learning and reinforcement learning methods.Different from the end-to-end learning method, our method breaks the vision-based lateral control system down into a perception module and a control module.The perception module which is based on a multi-task learning neural network first takes a driver-view image as its input and predicts the track features.The control module which is based on reinforcement learning then makes a control decision based on these features.In order to improve the data efficiency, we propose visual TORCS (VTORCS), a deep reinforcement learning environment which is based on the open racing car simulator (TORCS).By means of the provided functions, one can train an agent with the input of an image or various physical sensor measurement, or evaluate the perception algorithm on this simulator.The trained reinforcement learning controller outperforms the linear quadratic regulator (LQR) controller and model predictive control (MPC) controller on different tracks.The experiments demonstrate that the perception module shows promising performance and the controller is capable of controlling the vehicle drive well along the track center with visual input.
Deep learning for automatic Gleason pattern classification for grade group determination of prostate biopsies

Histopathologic grading of prostate cancer using Gleason patterns (GPs) is subject to a large inter-observer variability, which may result in suboptimal treatment of patients. With the introduction of digitization and whole-slide images of prostate biopsies, computer-aided grading becomes feasible. Computer-aided grading has the potential to improve histopathological grading and treatment selection for prostate cancer. Automated detection of GPs and determination of the grade groups (GG) using a convolutional neural network. In total, 96 prostate biopsies from 38 patients are annotated on pixel-level. Automated detection of GP 3 and GP ≥ 4 in digitized prostate biopsies is performed by re-training the Inception-v3 convolutional neural network (CNN). The outcome of the CNN is subsequently converted into probability maps of GP ≥ 3 and GP ≥ 4, and the GG of the whole biopsy is obtained according to these probability maps. Differentiation between non-atypical and malignant (GP ≥ 3) areas resulted in an accuracy of 92% with a sensitivity and specificity of 90 and 93%, respectively. The differentiation between GP ≥ 4 and GP ≤ 3 was accurate for 90%, with a sensitivity and specificity of 77 and 94%, respectively. Concordance of our automated GG determination method with a genitourinary pathologist was obtained in 65% (κ = 0.70), indicating substantial agreement. A CNN allows for accurate differentiation between non-atypical and malignant areas as defined by GPs, leading to a substantial agreement with the pathologist in defining the GG.
EEMEFN: Low-Light Image Enhancement via Edge-Enhanced Multi-Exposure Fusion Network

This work focuses on the extremely low-light image enhancement, which aims to improve image brightness and reveal hidden information in darken areas. Recently, image enhancement approaches have yielded impressive progress. However, existing methods still suffer from three main problems: (1) low-light images usually are high-contrast. Existing methods may fail to recover images details in extremely dark or bright areas; (2) current methods cannot precisely correct the color of low-light images; (3) when the object edges are unclear, the pixel-wise loss may treat pixels of different objects equally and produce blurry images. In this paper, we propose a two-stage method called Edge-Enhanced Multi-Exposure Fusion Network (EEMEFN) to enhance extremely low-light images. In the first stage, we employ a multi-exposure fusion module to address the high contrast and color bias issues. We synthesize a set of images with different exposure time from a single image and construct an accurate normal-light image by combining well-exposed areas under different illumination conditions. Thus, it can produce realistic initial images with correct color from extremely noisy and low-light images. Secondly, we introduce an edge enhancement module to refine the initial images with the help of the edge information. Therefore, our method can reconstruct high-quality images with sharp edges when minimizing the pixel-wise loss. Experiments on the See-in-the-Dark dataset indicate that our EEMEFN approach achieves state-of-the-art performance.
Learning to Solve NP-Complete Problems: A Graph Neural Network for Decision TSP

Graph Neural Networks (GNN) are a promising technique for bridging differential programming and combinatorial domains. GNNs employ trainable modules which can be assembled in different configurations that reflect the relational structure of each problem instance. In this paper, we show that GNNs can learn to solve, with very little supervision, the decision variant of the Traveling Salesperson Problem (TSP), a highly relevant NP-Complete problem. Our model is trained to function as an effective message-passing algorithm in which edges (embedded with their weights) communicate with vertices for a number of iterations after which the model is asked to decide whether a route with cost &lt; C exists. We show that such a network can be trained with sets of dual examples: given the optimal tour cost C∗, we produce one decision instance with target cost x% smaller and one with target cost x% larger than C∗. We were able to obtain 80% accuracy training with −2%,+2% deviations, and the same trained model can generalize for more relaxed deviations with increasing performance. We also show that the model is capable of generalizing for larger problem sizes. Finally, we provide a method for predicting the optimal route cost within 2% deviation from the ground truth. In summary, our work shows that Graph Neural Networks are powerful enough to solve NP-Complete problems which combine symbolic and numeric data.
Federated learning for computational pathology on gigapixel whole slide images

Deep Learning-based computational pathology algorithms have demonstrated profound ability to excel in a wide array of tasks that range from characterization of well known morphological phenotypes to predicting non human-identifiable features from histology such as molecular alterations. However, the development of robust, adaptable and accurate deep learning-based models often rely on the collection and time-costly curation large high-quality annotated training data that should ideally come from diverse sources and patient populations to cater for the heterogeneity that exists in such datasets. Multi-centric and collaborative integration of medical data across multiple institutions can naturally help overcome this challenge and boost the model performance but is limited by privacy concerns among other difficulties that may arise in the complex data sharing process as models scale towards using hundreds of thousands of gigapixel whole slide images. In this paper, we introduce privacy-preserving federated learning for gigapixel whole slide images in computational pathology using weakly-supervised attention multiple instance learning and differential privacy. We evaluated our approach on two different diagnostic problems using thousands of histology whole slide images with only slide-level labels. Additionally, we present a weakly-supervised learning framework for survival prediction and patient stratification from whole slide images and demonstrate its effectiveness in a federated setting. Our results show that using federated learning, we can effectively develop accurate weakly-supervised deep learning models from distributed data silos without direct data sharing and its associated complexities, while also preserving differential privacy using randomized noise generation. We also make available an easy-to-use federated learning for computational pathology software package: http://github.com/mahmoodlab/HistoFL.
Training Complex Models with Multi-Task Weak Supervision

As machine learning models continue to increase in complexity, collecting large hand-labeled training sets has become one of the biggest roadblocks in practice. Instead, weaker forms of supervision that provide noisier but cheaper labels are often used. However, these weak supervision sources have diverse and unknown accuracies, may output correlated labels, and may label different tasks or apply at different levels of granularity. We propose a framework for integrating and modeling such weak supervision sources by viewing them as labeling different related sub-tasks of a problem, which we refer to as the multi-task weak supervision setting. We show that by solving a matrix completion-style problem, we can recover the accuracies of these multi-task sources given their dependency structure, but without any labeled data, leading to higher-quality supervision for training an end model. Theoretically, we show that the generalization error of models trained with this approach improves with the number of unlabeled data points, and characterize the scaling with respect to the task and dependency structures. On three fine-grained classification problems, we show that our approach leads to average gains of 20.2 points in accuracy over a traditional supervised approach, 6.8 points over a majority vote baseline, and 4.1 points over a previously proposed weak supervision method that models tasks separately.
Error Feedback Fixes SignSGD and other Gradient Compression Schemes.

Sign-based algorithms (e.g. signSGD) have been proposed as a biased gradient compression technique to alleviate the communication bottleneck in training large neural networks across multiple workers. We show simple convex counter-examples where signSGD does not converge to the optimum. Further, even when it does converge, signSGD may generalize poorly when compared with SGD. These issues arise because of the biased nature of the sign compression operator. We then show that using error-feedback, i.e. incorporating the error made by the compression operator into the next step, overcomes these issues. We prove that our algorithm EF-SGD with arbitrary compression operator achieves the same rate of convergence as SGD without any additional assumptions. Thus EF-SGD achieves gradient compression for free. Our experiments thoroughly substantiate the theory and show that error-feedback improves both convergence and generalization. Code can be found at \url{this https URL}.
Aligning Cross-Lingual Entities with Multi-Aspect Information

Multilingual knowledge graphs (KGs), such as YAGO and DBpedia, represent entities in different languages.The task of cross-lingual entity alignment is to match entities in a source language with their counterparts in target languages.In this work, we investigate embedding-based approaches to encode entities from multilingual KGs into the same vector space, where equivalent entities are close to each other.Specifically, we apply graph convolutional networks (GCNs) to combine multiaspect information of entities, including topological connections, relations, and attributes of entities, to learn entity embeddings.To exploit the literal descriptions of entities expressed in different languages, we propose two uses of a pretrained multilingual BERT model to bridge cross-lingual gaps.We further propose two strategies to integrate GCN-based and BERTbased modules to boost performance.Extensive experiments on two benchmark datasets demonstrate that our method significantly outperforms existing systems.
MarioNETte: Few-Shot Face Reenactment Preserving Identity of Unseen Targets

When there is a mismatch between the target identity and the driver identity, face reenactment suffers severe degradation in the quality of the result, especially in a few-shot setting. The identity preservation problem, where the model loses the detailed information of the target leading to a defective output, is the most common failure mode. The problem has several potential sources such as the identity of the driver leaking due to the identity mismatch, or dealing with unseen large poses. To overcome such problems, we introduce components that address the mentioned problem: image attention block, target feature alignment, and landmark transformer. Through attending and warping the relevant features, the proposed architecture, called MarioNETte, produces high-quality reenactments of unseen identities in a few-shot setting. In addition, the landmark transformer dramatically alleviates the identity preservation problem by isolating the expression geometry through landmark disentanglement. Comprehensive experiments are performed to verify that the proposed framework can generate highly realistic faces, outperforming all other baselines, even under a significant mismatch of facial characteristics between the target and the driver.
XAI Meets Mobile Traffic Classification: Understanding and Improving Multimodal Deep Learning Architectures

The increasing diffusion of mobile devices has dramatically changed the network traffic landscape, with Traffic Classification (TC) surging into a fundamental role while facing new and unprecedented challenges. The recent and appealing adoption of Deep Learning (DL) techniques has risen as the solution overcoming the performance of ML techniques based on tedious and time-consuming handcrafted feature design. Still, the black-box nature of DL models prevents its practical and trustful adoption in critical scenarios where the reliability/interpretation of results/policies is of key importance. To cope with these limitations, eXplainable Artificial Intelligence (XAI) techniques have recently acquired the interest of the community. Accordingly, in this work we investigate trustworthiness and interpretability via XAI-based techniques to understand, interpret and improve the behavior of state-of-the-art multimodal DL traffic classifiers. The proposed methodology, as opposed to common results seen in XAI, attempts to provide global interpretation, rather than sample-based ones. Results, based on an open dataset, allow to complement the above findings with domain knowledge.