Reviewerly/exHarmony huggingface dataset:
SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check

Chinese Spelling Check (CSC) is a task to detect and correct spelling errors in Chinese natural language. Existing methods have made attempts to incorporate the similarity knowledge between Chinese characters. However, they take the similarity knowledge as either an external input resource or just heuristic rules. This paper proposes to incorporate phonological and visual similarity knowledge into language models for CSC via a specialized graph convolutional network (SpellGCN). The model builds a graph over the characters, and SpellGCN is learned to map this graph into a set of inter-dependent character classifiers. These classifiers are applied to the representations extracted by another network, such as BERT, enabling the whole network to be end-to-end trainable. Experiments are conducted on three human-annotated datasets. Our method achieves superior performance against previous models by a large margin.
In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation

The comparative losses (typically, triplet loss) are appealing choices for learning person re-identification (ReID) features. However, the triplet loss is computationally much more expensive than the (practically more popular) classification loss, limiting their wider usage in massive datasets. Moreover, the abundance of label noise and outliers in ReID datasets may also put the margin-based loss in jeopardy. This work addresses the above two shortcomings of triplet loss, extending its effectiveness to large-scale ReID datasets with potentially noisy labels. We propose a fast-approximated triplet (FAT) loss, which provably converts the point-wise triplet loss into its upper bound form, consisting of a point-to-set loss term plus cluster compactness regularization. It preserves the effectiveness of triplet loss, while leading to linear complexity to the training set size. A label distillation strategy is further designed to learn refined soft-labels in place of the potentially noisy labels, from only an identified subset of confident examples, through teacher-student networks. We conduct extensive experiments on three most popular ReID benchmarks (Market-1501, DukeMTMC-reID, and MSMT17), and demonstrate that FAT loss with distilled labels lead to ReID features with remarkable accuracy, efficiency, robustness, and direct transferability to unseen datasets.
Text Sentiment Orientation Analysis Based on Multi-Channel CNN and Bidirectional GRU With Attention Mechanism

Convolutional Neural Network(CNN) and Recurrent Neural Network(RNN) have been widely used in the field of text sentiment analysis and have achieved good results. However, there is an anteroposterior dependency between texts, although CNN can extract local information between consecutive words of a sentence, it ignores the contextual semantic information between words. Bidirectional GRU can make up for the shortcomings that CNN can't extract contextual semantic information of long text, but it can't extract the local features of the text as well as CNN. Therefore, we propose a multi-channel model that combines the CNN and the bidirectional gated recurrent unit network with attention mechanism (MC-AttCNN-AttBiGRU). The model can pay attention to the words that are important to the sentiment polarity classification in the sentence through the attention mechanism and combine the advantages of CNN to extract local features of text and bidirectional GRU to extract contextual semantic information of long text, which improves the text feature extraction ability of the model. The experimental results on the IMDB dataset and Yelp 2015 dataset show that the proposed model can extract more rich text features than other baseline models, and can achieve better results than other baseline models.
Dependable Scheduling for Real-Time Workflows on Cyber–Physical Cloud Systems

Cyber-physical cloud systems (CPCS) are integrations of cyber-physical systems (CPS) and cloud computing infrastructures. Integrating CPS into cloud computing infrastructures could improve the performance in many aspects. However, new reliability and security challenges are also introduced. This fact highlights the need to develop novel methodologies to tackle these challenges in CPCS. To this end, this article is oriented toward enhancing the soft-error reliability of real-time workflows on CPCS while satisfying the lifetime reliability, security, and real-time constraints. In this article, we propose a dependable algorithm for scheduling workflow applications on CPCS. The proposed algorithm uses slack to recover failed tasks and allows all tasks to share the available slack in the system. To improve soft-error reliability, the algorithm first determines the priority of tasks, then assigns the maximum frequency to each task, and finally assigns the recoveries to tasks dynamically. Slack also can be used to utilize security services for satisfying system security requirements. The lifetime reliability constraint is met by dynamically scaling down the operating frequency of low-priority tasks. Extensive experiments on real-world workflow benchmarks demonstrate that the proposed scheme reduces the probability of failure by up to $52.1\%$ and improves the scheduling feasibility by up to $83.5\%$ compared to a number of representative approaches.
Evolutionary artificial neural networks for accurate solar radiation prediction

This paper evaluates the performance of different evolutionary neural network models in a problem of solar radiation prediction at Toledo, Spain. The prediction problem has been tackled exclusively from satellite-based measurements and variables, which avoids the use of data from ground stations or atmospheric soundings. Specifically, three types of neural computation approaches are considered: neural networks with sigmoid-based neurons, radial basis function units and product units. In all cases these neural computation algorithms are trained by means of evolutionary algorithms, leading to robust and accurate models for solar radiation prediction. The results obtained in the solar radiation estimation at the radiometric station of Toledo show an excellent performance of evolutionary neural networks tested. The structure sigmoid unit-product unit with evolutionary training has been shown as the best model among all tested in this paper, able to obtain an extremely accurate prediction of the solar radiation from satellite images data, and outperforming all other evolutionary neural networks tested, and alternative Machine Learning approaches such as Support Vector Regressors or Extreme Learning Machines.
Small Object Detection in Remote Sensing Images Based on Super-Resolution with Auxiliary Generative Adversarial Networks

This article tackles the problem of detecting small objects in satellite or aerial remote sensing images by relying on super-resolution to increase image spatial resolution, thus the size and details of objects to be detected. We show how to improve the super-resolution framework starting from the learning of a generative adversarial network (GAN) based on residual blocks and then its integration into a cycle model. Furthermore, by adding to the framework an auxiliary network tailored for object detection, we considerably improve the learning and the quality of our final super-resolution architecture, and more importantly increase the object detection performance. Besides the improvement dedicated to the network architecture, we also focus on the training of super-resolution on target objects, leading to an object-focused approach. Furthermore, the proposed strategies do not depend on the choice of a baseline super-resolution framework, hence could be adopted for current and future state-of-the-art models. Our experimental study on small vehicle detection in remote sensing data conducted on both aerial and satellite images (i.e., ISPRS Potsdam and xView datasets) confirms the effectiveness of the improved super-resolution methods to assist with the small object detection tasks.
Edge-Enhanced Graph Convolution Networks for Event Detection with Syntactic Relation

Event detection (ED), a key subtask of information extraction, aims to recognize instances of specific event types in text.Previous studies on the task have verified the effectiveness of integrating syntactic dependency into graph convolutional networks.However, these methods usually ignore dependency label information, which conveys rich and useful linguistic knowledge for ED.In this paper, we propose a novel architecture named Edge-Enhanced Graph Convolution Networks (EE-GCN), which simultaneously exploits syntactic structure and typed dependency label information to perform ED.Specifically, an edge-aware node update module is designed to generate expressive word representations by aggregating syntactically-connected words through specific dependency types.Furthermore, to fully explore clues hidden in dependency edges, a node-aware edge update module is introduced, which refines the relation representations with contextual information.These two modules are complementary to each other and work in a mutual promotion way.We conduct experiments on the widely used ACE2005 dataset and the results show significant improvement over competitive baseline methods 1 .
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models

Existing pre-trained large language models have shown unparalleled generative capabilities.However, they are not controllable.In this paper, we propose MEGATRON-CNTRL, a novel framework that uses large-scale language models and adds control to text generation by incorporating an external knowledge base.Our framework consists of a keyword predictor, a knowledge retriever, a contextual knowledge ranker, and a conditional text generator.As we do not have access to groundtruth supervision for the knowledge ranker, we make use of weak supervision from sentence embedding.The empirical results show that our model generates more fluent, consistent, and coherent stories with less repetition and higher diversity compared to prior work on the ROC story dataset.We showcase the controllability of our model by replacing the keywords used to generate stories and re-running the generation process.Human evaluation results show that 77.5% of these stories are successfully controlled by the new keywords.Furthermore, by scaling our model from 124 million to 8.3 billion parameters we demonstrate that larger models improve both the quality of generation (from 74.5% to 93.0% for consistency) and controllability (from 77.5% to 91.5%).
Mixture Hidden Markov Models for Sequence Data: The <b>seqHMM</b> Package in <i>R</i>

Sequence analysis is being more and more widely used for the analysis of social sequences and other multivariate categorical time series data.However, it is often complex to describe, visualize, and compare large sequence data, especially when there are multiple parallel sequences per subject.Hidden (latent) Markov models (HMMs) are able to detect underlying latent structures and they can be used in various longitudinal settings: to account for measurement error, to detect unobservable states, or to compress information across several types of observations.Extending to mixture hidden Markov models (MHMMs) allows clustering data into homogeneous subsets, with or without external covariates.The seqHMM package in R is designed for the efficient modeling of sequences and other categorical time series data containing one or multiple subjects with one or multiple interdependent sequences using HMMs and MHMMs.Also other restricted variants of the MHMM can be fitted, e.g., latent class models, Markov models, mixture Markov models, or even ordinary multinomial regression models with suitable parameterization of the HMM.Good graphical presentations of data and models are useful during the whole analysis process from the first glimpse at the data to model fitting and presentation of results.The package provides easy options for plotting parallel sequence data, and proposes visualizing HMMs as directed graphs.
BLEU might be Guilty but References are not Innocent

The quality of automatic metrics for machine translation has been increasingly called into question, especially for high-quality systems.This paper demonstrates that, while choice of metric is important, the nature of the references is also critical.We study different methods to collect references and compare their value in automated evaluation by reporting correlation with human evaluation for a variety of systems and metrics.Motivated by the finding that typical references exhibit poor diversity, concentrating around translationese language, we develop a paraphrasing task for linguists to perform on existing reference translations, which counteracts this bias.Our method yields higher correlation with human judgment not only for the submissions of WMT 2019 English→German, but also for Back-translation and APE augmented MT output, which have been shown to have low correlation with automatic metrics using standard references.We demonstrate that our methodology improves correlation with all modern evaluation metrics we look at, including embedding-based methods.To complete this picture, we reveal that multireference BLEU does not improve the correlation for high quality output, and present an alternative multi-reference formulation that is more effective.
DB-LSTM: Densely-connected Bi-directional LSTM for human action recognition

Although deep learning has achieved promising progress recently, action recognition remains a challenging task, due to cluttered backgrounds, diverse scenes, occlusions, viewpoint variations and camera motions. In this paper, we propose a novel deep learning model to capture the spatial and temporal patterns of human actions from videos. Sample representation learner is proposed to extract the video-level temporal feature, which combines the sparse temporal sampling and long-range temporal learning to form an efficient and effective training strategy. To boost the effectiveness and robustness of modeling long-range action recognition, a Densely-connected Bi-directional LSTM (DB-LSTM) network is novelly proposed to model the visual and temporal associations in both forward and backward directions. They are stacked and integrated with the dense skip-connections to improve the capability of temporal pattern modeling. Two modalities from appearance and motion are integrated with a fusion module to further improve the performance. Experiments conducted on two benchmark datasets, UCF101 and HMDB51, demonstrate that the proposed DB-LSTM model achieves promising performance, which outperforms the state-of-the-art approaches for action recognition.
Scalable Active Learning for Object Detection

Deep Neural Networks trained in a fully supervised fashion are the dominant technology in perception-based autonomous driving systems. While collecting large amounts of unlabeled data is already a major undertaking, only a subset of it can be labeled by humans due to the effort needed for high-quality annotation. Therefore, finding the right data to label has become a key challenge. Active learning is a powerful technique to improve data efficiency for supervised learning methods, as it aims at selecting the smallest possible training set to reach a required performance. We have built a scalable production system for active learning in the domain of autonomous driving. In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, present our current results at scale, and briefly describe the open problems and future directions.
Domain Knowledge Powered Deep Learning for Breast Cancer Diagnosis Based on Contrast-Enhanced Ultrasound Videos

In recent years, deep learning has been widely used in breast cancer diagnosis, and many high-performance models have emerged. However, most of the existing deep learning models are mainly based on static breast ultrasound (US) images. In actual diagnostic process, contrast-enhanced ultrasound (CEUS) is a commonly used technique by radiologists. Compared with static breast US images, CEUS videos can provide more detailed blood supply information of tumors, and therefore can help radiologists make a more accurate diagnosis. In this paper, we propose a novel diagnosis model based on CEUS videos. The backbone of the model is a 3D convolutional neural network. More specifically, we notice that radiologists generally follow two specific patterns when browsing CEUS videos. One pattern is that they focus on specific time slots, and the other is that they pay attention to the differences between the CEUS frames and the corresponding US images. To incorporate these two patterns into our deep learning model, we design a domain-knowledge-guided temporal attention module and a channel attention module. We validate our model on our Breast-CEUS dataset composed of 221 cases. The result shows that our model can achieve a sensitivity of 97.2% and an accuracy of 86.3%. In particular, the incorporation of domain knowledge leads to a 3.5% improvement in sensitivity and a 6.0% improvement in specificity. Finally, we also prove the validity of two domain knowledge modules in the 3D convolutional neural network (C3D) and the 3D ResNet (R3D).
A Threshold for Quantum Advantage in Derivative Pricing

We give an upper bound on the resources required for valuable quantum advantage in pricing derivatives. To do so, we give the first complete resource estimates for useful quantum derivative pricing, using autocallable and Target Accrual Redemption Forward (TARF) derivatives as benchmark use cases. We uncover blocking challenges in known approaches and introduce a new method for quantum derivative pricing – the<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mtext class="MJX-tex-mathit" mathvariant="italic">re-parameterization method</mml:mtext></mml:mrow></mml:math>– that avoids them. This method combines pre-trained variational circuits with fault-tolerant quantum computing to dramatically reduce resource requirements. We find that the benchmark use cases we examine require 8k logical qubits and a T-depth of 54 million. We estimate that quantum advantage would require executing this program at the order of a second. While the resource requirements given here are out of reach of current systems, we hope they will provide a roadmap for further improvements in algorithms, implementations, and planned hardware architectures.
Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction

Aspect Sentiment Triplet Extraction (ASTE) is the most recent subtask of ABSA which outputs triplets of an aspect target, its associated sentiment, and the corresponding opinion term.Recent models perform the triplet extraction in an end-to-end manner but heavily rely on the interactions between each target word and opinion word.Thereby, they cannot perform well on targets and opinions which contain multiple words.Our proposed span-level approach explicitly considers the interaction between the whole spans of targets and opinions when predicting their sentiment relation.Thus, it can make predictions with the semantics of whole spans, ensuring better sentiment consistency.To ease the high computational cost caused by span enumeration, we propose a dual-channel span pruning strategy by incorporating supervision from the Aspect Term Extraction (ATE) and Opinion Term Extraction (OTE) tasks.This strategy not only improves computational efficiency but also distinguishes the opinion and target spans more properly.Our framework simultaneously achieves strong performance for the ASTE as well as ATE and OTE tasks.In particular, our analysis shows that our spanlevel approach achieves more significant improvements over the baselines on triplets with multi-word targets or opinions. 1
A universal fully reconfigurable 12-mode quantum photonic processor

Abstract Photonic processors are pivotal for both quantum and classical information processing tasks using light. In particular, linear optical quantum information processing requires both large-scale and low-loss programmable photonic processors. In this paper, we report the demonstration of the largest universal quantum photonic processor to date: a low-loss 12-mode fully tunable linear interferometer with all-to-all mode coupling based on stoichiometric silicon nitride waveguides.
Deep Network Approximation for Smooth Functions

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 22 June 2020Accepted: 21 June 2021Published online: 30 September 2021Keywordsdeep ReLU network, smooth function, polynomial approximation, function composition, curse of dimensionalityAMS Subject Headings78M32, 41A25, 41A63Publication DataISSN (print): 0036-1410ISSN (online): 1095-7154Publisher: Society for Industrial and Applied MathematicsCODEN: sjmaah
Deep Learning in Cancer Diagnosis and Prognosis Prediction: A Minireview on Challenges, Recent Trends, and Future Directions

Deep learning (DL) is a branch of machine learning and artificial intelligence that has been applied to many areas in different domains such as health care and drug design. Cancer prognosis estimates the ultimate fate of a cancer subject and provides survival estimation of the subjects. An accurate and timely diagnostic and prognostic decision will greatly benefit cancer subjects. DL has emerged as a technology of choice due to the availability of high computational resources. The main components in a standard computer-aided design (CAD) system are preprocessing, feature recognition, extraction and selection, categorization, and performance assessment. Reduction of costs associated with sequencing systems offers a myriad of opportunities for building precise models for cancer diagnosis and prognosis prediction. In this survey, we provided a summary of current works where DL has helped to determine the best models for the cancer diagnosis and prognosis prediction tasks. DL is a generic model requiring minimal data manipulations and achieves better results while working with enormous volumes of data. Aims are to scrutinize the influence of DL systems using histopathology images, present a summary of state-of-the-art DL methods, and give directions to future researchers to refine the existing methods.
Deep adaptive learning for writer identification based on single handwritten word images

There are two types of information in each handwritten word image: explicit information which can be easily read or derived directly, such as lexical content or word length, and implicit attributes such as the author’s identity. Whether features learned by a neural network for one task can be used for another task remains an open question. In this paper, we present a deep adaptive learning method for writer identification based on single-word images using multi-task learning. An auxiliary task is added to the training process to enforce the emergence of reusable features. Our proposed method transfers the benefits of the learned features of a convolutional neural network from an auxiliary task such as explicit content recognition to the main task of writer identification in a single procedure. Specifically, we propose a new adaptive convolutional layer to exploit the learned deep features. A multi-task neural network with one or several adaptive convolutional layers is trained end-to-end, to exploit robust generic features for a specific main task, i.e., writer identification. Three auxiliary tasks, corresponding to three explicit attributes of handwritten word images (lexical content, word length and character attributes), are evaluated. Experimental results on two benchmark datasets show that the proposed deep adaptive learning method can improve the performance of writer identification based on single-word images, compared to non-adaptive and simple linear-adaptive approaches.