Reviewerly/exHarmony huggingface dataset:
Natural Language Processing for Clinical Laboratory Data Repository Systems: Implementation and Evaluation for Respiratory Viruses

Background With the growing volume and complexity of laboratory repositories, it has become tedious to parse unstructured data into structured and tabulated formats for secondary uses such as decision support, quality assurance, and outcome analysis. However, advances in natural language processing (NLP) approaches have enabled efficient and automated extraction of clinically meaningful medical concepts from unstructured reports. Objective In this study, we aimed to determine the feasibility of using the NLP model for information extraction as an alternative approach to a time-consuming and operationally resource-intensive handcrafted rule-based tool. Therefore, we sought to develop and evaluate a deep learning–based NLP model to derive knowledge and extract information from text-based laboratory reports sourced from a provincial laboratory repository system. Methods The NLP model, a hierarchical multilabel classifier, was trained on a corpus of laboratory reports covering testing for 14 different respiratory viruses and viral subtypes. The corpus includes 87,500 unique laboratory reports annotated by 8 subject matter experts (SMEs). The classification task involved assigning the laboratory reports to labels at 2 levels: 24 fine-grained labels in level 1 and 6 coarse-grained labels in level 2. A “label” also refers to the status of a specific virus or strain being tested or detected (eg, influenza A is detected). The model’s performance stability and variation were analyzed across all labels in the classification task. Additionally, the model's generalizability was evaluated internally and externally on various test sets. Results Overall, the NLP model performed well on internal, out-of-time (pre–COVID-19), and external (different laboratories) test sets with microaveraged F1-scores &gt;94% across all classes. Higher precision and recall scores with less variability were observed for the internal and pre–COVID-19 test sets. As expected, the model’s performance varied across categories and virus types due to the imbalanced nature of the corpus and sample sizes per class. There were intrinsically fewer classes of viruses being detected than those tested; therefore, the model's performance (lowest F1-score of 57%) was noticeably lower in the detected cases. Conclusions We demonstrated that deep learning–based NLP models are promising solutions for information extraction from text-based laboratory reports. These approaches enable scalable, timely, and practical access to high-quality and encoded laboratory data if integrated into laboratory information system repositories.
RETSR: An Effective Review-Enhanced and Time-Aware Sequential Recommendation Framework

Side information has been demonstrated effective for recommendation by many existing works. However, even recent advancements can hardly extract the rich interactive knowledge and accurately model users' true interests due to ignorance of irregularities of reviews or temporal information, e.g., uneven time intervals. Therefore in this paper, we propose a Review-Enhanced Time-aware Sequential Recommendation framework (called RETSR) including review selector, long/short-term preferences extractors, etc. It can effectively exploit rich sequential, semantic, and temporal information to solve the aforementioned issues. More specifically, a review selector is first designed to select some important comments for obtaining users' long-term preferences and items' essential properties. Furthermore, users' short-term preferences are obtained by extracting meaningful time-sensitive features from two perspectives (i.e., reviews and interactions). Finally, extensive experiments show our framework significantly outperforms eleven baselines or state-of-the-art models on three popular real-world benchmarks, revealing its power for sequential recommendation tasks.
Video Face Recognition Using Neural Aggregation Networks with Mutual Relational Learning

Video face recognition benefits profoundly from deep convolutional neural networks (CNNs), which learn robust feature embeddings. However, due to their fixed geometric structures, CNNs are inherently limited in modeling the significant variations from the angle, pose, occlusion and other factors of face images. In this paper, a neural aggregation network based on mutual relation learning is proposed for video face recognition. First, Intra-frame Relational Learning network (Intra-Net) is introduced, which models the interdependencies between the re-gional components of individual features and develops relevance between fine-grained features. Such processing can determine the region of interest adaptively according to the quality of the input face image to achieve the extraction of valuable information. Secondly, we introduce Inter-frame Relational Learning Network (Inter-Net), which considers the most significant appearance representation in the overall structure of the face image to cor-relate the complementarity of features between frames. Finally, information aggregation is performed by combining Inter-Net and Intra-Net. Joint optimization of the two branches allows our model to effectively exploit the complementary information between them to improve the aggregation capability. We validate the effectiveness of our model for video face recognition, proving its superiority over state-of-the-art methods on two benchmark datasets.
Image Inpainting with Context Flow Network

Image inpainting using deep and complicated convolutional neural networks(CNN) has recently produced outstanding results. Several researchers have considered employing large receptive fields and deep networks for long-distance information transfer to obtain semantically coherent inpainting results. As a side effect, these strategies would lead to the loss of detail and other artifacts. Motivated by the attention mechanism and sequence-to-sequence model, a novel convolution structure called context flow module is introduced into a coarse-to-fine two stages network, extracting information from distant regions without extra network layers or details loss. The context flow module in the refinement network can effectively gather both spatial and contextual data in the distance, and flow information to the next layer patch by patch. The coarse and refinement networks' backbones are encoder-decoder architecture stacked with gated and dilated convolutions. The refinement network encloses two extra elements: the context flow module and a feature-sharing space. The coarse network generates semantically consistent images with no gaps. The refinement network enhances the sharpness and enriches the details of the initial results. Moreover, a patch-based GAN is applied to stabilize training and generate semantically reasonable results. Experimental results show that our method excels at the performance of the state-of-the-art methods on faces(CelebA), buildings(Paris Street View), and natural images(Places2) datasets. The proposed context flow module can be easily integrated with any existing networks to improve their inpainting performance.
EDSC: Event Detection with Superior Conceptual Information

Event detection (ED) plays a critical role in information extraction, which aims to identify instances of specific types of events in text. The ambiguity in language expressions poses a great challenge for event detection, and many methods have been proposed to disambiguate event types. However, most existing methods for this task classify each event type independently, ignoring the potential hierarchical association between events. To address this issue, we propose a novel approach - dubbed the event detection with superior conceptual information (EDSC) framework - to use conceptual hierarchy information to improve the performance of event detection. The proposed model conducts event detection based on the hybrid neural network. Specifically, to obtain high-quality superior conceptual information, we propose an effective model, called a heterogeneous graph self-attention network based on dependency parsing, which can delve into syntactic and semantic information simultaneously. Furthermore, the superior conceptual information is integrated into the event detection process, which can provide effective inductive bias for better classification performance. Experiments on the ACE2005 dataset show that the proposed model achieves comparable results against the strong baselines in event detection. It also indicates that the conceptual hierarchy information between events can indeed improve the performance of event detection models.
Is it a bug or a feature? Identifying software bugs using graph attention networks

This paper proposes a novel approach for identifying software bugs by building on a meaningful combination of word embeddings, graph-based text representations and graph attention networks. Existing approaches aim to advance each of the above components individually, without considering an integrative approach. As a result, they ignore information that is related to either the structure of a given text or an individual word of the text. Instead, our approach seamlessly incorporates both semantic and structural characteristics into a graph, which are then fed to a graph attention network in order to classify GitHub issues as bugs or features. Our experimental results demonstrate a significant improvement in terms of accuracy, precision and recall of the proposed approach compared to a list of classical and graph-based machine learning models. The dataset for the experiments reported in this paper has been retrieved from the kaggle.com platform and concerns GitHub issues with short-text attributes.
Correlation Enhancement with Graph Convolutional Network for Pedestrian Attribute Recognition

Pedestrian attribute recognition aims to predict human attributes in surveillance video. Since the correlations are different among attribute groups, mining the correlation variation between attributes is an effective way to improve the performance of pedestrian attribute recognition. In this paper, we propose a Feature Enhanced Multi-scale and Graph Convolutional Compound Network named FEM-GCNet to learn the enhanced correlations between attributes. The FEM-GCNet contains an enhanced feature extractor and a graph convolutional network. The enhanced feature extractor learns the enhanced features of attributes, improving the feature representation. The graph convolutional network exploits attribute correlation rules to mine the correlation features between attributes, which are embedded into the enhanced features to obtain the enhanced attribute correlations. Finally, a multi-label loss is used to train our model to identify pedestrian attributes. Experiments on RAPv1 and RAPv2 datasets demonstrate the robustness of the enhanced attribute correlations and the better performance of the FEM-GCNet.
QDN: An Efficient Value Decomposition Method for Cooperative Multi-agent Deep Reinforcement Learning

Multi-agent systems have recently received significant attention from researchers in many scientific fields. The value factorization method is popular for scaling up cooperative reinforcement learning in multi-agent environments. However, the approximation of the joint value function may introduce a significant disparity between the estimated and actual joint reward value function, leading to a local optimum for cooperative multi-agent deep reinforcement learning. In addition, as the number of agents increases, the input space grows exponentially, negatively impacting the convergence performance of multi-agent algorithms. This work proposes an efficient multi-agent rein-forcement learning algorithm, QDN, to enhance the convergence performance in cooperative multi-agent tasks. The proposed QDN scheme utilizes a competitive network to enable the agents to learn the value of the environmental state without the influence of actions. Hence, the error between the estimated joint reward value function and the actual joint reward value function can be significantly reduced, preventing the emergence of sub-optimal actions. Meanwhile, the proposed QDN algorithm utilizes the parametric noise on the network weights to introduce random-ness in the network's weights so that the agents can explore the environments and states effectively, thereby improving the convergence performance of the QDN algorithm. We evaluate the proposed QDN scheme using the SMAC challenges with various map difficulties. Experimental results show that the QDN algorithm excels in the convergence speed and the success rate in all scenarios compared to some state-of-the-art methods. Further experiments using four additional multi-agent tasks demonstrate that the QDN algorithm is robust in various multi-agent tasks and can significantly improve the training convergence performance compared with the state-of-the-art methods.
Exploring Policy Diversity in Parallel Actor-Critic Learning

Exploration is a critical challenge for deep reinforcement learning methods. Although existing works such as actor-critic algorithms have made much progress, most still suffer from the sample inefficiency problem in complex environments where rewards are sparse. Parallel sampling, which uses multiple actors with the same policy interacting with the environment, is an effective approach to improve sample efficiency. However, parallel parameter-sharing actors collect similar samples, which generally hinders the improvement of the overall exploration process. In this paper, we propose a Policy Diversity enhanced approach for parallel Actor-Critic (PDAC). Specifically, we extend the parallel actor-critic architecture to the PDAC framework composed of a shared critic and parallel distinct actors. Then we introduce the KL-divergence of the action probability distribution between parallel actors as the intrinsic reward to encourage actors to explore diverse strategies. We evaluate our approach in multiple challenging procedurally-generated tasks and compare it with state-of-the-art algorithms. Experiments show that PDAC makes significant progress in the comparison, in terms of cumulative rewards and sample efficiency.
Using Noise and External Knowledge to Enhance Chinese Pre-trained Model

Pre-trained language models (PLMs) have the risk of overfitting pre-trained tasks and data in fine-tuning, while Chinese PLMs often ignore external knowledge such as word and sentence to learn representations. Therefore, we propose a Chinese PLM enhancement method using noise and external knowledge (NEK). NEK first adds different uniform noises to the PLM according to the standard deviation of different parameter matrices, so as to obtain the perturbed PLM. In the fine-tuning phase, NEK builds a heterogeneous linguistic graph based on external knowledge. This module adopts a graph-based approach to generalize information of different granularities in Chinese linguistics, and enhances Chinese PLM on this basis. Experimental results show that NEK brings performance improvements to a variety of different Chinese PLMs on six natural language processing tasks on eight benchmark datasets.
A Dual-View Knowledge Enhancing Self-Attention Network for Sequential Recommendation

Modeling user preferences from users' historical sequences is one of the core problems of sequential recommendation. Previous studies only considered transition patterns between user-items, ignoring transition patterns between item features and item-item interactions. Recently, there has been interest in integrating knowledge graphs as auxiliary information into sequential recommendation. Most of the existing methods deal with the heterogeneous information in the knowledge graph in a coarse-grained manner. We believe that fine-grained processing of information in knowledge graphs can help recommendation systems understand changes in user preferences, i.e. dividing hetero-geneous information into item-to-item relationships and item-to-attribute relationships. In this paper, we propose a dual-level self-attention network for sequential recommendation. Specifically, we divide the knowledge graph heterogeneous information about items into relation level and attribute level, representing item-item relationship and item-attribute relationship, respectively. Afterwards, the self-attention network is used to learn user preferences at dual-level, respectively. Then, the outputs of the above dual levels are integrated for next item recommendation. Based on extensive experiments on three real-world data sets, our model achieves significant improvements compared to state-of-the-art baseline methods.
A Deep Reinforcement Learning Heuristic for SAT based on Antagonist Graph Neural Networks

Heuristics are one of the most important tools to guide search to solve combinatorial problems. They are often specifically designed for one single problem and require both expertise and implementation work. Generic frameworks like SAT or CSP have developed heuristics that obey general principles like first fail or are able to learn and adapt from the exploration of the search tree like Dom/wDeg. In SAT, the classic VSIDS heuristic falls into both categories. The question of whether it is possible to learn from solving existing problems has been addressed for a long time by portfolio solvers where the best heuristic is chosen by Machine Learning from hand-crafted features, and more recently with Deep Learning by embedding this knowledge into a Graph Neural Network (GNN). In this paper, we build upon the latter category by proposing a new heuristic based on Deep Reinforcement Learning using two GNNs with adversarial rewards. We show that our method reduces the number of fails to get the first solution by more than 50% compared to MiniSat. This work shows the advantages of this type of techniques to extract structural and contextual knowledge from past solving experience.
A Dilated Transformer Network for Time Series Anomaly Detection

Unsupervised anomaly detection for time series has been an active research area due to its enormous potential for wireless network management. Existing works have made extraordinary progress in time series representation, reconstruction and forecasting. However, long-term temporal patterns prohibit the model from learning reliable dependencies. To this end, we propose a novel approach based on Transformer with dilated convolution for time anomaly detection. Specifically, we provide a dilated convolution module to extract long-term dependence features. Extensive experiments on various public benchmarks demonstrate that our method has achieved the state-of-the-art performance.
Improving Surrogate Model Prediction by Noise Injection into Autoencoder Latent Space

Autoencoders (AEs) represent a powerful tool for enhancing data-driven surrogate modeling by learning a lower-dimensional representation of high-dimensional data in an encoding-reconstructing fashion. Variational autoencoders (VAEs) improve interpolation capabilities of autoencoders by structuring the latent space with the Kullback-Liebler regularization term. However, learning a VAE poses practical challenges due to the difficulties on balancing the quality of prediction and the interpolation capability. Thus, a compromise between AEs and VAEs is needed to deliver robust predictive models. In this paper, an effective strategy, consisting on the injection of noise into the latent space of AEs, is proposed to improve the smoothness of the latent space of autoencoders while preserving the quality of reconstruction. The experimental results show that the model with the proposed noise injection technique outperforms AEs, VAEs and other alternatives in terms of quality of predictions.
Two-stage Modeling for Prediction with Confidence

The use of neural networks has been very successful in a wide variety of applications. However, it has recently been observed that it is difficult to generalize the performance of neural networks under the condition of distributional shift. Several efforts have been made to identify potential out-of-distribution inputs. Although existing literature has made significant progress with regard to images and textual data, finance has been overlooked. The aim of this paper is to investigate the distribution shift in the credit scoring problem, one of the most important applications of finance. For the potential distribution shift problem, we propose a novel two-stage model. Using the out-of-distribution detection method, data is first separated into confident and unconfident sets. As a second step, we utilize the domain knowledge with a mean-variance optimization in order to provide reliable bounds for unconfident samples. Using empirical results, we demonstrate that our model offers reliable predictions for the vast majority of datasets. It is only a small portion of the dataset that is inherently difficult to judge, and we leave them to the judgment of human beings. Based on the two-stage model, highly confident predictions have been made and potential risks associated with the model have been significantly reduced.
Towards building reliable deep learning based driver identification systems

Recent studies have shown the potential of leveraging neural networks to achieve high levels of accuracy in re-identifying drivers by learning latent features from vehicular sensor data. However, deploying such networks in real-world applications (like theft detection or fleet management) requires re-training the networks with new data to transfer the learnings from the initial dataset to the target drivers. In this paper, we highlight the importance of the evaluation of such networks in both phases, initial training and transfer learning. Our evaluation shows that the performance of existing solutions drops significantly, when applied to new drivers that have not been seen by the networks in the initial training phase. Moreover, we propose a deep neural network that outperforms state-of-the-art solutions in both phases. For the evaluation of the transfer learning phase, we use a dataset from a real-world ride-sharing service that has not been used in the initial training.
Multi-scale Intermediate Flow Estimation for Video Frame Interpolation

Video frame interpolation is one of the most chal-lenging tasks in video processing, which aims to synthesize intermediate frames between consecutive frames. In this work, we propose a flow-based approach called Multi-scale Intermediate Flow Estimation (MIFE) to balance the fineness and estimation range of the flows. MIFE consists of two main modules. Specifically, (1) Refined Flow Estimation uses a shifted window to estimate low-resolution intermediate flows at three levels. The refined full-resolution flow of each level is a weighted combination of nearby low-resolution flows, where the weights are determined by the similarity scores of the input frames and the reliability scores of the flows. (2) Multi-scale Flow Fusion generates fusion masks based on the estimable flow range and the estimated flow size. It fuses three levels of flows and refines the results. Experimental results show that the proposed method achieves good performance on various datasets. The source code is available at https://github.com/fzh169/MIFE.