Reviewerly/exHarmony huggingface dataset:
A Comparative Study of Imputation Methods for Multivariate Ordinal Data

Abstract Missing data remains a very common problem in large datasets, including survey and census data containing many ordinal responses, such as political polls and opinion surveys. Multiple imputation (MI) is usually the go-to approach for analyzing such incomplete datasets, and there are indeed several implementations of MI, including methods using generalized linear models, tree-based models, and Bayesian non-parametric models. However, there is limited research on the statistical performance of these methods for multivariate ordinal data. In this article, we perform an empirical evaluation of several MI methods, including MI by chained equations (MICE) using multinomial logistic regression models, MICE using proportional odds logistic regression models, MICE using classification and regression trees, MICE using random forest, MI using Dirichlet process (DP) mixtures of products of multinomial distributions, and MI using DP mixtures of multivariate normal distributions. We evaluate the methods using simulation studies based on ordinal variables selected from the 2018 American Community Survey. Under our simulation settings, the results suggest that MI using proportional odds logistic regression models, classification and regression trees, and DP mixtures of multinomial distributions generally outperform the other methods. In certain settings, MI using multinomial logistic regression models is able to achieve comparable performance, depending on the missing data mechanism and amount of missing data.
Thermal Image Processing via Physics-Inspired Deep Networks

We introduce DeepIR, a new thermal image processing framework that combines physically accurate sensor modeling with deep network-based image representation. Our key enabling observations are that the images captured by thermal sensors can be factored into slowly changing, scene-independent sensor non-uniformities (that can be accurately modeled using physics) and a scene-specific radiance flux (that is well-represented using a deep network-based regularizer). DeepIR requires neither training data nor periodic ground-truth calibration with a known black body target–making it well suited for practical computer vision tasks. We demonstrate the power of going DeepIR by developing new denoising and super-resolution algorithms that exploit multiple images of the scene captured with camera jitter. Simulated and real data experiments demonstrate that DeepIR can perform high-quality non-uniformity correction with as few as three images, achieving a 10dB PSNR improvement over competing approaches.
Classifying issue reports according to feature descriptions in a user manual based on a deep learning model

Issue reports are documents with which users report problems and state their opinions on a software system. Issue reports are useful for software maintenance, but managing them requires developers’ considerable manual effort. To reduce such effort, previous studies have mostly suggested methods for automatically classifying issue reports. However, most of those studies classify issue reports according to issue types, based only on whether the report is relevant to a bug, whether the report is duplicated, or whether the issue is functional or nonfunctional. In this paper, we intend to link issue reports and a user manual and so propose a deep learning model-based method that classifies issue reports according to software features that are described in the user manual in order to help developers relate issue reports to features to make changes to a software system. In order to classify issue reports according to the feature descriptions in a user manual, our method uses a deep learning technique with a word embedding technique. The key insight in our method is that the sections of a user manual that describe software features contain the words and sentences similar to those in issue reports. Based on the insight, we construct a classification model that learns the feature descriptions (i.e. sections) in a user manual and classifies issue reports according to the feature descriptions. We evaluate the proposed method by comparing its classification performance with that of the state-of-the-art method, TicketTagger. The experimental results show that the proposed method yields 10% ∼ 24% higher classification f1-score than that of TicketTagger. We also experiment with two deep learning models and four word embedding techniques and find out that the Convolution Neural Network model with FastText (or GloVe) yields the best performance. Our study shows the feasibility of classifying issue reports according to software features, which can be the basis for successive studies to classify issue reports into software features.
Time-series deep survival prediction for hemodialysis patients using an attention-based Bi-GRU network

The number of end-stage renal disease (ESRD) patients treated with hemodialysis (HD) has significantly increased, but the prognosis remains poor. Time-series features have been included in only a few studies to predict HD patient survival, and how to utilize such features effectively remains unclear. This article aims to develop a more accurate, interpretable, and clinically practical personalized survival prediction model for HD patients.This study proposed and evaluated an attention-based Bi-GRU network using time-series features for survival prediction. A distance-based loss function was proposed to improve performance. We used data from 1232 ESRD patients who received regular hemodialysis treatment for ≥ 3 months from 2007 to 2016 at the First Affiliated Hospital of Zhejiang University. The proposed model was compared with representative sequence modeling deep learning architectures and existing survival analysis methods in terms of the C-index and IBS value. Post hoc tests were used to test statistical significance. The attention map was used to assess feature importance over time. The impact of time-series changes on survival was investigated after controlling initial values (using BMI as an example).The proposed method outperformed other sequence modeling architectures and the state-of-the-art survival analysis approaches in terms of the C-index and the integrated Brier score (IBS) value. Our method achieved a C-index of 0.7680 (95% confidence intervals [CI]: 0.7645, 0.7716) and an IBS of 0.1302 (95% confidence intervals [CI]: 0.1292, 0.1313), showing an improvement of up to 5.4% in terms of the C-index and a decrease of 3.2% in terms of the IBS value. The addition of the distance-based loss function improved the performance. The predicted risk and actual risk levels closely agreed. This study also found that even after controlling the initial body mass index (BMI) values, different 3-month BMI trends could produce different survival outcomes.This study proposed a more effective and interpretable method to use time-series information in survival analysis. The proposed method may help promote personalized medicine and improve patient prognosis.
Pixel-wise Graph Attention Networks for Person Re-identification

Graph convolutional networks (GCN) is widely used to handle irregular data since it updates node features by using the structure information of graph. With the help of iterated GCN, high-order information can be obtained to further enhance the representation of nodes. However, how to apply GCN to structured data (such as pictures) has not been deeply studied. In this paper, we explore the application of graph attention networks (GAT) in image feature extraction. First of all, we propose a novel graph generation algorithm to convert images into graphs through matrix transformation. It is one magnitude faster than the algorithm based on K Nearest Neighbors (KNN). Then, GAT is used on the generated graph to update the node features. Thus, a more robust representation is obtained. These two steps are combined into a module called pixel-wise graph attention module (PGA). Since the graph obtained by our graph generation algorithm can still be transformed into a picture after processing, PGA can be well combined with CNN. Based on these two modules, we consulted the ResNet and design a pixel-wise graph attention network (PGANet). The PGANet is applied to the task of person re-identification in the datasets Market1501, DukeMTMC-reID and Occluded-DukeMTMC (outperforms state-of-the-art by 0.8%, 1.1% and 11% respectively, in mAP scores). Experiment results show that it achieves the state-of-the-art performance.
Bottom-Up and Bidirectional Alignment for Referring Expression Comprehension

In this paper, we propose a one-stage approach to improve referring expression comprehension (REC) which aims at grounding the referent according to a natural language expression. We observe that humans understand referring expressions through a fine-to-coarse bottom-up way, and bidirectionally obtain vision-language information between image and text. Inspired by this, we define the language granularity and the vision granularity. Otherwise, existing methods do not follow the mentioned way of human understanding in referring expression. Motivated by our observation and to address the limitations of existing methods, we propose a bottom-up and bidirectional alignment (BBA) framework. Our method constructs the cross-modal alignment starting from fine-grained representation to coarse-grained representation and bidirectionally obtains vision-language information between image and text. Based on the structure of BBA, we further propose a progressive visual attribute decomposing approach to decompose visual proposals into several independent spaces to enhance the bottom-up alignment framework. Experiments on five benchmark datasets of RefCOCO, RefCOCO+, ReferItGame, RefCOCOg and Flick30K show that our approach obtains +2.16%, +4.47%, +2.85%, +3.44%, and +2.91% improvements over the one-stage SOTA approaches, which validates the effectiveness of our approach.
Adversarial Differentiable Data Augmentation for Autonomous Systems

Autonomous systems often rely on neural networks to achieve high performance on planning and control problems. Unfortunately, neural networks suffer severely when input images become degraded in ways that are not reflected in the training data. This is particularly problematic for robotic systems like autonomous vehicles (AV) for which reliability is paramount. In this work, we consider robust optimization methods for hardening control systems against image corruptions and other unexpected domain shifts. Recent work on robust optimization for neural nets has been focused largely on combating adversarial attacks. In this work, we borrow ideas from the adversarial training and data augmentation literature to enhance robustness to image corruptions and domain shifts. To this end, we train networks while augmenting image data with a battery of image degradations. Unlike traditional augmentation methods, we choose the parameters for each degradation adversarially so as to maximize system performance. By formulating image degradations in a way that is differentiable with respect to degradation parameters, we enable the use of efficient optimization methods (PGD) for choosing worst-case augmentation parameters. We demonstrate the efficacy of this method on the learning to steer task for AVs. By adversarially training against image corruptions, we produce networks that are highly robust to image corruptions. We show that the proposed differentiable augmentation schemes result in higher levels of robustness and accuracy for a range of settings as compared to baseline and state-of-the-art augmentation methods.
Does it matter who pays back Technical Debt? An empirical study of self-fixed TD

Technical Debt (TD) can be paid back either by those that incurred it or by others. We call the former self-fixed TD, and it can be particularly effective, as developers are experts in their own code and are well-suited to fix the corresponding TD issues. The goal of our study is to investigate self-fixed technical debt, especially the extent in which TD is self-fixed, which types of TD are more likely to be self-fixed, whether the remediation time of self-fixed TD is shorter than non-self-fixed TD and how development behaviors are related to self-fixed TD. We report on an empirical study that analyzes the self-fixed issues of five types of TD (i.e., Code, Defect, Design, Documentation and Test), captured via static analysis, in more than 44,000 commits obtained from 20 Python and 16 Java projects of the Apache Software Foundation. The results show that about half of the fixed issues are self-fixed and that the likelihood of contained TD issues being self-fixed is negatively correlated with project size, the number of developers and total issues. Moreover, there is no significant difference of the survival time between self-fixed and non-self-fixed issues. Furthermore, developers are more keen to pay back their own TD when it is related to lower code level issues, e.g., Defect Debt and Code Debt. Finally, developers who are more dedicated to or knowledgeable about the project contribute to a higher chance of self-fixing TD. These results can benefit both researchers and practitioners by aiding the prioritization of TD remediation activities and refining strategies within development teams, and by informing the development of TD management tools.
Local Exceptionality Detection in Time Series Using Subgroup Discovery: An Approach Exemplified on Team Interaction Data

In this paper, we present a novel approach for local exceptionality detection on time series data. This method provides the ability to discover interpretable patterns in the data, which can be used to understand and predict the progression of a time series. As an exploratory approach, the results can be used to generate hypotheses about the relationships between the variables describing a specific process and its dynamics. We detail our approach in a concrete instantiation and exemplary implementation, specifically in the field of teamwork research. Using a real-world dataset of team interactions we discuss the results and showcase the presented novel analysis options. In addition, we outline possible implications of the results in terms of understanding teamwork.
HDR Image Reconstruction Using Segmented Image Learning

Converting a low dynamic range (LDR) image into a high dynamic range (HDR) image produces an image that closely depicts the real world without requiring expensive devices. Recent deep learning developments can produce highly realistic and sophisticated HDR images. This paper proposes a deep learning method to segment the bright and dark regions from an input LDR image and reconstruct the corresponding HDR image with similar dynamic ranges in the real world. The proposed multi-stage deep learning network brightens bright regions and darkens dark regions, and features with extended brightness range are combined to form the HDR image. Dividing the LDR image into the bright and dark regions effectively implements information on lost over-exposed and under-exposed areas, reconstructing a natural HDR image with color and appearance that is similar to reality. Experimental results confirm that the proposed method achieves an 8.52% higher HDR visual difference predictor (HDR-VDP) and a 41.2% higher log exposure range than current methods. Qualitative evaluation also verifies that the proposed method generates images that are close in quality to the ground truth.
Generating Disentangled Arguments with Prompts: A Simple Event Extraction Framework That Works

Event Extraction bridges the gap between text and event signals. Based on the assumption of trigger-argument dependency, existing approaches have achieved state-of-the-art performance with expert-designed templates or complicated decoding constraints. In this paper, for the first time we introduce the prompt-based learning strategy to the domain of Event Extraction, which empowers the automatic exploitation of label semantics on both input and output sides. To validate the effectiveness of the proposed generative method, we conduct extensive experiments with 11 diverse baselines. Empirical results show that, in terms of F1 score on Argument Extraction, our simple architecture is stronger than any other generative counterpart and even competitive with algorithms that require template engineering. Regarding the measure of recall, it sets new overall records for both Argument and Trigger Extractions. We hereby recommend this framework to the community, with the code publicly available at https://github.com/RingBDStack/GDAP.
Off-TANet: A Lightweight Neural Micro-expression Recognizer with Optical Flow Features and Integrated Attention Mechanism

Micro-expression recognition is a video sentiment classification task with extremely small sample size. The transience and spatial locality of micro-expressions bring difficulties to constructing large micro-expression databases and designing micro-expression recognition algorithms. To reach the balance between classification accuracy and model complexity in this domain, we propose a lightweight neural micro-expression recognizer, Off-TANet, which is based on apex-onset optical flow features. The neural network contains a simple yet powerful triplet attention mechanism, and the powerfulness of this design could be interpreted in 2 aspects, FACS AU and matrix sparseness. The model evaluation is conducted with a LOSO cross-validation strategy on a combined database including 3 mainstream micro-expression databases. With obviously fewer total parameters (59,403), the results of the experiment indicate that the model achieves an average recall of 0.7315 and an average F1-score of 0.7242, exceeding other major architectures in this domain. A series of ablation experiments are also conducted to ensure the validity of our model design.
Non-convex fractional-order derivative for single image blind restoration

This paper considers a variational model for single image blind restoration. By exploiting the fractional-order total variation (FTV) and Lp quasinorm relaxation, a non-convex fractional-order variational model is proposed to restore the image blurred by an unknown blur kernel. A quaternion FTV model is first put forward to exploring more directional information of an image. The new model utilizes non-convex and non-smooth quaternion FTV with Lp quasinorm to constrain the image and L1-norm to constrain the blur kernel, which are unified to a unified regularization framework. Further, an efficient algorithm is proposed to solve the non-convex problem via using the alternating direction minimization. The extensive experiments demonstrate the efficiency and viability of the proposed method and reveal superior performances in comparison with several existing methods.
An automated feature selection and classification pipeline to improve explainability of clinical prediction models

Artificial Intelligence is becoming recently a promising tool to achieve the deployment of personalized medicine in clinical practice. However, healthcare professionals are demanding clinical prediction models with better interpretability of the results in order to achieve an actual adoption and use of these solutions. The eXplainable Artificial Intelligence tackle this issue by offering feature relevance explanations of the model, among other techniques, where the selection of the important features and elimination of the redundant are cornerstones. This work presents a data management pipeline that allows automating the selection of those relevant features as well as the classifier technique that provides the best performance in terms of classification. The pipeline developed, named SCI-XAI (feature Selection and Classification for Improving eXplainable Artificial Intelligence) has been evaluated with 6 clinical datasets in a cross-validation approach as well as in a test set with unseen data. Next, an explainability evaluation has been carried out of the best models obtained by applying the SCI-XAI pipeline. Results obtained show that SCI-XAI achieves the best classification performance by applying different feature selection techniques depending on the variable type of the feature which reduces significantly the features processed by the model. Thus, feature reduction allows increasing the explainability of the models.
VR-Based Job Training System Using Tangible Interactions

Virtual training systems are in an increasing demand because of real-world training, which requires a high cost or accompanying risk, and can be conducted safely through virtual environments. For virtual training to be effective for users, it is important to provide realistic training situations; however, virtual reality (VR) content using VR controllers for experiential learning differ significantly from real content in terms of tangible interactions. In this paper, we propose a method for enhancing the presence and immersion during virtual training by applying various sensors to tangible virtual training as a way to track the movement of real tools used during training and virtualizing the entire body of the actual user for transfer to a virtual environment. The proposed training system connects virtual and real-world spaces through an actual object (e.g., an automobile) to provide the feeling of actual touch during virtual training. Furthermore, the system measures the posture of the tools (steam gun and mop) and the degree of touch and applies them during training (e.g., a steam car wash.) User-testing is conducted to validate the increase in the effectiveness of virtual job training.
Generative Face Parsing Map Guided 3D Face Reconstruction Under Occluded Scenes

Over the past few years, single-view 3D face reconstruction methods can produce beautiful 3D models. Nevertheless, the input of these works is unobstructed faces. We describe a system designed to reconstruct convincing face texture in the case of occlusion. Motivated by parsing facial features, we propose a complete face parsing map generation method guided by landmarks. We estimate the 2D face structure of the reasonable position of the occlusion area, which is used for the construction of 3D texture. An excellent anti-occlusion face reconstruction method should ensure the authenticity of the output, including the topological structure between the eyes, nose, and mouth. We extensively tested our method and its components, qualitatively demonstrating the rationality of our estimated facial structure. We conduct extensive experiments on general 3D face reconstruction tasks as concrete examples to demonstrate the method’s superior regulation ability over existing methods often break down. We further provide numerous quantitative examples showing that our method advances both the quality and the robustness of 3D face reconstruction under occlusion scenes.
PLG-IN: Pluggable Geometric Consistency Loss with Wasserstein Distance in Monocular Depth Estimation

We propose a novel objective for penalizing geometric inconsistencies and improving the depth and pose estimation performance of monocular camera images. Our objective is designed using the Wasserstein distance between two point clouds, estimated from images with different camera poses. The Wasserstein distance can impose a soft and symmetric coupling between two point clouds, which suitably maintains geometric constraints and results in a differentiable objective. By adding our objective to those of other state-of-the-art methods, we can effectively penalize geometric inconsistencies and obtain highly accurate depth and pose estimations. Our proposed method was evaluated using the KITTI dataset.