Miracle-dz/UniFAR-FTUs huggingface dataset:
On the Need of Graph Support for Developer Identification in Software Repositories

Software repositories from open-source projects provide a rich source of information for a wide range of tasks. However, one issue to overcome in order to make this information useful is the accurate identification of developers. This is a particular challenge, as developers usually use different IDs in different repositories of one project, but usually there is no kind of dictionary or similar available to map the different IDs to realworld persons. Often, they even use different IDs in the same repository. We show that the few methods suggested so far are not always appropriate to overcome this problem. Further, we highlight related techniques from other areas and discuss how they can be applied in this context. We particularly focus on the idea of applying graph-based methods and argue for the benefits we expect from that.
Deep Sequential Multi-camera Feature Fusion for Person Re-identification

Given a target image as query, person re-identification systems retrieve a ranked list of candidate matches from other camera field-of-views. Re-identification is typically performed independently for each gallery camera across the network. However, images of the same target from a subset of cameras often provide complementary appearance information which, if aggregated judiciously, can lead to better re-id performance in the remaining cameras. This motivates us to investigate the novel problem of multi-camera feature fusion for person re-id. We propose an intelligent sequential fusion technique, designed to not only improve re-id accuracy but to also learn increasingly better feature representations as observations from additional cameras are fused. The fusion function is modeled using a Gated Recurrent Unit (GRU) with modification on default GRU training regime to achieve the aforementioned improvements. Extensive experimentation validates that the proposed fusion scheme substantially boosts identification performance when combined with off-the-shelf feature extraction methods for re-id.
On the Equivalence between Objective Intelligibility and Mean-Squared Error for Deep Neural Network based Speech Enhancement

Although speech enhancement algorithms based on deep neural networks (DNNs) have shown impressive results, it is unclear, if they are anywhere near optimal in terms of aspects related to human auditory perception, e.g. speech intelligibility. The reason is that the vast majority of DNN based speech enhancement algorithms rely on the mean squared error (MSE) criterion of short-time spectral amplitudes (STSA). State-of-the-art speech intelligibility estimators, on the other hand, rely on linear correlation of speech temporal envelopes. This raises the question if a DNN training criterion based on envelope linear correlation (ELC) can lead to improved intelligibility performance of DNN based speech enhancement algorithms compared to algorithms based on the STSA-MSE criterion. In this paper we derive that, under certain general conditions, the STSA-MSE and ELC criteria are practically equivalent, and we provide empirical data to support our theoretical results. The important implication of our findings is that the standard STSA minimum-MSE estimator is optimal, if the objective is to perform optimally with respect to a state-of-the-art speech intelligibility estimator.
Similarity Join and Self-Join Size Estimation in a Streaming Environment

We study the problem of similarity self-join and similarity join size estimation in a streaming setting where the goal is to estimate, in one scan of the input and with sublinear space in the input size, the number of record pairs that have a similarity within a given threshold. The problem has many applications in data cleaning and query plan generation, where the cost of a similarity join may be estimated before actually doing the join. On unary input where two records either match or don't match, the problem becomes join and self-join size estimation for which one-pass algorithms are readily available. Our work addresses the problem for d-ary input, for d >= 1, where the degree of similarity can vary from 1 to d. We show that our proposed algorithm gives an accurate estimate and scales well with the input size. We provide error bounds and time and space costs, and conduct an extensive experimental evaluation of our algorithm, comparing its estimation accuracy to a few competitors, including some multi-pass algorithms. Our results show that given the same space, the proposed algorithm has an order of magnitude less error for a large range of similarity thresholds.
A Validation and Quality Assessment Method with Metamorphic Relations for Unsupervised Machine Learning Software

Unsupervised machine learning is a task of modeling the underlying structure of ``unlabeled data''. Since learning algorithms have been incorporated into many real-world applications, validating the implementations of those algorithms becomes much more important in the aim of software quality assurance. However, validating unsupervised machine learning programs is challenging because there lacks of priori knowledge. Along this line, in this paper, we present a metamorphic testing based method for validating and characterizing unsupervised machine learning programs, and conduct an empirical study on a real-world machine learning tool. The results demonstrate to what extent a program may fit to the profile of a specific scenario, which help end-users or software practitioners comprehend its performance in a vivid and light-weight way. And the experimental findings also reveal the gap between theory and implementation in a software artifact which could be easily ignored by people without much practical experience. In our method, metamorphic relations can serve as one type of quality measure, and one type of guidelines for selecting suitable programs.
Learning uncertainty in regression tasks by deep neural networks

We suggest a general approach to quantification of different types of uncertainty in regression tasks performed by deep neural networks. It is based on the simultaneous training of two neural networks with a joint loss function. One of the networks performs regression and the other quantifies the uncertainty of predictions of the first one. Unlike in many standard uncertainty quantification methods, the targets are not assumed to be sampled from an a priori given probability distribution. We analyze how the hyperparameters affect the learning process and, additionally, show that our method even allows for better predictions compared to standard neural networks without uncertainty counterparts. Finally, we show that a particular case of our approach is the mean-variance estimation given by a Gaussian network.
Hydrodynamic limit of lattice Boltzmann equations

An analysis of the lattice Boltzmann (LB) method is conducted, and various conclusions are drawn on how to exploit this method for the numerical resolution of the Navier-Stokes equation. A novel LB model is introduced, first for the simulation of advection-diffusion problems, and then for the resolution of the Navier-Stokes equation. The latter model, presented under the name of "regularized lattice Boltzmann", is shown to substantially increase the stability and accuracy of LB models in numerical simulaiton. The model is furthermore found to be innovative due to the fact that it possesses an accurate dimensionless formulation in terms of macroscopic variables. Based on this observation, the regularized model is investigated to address various challenges of LB, such as the implementaiton of boundary conditions and spatially refined grids.
Sleep Staging by Modeling Sleep Stage Transitions using Deep CRF

Sleep plays a vital role in human health, both mental and physical. Sleep disorders like sleep apnea are increasing in prevalence, with the rapid increase in factors like obesity. Sleep apnea is most commonly treated with Continuous Positive Air Pressure (CPAP) therapy, which maintains the appropriate pressure to ensure continuous airflow. It is widely accepted that in addition to preventing air passage collapse, increase in deep and REM sleep stages would be good metrics for how well the CPAP therapy is working in improving sleep health. Presently, however, there is no mechanism to easily detect a patient's sleep stages from CPAP flow data alone. We propose, for the first time, an automated sleep staging model based only on the flow signal. Recently deep neural networks have shown high accuracy on sleep staging by eliminating handcrafted features. However, these methods focus exclusively on extracting informative features from the input signal, without paying much attention to the dynamics of sleep stages in the output sequence. We propose an end-to-end framework that uses a deep convolution-recurrent neural networks to extract high-level features from raw flow signal and then uses a structured output layer based on a conditional random field to model the temporal transition structure of the sleep stages. We improve upon the previous methods by 10% using our model, that can be augmented to the previous sleep staging deep learning methods. We also show that our method can be used to accurately track sleep metrics like sleep efficiency calculated from sleep stages that can be deployed for monitoring the response of CPAP therapy on sleep apnea patients. Apart from the technical contributions, we expect this study to motivate new research questions in sleep science, especially towards the understanding of sleep architecture trajectory among patients under CPAP therapy.
Online Archives for Peer-Reviewed Journal Publications

Peer-reviewed journals used to perform two functions for research and researchers -- (1) peer review and (2) distribution -- and research libraries used to perform two more -- (3) archiving and (4) access provision. In the online age, journals will need only to provide the peer-review service. Authors will self-archive their papers, both before and after peer review, in their institutional Eprint Archives, which will all be interoperable with one another, providing open access to all peer-reviewed research output as if it were all in one global archive.
Hill Climbing Optimized Twin Classification Using Resting-State Functional MRI

Twin imaging studies are an important part of human brain research that can reveal the importance of genetic influences on different aspects of brain behavior and disorders. Accurate characterization of identical and fraternal twins allows inference to be performed on the genetic influence in a population. In this paper, we propose a novel pairwise feature representation to classify the zygosity of twin pairs using resting state functional magnetic resonance images (rs-fMRI). Specifically, we project an fMRI signal to a set of cosine series basis, and use the projection coefficients as the compact and discriminative feature representation of noisy fMRI data. The pairwise relation is encoded by a set of twin-wise correlations between the new feature representations across brain regions. We further employ Hill Climbing variable selection to identify the brain regions that are most genetically affected. The proposed framework has been applied to 208 twin pairs in the Human Connectome Project (HCP) and we achieved 94.19($\pm$3.53)% classification accuracy in determining the zygosity of paired images.
Neural Sequence Prediction by Coaching

In sequence prediction task, there mainly exist two types of learning algorithms. One genre based on maximum likelihood estimation (MLE) supervises the training in every time step via a surrogate loss function but leads to train-test discrepancy and hard-to-train problem. The other genre based on Reinforcement Learning gives the model freedom to generate samples and guides the training by feeding back task-level reward but suffers from high variance and instability problem. In general, these two frameworks both have their own tradeoffs and complement each other. In this paper, we introduce a hybrid-coach framework to combine these two existing algorithms so that the supervision from MLE can stabilize RL training while RL can incorporate task-level loss into MLE training. To augment more easy-to-learn samples for MLE training and denser reward for RL training, the coach is encouraged to be close to both the data and model distribution. During training, the coach acts as a proxy target and pulls the model distribution to the data distribution, as the model makes progress it correspondingly upgrades itself to become harsher and finally anneal to the ultimate target distribution. With the coordination of the coach system, our hybrid-coach framework can efficiently resolve RL’s reward sparsity problem and MLE’s data sparsity problem. We apply our algorithm on two Machine Translation Task and show significant improvements over the state-of-art systems.
FutureGAN: Anticipating the Future Frames of Video Sequences using Spatio-Temporal 3d Convolutions in Progressively Growing Autoencoder GANs

We propose a new Autoencoder GAN model, FutureGAN, that predicts future frames of a video sequence given a sequence of past frames. Our approach extends the recently introduced progressive growing of GANs (PGGAN) architecture by Karras et al. [18]. During training, the resolution of the input and output frames is gradually increased by progressively adding layers in both the discriminator and the generator network. To learn representations that effectively capture the spatial and temporal components of a frame sequence, we use spatio-temporal 3d convolutions. We already achieve promising results for frame resolutions of 128× 128 px over a variety of datasets ranging from synthetic to natural frame sequences, while theoretically not being limited to a specific frame resolution. The FutureGAN learns to generate plausible futures, learning representations that seem to effectively capture the spatial and the temporal transformations of the input frames. A great advantage of our architecture, in comparison to the majority of other video prediction models, is its simplicity. The model receives solely the raw pixel values as an input, generating output frames effectively, without relying on additional constraints, conditions, or complex pixel-based error loss metrics.
Practical Neural Network Performance Prediction for Early Stopping

In the neural network domain, methods for hyperparameter optimization and meta-modeling are computationally expensive due to the need to train a large number of neural network configurations. In this paper, we show that a simple regression model, based on support vector machines, can predict the final performance of partially trained neural network configurations using features based on network architectures, hyperparameters, and time-series validation performance data. We use this regression model to develop an early stopping strategy for neural network configurations. With this early stopping strategy, we obtain significant speedups in both hyperparameter optimization and meta-modeling. Particularly in the context of meta-modeling, our method can learn to predict the performance of drastically different architectures and is seamlessly incorporated into reinforcement learning-based architecture selection algorithms. Finally, we show that our method is simpler, faster, and more accurate than Bayesian methods for learning curve prediction.
Bottom-Up and Top-Down Attention for Image Captioning and VQA

Top-down visual attention mechanisms have been used extensively in image captioning and visual question answering (VQA) to enable deeper image understanding through ﬁne-grained analysis and even multiple steps of reasoning. In this work, we propose a combined bottom-up and top-down attention mechanism that enables attention to be calculated at the level of objects and other salient image regions. This is the natural basis for attention to be considered. Within our approach, the bottom-up mechanism (based on Faster R-CNN) proposes image regions, each with an associated feature vector, while the top-down mechanism determines feature weightings. Applying this approach to image captioning, our results on the MSCOCO test server establish a new state-of-the-art for the task, improving the best published result in terms of CIDEr score from 114.7 to 117.9 and BLEU-4 from 35.2 to 36.9. Demon-strating the broad applicability of the method, applying the same approach to VQA we obtain a new state-of-the-art on the VQA v2.0 dataset with 70.2% overall accuracy.
Deep Multi-View Learning with Stochastic Decorrelation Loss

Multi-view learning aims to learn an embedding space where multiple views are either maximally correlated for cross-view recognition, or decorrelated for latent factor disentanglement. A key challenge for deep multi-view representation learning is scalability. To correlate or decorrelate multi-view signals, the covariance of the whole training set should be computed which does not fit well with the mini-batch based training strategy, and moreover (de)correlation should be done in a way that is free of SVD-based computation in order to scale to contemporary layer sizes. In this work, a unified approach is proposed for efficient and scalable deep multi-view learning. Specifically, a mini-batch based Stochastic Decorrelation Loss (SDL) is proposed which can be applied to any network layer to provide soft decorrelation of the layer's activations. This reveals the connection between deep multi-view learning models such as Deep Canonical Correlation Analysis (DCCA) and Factorisation Autoencoder (FAE), and allows them to be easily implemented. We further show that SDL is superior to other decorrelation losses in terms of efficacy and scalability.
Deep convolutional filter banks for texture recognition and segmentation

Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications. In this work we conduct a first study of material and describable texture at- tributes recognition in clutter, using a new dataset derived from the OpenSurface texture repository. Motivated by the challenge posed by this problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN substantially improves the state-of-the-art in texture, mate- rial and scene recognition. Our approach achieves 82.3% accuracy on Flickr material dataset and 81.1% accuracy on MIT indoor scenes, providing absolute gains of more than 10% over existing approaches. D-CNN easily trans- fers across domains without requiring feature adaptation as for methods that build on the fully-connected layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale information and describe regions of arbitrary shapes and sizes. Our approach is particularly suited at lo- calizing stuff categories and obtains state-of-the-art re- sults on MSRC segmentation dataset, as well as promising results on recognizing materials and surface attributes in clutter on the OpenSurfaces dataset.
ReTiCaM: Real-time Human Performance Capture from Monocular Video

We present the first real-time human performance capture approach that reconstructs dense, space-time coherent deforming geometry of entire humans in general everyday clothing from just a single RGB video. We propose a novel two-stage analysis-by-synthesis optimization whose formulation and implementation are designed for high performance. In the first stage, a skinned template model is jointly fitted to background subtracted input video, 2D and 3D skeleton joint positions found using a deep neural network, and a set of sparse facial landmark detections. In the second stage, dense non-rigid 3D deformations of skin and even loose apparel are captured based on a novel real-time capable algorithm for non-rigid tracking using dense photometric and silhouette constraints. Our novel energy formulation leverages automatically identified material regions on the template to model the differing non-rigid deformation behavior of skin and apparel. The two resulting non-linear optimization problems per-frame are solved with specially-tailored data-parallel Gauss-Newton solvers. In order to achieve real-time performance of over 25Hz, we design a pipelined parallel architecture using the CPU and two commodity GPUs. Our method is the first real-time monocular approach for full-body performance capture. Our method yields comparable accuracy with off-line performance capture techniques, while being orders of magnitude faster.
Near or Far, Wide Range Zero-Shot Cross-Lingual Dependency Parsing

Cross-lingual transfer is the major means toleverage knowledge from high-resource lan-guages to help low-resource languages. In this paper, we investigate cross-lingual trans-fer across a broad spectrum of language dis-tances. We posit that Recurrent Neural Net-works (RNNs)-based encoders, since explic-itly incorporating surface word order, are brit-tle for transferring across distant languages,while self-attentive models are more flexibleon modeling word order information; thusthey would be more robust in the cross-lingualtransfer setting. We test our hypothesis bytraining dependency parsers on only Englishcorpus and evaluating them on 31 other lan-guages. With detailed analysis, we find inter-esting patterns showing that RNNs-based ar-chitectures can transfer well for languages thatare close to English, while self-attentive mod-els are have better cross-lingual transferabilityacross a wide range of languages.
A Fast Foveated Fully Convolutional Network Model for Human Peripheral Vision

Visualizing the information available to a human observer in a single glance at an image provides a powerful tool for evaluating models of full-field human vision. The hard part is human-realistic visualization of the periphery. Degradation of information with distance from fixation is far more complex than a mere reduction of acuity that might be mimicked using blur with a standard deviation that linearly increases with eccentricity. Rather, behaviorally-validated models hypothesize that peripheral vision measures a large number of local texture statistics in pooling regions that overlap, grow with eccentricity, and tile the visual field. We propose a "foveated" variant of a fully convolutional network that approximates one such model. Our approach achieves a 21,000 fold reduction in average running time (from 4.2 hours to 0.7 seconds per image), and statistically similar results to the behaviorally-validated model.