Miracle-dz/UniFAR-FTUs huggingface dataset:
Face Alignment by Combining Residual Features in Cascaded Hourglass Network

Fully Convolutional Networks (FCN) are popular in face alignment thanks to its capacity to retain accurate spatial information. In this work we study the effect of kernel functions of FCN for face alignment. We claim that neither the cross entropy nor the pixel-wise L2 losses can reflect the alignment error accurately if we generate the ground truth probability matrix with kernel functions. Based on this analysis, firstly, we develop a Cascaded Hourglass Network (CHN) as our baseline, and then regress the residual face shape via features obtained from the middle layer of the network, which are called Residual Features (RF). The proposed RF-CHN method obtains Normalized Mean Error (NME) of 6.84, which gives an error reduction of 0.21 compared to the current state-of-the-art on the challenging 300-W database.
A Novel CNN Segmentation Framework Based on Using New Shape and Appearance Features

To improve the accuracy of segmenting medical images from different modalities we propose to integrate three types of comprehensive quantitative image descriptors with a deep 3D convolutional neural network. The descriptors include: (i) a Gibbs energy for a prelearned 7th-order Markov-Gibbs random field (MGRF) model of visual appearance, (ii) a relearned adaptive shape prior model, and a first-order conditional random field model of visual appearance of regions at each current stage of segmentation. The neural network fuses the computed descriptors, together with the raw image data, for obtaining the final voxel-wise probabilities of the goal regions. Quantitative assessment of our framework in terms of Dice similarity coefficients, 95-percentile bidirectional Hausdorff distances, and percentage volume differences confirms the high accuracy of our model on 95 CT lung images $(98.37\pm_{0.68}\%,2.79\pm_{1.32}\ mm,3.94\pm_{2.11}\%)$ and 95 diffusion weighted kidney MRI $(96.65\pm_{2.15}\%,4.32\pm_{3.09}\ mm,5.61\pm_{3.37}\%)$, respectively.
ISI ASR System for the Low Resource Speech Recognition Challenge for Indian Languages

This paper describes the ISI ASR system used to generate ISI’s submissions across Gujarati, Tamil and Telugu speech recognition tasks as part of the Low Resource Speech Recognition Challenge for Indian Languages. The key constraints on this task were limited training data, and the restriction that no ex-ternal data be used. The ISI ASR system leverages our earlier work on data augmentation and dropout approaches and current work on multilingual training within a Eesen based end-to-end Long Short Term Memory (LSTM) based automatic speech recognition (ASR) system trained with the Connectionist Temporal Classiﬁcation (CTC) loss criterion, and demonstrates, to the best of our knowledge, one of the ﬁrst times such systems have been applied to low resource languages with performance comparable and some cases better than hybrid DNN systems. Our best monolingual systems show between 6.5% to 25.5% relative reduction in word error rate (WER) compared to the challenge organizer’s Time Delay Neural Network (TDNN) based baseline WERs. We further extend these systems with multilingual training approaches that lead to an additional 4.5% to 11.1% relative reduction in WER as measured on the development set.
Salient Object Detection Via Deformed Smoothness Constraint

In recent years, various graph-based salient object detection methods have been successfully proposed. Since existing methods may miss some object regions with low contrast to background, a novel propagation model via deformed smoothness constraint is proposed to address this problem. By regularizing nodes and their neighbors locally, the deformed smoothness constraint is able to prevent erroneous label propagation. Thus, the object regions with low contrast to background can be emerged. Besides, the deformed smoothness constraint is further utilized in a map refinement model, which can suppress the background noises in label propagation result. Experiments on three public datasets show that the proposed method outperforms eleven state-of-the-art salient object detection methods.
How to Learn the Effect of Non-Uniform Distortion on Perceived Visual Quality? Case Study Using Convolutional Sparse Coding for Quality Assessment of Synthesized Views

Machine learning has been attached greater attention in the field of quality assessment and related models have been developed by assuming that any region within images share the same quality score. However, this assumption may no longer stand when non-uniform distortions exist, e.g. geometric distortion within synthesized views in the case of free- view point video TV (FTV). In this paper, we explore to use Convolutional Sparse Coding (CSC), which computes a sparse representation for an entire image with the sum of a set of convolutions with dictionary filters instead of a linear combination of a set of dictionary atoms, to learn a visible codebook and propose a methodology to quantify the geometric distortion without using the reference. Experimental results show that the proposed no reference metric is the most visual friendly and reliable metric among the compared blind image metrics designed for synthesized images.
An Interactive Content-Based 3D Shape Retrieval System for on-Site Cultural Heritage Analysis

In this paper, we analyse the process of designing a Content-Based 3D shape Retrieval (CB3DR) adapted for non-experts. Our CB3DR solution aims at scanning an object on the fly with a low-cost 3D sensor and retrieve similar shapes from a database using the 3D point cloud acquired. Our system should meet the requirements of archaeologists who would like to be able to acquire artefacts without prior expertise in scanning, then query easily from the field knowledge bases for Cultural Heritage, and thus retrieve artefacts (i.e. objects or parts of objects) with similar shape without carrying or even moving the artefact found on the site. This context is definitely a retrieval context rather than a classification one since the found artefact may be unknown. At the era of Deep Learning techniques, we investigate in this paper how to design a system which could benefit from the latest advances in 3D object recognition. We thus design a classic CBR workflow starting with a similarity search step followed by an interactive learning SVM which requires very simple positive-negative annotations from the users to refine the ranking provided by the similarity search. The 3D shapes are first described using data representations. We compare first the “old fashion” hand-crafted 3D descriptors (which do not require to be learnt or trained) with PointNet, one of the most recent 3D object deep representation. We evaluate our retrieval baseline against recent Deep Learning approaches on ModelNet10 and explore the potential of deep representations in our interactive learning framework.
Risk factors for conversion to snare resection during colorectal endoscopic submucosal dissection in an expert Western center

Abstract Background There are limited data regarding the risk factors and consequences of conversion to endoscopic mucosal resection (rescue EMR) during colorectal endoscopic submucosal dissection (ESD) in Western centers. Methods This was a retrospective analysis of a prospectively collected database, from which 225 consecutive ESDs performed between 2013 and 2017 were selected. Of the included patients, 39 (18.6 %) required rescue EMR. Pre- and per-procedure characteristics were evaluated to determine the features associated with the need for rescue EMR. Outcomes and complications were also assessed. Results 210 patients were included, with median tumor size of 40 mm (range 20 – 110) and most tumors being in a non-rectal location (66.2 %). When compared with full ESD, rescue EMR was significantly associated with lower rates of en bloc resection (43.6 % vs. 100 %) and complete resection (R0 status; 28.2 % vs. 88.9 %), and with a higher rate of recurrence (5.1 % vs. 0 %) and more need for surgery (15.4 % vs. 3.5 %). In multivariable analysis, non-lifting (adjusted odds ratio [ORa] 3.06, 95 % confidence interval [CI] 1.23 – 7.66; P = 0.02), nongranular-type laterally spreading tumor (LST-NG; ORa 2.56, 95 %CI 1.10 – 5.99; P = 0.03), and difficult retroflexion (OR 3.22, 95 %CI 1.01 – 10.28; P = 0.049) were independent risk factors associated with conversion to rescue EMR, while tumor size and location were not. Conclusions During ESD, the presence of poor lifting, LST-NG morphology, and a difficult retroflexed approach were factors associated with the need to convert to rescue EMR. Conversion to rescue EMR remains a valuable strategy.
Increasingly Specialized Ensemble of Convolutional Neural Networks for Fine-Grained Recognition

Finegrained recognition focuses on the challenging task of automatically identifying the subtle differences between similar categories. Current state-of-the-art approaches require elaborated feature learning procedures, involving tuning several hyper-parameters, or rely on expensive human annotations such as objects or parts location. In this paper we propose a simple method for fine-grained recognition that exploits a nearly cost-free attention-based focus operation to construct an ensemble of increasingly specialized Convolutional Neural Networks. Our method achieves state-of-the-art results on three of the most popular datasets used for fine-grained classification namely CUB Birds 200–2011, FGVC-Aircraft and Stanford Cars requiring minimal hyperparameter tuning and no annotations.
Text-Dependent Speech Enhancement for Small-Footprint Robust Keyword Detection

Keyword detection (KWD), also known as keyword spotting, is in great demand in small devices in the era of Internet of Things. Albeit recent progresses, the performance of KWD, measured in terms of precision and recall rate, may still degrade signiﬁ-cantly when either the non-speech ambient noises or the human voice and speech-like interferences (e.g., TV, background competing talkers) exists. In this paper, we propose a general so-lution to address all kinds of environmental interferences. A novel text-dependent speech enhancement (TDSE) technique using a recurrent neural network (RNN) with long short-term memory (LSTM) is presented for improving the robustness of the small-footprint KWD task in the presence of environmental noises and interfering talkers. On our large simulated and recorded noisy and far-ﬁeld evaluation sets, we show that TDSE signiﬁcantly improves the quality of the target keyword speech and performs particularly well under speech interference conditions. We demonstrate that KWD with TDSE frontend signiﬁ-cantly outperforms the baseline KWD system with or without a generic speech enhancement in terms of equal error rate (EER) in the keyword detection evaluation.
Investigation on Estimation of Sentence Probability by Combining Forward, Backward and Bi-directional LSTM-RNNs

A combination of forward and backward long short-term memory (LSTM) recurrent neural network (RNN) language models is a popular model combination approach to improve the estimation of the sequence probability in the second pass N-best list rescoring in automatic speech recognition (ASR). In this work, we further push such an idea by proposing a combination of three models: a forward LSTM language model, a backward LSTM language model and a bi-directional LSTM based gap completion model. We derive such a combination method from a forward backward decomposition of the sequence probability. We carry out experiments on the Switchboard speech recognition task. While we empirically ﬁnd that such a combination gives slight improvements in perplexity over the combination of forward and backward models, we ﬁnally show that a combination of the same number of forward models gives the best perplexity and word error rate (WER) overall.
Semantic-Fusion Gans for Semi-Supervised Satellite Image Classification

Most of the public satellite image datasets contain only a small number of annotated images. The lack of a sufficient quantity of labeled data for training is a bottleneck for the use of modern deep-learning based classification approaches in this domain. In this paper we propose a semi -supervised approach to deal with this problem. We use the discriminator $(D)$ of a Generative Adversarial Network (GAN) as the final classifier, and we train $D$ using both labeled and unlabeled data. The main novelty we introduce is the representation of the visual information fed to $D$ by means of two different channels: the original image and its “semantic” representation, the latter being obtained by means of an external network trained on ImageNet. The two channels are fused in $D$ and jointly used to classify fake images, real labeled and real unlabeled images. We show that using only 100 labeled images, the proposed approach achieves an accuracy close to 69% and a significant improvement with respect to other GAN-based semi-supervised methods. Although we have tested our approach only on satellite images, we do not use any domain-specific knowledge. Thus, our method can be applied to other semi-supervised domains.
Task‐enhanced arterial spin labeled perfusion MRI predicts longitudinal neurodegeneration in mild cognitive impairment

Mild cognitive impairment (MCI) is considered a prodromal stage of Alzheimer's disease (AD), but is also recognized to be a heterogeneous condition. Biomarkers that predict AD progression in MCI are of clinical significance because they can be used to better identify appropriate candidates for therapeutic intervention studies. It has been hypothesized that comparing to structural measurements, functional ones may be more sensitive to early disease abnormalities and the sensitivity could be further enhanced when combined with cognitive task, a “brain stress test. ” In this study, we investigated the value of regional cerebral blood flow (CBF), measured by arterial spin labeled perfusion MRI (ASL MRI) during a memory‐encoding task, in predicting the estimated rate of hippocampal atrophy, an established marker of AD progression. Thirty‐one amnestic MCI patients (20 male and 11 female; age: 70.9 ± 6.5 years, range from 56 to 83 years; mini mental status examination: 27.8 ± 1.8) and 42 normal control subjects (13 male and 29 female; age: 70.6 ± 8.8 years, range from 55 to 88 years; mini mental status examination: 29.1 ± 1.2) were included in this study. We compared the predictive value of CBF during task to CBF during rest and structural volumetry. Both region‐of‐interest and voxelwise analyses showed that baseline CBF measurements during task (strongest effect in fusiform gyrus, region‐of‐interest analysis statistics: r = 0.56, p = .003), but not resting ASL MRI or structural volumetry, were correlated with the estimated rate of hippocampal atrophy in amnestic MCI patients. Further, stepwise linear regression demonstrated that resting ASL MRI and volumetry did not provide complementary information in prediction. These results support the notion that physiologic measures during a cognitive challenge may increase the ability to detect subtle functional changes that predict progression. As such, ASL MRI could have important utility in stratifying candidates for AD treatment trials.
Depression Detection from Short Utterances via Diverse Smartphones in Natural Environmental Conditions

Depression is a leading cause of disease burden worldwide, however there is an unmet need for screening and diagnostic measures that can be widely deployed in real-world environments. Voice-based diagnostic methods are convenient, non-invasive to elicit, and can be collected and processed in near real-time using modern smartphones, smart speakers, and other devices. Studies in voice-based depression detection to date have primarily focused on laboratory-collected voice samples, which are not representative of typical user environments or devices. This paper conducts the first investigation of voice-based depression assessment techniques on real-world data from 887 speakers, recorded using a variety of different smartphones. Evaluations on 16 hours of speech show that conservative segment selection strategies using highly thresholded voice activity detection, coupled with tailored normalization approaches are effective for mitigating smartphone channel variability and background environmental noise. Together, these strategies can achieve F1 scores comparable with or better than those from a combination of clean recordings, a single recording environment and long utterances. The scalability of speech elicitation via smartphone allows detailed models dependent on gender, smartphone manufacturer and/or elicitation task. Interestingly, results herein suggest that normalization based on these criteria may be more effective than tailored models for detecting depressed speech.
Content Based Video Summarization: Finding Interesting Temporal Sequences of Frames

We present a novel video summarization model to generate coherent video summaries capturing the most interesting parts of a video, using CNN and bidirectional LSTMs to generate deep features for frame representation and to model variable-range temporal sequences. Further, we introduce a parameterized loss function minimizing KL-divergence between GMMs to learn relative orders of frame importances. We conduct extensive evaluation on several benchmarks (TV-Sum, SumMe and YouTube) to demonstrate the effectiveness of our model, where our approach significantly outperforms state-of-the-art methods in several settings. Given the enormous growth in user-generated videos, video summarization has increasing importance in being able to navigate, browse, and search videos efficiently. Our research could see direct applications in tackling problems like detecting break - ins from surveillance videos, generating sporting event highlights, etc.
Carbamylated Erythropoietin Decreased Proliferation and Neurogenesis in the Subventricular Zone, but Not the Dentate Gyrus, After Irradiation to the Developing Rat Brain

Cranial radiotherapy for pediatric brain tumors causes progressive, debilitating late effects, including cognitive decline. Erythropoietin (EPO) has been shown to be neuroprotective and to promote neuroregeneration. Carbamylated erythropoietin (CEPO) retains the protective properties of EPO but is not erythrogenic. To study the effects of CEPO on the developing brain exposed to radiotherapy, a single irradiation (IR) dose of 6 Gy was administered to the brains of postnatal day 9 (P9) rats, and CEPO (40 μg/kg s.c.) was injected on P8, P9, P11, P13, and P15. To examine proliferation, 5-Bromo-2-deoxyuridine (BrdU) was injected on P15, P16, and P17. CEPO administration did not affect BrdU incorporation in the granule cell layer (GCL) of the hippocampus or in the subventricular zone (SVZ) as quantified 7 days after the last BrdU injection, whereas IR decreased BrdU incorporation in the GCL and SVZ by 63% and 18%, respectively. CEPO did not affect BrdU incorporation in the GCL of irradiated brains, although it was reduced even further (to 31%) in the SVZ. To evaluate the effect of CEPO on neurogenesis, BrdU/doublecortin double-positive cells were quantified. CEPO did not affect neurogenesis in non-irradiated brains, whereas IR decreased neurogenesis by 58% in the dentate gyrus (DG) but did not affect it in the SVZ. In the DG, CEPO did not affect the rate of neurogenesis following IR, whereas in the SVZ, the rate decreased by 30% following IR compared with the rate in vehicle-treated rats. Neither CEPO nor IR changed the number of microglia. In summary, CEPO did not promote neurogenesis in non-irradiated or irradiated rat brains and even aggravated the decreased neurogenesis in the SVZ. This raises concerns regarding the use of EPO-related compounds following radiotherapy.
Postfiltering with Complex Spectral Correlations for Speech and Audio Coding

State-of-the-art speech codecs achieve a good compromise be-tween quality, bitrate and complexity. However, retaining performance outside the target bitrate range remains challenging. To improve performance, many codecs use pre-and post-ﬁltering techniques to reduce the perceptual effect of quantization-noise. In this paper, we propose a postﬁltering method to attenuate quantization noise which uses the complex spectral correlations of speech signals. Since conventional speech codecs cannot transmit information with temporal dependencies as transmission errors could result in severe error propagation, we model the correlation ofﬂine and employ them at the decoder, hence removing the need to transmit any side information. Objective evaluation indicates an average 4 dB improvement in the perceptual SNR of signals using the context-based post-ﬁlter, with respect to the noisy signal, and an average 2 dB improvement relative to the conventional Wiener ﬁl-ter. These results are conﬁrmed by an improvement of up to 30 MUSHRA points in a subjective listening test.
Robust Spoken Language Understanding via Paraphrasing

Learning intents and slot labels from user utterances is a fundamental step in all spoken language understanding (SLU) and dialog systems. State-of-the-art neural network based methods, after deployment, often suffer from performance degradation on encountering paraphrased utterances, and out-of-vocabulary words, rarely observed in their training set. We address this challenging problem by introducing a novel paraphrasing based SLU model which can be integrated with any existing SLU model in order to improve their overall performance. We propose two new paraphrase generators using RNN and sequence-to-sequence based neural networks, which are suitable for our application. Our experiments on existing benchmark and in house datasets demonstrate the robustness of our models to rare and complex paraphrased utterances, even under adversarial test distributions.
Fully Automatic Camera Calibration for Principal Point Using Flat Monitors

In this paper, a novel, fully automatic camera calibration procedure is proposed to estimate the camera principal point, i.e., the intersection of the optical axis and the image plane. The basic idea is to derive the orthogonal projection of the camera optical axis onto the flat-screen monitor via changing/analyzing simple edge patterns displayed on the flat monitor. The principal point can then be estimated as the intersection of images of line features thus derived on the screen, called calibration lines, from multiple monitor configurations. Thus, the calibration can be performed automatically using fixed camera and multiple monitors, or using multiple poses of a movable monitor. Several experiments are developed to evaluate the proposed method for accuracy as well as robustness, and the results show that the proposed approach compares favorably with some previous works, including the classic Zhang's algorithm that derives the principal point together with other camera parameters at the same time.
KalmanFlow: Efficient Kalman Filtering for Video Optical Flow

This paper proposes an efficient optical flow filtering method for video sequences. Motivated by the observation that motions in videos have strong temporal coherence, we use Kalman filtering to exploit this characteristic for more accurate flow fields. In the proposed system, pixel's motion flow is formulated as a time-variant state vector and optimally estimated by Kalman filter according to the noise level, which is evaluated using flow's temporal derivative, spatial gradient and matching error. Experiments on MPI Sintel video dataset demonstrate that the temporal coherence employed during Kalman filtering has the advantage of more consistent results, and can contribute to the state-of-the-art methods.