Miracle-dz/UniFAR-FTUs huggingface dataset:
Future Spaces: Reinventing the Home Network for Better Security and Automation in the IoT Era

Cyber-Physical Systems (CPSs) are complex systems comprising computation, physical, and networking assets. Used in various domains such as manufacturing, agriculture, vehicles, etc., they blend the control of the virtual and physical worlds. Smart homes are a peculiar type of CPS where the local networking fundamentals have seen little evolution in the past decades, while the context in which home networks operate has drastically evolved. With the advent of the Internet of Things (IoT), the number and diversity of devices connected to our home networks are exploding. Some of those devices are poorly secured and put users’ data privacy and security at risk. At the same time, administrating a home network has remained a tedious chore, requiring skills from un-savvy users. We present Future Spaces, an end-to-end hardware-software prototype providing fine-grained control over IoT connectivity to enable easy and secure management of smart homes. Relying on Software-Defined Networking-enabled home gateways and the virtualization of network functions in the cloud, we achieve advanced networking security and automation through the definition of isolated, usage-oriented slices. This disrupts how users discover, control and share their connected assets across multiple domains, smoothly adapting to various usage contexts.
Semi-supervised Cross-domain Visual Feature Learning for Audio-Visual Broadcast Speech Transcription

Visual information can be incorporated into automatic speech recognition (ASR) systems to improve their robustness in adverse acoustic conditions. Conventional audio-visual speech recognition (AVSR) systems require highly specialized audio-visual (AV) data in both system training and evaluation. For many real-world speech recognition applications, only audio information is available. This presents a major challenge to a wider application of AVSR systems. In order to address this challenge, this paper proposes a semi-supervised visual feature learning approach for developing AVSR systems on a DARPA GALE Mandarin broadcast transcription task. Audio to visual feature inversion long short-term memory neural networks (L-STMs) were initially constructed using limited amounts of out of domain AV data. The acoustic features domain mismatch against the broadcast data was further reduced using multi-level domain adaptive deep networks. Visual features were then automatically generated from the broadcast speech audio and used in both AVSR system training and testing time. Experimental results suggest a CNN based AVSR system using the proposed semi-supervised cross-domain audio-to-visual feature generation technique outperformed the baseline audio only CNN ASR system by an average CER reduction of 6.8% relative. In particular, on the most difﬁcult Phoenix TV subset, a CER reduction of 1.32% absolute (8.34% relative) was obtained.
Estimation of Fundamental Frequency from Singing Voice Using Harmonics of Impulse-like Excitation Source

This paper focuses on the problem of estimating fundamental frequency from singing voice. Estimation of fundamental frequency is a well studied topic in the speech research community. From the recent studies on fundamental frequency estimation from singing voice with state-of-art methods proposed for speech, there exists a signiﬁcant gap in accuracy for singing voice. This is mainly because of the wider and rapid variations in pitch in singing voice compared to that in speech. To overcome this, in this paper we propose a method to derive the fundamental frequency from singing voice by exploiting the harmonics of impulse-like excitation in sequence of glottal cycles. The proposed method is compared with the eight state-of-art methods such as YIN, SWIPE, YAAPT, RAPT, SRH, SFF CEP, PEFAC and SHRP on the LYRICS singing database. From the experimental results, it is observed that the accuracy of fundamental frequency by the proposed method is better than many state-of-art methods in various singing categories and laryngeal mechanisms.
Multi-Task Learning of Speech Recognition and Speech Synthesis Parameters for Ultrasound-based Silent Speech Interfaces

Silent Speech Interface systems apply two different strategies to solve the articulatory-to-acoustic conversion task. The recognition-and-synthesis approach applies speech recognition techniques to map the articulatory data to a textual transcript, which is then converted to speech by a conventional text-to-speech system. The direct synthesis approach seeks to convert the articulatory information directly to speech synthesis (vocoder) parameters. In both cases, deep neural networks are an evident and popular choice to learn the mapping task. Recognizing that the learning of speech recognition and speech synthesis targets (acoustic model states vs. vocoder parameters) are two closely related tasks over the same ultrasound tongue image input, here we experiment with the multi-task training of deep neural networks, which seeks to solve the two tasks simultaneously. Our results show that the parallel learning of the two types of targets is indeed beneﬁcial for both tasks. Moreover, we obtained further improvements by using multi-task training as a weight initialization step before task-speciﬁc training. Overall, we report a relative error rate reduction of about 7% in both the speech recognition and the speech synthesis tasks.
Portrait-Aware Artistic Style Transfer

The goal of artistic style transfer is to transfer the style of artistic works into photos. However, the performances of existing style transfer algorithms on portraits are not very satisfactory, because the synthetic photo is either not sufficiently stylized or distorted severely in the portrait domain (i.e., foreground), which limits the use of style transfer for portraits. In this paper, we propose a novel portrait-aware artistic style transfer algorithm, which treats foreground and background differently. Particularly, we separate the foreground from the background, and apply fine-grained style transfer to the background and coarse-grained style transfer to the entire image at the same time, so that the artistic style of entire image can be transferred with the details of the portrait well preserved. Extensive experiments demonstrate the effectiveness of our proposed method.
Unsupervised Feature Selection with Local Structure Learning

Conventional graph-based unsupervised feature selection approaches carry out the feature selection requiring two stages: first, constructing the data similarity matrix and next performing feature selection. In this way, the similarity matrix is invariably kept unchanged, totally separated from the process of feature selection and the performance of feature selection highly depends on the initially constructed similarity matrix. In order to address this problem, a novel unsupervised feature selection method is proposed in this paper where constructing similarity matrix and performing feature selection are together incorporated into a coherent model. Besides, the constructed similarity matrix has $k$ connected components ($k$ is the number of data clusters). At last, five state-of-the-art unsupervised feature selection methods are compared to validate the effectiveness of the proposed method.
How the Region of Interest Impacts Contact Free Heart Rate Estimation Algorithms

The contact free camera-based estimation of human vital signs is more comfortable than the classical contact-based methods. Current methods suffer in realistic environments from e.g. occlusions by hair or glasses. Several approaches use complex methods to extract the pulse signals from the skin, but use basic geometric defined regions of the face, which ignore possible occlusions. In this paper we compare for the first time the influence of several region based methods and a newly proposed parameter-free skin segmentation approach on the performance of the heart rate estimation for nine different algorithms from the literature on two datasets with strong facial and head movement.
Unsupervised Detection of White Matter Fiber Bundles with Stochastic Neural Networks

Exploring the human structural connectome often involves dealing with millions of white matter tracts reconstructed from diffusion MRI. Reducing the dimensionality of such data by grouping tracts into bundles can prove essential for subsequent analyses. Many unsupervised clustering algorithms aim at providing such bundles but often require the choice of a distance metric and suffer from memory storage issues relating to the size of the datasets. We propose for the first time a neural network approach for the unsupervised clustering of white matter tracts. It has the main properties of learning automatically the tract features and scaling well with the data size. As a proof of concept, we compare both quantitatively and qualitatively the computed tract clusters with a commonly used clustering method. The proposed approach shows results similar to the reference approach while not using any distance matrix or similarity metric.
Knowledge Distillation for Sequence Model

Knowledge distillation , or teacher-student training , has been effectively used to improve the performance of a relatively simpler deep learning model (the student) using a more complex model (the teacher). It is usually done by minimizing the Kullback-Leibler divergence (KLD) between the output distributions of the student and the teacher at each frame. However, the gain from frame-level knowledge distillation is limited for sequence models such as Connectionist Temporal Classiﬁcation (CTC), due to the mismatch between the sequence-level criterion used in teacher model training and the frame-level criterion used in distillation. In this paper, sequence-level knowledge distillation is proposed to achieve better distillation performance. Instead of calculating a teacher posterior distribution given the feature vector of the current frame, sequence training criterion is employed to calculate the posterior distribution given the whole utterance and the teacher model. Experiments are conducted on both English Switchboard corpus and a large Chinese corpus. The proposed approach achieves signiﬁcant and consistent improvements over the traditional frame-level knowledge distillation using both labeled and unlabeled data.
Image Captioning with Word Level Attention

Image captioning is an attractive and challenging task to perform automatic image description and a number of works are designed for this task. Most of these researches are based on convolutional neural network (CNN) and recurrent neural network (RNN), where the primary input to language model for word prediction at the current time step is usually the linguistic word generated at the previous time step. In this work, a novel word level attention layer is designed to process image features with two modules for accurate word prediction. The first is a bidirectional spatial embedding module to handle feature maps, then the second module employs attention mechanism to extract word level attention which will be fed into language model. The experimental results on the benchmark MSCOCO dataset demonstrate that the proposed model achieves the state-of-the-art performances with 106.0 on CIDEr and 34.0 on B-4, respectively.
Towards Fast and Accurate Ellipse and Semi-Ellipse Detection

Ellipse detection is an important task in machine vision and can be used to solve other high-level visual problems. In the past, many studies have been conducted by researchers at high precision ellipse fitting, but most of the proposed solutions are not fast enough to be applied in real-time detection for larger images or with limited hardware resources. This paper improves the accuracy of ellipse detection by improving the candidate ellipse scoring method and geometric constraints. Combined with ellipse fitting algorithm, the proposed method can perform real-time ellipse and semi ellipse detection. The efficiency of the algorithm is proven by using three popular datasets which shows the state of the art ellipse and semiellipse detection performance.
Automatic Visual Augmentation for Concatenation Based Synthesized Articulatory Videos from Real-time MRI Data for Spoken Language Training

For the beneﬁt of spoken language training, concatenation based articulatory video synthesis has been proposed in the past to overcome the limitation in the articulatory data recording. For this, real time magnetic resonance imaging (rt-MRI) video image-frames (IFs) containing articulatory movements have been used. These IFs require a visual augmentation for better understanding. We, in this work, propose an augmentation method using pixel intensities in the regions enclosed by the articulatory boundaries obtained from air-tissue boundaries (ATBs). Since, the pixel intensities reﬂect the muscle movements in the articulators, the augmented IFs could provide realistic articulatory movements, when we color them accordingly. However, the ATB manual annotation is time consuming; hence, we propose to synthesize ATBs using the ATBs from a few selected frames that have been used in synthesizing the articulatory videos. We augment a set of synthesized articulatory videos for 50 words obtained from the MRI-TIMIT database. Subjective evaluation on the quality of the augmented videos using twenty-one subjects suggests that the videos are visually more appealing than the respective synthesized rt-MRI videos with a rating of 3.75 out of 5, where a score of 5 (1) indicates that the augmented video quality is excellent (poor).
Online Filter Clustering and Pruning for Efficient Convnets

Pruning filters is an effective method for accelerating deep neural networks (DNNs), but most existing approaches prune filters on a pre-trained network directly which limits in acceleration. Although each filter has its own effect in DNNs, but if two filters are same with each other, we could prune one safely. In this paper, we add an extra cluster loss term in the loss function which can force filters in each cluster to be similar online. After training, we keep one filter in each cluster and prune others and fine-tune the pruned network to compensate the loss. Particularly, the clusters in every layer can be defined firstly which is effective for pruning DNNs within residual blocks. Extensive experiments on CIFAR10 and CIFR100 benchmarks demonstrate the competitive performance of our proposed filter pruning method.
Management of clinical and subclinical actinic keratoses with histological and immunohistochemical assessments by confocal microscopy

Reflectance confocal microscopy has been used to monitor treatment efficacy in non‐melanoma skin cancer, but few studies have compared different therapies using the same confocal criteria. We compare a single score designed in previous study to evaluate confocal, histological, and immunohistochemical results obtained before and after two treatments of actinic keratosis (AK) and in field cancerisation area. Thirty volunteer male patients, aged >50‐year old, with at least three clinically visible AKs on the bald scalp, were included in this study: 15 patients were submitted to 3% diclofenac therapy and 15 to 5% fluorouracil therapy. Confocal imaging was performed on a 5 × 5 cm skin area. In five patients of each group we performed cutaneous biopsies. Scaling, upper nucleated cells, and inflammatory cells showed a higher percentage reduction in field cancerisation, especially in diclofenac treatment group, while in AK inflammatory cells showed a greater percentage reduction in 5‐fluorouracil treatment group. Both therapies are efficacious, but their effectiveness is different on the single parameters of the confocal, histology, immunohistochemically.
Investigating Speech Enhancement and Perceptual Quality for Speech Emotion Recognition

In this study, the performance of two enhancement algorithms is investigated in terms of perceptual quality as well as in respect to their impact on speech emotion recognition (SER). The SER system adopted is based on the same benchmark system provided for the AVEC Challenge 2016. The three objective measures adopted are the speech-to-reverberation modulation energy ratio (SRMR), the perceptual evaluation of speech quality (PESQ) and the perceptual objective listening quality assessment (POLQA). Evaluations are conducted on speech ﬁles from the RECOLA dataset, which provides spontaneous interactions in French of 27 subjects. Clean speech ﬁles are corrupted with different levels of background noise and reverberation. Results show that applying enhancement prior to the SER task can improve SER performance in more degraded scenarios. We also show that quality measures can be an important asset as indicator of enhancement algorithms performance towards SER, with SRMR and POLQA providing the most reliable results.
Multiple Combined Constraints for Image Stitching

Several approaches to image stitching use different constraints to estimate the motion model between image pairs. These constraints can be roughly divided into two categories: geometric constraints and photometric constraints. In this paper, geometric and photometric constraints are combined to improve the alignment quality, which is based on the observation that these two kinds of constraints are complementary. On the one hand, geometric constraints (e.g., point and line correspondences) are usually spatially biased and are insufficient in some extreme scenes, while photometric constraints are always evenly and densely distributed. On the other hand, photometric constraints are sensitive to displacements and are not suitable for images with large parallaxes, while geometric constraints are usually imposed by feature matching and are more robust to handle parallaxes. The proposed method therefore combines them together in an efficient mesh-based image warping framework. It achieves better alignment quality than methods only with geometric constraints, and can handle larger parallax than photometric-constraint-based method. Experimental results on various images illustrate that the proposed method outperforms representative state-of-the-art image stitching methods reported in the literature.
Adaptive Patch Based Convolutional Neural Network for Robust Dehazing

We present a novel deep learning-based dehazing method using adaptive patch splits. Our method applies quad-tree decomposition to an input image, yielding multiple patches with adaptive sizes. Then, each patch is fed into a Convolutional Neural Network (CNN) and classified into a single transmission value, in which a transmission map comprises transmission values from all patches. Homogeneous regions in the image are typically decomposed into large patches. Thus the method can save computational cost. Non-homogeneous regions are divided into small patches, which helps preserve local details in a transmission map. To train CNN, we synthesize numerous hazy images from haze-free images. Experimental results demonstrate our method surpasses state- of-the-art deep learning based algorithms quantitatively and qualitatively.
Deep Discriminative Representation Learning for Face Verification and Person Re-Identification on Unconstrained Condition

In this paper, we address face verification and person re-identification tasks under the unconstrained condition. Both tasks under the unconstrained condition are difficult since the testing dataset can contain the identities not appeared in training dataset. To overcome the difficulty, we propose deep discriminative representation learning (DDRL) to learn a discriminative representation which can cover not only trained representation but the appearance of images which are not trained. DDRL can be viewed as imposing discriminative constraints on the learnt representation via joint optimization for verification and identification objectives. The experimental results for face verification and person re-identification shows the superiority of DDRL in both tasks.
Structured Scene Decoding with Finite State Machines

We introduce in this work a novel stochastic inference process, for scene annotation and object class segmentation, based on finite state machines (FSMs). The design principle of our framework is generative and based on building, for a given scene, finite state machines that encode annotation lattices, and inference consists in finding and scoring the best configurations in these lattices. Different novel operations are defined using our FSM framework including reordering, segmentation, visual transduction, and label dependency modeling. All these operations are combined together in order to achieve annotation as well as object class segmentation.
Proteomics Analysis Reveals Non-Controlled Activation of Photosynthesis and Protein Synthesis in a Rice npp1 Mutant under High Temperature and Elevated CO2 Conditions

Rice nucleotide pyrophosphatase/phosphodiesterase 1 (NPP1) catalyzes the hydrolytic breakdown of the pyrophosphate and phosphodiester bonds of a number of nucleotides including ADP-glucose and ATP. Under high temperature and elevated CO2 conditions (HT + ECO2), the npp1 knockout rice mutant displayed rapid growth and high starch content phenotypes, indicating that NPP1 exerts a negative effect on starch accumulation and growth. To gain further insight into the mechanisms involved in the NPP1 downregulation induced starch overaccumulation, in this study we conducted photosynthesis, leaf proteomic, and chloroplast phosphoproteomic analyses of wild-type (WT) and npp1 plants cultured under HT + ECO2. Photosynthesis in npp1 leaves was significantly higher than in WT. Additionally, npp1 leaves accumulated higher levels of sucrose than WT. The proteomic analyses revealed upregulation of proteins related to carbohydrate metabolism and the protein synthesis system in npp1 plants. Further, our data indicate the induction of 14-3-3 proteins in npp1 plants. Our finding demonstrates a higher level of protein phosphorylation in npp1 chloroplasts, which may play an important role in carbohydrate accumulation. Together, these results offer novel targets and provide additional insights into carbohydrate metabolism regulation under ambient and adverse conditions.
BSCGAN: Deep Background Subtraction with Conditional Generative Adversarial Networks

This paper proposes a deep background subtraction method based on conditional Generative Adversarial Network (cGAN). The proposed model consists of two successive networks: generator and discriminator. The generator learns the mapping from the observing input (i.e., image and background), to the output (i.e., foreground mask). Then, the discriminator learns a loss function to train this mapping by comparing real foreground (i.e., ground-truth) and fake foreground (i.e., predicted output) with observing the input image and background. Evaluating the model performance with two public datasets, CDnet 2014 and BMC, shows that the proposed model outperforms the state-of-the-art methods.
Music Genre Recognition Using Deep Neural Networks and Transfer Learning

Music genre recognition is a very interesting area of research in the broad scope of music information retrieval and audio signal processing. In this work we propose a novel approach for music genre recognition using an ensemble of convolutional long short term memory based neural networks (CNN LSTM) and a transfer learning model. The neural network models are trained on a diverse set of spectral and rhythmic features whereas the transfer learning model was originally trained on the task of music tagging. We compare our system with a number of recently published works and show that our model outperforms them and achieves new state of the art results