Reviewerly/exHarmony huggingface dataset:
Sideways: Depth-Parallel Training of Video Models

We propose Sideways, an approximate backpropagation scheme for training video models. In standard backpropagation, the gradients and activations at every computation step through the model are temporally synchronized. The forward activations need to be stored until the backward pass is executed, preventing inter-layer (depth) parallelization. However, can we leverage smooth, redundant input streams such as videos to develop a more efficient training scheme? Here, we explore an alternative to backpropagation; we overwrite network activations whenever new ones, i.e., from new frames, become available. Such a more gradual accumulation of information from both passes breaks the precise correspondence between gradients and activations, leading to theoretically more noisy weight updates. Counter-intuitively, we show that Sideways training of deep convolutional video networks not only still converges, but can also potentially exhibit better generalization compared to standard synchronized backpropagation.
Low-rate Image Compression with Super-resolution Learning

In this paper, we propose an end-to-end learned image compression framework for low-rate scenarios. Based on variational autoencoder, our method features a pair of compact-resolution and super-resolution networks, a set of hyper and main codec networks, and a conditional context model. The learning process of this framework is facilitated with integrated non-local attention modules and phase congruency priors. Multiple models are obtained from training with different hyper-parameters, and are jointly used in the image-level model selection process for rate control, which ensures that the bit rate constraint of the CLIC challenge is satisfied. Experimental results demonstrate that the proposed method can achieve an averaged multi-scale structural similarity (MS-SSIM) score of 0.9648 with bit rate consumption of 0.1499 bits per pixel, which outperforms the BPG image coding method significantly.
Using Fuzzy Set Similarity in Sentence Similarity Measures

Sentence similarity measures the similarity between two blocks of text. A semantic similarity measure between individual pairs of words, each taken from the two blocks of text, has been used in STASIS. Word similarity is measured based on the distance between the words in the WordNet ontology. If the vague words, referred to as fuzzy words, are not found in WordNet, their semantic similarity cannot be used in the sentence similarity measure. FAST and FUSE transform these vague words into fuzzy set representations, type-1 and type-2 respectively, to create ontological structures where the same semantic similarity measure used in WordNet can then be used. This paper investigates eliminating the process of building an ontology with the fuzzy words and instead directly using fuzzy set similarity measures between the fuzzy words in the task of sentence similarity measurement. Their performance is evaluated based on their correlation with human judgments of sentence similarity. In addition, statistical tests showed there is not any significant difference in the sentence similarity values produced using fuzzy set similarity measures between fuzzy sets representing fuzzy words and using FAST semantic similarity within ontologies representing fuzzy words.
Reconciliation Of Group Sparsity And Low-Rank Models For Image Restoration

Image nonlocal self-similarity (NSS) property has been widely exploited via various sparsity models such as joint sparsity (JS) and group sparse coding (GSC). However, the existing NSS-based sparsity models are either too restrictive, i.e., JS enforces the sparse codes to share the same support, or too general, i.e., GSC imposes only plain sparsity on the group coefficients, which limit their effectiveness for modeling real images. In this paper, we propose a novel NSS-based sparsity model, namely low-rank regularized group sparse coding (LR-GSC), to bridge the gap between the popular GSC and JS. The proposed LR-GSC model simultaneously exploits the sparsity and low-rankness of the dictionary-domain coefficients for each group of similar patches. To make the proposed scheme tractable and robust, an alternating minimization with an adaptive adjusted parameter strategy is developed to solve the proposed optimization problem. Experimental results on both image deblocking and denoising demonstrate that the proposed LR-GSC image restoration algorithms outperform many popular or state-of-the-art methods, in terms of both the objective and perceptual quality.
Personality- and Value-Aware Scheduling of User Requests in Cloud for Profit Maximization

The main goal of a cloud provider is to make profits by providing services to users. Existing profit optimization strategies employ homogeneous user models in which user personality is ignored, resulting in fewer profits and particularly notably lower user satisfaction that in turn, leads to fewer users and reduced profits. In this article, we propose efficient personality-aware request scheduling schemes to maximize the profit of the cloud provider under the constraint of user satisfaction. Specifically, we first model the service requests at the granularity of individual personality and propose a personalized user satisfaction prediction model based on questionnaires. Subsequently, we design a personality-guided integer linear programming (ILP)-based request scheduling algorithm to maximize the profit under the constraint of user satisfaction, which is followed by an approximate but lightweight value assessment and cross entropy (VACE)-based profit improvement scheme. The VACE-based scheme is especially tailored for applications with high scheduling resolution. Extensive simulation results show that our satisfaction prediction model can achieve the accuracy of up to 83 percent, and our profit optimization schemes can improve the profit by at least 3.96 percent as compared to the benchmarking methods while still obtaining a speedup of at least 1.68x.
In-Vehicle Infotainment Systems: Using Bayesian Networks to Model Cognitive Selection of Music Genres

In the world of transportation, the ceaseless evolution of Information and Communication Technologies (ICT) is reflected on their migration towards In-Vehicle-Infotainment (IVI) systems, which are characterized by innovative applications and services. Such systems aim to support drivers/passengers with a varying set of functions, enhancing the quality of driving through entertainment. The goal of this study is to introduce a novel IVI cognitive functionality that automatically and dynamically proposes the optimal music genre to the drivers/users when they want to make a certain journey with their owned (or shared) vehicles. The proposed recommender functionality utilizes in an automated manner (i) drivers/users’ profile data and current situation, (ii) drivers/users’ personal preferences, (iii) external environment information obtained from sensor measurements, and (iv) previous knowledge and experience. Knowledge is obtained through the exploitation of Bayesian networking principles in combination with a practical implementation of the Naive-Bayes model. Indicative simulation results showcase the efficiency of the proposed infotainment functionality, in terms of accuracy and speed of convergence, in proactively identifying the optimal music genre and accordingly notifying the drivers/users.
On Improving the Generalization of Face Recognition in the Presence of Occlusions

In this paper, we address a key limitation of existing 2D face recognition methods: robustness to occlusions. To accomplish this task, we systematically analyzed the impact of facial attributes on the performance of a state-of-the-art face recognition method and through extensive experimentation, quantitatively analyzed the performance degradation under different types of occlusion. Our proposed Occlusion-aware face REcOgnition (OREO) approach learned discriminative facial templates despite the presence of such occlusions. First, an attention mechanism was proposed that extracted local identity-related region. The local features were then aggregated with the global representations to form a single template. Second, a simple, yet effective, training strategy was introduced to balance the non-occluded and occluded facial images. Extensive experiments demonstrated that OREO improved the generalization ability of face recognition under occlusions by 10.17% in a single-image-based setting and outperformed the baseline by approximately 2% in terms of rank-1 accuracy in an image-set-based scenario.
Person Re-Identification Via Pose-Aware Multi-Semantic Learning

Person re-identification (ReID) remains an open-ended research topic, with its variety of substantial applications such as tracking, searching, etc. Existing methods mostly explore the highest-semantic feature embedding, ignoring the insights hidden among the earlier layers. Moreover, owing to the misalignment and pose variations, pose-related information is of great significance and needs to be comprehensively utilized. In this paper, we present a novel person ReID framework called Pose-aware Multi-semantic Fusion Network (PMFN). First, taking into account multiple semantics, we propose Multi-semantic Fusion Network (MFN) as the backbone, employing several shortcuts to reserve bypass feature maps for subsequent fusion. Second, to learn a pose-sensitive embedding, pose-aware clues are considered, forming the complete PMFN and investigating the well-aligned global and local body regions. Finally, the center loss is introduced for enhancing the feature discriminability. Exhaustive experiments on two large-scale person ReID benchmarks demonstrate the strengths of our approach over recent state-of-the-art works.
Averaging Essential and Fundamental Matrices in Collinear Camera Settings

Global methods to Structure from Motion have gained popularity in recent years. A significant drawback of global methods is their sensitivity to collinear camera settings. In this paper, we introduce an analysis and algorithms for averaging bifocal tensors (essential or fundamental matrices) when either subsets or all of the camera centers are collinear. We provide a complete spectral characterization of bifocal tensors in collinear scenarios and further propose two averaging algorithms. The first algorithm uses rank constrained minimization to recover camera matrices in fully collinear settings. The second algorithm enriches the set of possibly mixed collinear and non-collinear cameras with additional, ``virtual cameras," which are placed in general position, enabling the application of existing averaging methods to the enriched set of bifocal tensors. Our algorithms are shown to achieve state of the art results on various benchmarks that include autonomous car datasets and unordered image collections in both calibrated and unclibrated settings.
Facial Recognition using Machine Learning Algorithms on Raspberry Pi

Facial recognition is a non-invasive method of biometric authentication and useful for numerous applications. The real time implementation of the algorithm with adequate accuracy is required, with hardware timing into consideration. This paper deals with the implementation of machine learning algorithm for real time facial image recognition. Two dominant methods out of many facial recognition methods are discussed, simulated and implemented using Raspberry Pi. A rigorous comparative analysis is presented considering various limitations which may be the case required for innumerable application which utilize facial recognition. The drawbacks and different use cases of each method is highlighted. The facial recognition software uses algorithms to compare a digital image captured through a camera, to the stored face print so as to authenticate a person's identity. The Haar-Cascade method was one of the first methods developed for facial recognition. The HOG (Histogram of Oriented Gradients) method has worked very effectively for object recognition and thus suitable for facial recognition also. Both the methods are compared with Eigen feature-based face recognition algorithm. Various important features are experimented like speed of operation, lighting condition, frontal face profile, side profiles, distance of image, size of image etc. The facial recognition model is implemented to detect and recognize faces in real-time by means of Raspberry Pi and Pi camera for the user defined database in addition to the available databases.
Utilizing Mask R-CNN for Waterline Detection in Canoe Sprint Video Analysis

Determining a waterline in images recorded in canoe sprint training is an important component for the kinematic parameter analysis to assess an athlete's performance. Here, we propose an approach for the automated waterline detection. First, we utilized a pre-trained Mask R-CNN by means of transfer learning for canoe segmentation. Second, we developed a multi-stage approach to estimate a waterline from the outline of the segments. It consists of two linear regression stages and the systematic selection of canoe parts. We then introduced a parameterization of the waterline as a basis for further evaluations. Next, we conducted a study among several experts to estimate the ground truth waterlines. This not only included an average waterline drawn from the individual experts annotations but, more importantly, a measure for the uncertainty between individual results. Finally, we assessed our method with respect to the question whether the predicted waterlines are in accordance with the experts annotations. Our method demonstrated a high performance and provides opportunities for new applications in the field of automated video analysis in canoe sprint.
A Simple Distortion-Free Method to Handle Variable Length Sequences for Recurrent Neural Networks in Text Dependent Speaker Verification

Recurrent neural networks (RNNs) can model the time-dependency of time-series data. It has also been widely used in text-dependent speaker verification to extract speaker-and-phrase-discriminant embeddings. As with other neural networks, RNNs are trained in mini-batch units. In order to feed input sequences into an RNN in mini-batch units, all the sequences in each mini-batch must have the same length. However, the sequences have variable lengths and we have no knowledge of these lengths in advance. Truncation/padding are most commonly used to make all sequences the same length. However, the truncation/padding causes information distortion because some information is lost and/or unnecessary information is added, which can degrade the performance of text-dependent speaker verification. In this paper, we propose a method to handle variable length sequences for RNNs without adding information distortion by truncating the output sequence so that it has the same length as corresponding original input sequence. The experimental results for the text-dependent speaker verification task in part 2 of RSR 2015 show that our method reduces the relative equal error rate by approximately 1.3% to 27.1%, depending on the task, compared to the baselines but with an associated, small overhead in execution time.
Imparting Fairness to Pre-Trained Biased Representations

Adversarial representation learning is a promising paradigm for obtaining data representations that are invariant to certain sensitive attributes while retaining the information necessary for predicting target attributes. Existing approaches solve this problem through iterative adversarial minimax optimization and lack theoretical guarantees. In this paper, we first study the "linear" form of this problem i.e., the setting where all the players are linear functions. We show that the resulting optimization problem is both non-convex and non-differentiable. We obtain an exact closed-form expression for its global optima through spectral learning. We then extend this solution and analysis to non-linear functions through kernel representation. Numerical experiments on UCI and CIFAR-100 datasets indicate that, (a) practically, our solution is ideal for "imparting" provable invariance to any biased pre-trained data representation, and (b) empirically, the trade-off between utility and invariance provided by our solution is comparable to iterative minimax optimization of existing deep neural network based approaches.Code is available at Human Analysis Lab.
The Implementation and Optimization of Matrix Decomposition Based Collaborative Filtering Task on X86 Platform

With the rapid development of the information age, the recommendation system becomes more and more significant to help people find hidden information from the big dataset in daily lives. Collaborative filtering is a popular technology often used in recommendation systems, which recommend items to users according to other users having the similar behaviors with the target user or according to the items having the alike properties with the target item. In this paper, we implement a parallel collaborative filtering algorithm called ALS-WR on the AMD x86 platform and use an adaptive granularity tuning method to obtain the best performance of 124.86 s in 30 training rounds.
High-Performance Constant-Time Discrete Gaussian Sampling

Discrete Gaussian distribution plays an essential role in lattice cryptography whereas naive implementations suffer from timing attacks. Unfortunately, conversion to secure constant-time variant incurs severe deterioration in performance. In Knuth-Yao sampling, we demonstrate several properties of the discrete distribution generation tree involving structural features and finite node height. Accordingly we propose a generic method independent of standard deviations, which focuses on minimizing the Boolean expressions for the mapping from input bit strings to output sample values, along with an in-depth efficiency analysis. Two optimization techniques are devised to further propel the minimization by replacing and adjusting nodes. To strike the balance of computational overhead and closeness to optimum, heuristic strategies are introduced. Finally, performance evaluation is conducted both in software and hardware. Running on a 3.4GHz Intel Core i7-6700 processor, our method improves sampling rate by up to 29.5 percent compared to the latest technique. Targeting hardware FPGA devices, our approach can be 2.7 times faster and achieves 57.3 percent resource reduction than the original constant-time Knuth-Yao sampling. Compared to the Cumulative Distribution Table algorithm with fixed step binary search, our sampler can be at least 12.6 times faster and gains 79/61 percent better area-time product than its counterpart without/with BRAM.
Subjective Assessment of Global Picture-Wise Just Noticeable Difference

The picture-wise just noticeable difference (PJND) for a given image and a compression scheme is a statistical quantity giving the smallest distortion that a subject can perceive when the image is compressed with the compression scheme. The PJND is determined with subjective assessment tests for a sample of subjects. We introduce and apply two methods of adjustment where the subject interactively selects the distortion level at the PJND using either a slider or keystrokes. We compare the results and times required to those of the adaptive binary search type approach, in which image pairs with distortions that bracket the PJND are displayed and the difference in distortion levels is reduced until the PJND is identified. For the three methods, two images are compared using the flicker test in which the displayed images alternate at a frequency of 8 Hz. Unlike previous work, our goal is a global one, determining the PJND not only for the original pristine image but also for a sequence of compressed versions. Results for the MCL-JCI dataset show that the PJND measurements based on adjustment are comparable with those of the traditional approach using binary search, yet significantly faster. Moreover, we conducted a crowdsourcing study with side-byside comparisons and forced choice, which suggests that the flicker test is more sensitive than a side-by-side comparison.
Text Steganalysis with Attentional L STM-CNN

With the rapid development of Natural Language Processing (NLP) technologies, text steganography methods have been significantly innovated recently, which poses a great threat to cybersecurity. In this paper, we propose a novel attentional LSTM-CNN model to tackle the text steganalysis problem. The proposed method firstly maps words into semantic space for better exploitation of the semantic feature in texts and then utilizes a combination of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) recurrent neural networks to capture both local and long-distance contextual information in steganography texts. In addition, we apply attention mechanism to recognize and attend to important clues within suspicious sentences. After merge feature clues from Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), we use a softmax layer to categorize the input text as cover or stego. Experiments showed that our model can achieve the state-of-art result in the text steganalysis task.
I Think Therefore You Are

Cyber-physical systems operate in our real world, constantly interacting with the environment and collaborating with other systems. The increasing number of devices will make it infeasible to control each one individually. It will also be infeasible to prepare each of them for every imaginable rapidly unfolding situation. Therefore, we must increase the autonomy of future Cyber-physical Systems. Making these systems self-aware allows them to reason about their own capabilities and their immediate environment. In this article, we extend the idea of the self-awareness of individual systems toward networked self-awareness . This gives systems the ability to reason about how they are being affected by the actions and interactions of others within their perceived environment, as well as in the extended environment that is beyond their direct perception. We propose that different levels of networked self-awareness can develop over time in systems as they do in humans. Furthermore, we propose that this could have the same benefits for networks of systems that it has had for communities of humans, increasing performance and adaptability.
Online Signature Verification Using Bidirectional Recurrent Neural Network

Signature verification is one of the most widely used ways to authenticate any person in today's world. Due to this reason, signature forgery is being widely attempted by attackers. Signature verification is classified into two categories online signature verification and other is offline signature verification. This paper focuses on identifying forgery done during the online signature verification. In this paper, we used the signature verification competition 2004 (SVC2004) dataset for our experiments. In the proposed method we used Discrete Fourier transform which is used to extract distinguishable features between forged and genuine signatures. In the next step, we used Recurrent Neural Networks methods Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM) for classification. We used bidirectional LSTM and bidirectional GRU as here we do have past as well as future results. Final results using bidirectional methods outperformed our overall results.