Reviewerly/exHarmony huggingface dataset:
360-Degree Image Completion by Two-Stage Conditional Gans

The latest generative adversarial networks (GANs) can generate realistic high resolution images. However, to the best of our knowledge, there are no GANs for generating 360-degree images. Therefore, this paper proposes the novel problem setting that by using a known area from the 360-degree image as an input, the remainder of the image can be completed with the GANs. To do so, we propose the approach of two-stage generation using network architecture with series-parallel dilated convolution layers. Moreover, we present how to rearrange images for data augmentation, simplify the problem, and make inputs for training the 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nd</sup> stage generator. Our experiments show that these methods generate the distortion seen in 360-degree images in the outlines of buildings and roads, and their boundaries are clearer than those of baseline methods. Furthermore, we discuss and clarify the difficulty of our proposed problem. Our work is the first step towards GANs predicting an unseen area within a 360-degree space.
Decoding High-Throughput Jpeg2000 (HTJ2K) On A G

High-throughput JPEG2000 (HTJ2K), also known as JPEG 2000 Part 15, is the most recent addition to the JPEG2000 suite of coding tools. Ee file extension JPH has been designated for compressed images employing this new part of the standard. Eis new part describes a "fast" block coder for the JPEG 2000 format, while retaining most other JPEG2000 features and capabilities intact. Ee HTJ2K block coder is amenable to parallelizable high-speed encoding and decoding implementations; moreover, it is designed to allow lossless transcoding of already compressed JPEG2000 images that employ the regular block coder. HTJ2K supports the scalability options available in the JPEG2000 format except for quality scalability, which is available only to a limited extent. Eis work gives a high-level overview of this new block coder; we also present preliminary performance results for a GPU implementation. We show that a low-end GPU can decode 4K 4:4:4 12-bit videos at more than 60 frames per second (fps) while a high-end GPU can decode 8K HDR videos at more than 120 fps.
Lap-Based Video Frame Interpolation

High-quality video frame interpolation often necessitates accurate motion estimation, which can be obtained using modern optical flow methods. In this paper, we use the recently proposed Local All-Pass (LAP) algorithm to compute the optical flow between two consecutive frames. The resulting flow field is used to perform interpolation using cubic splines. We compare the interpolation results against a well-known optical flow estimation algorithm as well as against a recent con-volutional neural network scheme for video frame interpolation. Qualitative and quantitative results show that the LAP algorithm performs fast, high-quality video frame interpolation, and perceptually outperforms the neural network and the Lucas-Kanade method on a variety of test sequences.
Improving Multi-scale Face Recognition Using VGGFace2

Convolutional neural networks have reached extremely high performances on the Face Recognition task. These models are commonly trained by using high-resolution images and for this reason, their discrimination ability is usually degraded when they are tested against low-resolution images. Thus, Low-Resolution Face Recognition remains an open challenge for deep learning models. Such a scenario is of particular interest for surveillance systems in which it usually happens that a low-resolution probe has to be matched with higher resolution galleries. This task can be especially hard to accomplish since the probe can have resolutions as low as 8, 16 and 24 pixels per side while the typical input of state-of-the-art neural network is 224. In this paper, we described the training campaign we used to fine-tune a ResNet-50 architecture, with Squeeze-and-Excitation blocks, on the tasks of very low and mixed resolutions face recognition. For the training process we used the VGGFace2 dataset and then we tested the performance of the final model on the IJB-B dataset; in particular, we tested the neural network on the 1:1 verification task. In our experiments we considered two different scenarios: (1) probe and gallery with same resolution; (2) probe and gallery with mixed resolutions. Experimental results show that with our approach it is possible to improve upon state-of-the-art models performance on the low and mixed resolution face recognition tasks with a negligible loss at very high resolutions.
Quadruplet Selection Methods for Deep Embedding Learning

Recognition of objects with subtle differences has been used in many practical applications, such as car model recognition and maritime vessel identification. For discrimination of the objects in fine-grained detail, we focus on deep embedding learning by using a multi-task learning framework, in which the hierarchical labels (coarse and fine labels) of the samples are utilized both for classification and a quadruplet-based loss function. In order to improve the recognition strength of the learned features, we present a novel feature selection method specifically designed for four training samples of a quadruplet. By experiments, it is observed that the selection of very hard negative samples with relatively easy positive ones from the same coarse and fine classes significantly increases some performance metrics in a fine-grained dataset when compared to selecting the quadruplet samples randomly. The feature embedding learned by the proposed method achieves favorable performance against its state-of-the-art counterparts.
Learning a Cascade Regression for No-Reference Super-Resolution Image Quality Assessment

No-reference super-resolution image quality assessment (NRSRIQA) technique has been recognized an effective way to evaluate the quality of SR images and the performance of SR algorithms. In this paper, we propose a novel NR-SRIQA method by learning a two-layer regression model to establish the mapping relationship between the multiple natural statistical features and visual perceptual scores. First, we exploit three types of statistical features to quantify the degradation of SR images. Next, a cascade two-layer regression model, which integrates AdaBoost Decision Tree Regression and ridge regression, is trained to predict the quality of SR images in a coarse-to-fine manner. The experimental results demonstrate that the proposed method is superior to other previous SR quality evaluation approaches and shows better consistency with visual perception quality.
Robustness of Saak Transform Against Adversarial Attacks

Image classification is vulnerable to adversarial attacks. This work investigates the robustness of Saak transform against adversarial attacks towards high performance image classification. We develop a complete image classification system based on multi-stage Saak transform. In the Saak transform domain, clean and adversarial images demonstrate different distributions at different spectral dimensions. Selection of the spectral dimensions at every stage can be viewed as an automatic denoising process. Motivated by this observation, we carefully design strategies of feature extraction, representation and classification that increase adversarial robustness. The performances with well-known datasets and attacks are demonstrated by extensive experimental evaluations.
Classifier Refinement for Weakly Supervised Object Detection with Class-Specific Activation Map

Weakly Supervised Object Detection (WSOD) is a challenging visual understanding task due to the absence of expensive human annotations like bounding boxes and segmentation. Recent WSOD methods usually generate a bounding box for the most contrastive part of the object rather than the entire object. Some methods alleviate the incompleteness problem of detection object with segmentation supplement or classifier refinement. However, segmentation requires high model cost and classifier refinement highly relies on the quality of initial candidate boxes at the beginning. In this paper, we propose a WSOD classifier refinement approach to overcome candidate boxes initialization and high segmentation model cost problems. Our approach can get high-quality class-specific activation maps for the objects and generate nail boxes at the maximum response point of the activation map to suppress incorrect refinement direction. Compared with previous methods, our WSOD classifier refinement approach can achieve 42.1% mAP on PASCAL VOC 2007 benchmarks without high segmentation model cost.
Instance Stixels: Segmenting and Grouping Stixels into Objects

State-of-the-art stixel methods fuse dense stereo and semantic class information, e.g. from a Convolutional Neural Network (CNN), into a compact representation of driveable space, obstacles, and background. However, they do not explicitly differentiate instances within the same class. We investigate several ways to augment single-frame stixels with instance information, which can similarly be extracted by a CNN from the color input. As a result, our novel Instance Stixels method efficiently computes stixels that do account for boundaries of individual objects, and represents individual instances as grouped stixels that express connectivity. Experiments on Cityscapes demonstrate that including instance information into the stixel computation itself, rather than as a post-processing step, increases Instance AP performance with approximately the same number of stixels. Qualitative results confirm that segmentation improves, especially for overlapping objects of the same class. Additional tests with ground truth instead of CNN output show that the approach has potential for even larger gains. Our Instance Stixels software is made freely available for non-commercial research purposes.
Pose Guided Global and Local GAN for Appearance Preserving Human Video Prediction

We propose a pose-guided approach for appearance preserving video prediction by combining global and local information using Generative Adversarial Networks (GANs). The aim is to predict the subsequent frames based on previous frames of human action videos. Considering that human action videos contain both background scenes which are relatively time-invariant among frames, and human actions which are time-varying components, we use a global GAN to model the time-invariant background and coarse human profiles. Then, a local GAN is utilized to further refine the time-varying human parts. Finally, we use a 3D auto-encoder to fine-tune the frame-by-frame images to obtain the whole predicted video. We evaluate our model on the Penn Action and J-HMDB datasets and demonstrate the superiority of our proposed method over other state-of-the-art methods.
Sentiment Analysis on Time-Series Data Using Weight Priority Method on Deep Learning

Sentiment Analysis (SA)is the process to gain an overview of the public opinion on certain topics and it is useful in commerce and social media. The preference on certain topics can be varied on different time periods. To analyze the sentiments on topics in different time periods, priority weight based deep learning approaches like Convolutional-Long Short-Term Memory (C-LSTM)and Stacked- Long Short-Term Memory (S-LSTM)is explored and analyzed in this research. The research method focuses on three phases. In the first phase text data (review given by the customers on various products)is collected from social networking e-commerce site and temporal ordering is done. In the second phase, different deep learning models are created and trained with different time-series data. In the final phase the weights are assigned based on temporal aspect of the data collected. For the obtained results verification and validation processes are carried out. Precision and recall measures are computed. Results obtained shows better performance in terms of classification accuracy and F1-score.
Performance Optimization of Autonomous Platforms in Unstructured Outdoor Environments Using a Novel Constrained Planning Approach

We propose a novel constrained planning approach for autonomous vehicles in unstructured outdoor environments. Our method enables autonomous off-road platforms to keep a predetermined track accurately, but in the same time allows the avoidance of static obstacles and dynamic objects. Two application scenarios are presented according to common transport behaviors with obstacle avoidance: Mule and Convoying. Our method is real-time integrated in a typical processing pipeline for autonomous driving in unstructured outdoor environments. It provides a constrained planning area, subsequently designated cost valley. Our cost valley keeps the trajectory of the vehicle both smooth in difficult passages and close to the desired track. The efficiency of our method is demonstrated on two autonomous platforms with a huge difference in kinematics, weight and size - a small electric platform and an off-road truck. It directly improves the behavior of autonomous vehicles, especially in critical passages.
Code Smells Survival Analysis in Web Apps

Web applications are heterogeneous, both in their target platform (split across client and server sides) and on the formalisms they are built with, usually a mixture of programming and formatting languages. This heterogeneity is perhaps an explanation why software evolution of web applications (apps) is a poorly addressed topic in the literature. In this paper we focus on web apps built with PHP, the most widely used server-side programming language. We analyzed the evolution of 6 code smells in 4 web applications, using the survival analysis technique. Since code smells are symptoms of poor design, it is relevant to study their survival, that is, how long did it take from their introduction to their removal. It is obviously desirable to minimize their survival. In our analysis we split code smells in two categories: scattered smells and localized smells, since we expect the former to be more harmful than the latter. Our results provide some evidence that the survival of PHP code smells depends on their spreadness. We have also analyzed whether the survival curve varies in the long term, for the same web application. Due to the increasing awareness on the potential harmfulness of code smells, we expected to observe a reduction in the survival rate in the long term. The results show that there is indeed a change, for all applications except one, which lead us to consider that other factors should be analyzed in the future, to explain the phenomenon.
Robust Learning for Deep Monocular Depth Estimation

Existing methods for deep monocular depth estimation are often trained with basic loss functions such as mean absolute error (MAE) or reverse Huber (BerHu). We revisit several basic loss functions to explore possibilities for improvement and show that the final depth estimation accuracy is dominated by pixels with small errors, which account for the vast majority. Based on this observation, we propose a new robust loss function that suppresses the contributions of pixels with higher errors by taking their square root. The loss function called the square root Huber (Ruber) is designed to be first-order differentiable on ℝ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;0</sup> , so it can be directly applied to the end-to-end learning of a general type of neural network. Unlike the widely-used robust loss function called Huber, the Ruber loss function facilitates further refinement of the pixels with smaller errors by giving larger gradient values to the errors that are close to zero. Moreover, we show that the estimation accuracy can be further improved by introducing a second training step based on an edge-preserving loss. Experimental results with public indoor scene datasets demonstrate that our method outperforms major loss functions and can yield better accuracies than existing approaches in terms of root mean square error (RMSE).
Human Action Recognition in Unconstrained Trimmed Videos Using Residual Attention Network and Joints Path Signature

Action recognition has been achieved great progress in recent years because of better feature representation learning and classification technology like convolutional neural networks (CNNs). However, most current deep learning approaches treat the action recognition as a black box, ignoring the specific domain knowledge of action itself. In this paper, by analyzing the characteristics of different actions, we proposed a new framework that involves residual-attention module and joint path-signature feature (JPSF) representation framework. The path signature theory was developed recently in the field of rough path and stochastic analysis, which provides a very efficient way to analyze any temporal sequence data. The proposed n-fold joint path signature features entail the Euclidean distances between joints and respective angles. For our experiment, JPSF for three modalities of joints (spatial location, bi-folds and tri-folds) are computed over the temporal length of action sequence. Then all these PSF are concatenated and fed to a CNN to give the recognition result. Experiments on three benchmark datasets, J-HMDB, HMDB-51 and UCF-101, indicate that our proposed method achieves state-of-the-art performance.
Depth-Guided Pixel Dimming With Saliency-Oriented Power-Saving Transformation for Stereoscope AMOLED Displays

With better experience of immersive scene, the virtual reality (VR) becomes the main stream of entertainments. Featuring high chromatic displaying performance, relatively higher efficiency in power and lower latency, the Active-Matrix Organic Lighting-Emitting Diode (AMOLED) is the best-choice display technology for VR. However, AMOLED still requires the higher power consumption amongst other components on smart phones. Thus, various studies attempted to leverage the property of AMOLED to lower the power consumption while preserving the visual quality. This paper proposes a perceptually saliency-oriented transformation by incorporating the concept of how human eyes work in VR environment to provide more suitable experience while saving more power consumption. Experimental results show the proposed method preserves up to 49.33% of the power demand with high perceptual quality as other existing techniques. Furthermore, the subjective experiment demonstrates the proposed method provides a better perceptual quality for VR environment.
An Extensible Approach for Integrating Health and Activity Wearables in Mobile IoT Apps

The amount of edge devices that are currently developed to support fitness and health monitoring is enormous. Many of them aim at measuring body parameters to offer care related services. At the same time, a lot of smart health applications are developed, often making decisions or offering feedback based on sensor data processing. Application developers often struggle to integrate and plug in novel sensor technologies becoming available on the market at a fast pace. However, being able to cope with hardware changes is essential when building sustainable software systems. Standardization efforts are very fragmented and continuously evolve. Hence, compliance with a particular standard seriously restricts the IoT device technologies that can be inserted in practice. Moreover, they do not guarantee any quality-of-service. Our work presents a complementary approach that supports software integrators in building flexible and open IoT ecosystems, helping them to avoid vendor lock-in. The paper further shows how the approach can be applied to build sustainable and easily reconfigurable heart rate monitoring apps.
Spatially-Aware Domain Adaptation for Semantic Segmentation of Urban Scenes

It is very expensive and time consuming to collect a large enough dataset with pixel-level annotations to train a semantic segmentation model. Synthetic datasets are common alternatives for training segmentation models, however models trained on synthetic data do not necessarily perform well on real world images due to the domain shift problem. Domain adaptation techniques address this problem by leveraging on adversarial training to align features. Prior works have mostly performed global feature alignment. They do not consider the positions of objects. However, objects in urban scenes are highly correlated with their spatial locations. For example, the sky will always appear on top while cars will usually appear in the middle of the image. Based on this insight, we propose a spatial-aware discriminator that accounts for the spatial prior on the objects in order to improve the feature alignment. We demonstrate in our experiments that our model outperforms several state-of-the-art baselines in terms of mean intersection over union (mIoU).
An online graph-based anomalous change detection strategy for unsupervised video surveillance

Due to various accidents and crime threats to an unspecified number of people, many surveillance technologies have been studied as an interest in individual security continues to increase throughout society. In particular, intelligent video surveillance technology is one of the most active research areas in the field of surveillance; this popularity has been spurred by recent advances in computer vision/image processing and machine learning. The main goal is to automatically detect, recognize, and analyze objects of interest from collected sensor information and then efficiently extract/utilize this useful information, such as by detecting abnormal events or intruders and recognizing objects. Anomalous event detection is a key component of security, and many existing anomaly detection algorithms rely on a foreground subtraction process to detect changes in the foreground scene. By comparing input image frames with a reference image, changed areas of the image can be efficiently detected. However, this technique can be insensitive to static changes and has difficulties in noisy environments since it depends on a reference image. We propose a new strategy for improved dynamic/static change detection that complements the weak points of existing detection methods, which have low robustness in noisy environments. To achieve this goal, we employed a self-organizing map (SOM) for data clustering and regarded the cluster distribution of neurons, represented by the weight of the optimized SOM, as a directed graph problem. We then applied the shortest path algorithm to recognize anomalous events. The real-time monitoring capability of the proposed change detection system was verified by applying it to self-produced test data and the CDnet-2014 dataset. This system showed robustness against noise that was superior to other surveillance systems in various environments.
Salient Object Detection With Capsule-Based Conditional Generative Adversarial Network

Salient Object Detection (SOD) is one significant research area which is closely correlated to the attention of human beings. Most of the nowadays CNN-based approaches for SOD are based on an U-Net architecture. In this paper, we propose a novel capsule-based salient object detection framework by integrating the novel capsule blocks into both the generator and discriminator of GAN architecture. The experimental result showed that our approach is able to generate accurate saliency maps, which also highlighted the effectiveness of the capsule blocks. We also provide a challenging dataset that contains 3,299 images for SOD with difficult foreground objects and complex background contents.