Reviewerly/exHarmony huggingface dataset:
Texture and Shape Biased Two-Stream Networks for Clothing Classification and Attribute Recognition

Clothes category classification and attribute recognition have achieved distinguished success with the development of deep learning. People have found that landmark detection plays a positive role in these tasks. However, little research is committed to analyzing these tasks from the perspective of clothing attributes. In our work, we explore the usefulness of landmarks and find that landmarks can assist in extracting shape features; and using landmarks for joint learning can increase classification and recognition accuracy effectively. We also find that texture features have an impelling effect on these tasks and that the pre-trained ImageNet model has good performance in extracting texture features. To this end, we propose to use two streams to enhance the extraction of shape and texture, respectively. In particular, this paper proposes a simple implementation, Texture and Shape biased Fashion Networks (TS-FashionNet). Comprehensive and rich experiments demonstrate our discoveries and the effectiveness of our model. We improve the top-3 classification accuracy by 0.83% and improve the top-3 attribute recognition recall rate by 1.39% compared to the state-of-the-art models.
Towards Coding For Human And Machine Vision: A Scalable Image Coding Approach

The past decades have witnessed the rapid development of image and video coding techniques in the era of big data. However, the signal fidelity-driven coding pipeline design limits the capability of the existing image/video coding frameworks to fulfill the needs of both machine and human vision. In this paper, we come up with a novel image coding framework by leveraging both the compressive and the generative models, to support machine vision and human perception tasks jointly. Given an input image, the feature analysis is first applied, and then the generative model is employed to perform image reconstruction with features and additional reference pixels, in which compact edge maps are extracted in this work to connect both kinds of vision in a scalable way. The compact edge map serves as the basic layer for machine vision tasks, and the reference pixels act as a sort of enhanced layer to guarantee signal fidelity for human vision. By introducing advanced generative models, we train a flexible network to reconstruct images from compact feature representations and the reference pixels. Experimental results demonstrate the superiority of our framework in both human visual quality and facial landmark detection, which provide useful evidence on the emerging standardization efforts on MPEG VCM (Video Coding for Machine). Our project website is available at https://williamyang1991.github.io/projects/VCM-Face/.
Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data

Transfer learning has proven to be a successful technique to train deep learning models in the domains where little training data is available. The dominant approach is to pretrain a model on a large generic dataset such as ImageNet and finetune its weights on the target domain. However, in the new era of an ever-increasing number of massive datasets, selecting the relevant data for pretraining is a critical issue. We introduce Neural Data Server (NDS), a large-scale search engine for finding the most useful transfer learning data to the target domain. NDS consists of a dataserver which indexes several large popular image datasets, and aims to recommend data to a client, an end-user with a target application with its own small labeled dataset. The dataserver represents large datasets with a much more compact mixture-of-experts model, and employs it to perform data search in a series of dataserver-client transactions at a low computational cost. We show the effectiveness of NDS in various transfer learning scenarios, demonstrating state-of-the-art performance on several target datasets and tasks such as image classification, object detection and instance segmentation. Neural Data Server is available as a web-service at http://aidemo s.cs.toronto.edu/nds/.
Visual Grounding in Video for Unsupervised Word Translation

There are thousands of actively spoken languages on Earth, but a single visual world. Grounding in this visual world has the potential to bridge the gap between all these languages. Our goal is to use visual grounding to improve unsupervised word mapping between languages. The key idea is to establish a common visual representation between two languages by learning embeddings from unpaired instructional videos narrated in the native language. Given this shared embedding we demonstrate that (i) we can map words between the languages, particularly the 'visual' words; (ii) that the shared embedding provides a good initialization for existing unsupervised text-based word translation techniques, forming the basis for our proposed hybrid visual-text mapping algorithm, MUVE; and (iii) our approach achieves superior performance by addressing the shortcomings of text-based methods -- it is more robust, handles datasets with less commonality, and is applicable to low-resource languages. We apply these methods to translate words from English to French, Korean, and Japanese -- all without any parallel corpora and simply by watching many videos of people speaking while doing things.
Variational-EM-Based Deep Learning for Noise-Blind Image Deblurring

Non-blind deblurring is an important problem encountered in many image restoration tasks. The focus of non-blind deblurring is on how to suppress noise magnification during deblurring. In practice, it often happens that the noise level of input image is unknown and varies among different images. This paper aims at developing a deep learning framework for deblurring images with unknown noise level. Based on the framework of variational expectation maximization (EM), an iterative noise-blind deblurring scheme is proposed which integrates the estimation of noise level and the quantification of image prior uncertainty. Then, the proposed scheme is unrolled to a neural network (NN) where image prior is modeled by NN with uncertainty quantification. Extensive experiments showed that the proposed method not only outperformed existing noise-blind deblurring methods by a large margin, but also outperformed those state-of-the-art image deblurring methods designed/trained with known noise level.
Foreign Direct Investment and Economic Growth in Africa: New Empirical Approach on the Role of Institutional Development

According to the predominant conclusion from the recent FDI-growth literature, the conflicting evidence concerning the effect of foreign direct investment on economic growth in host countries might be due to the fact that the gains from FDI are conditional on certain domestic factors. Among these factors is the quality of institutions. The aim of this study is therefore to investigate the role of institutional development in FDI-growth nexus in African countries. Using a panel smooth transition regression model with a sample made up of 27 African countries over the period 1990–2017, the results show that FDI promotes economic growth in countries where the level of institutional development is beyond a certain threshold. In countries that fall below the threshold, FDI has either a negative or null effect on economic growth. Specifically, we find that countries should be above the 65% threshold regarding government stability score, 55% for investment profile, 50% for democratic accountability, 45% for law and order, 35% for corruption, and above the 25% threshold for bureaucracy quality score to benefit from FDI in terms of economic growth.
Virtual Machine Placement Optimization in Mobile Cloud Gaming Through QoE-Oriented Resource Competition

Cloud gaming is a novel service provisioning paradigm, which hosts video games in the cloud and transmits interactive game streams to game players via the Internet. In such cloud gaming scenarios, the cloud is required to consume tremendous resources for video rendering and streaming, especially when the number of concurrent players reaches a certain level. On the other hand, different game players may have distinct requirements on Quality-of-Experience, such as high video quality, low delay, etc. Under this circumstance, how to satisfy players of different interests by efficiently leveraging cloud resources becomes a major challenge to existing cloud gaming services. In order to meet the overall requirements of players in a cost-effective manner, this article applies game theory to cloud gaming scenarios. It proposes a distributed algorithm to optimize virtual machine (VM) placement in mobile cloud gaming through resource competition. Further, by constructing a potential function, we prove that the resource competition game is a potential game, and the proposed algorithm scales well as the player population increases. We prove theoretically and verify experimentally that, with the proposed distributed VM placement algorithm, players can achieve a mutually satisfying state within a finite number of iterations.
Disparity-Aware Domain Adaptation in Stereo Image Restoration

Under stereo settings, the problems of disparity estimation, stereo magnification and stereo-view synthesis have gathered wide attention. However, the limited image quality brings non-negligible difficulties in developing related applications and becomes the main bottleneck of stereo images. To the best of our knowledge, stereo image restoration is rarely studied. Towards this end, this paper analyses how to effectively explore disparity information, and proposes a unified stereo image restoration framework. The proposed framework explicitly learn the inherent pixel correspondence between stereo views and restores stereo image with the cross-view information at image and feature level. A Feature Modulation Dense Block (FMDB) is introduced to insert disparity prior throughout the whole network. The experiments in terms of efficiency, objective and perceptual quality, and the accuracy of depth estimation demonstrates the superiority of the proposed framework on various stereo image restoration tasks.
Deep Degradation Prior for Low-Quality Image Classification

State-of-the-art image classification algorithms building upon convolutional neural networks (CNNs) are commonly trained on large annotated datasets of high-quality images. When applied to low-quality images, they will suffer a significant degradation in performance, since the structural and statistical properties of pixels in the neighborhood are obstructed by image degradation. To address this problem, this paper proposes a novel deep degradation prior for low-quality image classification. It is based on statistical observations that, in the deep representation space, image patches with structural similarity have uniform distribution even if they come from different images, and the distributions of corresponding patches in low- and high-quality images have uniform margins under the same degradation condition. Therefore, we propose a feature de-drifting module (FDM) to learn the mapping relationship between deep representations of low- and high- quality images, and leverage it as a deep degradation prior (DDP) for low-quality image classification. Since the statistical properties are independent to image content, deep degradation prior can be learned on a training set of limited images without supervision of semantic labels and served in a form of “plugging-in” module of the existing classification networks to improve their performance on degraded images. Evaluations on the benchmark dataset ImageNet-C demonstrate that our proposed DDP can improve the accuracy of the pre-trained network model by more than 20% under various degradation conditions. Even under the extreme setting that only 10 images from CUB-C dataset are used for the training of DDP, our method improves the accuracy of VGG16 on ImageNet-C from 37% to 55%.
An automated implementation of hybrid cloud for performance evaluation of distributed databases

A Hybrid cloud is an integration of resources between private and public clouds. It enables users to horizontally scale their on-premises infrastructure up to public clouds in order to improve performance and cut up-front investment cost. This model of applications deployment is called cloud bursting that allows data-intensive applications especially distributed database systems to have the benefit of both private and public clouds. In this work, we present an automated implementation of a hybrid cloud using (i) a robust and zero-cost Linux-based VPN to make a secure connection between private and public clouds, and (ii) Terraform as a software tool to deploy infrastructure resources based on the requirements of hybrid cloud. We also explore performance evaluation of cloud bursting for six modern and distributed database systems on the hybrid cloud spanning over local OpenStack and Microsoft Azure. Our results reveal that MongoDB and MySQL Cluster work efficient in terms of throughput and operations latency if they burst into a public cloud to supply their resources. In contrast, the performance of Cassandra, Riak, Redis, and Couchdb reduces if they significantly leverage their required resources via cloud bursting.
PuppeteerGAN: Arbitrary Portrait Animation With Semantic-Aware Appearance Transformation

Portrait animation, which aims to animate a still portrait to life using poses extracted from target frames, is an important technique for many real-world entertainment applications. Although recent works have achieved highly realistic results on synthesizing or controlling human head images, the puppeteering of arbitrary portraits is still confronted by the following challenges: 1) identity/personality mismatch; 2) training data/domain limitations; and 3) low-efficiency in training/fine-tuning. In this paper, we devised a novel two-stage framework called PuppeteerGAN for solving these challenges. Specifically, we first learn identity-preserved semantic segmentation animation which executes pose retargeting between any portraits. As a general representation, the semantic segmentation results could be adapted to different datasets, environmental conditions or appearance domains. Furthermore, the synthesized semantic segmentation is filled with the appearance of the source portrait. To this end, an appearance transformation network is presented to produce fidelity output by jointly considering the wrapping of semantic features and conditional generation. After training, the two networks can directly perform end-to-end inference on unseen subjects without any retraining or fine-tuning. Extensive experiments on cross-identity/domain/resolution situations demonstrate the superiority of the proposed PuppetterGAN over existing portrait animation methods in both generation quality and inference speed.
Document Image Binarization Using Dual Discriminator Generative Adversarial Networks

For document image analysis, image binarization is an important preprocessing step. Also, binarization can help in improving the readability of old and historical manuscripts. Such documents are generally degraded due to various reasons such as bleed-through, faded ink, or stains. Achieving good binarization performance on these documents is a challenging task. In this letter, a deep learning based model for document image binarization has been proposed, comprising a Dual Discriminator Generative Adversarial Network (DD-GAN) which uses Focal Loss as generator loss. The DD-GAN consists of two discriminator networks - one looks for the global similarity i.e. on the whole image, and another one explores the image in small patches i.e. local similarity. At the final stage, simple thresholding is performed on the generated images. The method has been tested on five recent DIBCO datasets. It has been found that the method is robust and it provides results comparable with state-of-the-art methods. The code for this letter is available at https://github.com/anuran-Chakraborty/BinarizationDualDiscriminatorGAN.
A Statistical Approach for Sarcasm Detection Using Twitter Data

Sarcasm is an alternate sort of assessment that comprises of words which by and large methods something contrary to what you truly need to state and can be utilized in various circumstances, to affront, to show disturbance, or be clever. Here and there to pass on understood data, sarcasm is utilized inside the message as an individual conveys. Sarcasm can be used for different capacities, similar to analysis or joke. In any case, sarcasm is extremely debilitating in any event, for people to recognize. Therefore, the detection of sarcasm leads to a better understanding of sentiment analysis of the user, from the data collected from websites such as Twitter and Facebook. The advantages of identifying sarcasm for opinion mining has prompted enthusiasm for automated sarcasm recognition as an exploration issue. In Automated sarcasm identification, the program tries to identify whether a text is sarcastic or neutral. The research paper is divided into two phases. In the first phase, it extracts features related to sentiments and punctuation, and then the chi-square test is used to shortlist the most useful features. In the second phase, 200 top tf-idf features are extracted and combined with sentiment related and punctuation related features to find sarcastic content within the tweet.In the first approach the best accuracy is achieved by using the Support Vector Machine algorithm which is equal is equal to 74.59% and in the second approach the maximum accuracy is achieved by the voting classifier and it increases the accuracy to 83.53%.
Exploiting a low-cost CNN with skip connection for robust automatic modulation classification

Recently, deep learning (DL) is an innovative machine learning (ML) technique that has gained the outstanding achievements in computer vision and natural language processing. This work takes advantage of DL for effectively handling automatic modulation classification (AMC), which is the fundamental function of numerous cognitive radio-based and spectrum sensing-based applications in many modern communication systems. Concretely, a novel deep convolutional neural network (DCNN) is proposed for learning a classification model from a massive amount of modulated signals, in which the network architecture has several convolutional blocks specialized to simultaneously capture the temporal intra-signal correlations and the spatial inter-signal relations. To this end, each block comprises various convolutional layers of asymmetric convolution kernels, whose outputs are gathered via a concatenation layer. For the enrichment of multi-scale deep feature and the prevention of gradient vanishing problem, these blocks are associated by skip connections to take into account the useful residual information. In experiments, the proposed CNN-based AMC method achieves the overall 24-modulation classification rate of 88.22% at 10dB SNR on the well-known DeepSig dataset.
Auction based Incentive Design for Efficient Federated Learning in Cellular Wireless Networks

Federated learning is an prominent machine learning technique that model is trained distributively by using local data of mobile users, which can preserve the privacy of users and still guarantee high learning performance. In this paper, we deal with the problem of incentive mechanism design for motivating users to participate in training. In this paper, we employ the randomized auction framework for incentive mechanism design in which the base station is a seller and mobile users are buyers. Concerning the energy cost incurred due to join the training, the users need to decide how many uplink subchannels, transmission power and CPU cycle frequency and then claim them in submitted bids to the base station. After receiving the submitted bids, the base station needs algorithms to select winners and determine the corresponding rewards so that the social cost is minimized. The proposed mechanism can guarantee three economic properties, i.e., truthfulness, individual rationality and efficiency. Finally, numerical results are provided to demonstrate the effectiveness, and efficiency of our scheme.
Decay suppression of spontaneous emission of a single emitter in a high- <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>Q</mml:mi></mml:math> cavity at exceptional points

The authors show that the peculiar spectral response of the electromagnetic field of a high-Q cavity operating at an exceptional point modifies the spontaneous emission of a single emitter in the strong coupling regime. The theory, based on macroscopic QED in dispersing and absorbing media, demonstrates much longer effective decay times in the exceptional point case in comparison to the single-mode case
Convergence and objective functions of noise-injected multilayer perceptrons with hidden multipliers

Artificial neural networks (ANNs) are known to be sensitive to the initial setting of parameters and the network architecture, such as the number of hidden nodes in multilayer perceptron (MLP). In this paper, we focus on a network structure which can help to find the proper number of hidden nodes in MLP. In this structure, so called Multilayer Perceptrons with Hidden Multipliers (MLPHM), each of the hidden nodes is associated with a tunable "gate" multiplier. With a specific regularization term, each gate tends to be opened or closed completely at the end of the training, and finally a pruned network is obtained. To study the fault tolerance and to improve the generalization of MLPHM, a noise-injected training scheme is proposed, with both multiplicative noise and additive noise taken into consideration. The objective functions and convergence theorems of the noise-injected training algorithms are obtained, and the latter have been verified by simulations. Applications to several UCI datasets have demonstrated that the proposed algorithms have efficient pruning ability and superior generalization ability.
Scalable KDE-based top-<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="d1e877" altimg="si742.svg"><mml:mi>n</mml:mi></mml:math> local outlier detection over large-scale data streams

The detection of local outliers over high-volume data streams is critical for diverse real-time applications in the real world, where the distributions in different subsets of the data tend to be skewed. However, existing methods are not scalable to large-scale high-volume data streams owing to the high complexity of the re-detection of data updates. In this work, we propose a top-n local outlier detection method based on Kernel Density Estimation (KDE) over large-scale high-volume data streams. First, we define a KDE-based Outlier Factor (KOF) to measure the local outlierness score for the data points. Then, we propose the upper bounds of the KOF and an upper-bound-based pruning strategy to quickly eliminate the majority of the inlier points by leveraging the upper bounds without computing the expensive KOF scores. Moreover, we design an Upper-bound pruning-based top-n KOF detection method (UKOF) over data streams to efficiently address the data updates in a sliding window environment. Furthermore, we propose a Lazy update method of UKOF (LUKOF) for bulk updates in high-speed large-scale data streams to further minimize the computation cost. Our comprehensive experimental study demonstrates that the proposed method outperforms the state-of-the-art methods by up to 3,600 times in speed, while achieving the best performance in detecting local outliers over data streams.