Reviewerly/exHarmony huggingface dataset:
Where's Wally Now? Deep Generative and Discriminative Embeddings for Novelty Detection

We develop a framework for novelty detection (ND) methods relying on deep embeddings, either discriminative or generative, and also propose a novel framework for assessing their performance. While much progress was made recently in these approaches, it has been accompanied by certain limitations: most methods were tested on relatively simple problems (low resolution images / small number of classes) or involved non-public data; comparative performance has often proven inconclusive because of lacking statistical significance; and evaluation has generally been done on non-canonical problem sets of differing complexity, making apples-to-apples comparative performance evaluation difficult. This has led to a relative confusing state of affairs. We address these challenges via the following contributions: We make a proposal for a novel framework to measure the performance of novelty detection methods using a trade-space demonstrating performance (measured by ROCAUC) as a function of problem complexity. We also make several proposals to formally characterize problem complexity. We conduct experiments with problems of higher complexity (higher image resolution / number of classes). To this end we design several canonical datasets built from CIFAR-10 and ImageNet (IN-125) which we make available to perform future benchmarks for novelty detection as well as other related tasks including semantic zero/adaptive shot and unsupervised learning. Finally, we demonstrate, as one of the methods in our ND framework, a generative novelty detection method whose performance exceeds that of all recent best-in-class generative ND methods.
DeCloud: Truthful Decentralized Double Auction for Edge Clouds

The sharing economy has made great inroads with services like Uber or Airbnb enabling people to share their unused resources with those needing them. The computing world, however, despite its abundance of excess computational resources has remained largely unaffected by this trend, save for few examples like SETI@home. We present DeCloud, a decentralized market framework bringing the sharing economy to on-demand computing where the offering of pay-as-you-go services will not be limited to large companies, but ad hoc clouds can be spontaneously formed on the edge of the network. We design incentive compatible double auction mechanism targeted specifically for distributed ledger trust model instead of relying on third-party auctioneer. DeCloud incorporates innovative matching heuristic capable of coping with the level of heterogeneity inherent for large-scale open systems. Evaluating DeCloud on Google cluster-usage data, we demonstrate that the system has a near-optimal performance from an economic point of view, additionally enhanced by the flexibility of matching.
Chamulteon: Coordinated Auto-Scaling of Micro-Services

Nowadays, in order to keep track of the fast changing requirements of Internet applications, auto-scaling is used as an essential mechanism for adapting the number of provisioned resources to the resource demand. The straightforward approach is to deploy a set of common and opensource single-service auto-scalers for each service independently. However, this deployment leads to problems such as bottleneck-shifting and increased oscillations. Existing auto-scalers that scale applications consisting of multiple services are kept closed-source. To face these challenges, we first survey existing auto-scalers and highlight current challenges. Then, we introduce Chamulteon, a redesign of our previously introduced mechanism, which can scale applications consisting of multiple services in a coordinated manner. We evaluate Chamulteon against four different well-cited auto-scalers in four sets of measurement-based experiments where we use diverse environments (VM vs. Docker), real-world traces, and vary the scale of the demanded resources. Overall, Chamulteon achieves the best auto-scaling performance based on established user-oriented and endorsed elasticity metrics.
From video games to solar energy: 3D shading simulation for PV using GPU

Photovoltaic (PV) systems can be affected by complex shading. Software solutions have been developed over time, offering an ever-increasing set of simulation possibilities to evaluate the energy losses induced by shading on PV systems. Yet, several practical cases cannot be satisfactorily solved by means of existing tools. This study explores the possibilities offered by the powerful graphics processing units (GPUs) that have been developed for the video game industry. It is shown that complex shading problems applicable to PV systems can be satisfactorily analyzed, both visually and quantitatively, with a focus on the rasterization process for an in-depth evaluation of the shading dynamics that affect the direct component of solar irradiance. This analysis can be conducted at high spatiotemporal resolution for maximum accuracy. Its application is illustrated based on several practical cases that are typically encountered in the world of PV systems engineering, such as building-integrated PV (BIPV) on large and complex buildings, urban PV planning, or PV plants equipped with tracking systems and installed on uneven ground. Additional advantages are also presented, including the full integration of the GPU-based shading simulation tool into a Web browser, and the use of online input information.
Toward Efficient Compute-Intensive Job Allocation for Green Data Centers: A Deep Reinforcement Learning Approach

Reducing the energy consumption of the servers in a data center via proper job allocation is desirable. Existing advanced job allocation algorithms, based on constrained optimization formulations capturing servers' complex power consumption and thermal dynamics, often scale poorly with the data center size and optimization horizon. This paper applies deep reinforcement learning (DRL) to build an allocation algorithm for long-lasting and compute-intensive jobs that are increasingly seen among today's computation demands. Specifically, a deep Q-network is trained to allocate jobs, aiming to maximize a cumulative reward over long horizons. The training is performed offline using a computational model based on long short-term memory networks that capture the servers' power and thermal dynamics. This offline training approach avoids slow online convergence, low energy efficiency, and potential server overheating during the DRL's extensive state-action space exploration if it directly interacts with the physical data center in the usually adopted online learning scheme. At run time, the trained Q-network is forward-propagated with little computation to allocate jobs. Evaluation based on 8 months' physical state and job arrival records from a national supercomputing data center hosting 1,152 processors shows that our solution reduces computing power consumption by nearly 10% and processor temperature by more than 3°C without sacrificing job processing throughput.
Selective feature connection mechanism: Concatenating multi-layer CNN features with a feature selector

Different layers of deep convolutional neural networks(CNNs) can encode different-level information. High-layer features always contain more semantic information, and low-layer features contain more detail information. However, low-layer features suffer from the background clutter and semantic ambiguity. During visual recognition, the feature combination of the low-layer and high-level features plays an important role in context modulation. If directly combining the high-layer and low-layer features, the background clutter and semantic ambiguity may be caused due to the introduction of detailed information. In this paper, we propose a general network architecture to concatenate CNN features of different layers in a simple and effective way, called Selective Feature Connection Mechanism (SFCM). Low-level features are selectively linked to high-level features with a feature selector which is generated by high-level features. The proposed connection mechanism can effectively overcome the above-mentioned drawbacks. We demonstrate the effectiveness, superiority, and universal applicability of this method on multiple challenging computer vision tasks, including image classification, scene text detection, and image-to-image translation.
Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.
Siamese Networks: The Tale of Two Manifolds

Siamese networks are non-linear deep models that have found their ways into a broad set of problems in learning theory, thanks to their embedding capabilities. In this paper, we study Siamese networks from a new perspective and question the validity of their training procedure. We show that in the majority of cases, the objective of a Siamese network is endowed with an invariance property. Neglecting the invariance property leads to a hindrance in training the Siamese networks. To alleviate this issue, we propose two Riemannian structures and generalize a well-established accelerated stochastic gradient descent method to take into account the proposed Riemannian structures. Our empirical evaluations suggest that by making use of the Riemannian geometry, we achieve state-of-the-art results against several algorithms for the challenging problem of fine-grained image classification.
Adaptive Importance Learning for Improving Lightweight Image Super-Resolution Network

Deep neural networks have achieved remarkable success in single image super-resolution (SISR). The computing and memory requirements of these methods have hindered their application to broad classes of real devices with limited computing power, however. One approach to this problem has been lightweight network architectures that balance the super-resolution performance and the computation burden. In this study, we revisit this problem from an orthogonal view, and propose a novel learning strategy to maximize the pixel-wise fitting ability of a given lightweight network architecture. Considering that the initial performance of the lightweight network is very limited, we present an adaptive importance learning scheme for SISR that trains the network with an easy-to-complex paradigm by dynamically updating the importance of image pixels on the basis of the training loss. Specifically, we formulate the network training and the importance learning into a joint optimization problem. With a carefully designed importance penalty function, the importance of individual pixels can be gradually increased through solving a convex optimization problem. The training process thus begins with pixels that are easy to reconstruct, and gradually proceeds to more complex pixels as fitting improves. Furthermore, the proposed learning scheme is able to seamlessly assimilate knowledge from a more powerful teacher network in the form of importance initialization, thus obtaining better initial performance for the network. Through learning the network parameters, and updating pixel importance, the proposed learning scheme enables smaller, lightweight, networks to achieve better performance than has previously been possible. Extensive experiments on four benchmark datasets demonstrate the potential benefits of the proposed learning strategy in lightweight SISR network enhancement. In some cases, our learned network with only $$25\%$$ of the parameters and computational complexity can produce comparable or even better results than the corresponding full-parameter network.
Shadow Inpainting and Removal Using Generative Adversarial Networks with Slice Convolutions

Abstract In this paper, we propose a two‐stage top‐down and bottom‐up Generative Adversarial Networks (TBGANs) for shadow inpainting and removal which uses a novel top‐down encoder and a bottom‐up decoder with slice convolutions. These slice convolutions can effectively extract and restore the long‐range spatial information for either down‐sampling or up‐sampling. Different from the previous shadow removal methods based on deep learning, we propose to inpaint shadow to handle the possible dark shadows to achieve a coarse shadow‐removal image at the first stage, and then further recover the details and enhance the color and texture details with a non‐local block to explore both local and global inter‐dependencies of pixels at the second stage. With such a two‐stage coarse‐to‐fine processing, the overall effect of shadow removal is greatly improved, and the effect of color retention in non‐shaded areas is significant. By comparing with a variety of mainstream shadow removal methods, we demonstrate that our proposed method outperforms the state‐of‐the‐art methods.
Long-tail Hashtag Recommendation for Micro-videos with Graph Convolutional Network

Hashtags, a user provides to a micro-video, are the ones which can well describe the semantics of the micro-video's content in his/her mind. At the same time, hashtags have been widely used to facilitate various micro-video retrieval scenarios (e.g., search, browse, and categorization). Despite their importance, numerous micro-videos lack hashtags or contain inaccurate or incomplete hashtags. In light of this, hashtag recommendation, which suggests a list of hashtags to a user when he/she wants to annotate a post, becomes a crucial research problem. However, little attention has been paid to micro-video hashtag recommendation, mainly due to the following three reasons: 1) lack of benchmark dataset; 2) the temporal and multi-modality characteristics of micro-videos; and 3) hashtag sparsity and long-tail distributions. In this paper, we recommend hashtags for micro-videos by presenting a novel multi-view representation interactive embedding model with graph-based information propagation. It is capable of boosting the performance of micro-videos hashtag recommendation by jointly considering the sequential feature learning, the video-user-hashtag interaction, and the hashtag correlations. Extensive experiments on a constructed dataset demonstrate our proposed method outperforms state-of-the-art baselines. As a side research contribution, we have released our dataset and codes to facilitate the research in this community.
A Survey of Deep Learning Applied to Story Generation

With the rapid development of deep learning in the fields of text abstraction and dialogue generation, researchers are now reconsidering the long-standing story-generation task from the 1970s. Deep learning methods are gradually being adopted to solve problems in traditional story generation, making story generation a new research hotspot in the field of text generation. However, in the field of story generation, the widely used seq2seq model is unable to provide adequate long-distance text modeling. As a result, this model struggles to solve the story-generation task, since the relation between long text should be considered, and coherency and vividness are critical. Thus, recent years have seen numerous proposals for better modeling methods. In this paper, we present the results of a comprehensive study of story generation. We first introduce the relevant concepts of story generation, its background, and the current state of research. We then summarize and analyze the standard methods of story generation. Based on various divisions of user constraints, the story-generation methods are divided into three categories: the theme-oriented model, the storyline-oriented model, and the human-machine-interaction–oriented model. On this basis, we discuss the basic ideas and main concerns of various methods and compare the strengths and weaknesses of each method. Finally, we finish by analyzing and forecasting future developments that could push story-generation research toward a new frontier.
Extractive single document summarization using binary differential evolution: Optimization of different sentence quality measures

With the increase in the amount of text information in different real-life applications, automatic text-summarization systems become more predominant in extracting relevant information. In the current study, we formulated the problem of extractive text-summarization as a binary optimization problem, and multi-objective binary differential evolution (DE) based optimization strategy is employed to solve this. The solutions of DE encode a possible subset of sentences to be present in the summary which is then evaluated based on some statistical features (objective functions) namely, the position of the sentence in the document, the similarity of a sentence with the title, length of the sentence, cohesion, readability, and coverage. These objective functions, measuring different aspects of summary, are optimized simultaneously using the search capability of DE. Some newly designed self-organizing map (SOM) based genetic operators are incorporated in the optimization process to improve the convergence. SOM generates a mating pool containing solutions and their neighborhoods. This mating pool takes part in the genetic operation (crossover and mutation) to create new solutions. To measure the similarity or dissimilarity between sentences, different existing measures like normalized Google distance, word mover distance, and cosine similarity are explored. For the purpose of evaluation, two standard summarization datasets namely, DUC2001, and DUC2002 are utilized, and the obtained results are compared with various supervised, unsupervised and optimization strategy based existing summarization techniques using ROUGE measures. Results illustrate the superiority of our approach in terms of convergence rate and ROUGE scores as compared to state-of-the-art methods. We have obtained 45% and 5% improvements over two recent state-of-the-art methods considering ROUGE-2 and ROUGE-1 scores, respectively, for the DUC2001 dataset. While for the DUC2002 dataset, improvements obtained by our approach are 20% and 5%, considering ROUGE-2 and ROUGE-1 scores, respectively. In addition to these standard datasets, CNN news dataset is also utilized to evaluate the efficacy of our proposed approach. It was also shown that the best performance not only depends on the objective functions used but also on the correct choice of similarity/dissimilarity measure between sentences.
Customizing Student Networks From Heterogeneous Teachers via Adaptive Knowledge Amalgamation

A massive number of well-trained deep networks have been released by developers online. These networks may focus on different tasks and in many cases are optimized for different datasets. In this paper, we study how to exploit such heterogeneous pre-trained networks, known as teachers, so as to train a customized student network that tackles a set of selective tasks defined by the user. We assume no human annotations are available, and each teacher may be either single- or multi-task. To this end, we introduce a dual-step strategy that first extracts the task-specific knowledge from the heterogeneous teachers sharing the same sub-task, and then amalgamates the extracted knowledge to build the student network. To facilitate the training, we employ a selective learning scheme where, for each unlabelled sample, the student learns adaptively from only the teacher with the least prediction ambiguity. We evaluate the proposed approach on several datasets and the experimental results demonstrate that the student, learned by such adaptive knowledge amalgamation, achieves performances even better than those of the teachers.
Proof of Concept of an Assistive Robotic Arm Control Using Artificial Stereovision and Eye-Tracking

Assistive robotic arms have become popular to help users with upper limb disabilities achieve autonomy in their daily tasks, such as drinking and grasping objects in general. Usually, these robotic arms are controlled with an adapted joystick. Joysticks are user-friendly when it comes to a general approach to an object. However, they are not as intuitive when having to accurately approach an object, especially when obstacles are present. Alternatively, the combined use of artificial stereovision and eye-tracking seems to be a promising solution, as the user's vision is usually dissociated from their upper limb disability. Hence, the objective of this study was to develop a proof of concept for the control of an assistive robotic arm using a low-cost combination of stereovision and eye-tracking. Using the developed control system, a typically developed person was able to control the robotic arm successfully reaching and grasping an object for 92% of the trials without obstacles with an average time of 13.8 seconds. Then, another set of trials with one obstacle had a success rate of 91% with an average time of 17.3 seconds. Finally, the last set of trials with two obstacles had a success rate of 98% with an average time of 18.4 seconds. Furthermore, the cost of an eye-tracker and stereovision remains below 400$.
Local-Adaptive Image Alignment Based on Triangular Facet Approximation

Accurate and efficient image alignment is the core problem in the research of panoramic stitching nowadays. This paper proposes a local-adaptive image alignment method based on triangular facet approximation, which directly manipulates the matching data in the camera coordinates, and therefore rises superior to the imaging model of cameras. A more robust planar transformation model is proposed and extended to be local-adaptive via combining it with two weighting strategies. By approximating the scene as a combination of adjacent triangular facets, the planar and spherical triangulation strategies are introduced to more efficiently align normal and fisheye images respectively. The efficiency of the proposed method are verified through the comparative experiments on several challenging cases both qualitatively and quantitatively.