Reviewerly/exHarmony huggingface dataset:
The relightables

We present "The Relightables", a volumetric capture system for photorealistic and high quality relightable full-body performance capture. While significant progress has been made on volumetric capture systems, focusing on 3D geometric reconstruction with high resolution textures, much less work has been done to recover photometric properties needed for relighting. Results from such systems lack high-frequency details and the subject's shading is prebaked into the texture. In contrast, a large body of work has addressed relightable acquisition for image-based approaches, which photograph the subject under a set of basis lighting conditions and recombine the images to show the subject as they would appear in a target lighting environment. However, to date, these approaches have not been adapted for use in the context of a high-resolution volumetric capture system. Our method combines this ability to realistically relight humans for arbitrary environments, with the benefits of free-viewpoint volumetric capture and new levels of geometric accuracy for dynamic performances. Our subjects are recorded inside a custom geodesic sphere outfitted with 331 custom color LED lights, an array of high-resolution cameras, and a set of custom high-resolution depth sensors. Our system innovates in multiple areas: First, we designed a novel active depth sensor to capture 12.4 MP depth maps, which we describe in detail. Second, we show how to design a hybrid geometric and machine learning reconstruction pipeline to process the high resolution input and output a volumetric video. Third, we generate temporally consistent reflectance maps for dynamic performers by leveraging the information contained in two alternating color gradient illumination images acquired at 60Hz. Multiple experiments, comparisons, and applications show that The Relightables significantly improves upon the level of realism in placing volumetrically captured human performances into arbitrary CG scenes.
GP-GAN

It is common but challenging to address high-resolution image blending in the automatic photo editing application. In this paper, we would like to focus on solving the problem of high-resolution image blending, where the composite images are provided. We propose a framework called Gaussian-Poisson Generative Adversarial Network (GP-GAN) to leverage the strengths of the classical gradient-based approach and Generative Adversarial Networks. To the best of our knowledge, it's the first work that explores the capability of GANs in high-resolution image blending task. Concretely, we propose Gaussian-Poisson Equation to formulate the high-resolution image blending problem, which is a joint optimization constrained by the gradient and color information. Inspired by the prior works, we obtain gradient information via applying gradient filters. To generate the color information, we propose a Blending GAN to learn the mapping between the composite images and the well-blended ones. Compared to the alternative methods, our approach can deliver high-resolution, realistic images with fewer bleedings and unpleasant artifacts. Experiments confirm that our approach achieves the state-of-the-art performance on Transient Attributes dataset. A user study on Amazon Mechanical Turk finds that the majority of workers are in favor of the proposed method. The source code is available in \urlhttps://github.com/wuhuikai/GP-GAN, and there's also an online demo in \urlhttp://wuhuikai.me/DeepJS.
Integrating prediction and reconstruction for anomaly detection

Anomaly detection in videos refers to identifying events that rarely or shouldn’t happen in a certain context. Among all existing methods, the idea of reconstruction or future frame prediction is predominant for detecting anomalies. Reconstruction methods try to minimize the reconstruction errors of training data, but cannot guarantee large reconstruction errors for abnormal events. Future frame prediction methods follow the concept that normal events are predictable while abnormal ones are unpredictable. However, the results may drop rapidly since prediction is not robust to the noise in real-world surveillance videos. In this paper, we propose an approach that combines the advantages and balances the disadvantages of these two methods. An end-to-end network is designed to conduct future frame prediction and reconstruction sequentially. Future frame prediction makes the reconstruction errors large enough to facilitate the identification of abnormal events, while reconstruction helps enhance the predicted future frames from normal events. Specifically, we connect two U-Net blocks in the generator. One block works in the form of frame prediction, and the other tries to reconstruct the frames generated by the former block. Experiments over several benchmark datasets demonstrate the superiority of our method over previous state-of-the-art approaches, while running in real-time at 30 frames per second.
Graph neural news recommendation with long-term and short-term interest modeling

With the information explosion of news articles, personalized news recommendation has become important for users to quickly find news that they are interested in. Existing methods on news recommendation mainly include collaborative filtering methods which rely on direct user-item interactions and content based methods which characterize the content of user reading history. Although these methods have achieved good performances, they still suffer from data sparse problem, since most of them fail to extensively exploit high-order structure information (similar users tend to read similar news articles) in news recommendation systems. In this paper, we propose to build a heterogeneous graph to explicitly model the interactions among users, news and latent topics. The incorporated topic information would help indicate a user’s interest and alleviate the sparsity of user-item interactions. Then we take advantage of graph neural networks to learn user and news representations that encode high-order structure information by propagating embeddings over the graph. The learned user embeddings with complete historic user clicks capture the users’ long-term interests. We also consider a user’s short-term interest using the recent reading history with an attention based LSTM model. Experimental results on real-world datasets show that our proposed model significantly outperforms state-of-the-art methods on news recommendation.
Smart contract architecture for decentralized energy trading and management based on blockchains

A blockchain-based smart contract has the potential to allow the performance of credible transactions without third parties. This paper presents a universal framework for a blockchain platform that enables peer-to-peer (P2P) energy trading in the retail electricity market. Focusing attention on seeking energy-matching pairs from the supply and demand sides, and encouraging direct energy trading between producers and consumers, the P2P energy trading mechanism is proposed. The designed multidimensional blockchain platform implements a complete energy trading process. As smart contracts strictly execute the trading and payment rules without human interaction, the security and fairness of energy trading are significantly enhanced. Case studies in the Ethereum private chain demonstrate that the proposed mechanism has obvious advantages in reflecting market quotations, balancing profits of players, and facilitating the utilization of renewables. Based on such characteristics, players are incentivized to participate in the P2P energy trading. Moreover, the authentic gas consumption and computational time to the smart contract indicate that this platform is able to achieve an efficient and effective transaction with multi-player participation.
Breast cancer detection by leveraging Machine Learning

India has witnessed 30% of the cases of breast cancer during the last few years and it is likely to increase. Breast cancer in India accounts that one woman is diagnosed every two minutes and every nine minutes, one woman dies. Early detection and diagnosis can save the lives of cancer patients. This paper presents a novel method to detect breast cancer by employing techniques of Machine Learning. The authors carried out an experimental analysis on a dataset to evaluate the performance. The proposed method has produced highly accurate and efficient results when compared to the existing methods.
CKAN

Since it can effectively address the problem of sparsity and cold start of collaborative filtering, knowledge graph (KG) is widely studied and employed as side information in the field of recommender systems. However, most of existing KG-based recommendation methods mainly focus on how to effectively encode the knowledge associations in KG, without highlighting the crucial collaborative signals which are latent in user-item interactions. As such, the learned embeddings underutilize the two kinds of pivotal information and are insufficient to effectively represent the latent semantics of users and items in vector space.
Gate-Shift Networks for Video Action Recognition

Deep 3D CNNs for video action recognition are designed to learn powerful representations in the joint spatiotemporal feature space.In practice however, because of the large number of parameters and computations involved, they may under-perform in the lack of sufficiently large datasets for training them at scale.In this paper we introduce spatial gating in spatial-temporal decomposition of 3D kernels.We implement this concept with Gate-Shift Module (GSM).GSM is lightweight and turns a 2D-CNN into a highly efficient spatio-temporal feature extractor.With GSM plugged in, a 2D-CNN learns to adaptively route features through time and combine them, at almost no additional parameters and computational overhead.We perform an extensive evaluation of the proposed module to study its effectiveness in video action recognition, achieving state-of-the-art results on Something Something-V1 and Diving48 datasets, and obtaining competitive results on EPIC-Kitchens with far less model complexity.
Spatial-Temporal Graph Convolutional Network for Video-Based Person Re-Identification

While video-based person re-identification (Re-ID) has drawn increasing attention and made great progress in recent years, it is still very challenging to effectively overcome the occlusion problem and the visual ambiguity problem for visually similar negative samples. On the other hand, we observe that different frames of a video can provide complementary information for each other, and the structural information of pedestrians can provide extra discriminative cues for appearance features. Thus, modeling the temporal relations of different frames and the spatial relations within a frame has the potential for solving the above problems. In this work, we propose a novel Spatial-Temporal Graph Convolutional Network (STGCN) to solve these problems. The STGCN includes two GCN branches, a spatial one and a temporal one. The spatial branch extracts structural information of a human body. The temporal branch mines discriminative cues from adjacent frames. By jointly optimizing these branches, our model extracts robust spatial-temporal information that is complementary with appearance information. As shown in the experiments, our model achieves state-of-the-art results on MARS and DukeMTMC-VideoReID datasets.
Google Landmarks Dataset v2 – A Large-Scale Benchmark for Instance-Level Recognition and Retrieval

While image retrieval and instance recognition techniques are progressing rapidly, there is a need for challenging datasets to accurately measure their performance -- while posing novel challenges that are relevant for practical applications. We introduce the Google Landmarks Dataset v2 (GLDv2), a new benchmark for large-scale, fine-grained instance recognition and image retrieval in the domain of human-made and natural landmarks. GLDv2 is the largest such dataset to date by a large margin, including over 5M images and 200k distinct instance labels. Its test set consists of 118k images with ground truth annotations for both the retrieval and recognition tasks. The ground truth construction involved over 800 hours of human annotator work. Our new dataset has several challenging properties inspired by real-world applications that previous datasets did not consider: An extremely long-tailed class distribution, a large fraction of out-of-domain test photos and large intra-class variability. The dataset is sourced from Wikimedia Commons, the world's largest crowdsourced collection of landmark photos. We provide baseline results for both recognition and retrieval tasks based on state-of-the-art methods as well as competitive results from a public challenge. We further demonstrate the suitability of the dataset for transfer learning by showing that image embeddings trained on it achieve competitive retrieval performance on independent datasets. The dataset images, ground-truth and metric scoring code are available at https://github.com/cvdfoundation/google-landmark.
Semantic Drift Compensation for Class-Incremental Learning

Class-incremental learning of deep networks sequentially increases the number of classes to be classified. During training, the network has only access to data of one task at a time, where each task contains several classes. In this setting, networks suffer from catastrophic forgetting which refers to the drastic drop in performance on previous tasks. The vast majority of methods have studied this scenario for classification networks, where for each new task the classification layer of the network must be augmented with additional weights to make room for the newly added classes. Embedding networks have the advantage that new classes can be naturally included into the network without adding new weights. Therefore, we study incremental learning for embedding networks. In addition, we propose a new method to estimate the drift, called semantic drift, of features and compensate for it without the need of any exemplars. We approximate the drift of previous tasks based on the drift that is experienced by current task data. We perform experiments on fine-grained datasets, CIFAR100 and ImageNet-Subset. We demonstrate that embedding networks suffer significantly less from catastrophic forgetting. We outperform existing methods which do not require exemplars and obtain competitive results compared to methods which store exemplars. Furthermore, we show that our proposed SDC when combined with existing methods to prevent forgetting consistently improves results.
Supply network design to address United Nations Sustainable Development Goals: A case study of blockchain implementation in Thai fish industry

Sustainable Development Goals present an opportunity for industries to (re)design their supply chains. It is understood that digital technologies like blockchain can be helpful in achieving certain Sustainable Development Goals linked to livelihoods, food security, and the environment, by identifying issues and implementing interventions in real-time. However, there is limited understanding over data structure requirements for blockchain technology implementation in digitally-enabled food supply chains. Therefore, this research studies the design of blockchain-centric food supply chains that promote Sustainable Development Goals, within the context of the Thai fish industry. Key findings suggest that data asymmetry exists in supply chains to achieve Sustainable Development Goals. This research presents four design principles and an integrated technology implementation framework, derived from empirical data, for blockchain-centric food supply chains. The research outcome contributes to the supply chain management field and could ultimately impact the resilience of fishery ecosystems and the achievement of Sustainable Development Goals.
Scheduling for Cellular Federated Edge Learning With Importance and Channel Awareness

In cellular federated edge learning (FEEL), multiple edge devices holding local data jointly train a neural network by communicating learning updates with an access point without exchanging their data samples. With very limited communication resources, it is beneficial to schedule the most informative local learning updates. This paper focuses on FEEL with gradient averaging over participating devices in each round of communication. A novel scheduling policy is proposed to exploit both diversity in multiuser channels and diversity in the “importance” of the edge devices' learning updates. First, a new probabilistic scheduling framework is developed to yield unbiased update aggregation in FEEL. The importance of a local learning update is measured by its gradient divergence. If one edge device is scheduled in each communication round, the scheduling policy is derived in closed form to achieve the optimal trade-off between channel quality and update importance. The probabilistic scheduling framework is then extended to allow scheduling multiple edge devices in each communication round. Numerical results obtained using popular models and learning datasets demonstrate that the proposed scheduling policy can achieve faster model convergence and higher learning accuracy than conventional scheduling policies that only exploit a single type of diversity.
Construction quality information management with blockchains

The information generated from a nonconformance can be used to determine the party responsible for ensuring that quality standards are assured. However, in the construction industry, the absence of a uniform and transparent system for managing quality information undermines the assurance process and may lead to disputes among stakeholders. In addressing this issue, we develop a blockchain-based framework for managing quality information, referred to as “Product Organization Process (POP) qualityChain”. A Hyperledger-Fabric-based architecture and a series of blockchain solutions (e.g., a POP-model-based quality information structure, a consensus mechanism, smart contracts for processing quality information, authorization sequences, and execution processes) are covered. Finally, we build a prototype system and use a case study to validate our framework. Results show that the proposed framework can decentralize the management of quality information, thereby achieving consistent and secure quality information management. Future work could seek more evidence from practice to further validate our framework.
Blockchain-based public auditing for big data in cloud storage

Cloud storage enables applications to efficiently manage their remote data but facing the risk of being tampered with. This paper utilizes blockchain technique to develop a novel public auditing scheme for verifying data integrity in cloud storage. In the proposed scheme, different from the existing works that involve three participatory entities, only two predefined entities (i.e. data owner and cloud service provider) who may not trust each other are involved, and the third party auditor for data auditing is removed. Specifically, data owners store the lightweight verification tags on the blockchain and generate a proof by constructing the Merkle Hash Tree using the hashtags to reduce the overhead of computation and communication for integrity verification. Besides, this work is able to achieve 100% confidence of auditing theoretically, as the hashtag of each data block is utilized to build the Merkle Hash Tree for the data integrity verification. Security analysis shows that the proposed scheme can defend against malicious entities and the 51% attack. Experimental results demonstrate the significant improvements on computation and communication.
Retrieval-based neural source code summarization

Source code summarization aims to automatically generate concise summaries of source code in natural language texts, in order to help developers better understand and maintain source code. Traditional work generates a source code summary by utilizing information retrieval techniques, which select terms from original source code or adapt summaries of similar code snippets. Recent studies adopt Neural Machine Translation techniques and generate summaries from code snippets using encoder-decoder neural networks. The neural-based approaches prefer the high-frequency words in the corpus and have trouble with the low-frequency ones. In this paper, we propose a retrieval-based neural source code summarization approach where we enhance the neural model with the most similar code snippets retrieved from the training set. Our approach can take advantages of both neural and retrieval-based techniques. Specifically, we first train an attentional encoder-decoder model based on the code snippets and the summaries in the training set; Second, given one input code snippet for testing, we retrieve its two most similar code snippets in the training set from the aspects of syntax and semantics, respectively; Third, we encode the input and two retrieved code snippets, and predict the summary by fusing them during decoding. We conduct extensive experiments to evaluate our approach and the experimental results show that our proposed approach can improve the state-of-the-art methods.
A novel Black Widow Optimization algorithm for multilevel thresholding image segmentation

Segmentation is a crucial step in image processing applications. This process separates pixels of the image into multiple classes that permits the analysis of the objects contained in the scene. Multilevel thresholding is a method that easily performs this task, the problem is to find the best set of thresholds that properly segment each image. Techniques as Otsu’s between class variance or Kapur’s entropy helps to find the best thresholds but they are computationally expensive for more than two thresholds. To overcome such problem this paper introduces the use of the novel meta-heuristic algorithm called Black Widow Optimization (BWO) to find the best threshold configuration using Otsu or Kapur as objective function. To evaluate the performance and effectiveness of the BWO-based method, it has been considered the use of a variety of benchmark images, and compared against six well-known meta-heuristic algorithms including; the Gray Wolf Optimization (GWO), Moth Flame Optimization (MFO), Whale Optimization Algorithm (WOA), Sine–Cosine Algorithm (SCA), Slap Swarm Algorithm (SSA), and Equilibrium Optimization (EO). The experimental results have revealed that the proposed BWO-based method outperform the competitor algorithms in terms of the fitness values as well as the others performance measures such as PSNR, SSIM and FSIM. The statistical analysis manifests that the BWO-based method achieves efficient and reliable results in comparison with the other methods. Therefore, BWO-based method was found to be most promising for multi-level image segmentation problem over other segmentation approaches that are currently used in the literature.
Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video

We present Non-Rigid Neural Radiance Fields (NR-NeRF), a reconstruction and novel view synthesis approach for general non-rigid dynamic scenes. Our approach takes RGB images of a dynamic scene as input (e.g., from a monocular video recording), and creates a high-quality space-time geometry and appearance representation. We show that a single handheld consumer-grade camera is sufficient to synthesize sophisticated renderings of a dynamic scene from novel virtual camera views, e.g. a ‘bullet-time’ video effect. NR-NeRF disentangles the dynamic scene into a canonical volume and its deformation. Scene deformation is implemented as ray bending, where straight rays are deformed non-rigidly. We also propose a novel rigidity network to better constrain rigid regions of the scene, leading to more stable results. The ray bending and rigidity network are trained without explicit supervision. Our formulation enables dense correspondence estimation across views and time, and compelling video editing applications such as motion exaggeration. Our code will be open sourced.