Reviewerly/exHarmony huggingface dataset:
AIM 2020: Scene Relighting and Illumination Estimation Challenge

We review the AIM 2020 challenge on virtual image relighting and illumination estimation. This paper presents the novel VIDIT dataset used in the challenge and the different proposed solutions and final evaluation results over the 3 challenge tracks. The first track considered one-to-one relighting; the objective was to relight an input photo of a scene with a different color temperature and illuminant orientation (i.e., light source position). The goal of the second track was to estimate illumination settings, namely the color temperature and orientation, from a given image. Lastly, the third track dealt with any-to-any relighting, thus a generalization of the first track. The target color temperature and orientation, rather than being pre-determined, are instead given by a guide image. Participants were allowed to make use of their track 1 and 2 solutions for track 3. The tracks had 94, 52, and 56 registered participants, respectively, leading to 20 confirmed submissions in the final competition stage.
Explainable recommendation based on knowledge graph and multi-objective optimization

Abstract Recommendation system is a technology that can mine user's preference for items. Explainable recommendation is to produce recommendations for target users and give reasons at the same time to reveal reasons for recommendations. The explainability of recommendations that can improve the transparency of recommendations and the probability of users choosing the recommended items. The merits about explainability of recommendations are obvious, but it is not enough to focus solely on explainability of recommendations in field of explainable recommendations. Therefore, it is essential to construct an explainable recommendation framework to improve the explainability of recommended items while maintaining accuracy and diversity. An explainable recommendation framework based on knowledge graph and multi-objective optimization is proposed that can optimize the precision, diversity and explainability about recommendations at the same time. Knowledge graph connects users and items through different relationships to obtain an explainable candidate list for target user, and the path between target user and recommended item is used as an explanation basis. The explainable candidate list is optimized through multi-objective optimization algorithm to obtain the final recommendation list. It is concluded from the results about experiments that presented explainable recommendation framework provides high-quality recommendations that contains high accuracy, diversity and explainability.
Product backorder prediction using deep neural network on imbalanced data

Taking backorders on products is a common scenario in inventory and supply chain management systems. The ability to predict the likelihood of backorders can surely minimise a company's losses. Because the number of backorders is much lower than the number of orders that ship on time, applying a predictive model for this domain is a challenging task. This paper proposes a model that uses a deep neural network to predict backorders; it handles the data imbalance between backorders and filled orders with efficient techniques. To make the dataset balanced, we employ different techniques that include minority class weight boosting, randomised oversampling, SMOTE oversampling, and a combination of oversampling and undersampling. The balanced training data are used in our proposed, fully connected deep neural networks model to train the predictive model. The predictive model learns the likelihood of product backorders by using the training samples. We conduct experiments on a large benchmark dataset to test the performance of our proposed deep neural network–based model. The experimental results achieve a new state-of-the-art performance and outperform some prominent classification models in terms of standard evaluation metrics and expected profit measure.
A deep reinforcement learning-based method applied for solving multi-agent defense and attack problems

Learning to cooperate among agents has always been an important research topic in artificial intelligence. Multi-agent defense and attack, one of the important issues in multi-agent cooperation, requires multiple agents in the environment to learn effective strategies to achieve their goals. Deep reinforcement learning (DRL) algorithms have natural advantages dealing with continuous control problems especially under situations with dynamic interactions, and have provided new solutions for those long-studied multi-agent cooperation problems. In this paper, we start from deep deterministic policy gradient (DDPG) algorithm and then introduce multi-agent DDPG (MADDPG) to solve the multi-agent defense and attack problem under different situations. We reconstruct the considered environment, redefine the continuous state space, continuous action space, reward functions accordingly, and then apply deep reinforcement learning algorithms to obtain effective decision strategies. Several experiments considering different confrontation scenarios are conducted to validate the feasibility and effectiveness of the DRL-based methods. Experimental results show that through learning the agents can make better decisions, and learning with MADDPG achieves superior performance than learning with other DRL-based models, which also explains the importance and necessity of mastering other agents’ information.
Self-Supervised Pretraining Improves Self-Supervised Pretraining

While self-supervised pretraining has proven beneficial for many computer vision tasks, it requires expensive and lengthy computation, large amounts of data, and is sensitive to data augmentation. Prior work demonstrates that models pretrained on datasets dissimilar to their target data, such as chest X-ray models trained on ImageNet, underperform models trained from scratch. Users that lack the resources to pretrain must use existing models with lower performance. This paper explores Hierarchical PreTraining (HPT), which decreases convergence time and improves accuracy by initializing the pretraining process with an existing pretrained model. Through experimentation on 16 diverse vision datasets, we show HPT converges up to 80× faster, improves accuracy across tasks, and improves the robustness of the self-supervised pretraining process to changes in the image augmentation policy or amount of pretraining data. Taken together, HPT provides a simple framework for obtaining better pretrained representations with less computational resources.
Human factor, a critical weak point in the information security of an organization's Internet of things

Internet of Things (IoT) presents opportunities for designing new technologies for organizations. Many organizations are beginning to accept these technologies for their daily work, where employees can be connected, both on the organization's premises and the “outside”, for business continuity. However, organizations continue to experience data breach incidents. Even though there is a plethora of researches in Information Security, there “seems” to be little or lack of interest from the research community, when it comes to human factors and its relationship to data breach incidents. The focus is usually on the technological component of Information Technology systems. Regardless of any technological solutions introduced, human factors continue to be an area that lacks the required attention. Making the assumption that people will follow expected secure behavioral patterns and therefore system security expectations will be satisfied, may not necessarily be true. Security is not something that can simply be purchased; human factors will always prove to be an important space to explore. Hence, human factors are without a doubt a critical point in Information Security. In this study, we propose an Organizational Information Security Framework For Human Factors applicable to the Internet of Things, which includes countermeasures that can help prevent or reduce data breach incidents as a result of human factors. Using linear regression on data breach incidents reported in the United States of America from 2009 to 2017, the study validates human factors as a weak-point in information security that can be extended to Internet of Things by predicting the relationship between human factors and data breach incidents, and the strength of these relationships. Our results show that five breach incidents out of the seven typified human factors to statistically and significantly predict data breach incidents. Furthermore, the results also show a positive correlation between human factors and these data breach incidents.
Self-Distillation: Towards Efficient and Compact Neural Networks

Remarkable achievements have been obtained by deep neural networks in the last several years. However, the breakthrough in neural networks accuracy is always accompanied by explosive growth of computation and parameters, which leads to a severe limitation of model deployment. In this paper, we propose a novel knowledge distillation technique named self-distillation to address this problem. Self-distillation attaches several attention modules and shallow classifiers at different depths of neural networks and distills knowledge from the deepest classifier to the shallower classifiers. Different from the conventional knowledge distillation methods where the knowledge of the teacher model is transferred to another student model, self-distillation can be considered as knowledge transfer in the same model - from the deeper layers to the shallow layers. Moreover, the additional classifiers in self-distillation allow the neural network to work in a dynamic manner, which leads to a much higher acceleration. Experiments demonstrate that self-distillation has consistent and significant effectiveness on various neural networks and datasets. On average, 3.49 and 2.32 percent accuracy boost are observed on CIFAR100 and ImageNet. Besides, experiments show that self-distillation can be combined with other model compression methods, including knowledge distillation, pruning and lightweight model design.
A dual neural network for object detection in UAV images

Computer vision analysis in Unmanned Aerial Vehicle (UAV) represents a major trend in the future development. Object detection in UAV images is one of the important techonlogies to analyze scene information. However, the UAV image includes small and dense targets, which easily leads to errors of missed detection. In this paper, we propose a dual neural network review method, which quickly screens the missed targets in the one-stage detection by classifying the secondary features of the suspected target regions, so as to achieve high quality detection of small targets. Firstly, the one-stage detector recognizes UAV images, and the result with confidence greater than or equal to the threshold is detected as the target. The result less than the threshold are considered as suspected areas containing missed targets. Secondly, the feature map of UAV image is extracted by VGG backbone. The feature map and the location information of the suspected areas are combined to secondary identification. Then, the features of the dual network are late fusion, and the re-identified results guide the initial confidence addition. After the addition, regions with confidence greater than the threshold are considered as targets. Finally, we synthesize the targets of initial detection and secondary identification to obtain the final detection results. Experimental results show that our method achieves breakthrough performance on VisDrone, UAVDT and MS COCO datasets.
Copy-Move Forgery Detection (CMFD) Using Deep Learning for Image and Video Forensics

With the exponential growth of high-quality fake images in social networks and media, it is necessary to develop recognition algorithms for this type of content. One of the most common types of image and video editing consists of duplicating areas of the image, known as the copy-move technique. Traditional image processing approaches manually look for patterns related to the duplicated content, limiting their use in mass data classification. In contrast, approaches based on deep learning have shown better performance and promising results, but they present generalization problems with a high dependence on training data and the need for appropriate selection of hyperparameters. To overcome this, we propose two approaches that use deep learning, a model by a custom architecture and a model by transfer learning. In each case, the impact of the depth of the network is analyzed in terms of precision (P), recall (R) and F1 score. Additionally, the problem of generalization is addressed with images from eight different open access datasets. Finally, the models are compared in terms of evaluation metrics, and training and inference times. The model by transfer learning of VGG-16 achieves metrics about 10% higher than the model by a custom architecture, however, it requires approximately twice as much inference time as the latter.
Anatomy-guided multimodal registration by learning segmentation without ground truth: Application to intraprocedural CBCT/MR liver segmentation and registration

Multimodal image registration has many applications in diagnostic medical imaging and image-guided interventions, such as Transcatheter Arterial Chemoembolization (TACE) of liver cancer guided by intraprocedural CBCT and pre-operative MR. The ability to register peri-procedurally acquired diagnostic images into the intraprocedural environment can potentially improve the intra-procedural tumor targeting, which will significantly improve therapeutic outcomes. However, the intra-procedural CBCT often suffers from suboptimal image quality due to lack of signal calibration for Hounsfield unit, limited FOV, and motion/metal artifacts. These non-ideal conditions make standard intensity-based multimodal registration methods infeasible to generate correct transformation across modalities. While registration based on anatomic structures, such as segmentation or landmarks, provides an efficient alternative, such anatomic structure information is not always available. One can train a deep learning-based anatomy extractor, but it requires large-scale manual annotations on specific modalities, which are often extremely time-consuming to obtain and require expert radiological readers. To tackle these issues, we leverage annotated datasets already existing in a source modality and propose an anatomy-preserving domain adaptation to segmentation network (APA2Seg-Net) for learning segmentation without target modality ground truth. The segmenters are then integrated into our anatomy-guided multimodal registration based on the robust point matching machine. Our experimental results on in-house TACE patient data demonstrated that our APA2Seg-Net can generate robust CBCT and MR liver segmentation, and the anatomy-guided registration framework with these segmenters can provide high-quality multimodal registrations.
Video Instance Segmentation with a Propose-Reduce Paradigm

Video instance segmentation (VIS) aims to segment and associate all instances of predefined classes for each frame in videos. Prior methods usually obtain segmentation for a frame or clip first, and merge the incomplete results by tracking or matching. These methods may cause error accumulation in the merging step. Contrarily, we propose a new paradigm – Propose-Reduce, to generate complete sequences for input videos by a single step. We further build a sequence propagation head on the existing image-level instance segmentation network for long-term propagation. To ensure robustness and high recall of our proposed framework, multiple sequences are proposed where redundant sequences of the same instance are reduced. We achieve state-of-the-art performance on two representative benchmark datasets – we obtain 47.6% in terms of AP on YouTube-VIS validation set and 70.4 % for J&F on DAVIS-UVOS validation set.
Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data

Photorealistic image generation has reached a new level of quality due to the breakthroughs of generative adversarial networks (GANs). Yet, the dark side of such deepfakes, the malicious use of generated media, raises concerns about visual misinformation. While existing research work on deepfake detection demonstrates high accuracy, it is subject to advances in generation techniques and adversarial iterations on detection countermeasure techniques. Thus, we seek a proactive and sustainable solution on deepfake detection, that is agnostic to the evolution of generative models, by introducing artificial fingerprints into the models.Our approach is simple and effective. We first embed artificial fingerprints into training data, then validate a surprising discovery on the transferability of such fingerprints from training data to generative models, which in turn appears in the generated deepfakes. Experiments show that our fingerprinting solution (1) holds for a variety of cutting-edge generative models, (2) leads to a negligible side effect on generation quality, (3) stays robust against image-level and model-level perturbations, (4) stays hard to be detected by adversaries, and (5) converts deepfake detection and attribution into trivial tasks and outperforms the recent state-of-the-art baselines. Our solution closes the responsibility loop between publishing pre-trained generative model inventions and their possible misuses, which makes it independent of the current arms race.
Human action recognition approaches with video datasets—A survey

Human Activity Recognition has recently attracted considerable attention. This has been triggered by the rapid development of advance technologies and learning methods. Human action recognition can be actively used in a number of application domains which may positively influence various aspects of the daily life. These include, (1) preventing dangerous activities and detection of crimes such as theft, murder, and property damage, and (2) predicting pedestrian activities in traffic, among others. To better serve these applications and the like, it is essential to highlight the various aspects related to the existing methods so that their actual users could realize and identify the good performing methods that work fast and are capable of recognizing the correct activities with high accuracy. The latter scope is covered in this survey which summarizes and analyzes the methods that perform learning and analysis processes on video datasets to grasp a new perspective on human action recognition. The survey also covers the major datasets commonly used in human activity recognition research. Accordingly, this survey could be recognized as a valuable source for researchers and practitioners.
A deep convolutional neural network to simultaneously localize and recognize waste types in images

Accurate waste classification is key to successful waste management. However, most current studies have focused exclusively on single-label waste classification from images, which goes against common sense. In this paper, we move beyond single-label waste classification and propose a benchmark for evaluating the multi-label waste classification and localization tasks to advance waste management via deep learning-based methods. We propose a multi-task learning architecture (MTLA) based on a convolutional neural network, which can be used to simultaneously identify and locate wastes in images. The MTLA comprises a backbone network with proposed attention modules, a novel multi-level feature pyramid network, and a group of joint learning multi-task subnets. To achieve joint optimization of waste identification and location, we designed the loss functions according to the concepts of focusing and joint. The proposed MTLA achieved performance similar to that of experts and had high scores for multiple tasks related to waste management. Its F1 score exceeded 95.50% (95.12% to 95.88%, with a 95% confidence interval) on the multi-label waste classification task, and the average precision score was over 81.50% (@IoU = 0.5) on the waste localization task. To improve interpretation, heatmaps were used to visualize the salient features extracted by the MTLA. The proposed MTLA is a promising auxiliary tool that can improve the automation of waste management systems.
A secure and efficient authentication and data sharing scheme for Internet of Things based on blockchain

Internet of Things (IoT) is a network convergence of multiple intelligent devices and advanced technologies aiming at connecting and exchanging data over the Internet. IoT is extensively applied in consumer, commercial, industrial, infrastructure and military spaces. With the prevalence of IoT applications, security issues such as identity authenticity and data privacy are increasingly become critical concerns. Authentication and confidential data sharing are the important measures towards secure IoT communication and applications. Blockchain is a burgeoning technology supporting for efficient authentication and secure data sharing. A secure and accountable data transmission scheme based on blockchain has been proposed by Hong et al. recently. But this scheme has security weaknesses of impersonation attack, man-in-the-middle attack, replay attack, denial of service attack (DoS) and key compromise attack. Thus we put forward an improved scheme to overcome the identified security flaws. Our scheme is provably secure and performance analysis shows that our scheme reduces 15.34% computation costs and 40.68% communication costs compared with Hong et al.’s scheme. Meanwhile, we also compare our scheme with other three recent and related researches, which finally indicates that our scheme realizes a well tradeoff between security and efficiency.
Comparison and Analysis of RNN-LSTMs and CNNs for Social Reviews Classification

This chapter presents and compares results of simple and efficient deep learning models to perform sentiment analysis and text classification. Natural language processing is a massive domain that enables in finding solutions for many day-to-day tasks, and sentiment analysis falls under this domain. A typical sentiment analysis task can be described as a process of classifying opinions expressed in a text as positive, negative, or neutral. This chapter employs two models; one model is built using recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) and the other with convolutional neural networks. In our first model, RNN-LSTMs model has been used to capture the semantic and syntactic relationships between words of a sentence with help of word2vec. In the second model, one-dimensional convolutional neural networks were used to learn structure in paragraphs of words and the techniques invariance to the specific position of features. The IMDB movie reviews dataset set is being used for sentiment analysis in both the models and the results are compared. Both the models yielded excellent results.