Reviewerly/exHarmony huggingface dataset:
R-MNet: A Perceptual Adversarial Network for Image Inpainting

Facial image inpainting is a problem that is widely studied, and in recent years the introduction of Generative Adversarial Networks, has led to improvements in the field. Unfortunately some issues persists, in particular when blending the missing pixels with the visible ones. We address the problem by proposing a Wasserstein GAN combined with a new reverse mask operator, namely Reverse Masking Network (R-MNet), a perceptual adversarial network for image inpainting. The reverse mask operator transfers the reverse masked image to the end of the encoder-decoder network leaving only valid pixels to be inpainted. Additionally, we propose a new loss function computed in feature space to target only valid pixels combined with adversarial training. These then capture data distributions and generate images similar to those in the training data with achieved realism (realistic and coherent) on the output images. We evaluate our method on publicly available dataset, and compare with state-of-the-art methods. We show that our method is able to generalize to high-resolution inpainting task, and further show more realistic outputs that are plausible to the human visual system when compared with the state-of-the-art methods. https://github.com/Jireh-Jam/R-MNet-Inpainting-keras
Zero Watermarking: Critical Analysis of Its Role in Current Medical Imaging

Abstract Medical image data plays a critical role in health care today. Whatever the context of image processing (research and educational activities, diagnostic or forensic purposes, etc.), they are supposed to be treated as highly sensitive. Unfortunately, currently available image-processing tools also enable very sophisticated malicious modification of their content. This paper is focused on the assessment of the effectiveness of selected so-called zero watermarking methods (ones that do not cause any modification of an image sample), in the protection of the integrity, and proving authorship, of these medical image studies. We have studied many zero watermarking methods and selected one representative from each type of known algorithms for comparison. We have conducted a series of simulations over a huge research database of anonymized medical image studies (patient examinations).
Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings

We describe an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text. We use multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. We validate our technique by extracting parallel sentence pairs on the BUCC 2017 bitext mining task and observe up to a 24.5 point increase (absolute) in F 1 scores over previous unsupervised methods. We then improve an XLM-based unsupervised neural MT system pre-trained on Wikipedia by supplementing it with pseudo-parallel text mined from the same corpus, boosting unsupervised translation performance by up to 3.5 BLEU on the WMT’14 French-English and WMT’16 German-English tasks and outperforming the previous state-of-the-art. Finally, we enrich the IWSLT’15 English-Vietnamese corpus with pseudo-parallel Wikipedia sentence pairs, yielding a 1.2 BLEU improvement on the low-resource MT task. We demonstrate that unsupervised bitext mining is an effective way of augmenting MT datasets and complements existing techniques like initializing with pre-trained contextual embeddings.
Real-time Localized Photorealistic Video Style Transfer

We present a novel algorithm for transferring artistic styles of semantically meaningful local regions of an image onto local regions of a target video while preserving its photorealism. Local regions may be selected either fully automatically from an image, through using video segmentation algorithms, or from casual user guidance such as scribbles. Our method, based on a deep neural network architecture inspired by recent work in photorealistic style transfer, is real-time and works on arbitrary inputs without runtime optimization once trained on a diverse dataset of artistic styles. By augmenting our video dataset with noisy semantic labels and jointly optimizing over style, content, mask, and temporal losses, our method can cope with a variety of imperfections in the input and produce temporally coherent videos without visual artifacts. We demonstrate our method on a variety of style images and target videos, including the ability to transfer different styles onto multiple objects simultaneously, and smoothly transition between styles in time.
Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learning

Current research directions in deep reinforcement learning include bridging the simulation-reality gap, improving sample efficiency of experiences in distributed multi-agent reinforcement learning, together with the development of robust methods against adversarial agents in distributed learning, among many others. In this work, we are particularly interested in analyzing how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems where the operation of the different robots is not necessarily homogeneous. These variations can happen due to sensing mismatches, inherent errors in terms of calibration of the mechanical joints, or simple differences in accuracy. While our results are simulation-based, we introduce the effect of sensing, calibration, and accuracy mismatches in distributed reinforcement learning with proximal policy optimization (PPO). We discuss on how both the different types of perturbances and how the number of agents experiencing those perturbances affect the collaborative learning effort. The simulations are carried out using a Kuka arm model in the Bullet physics engine. This is, to the best of our knowledge, the first work exploring the limitations of PPO in multi-robot systems when considering that different robots might be exposed to different environments where their sensors or actuators have induced errors. With the conclusions of this work, we set the initial point for future work on designing and developing methods to achieve robust reinforcement learning on the presence of real-world perturbances that might differ within a multi-robot system.
Quality Criteria of Fuzzy IF-THEN Rules and Their Calculations

Fuzzy IF-THEN rules are usually taken as specific characterization of dependencies among objects. The conception of IF-Then rules contributes all researchers and practitioners to equip conditional statements. It can directly relate with fuzzy inference methods and provides to create a comprehensive logical theory. This paper encompasses the quality criteria of fuzzy If-Then rules based on their calculations. Fuzzy If-Then rules supply the solution of decision-making problem. Numerical examples for the quality of fuzzy If-Then rules are described for their 5 criteria.
An access control model for the Internet of Things based on zero-knowledge token and blockchain

Abstract Information security has become a hot topic in Internet of Things (IoT), and traditional centralized access control models are faced with threats such as single point failure, internal attack, and central leak. In this paper, we propose a model to improve the access control security of the IoT, which is based on zero-knowledge proof and smart contract technology in the blockchain. Firstly, we deploy attribute information of access control in the blockchain, which relieves the pressure and credibility problem brought by the third-party information concentration. Secondly, encrypted access control token is used to gain the access permission of the resources, which makes the user's identity invisible and effectively avoids attribute ownership exposure problem. Besides, the use of smart contracts solves the problem of low computing efficiency of IoT devices and the waste of blockchain computing power resources. Finally, a prototype of IoT access control system based on blockchain and zero-knowledge proof technology is implemented. The test analysis results show that the model achieves effective attribute privacy protection, compared with the Attribute-Based Access Control model of the same security level, the access efficiency increases linearly with the increase of access scale.
Leveraging Task Modularity in Reinforcement Learning for Adaptable Industry 4.0 Automation

Abstract The vision of Industry 4.0 is to materialize the notion of a lot-size of one through enhanced adaptability and resilience of manufacturing and logistics operations to dynamic changes or deviations on the shop floor. This article is motivated by the lack of formal methods for efficient transfer of knowledge across different yet interrelated tasks, with special reference to collaborative robotic operations such as material handling, machine tending, assembly, and inspection. We propose a meta reinforcement learning framework to enhance the adaptability of collaborative robots to new tasks through task modularization and efficient transfer of policies from previously learned task modules. Our experiments on the OpenAI Gym Robotics environments Reach, Push, and Pick-and-Place indicate an average 75% reduction in the number of iterations to achieve a 60% success rate as well as a 50%-80% improvement in task completion efficiency, compared to the deep deterministic policy gradient (DDPG) algorithm as a baseline. The significant improvements achieved in the jumpstart and asymptotic performance of the robot create new opportunities for investigating the current limitations of learning robots in industrial settings, associated with sample inefficiency and specialization on one task through modularization and transfer learning.
Joint Visual and Semantic Optimization for zero-shot learning

Zero-shot learning (ZSL) aims to classify instances whose classes could be unseen during training. Most existing ZSL methods project visual or semantic features into the space of the other one, or into a common subspace. The main goal of projection is to find out the similar features in the latent subspace. However, existing methods barely consider common features that preserve knowledge, here we refer to these features as the shared concepts, which are essential to model the relationship between the visual and semantic spaces. In this paper, we exploit the underlying concepts shared by both visual and semantic features in a latent common subspace and propose to match their latent visual and semantic representations. To reduce domain shift and information loss, we introduce reconstruction losses for both visual and semantic features. As a result, the reconstruction regularizations are added to the similar features and thereby obtain knowledge preserving shared concepts via the proposed method. Mathematically, it is formulated as the minimization problem for mutual orthogonal projection to their latent common subspace. The problem involves two projection variables, thus we develop an algorithm based on the Gauss–Seidel iteration scheme and split the problem into two subproblems in the scheme. These two subproblems are further solved by searching algorithms based on the Barzilai–Borwein stepsize. Extensive experiments on six benchmark data sets are conducted to demonstrate that the accuracy of the proposed method is better than that of existing ZSL methods.
Let’s Get Dirty: GAN Based Data Augmentation for Camera Lens Soiling Detection in Autonomous Driving

Wide-angle fisheye cameras are commonly used in automated driving for parking and low-speed navigation tasks. Four of such cameras form a surround-view system that provides a complete and detailed view of the vehicle. These cameras are directly exposed to harsh environmental settings and can get soiled very easily by mud, dust, water, frost. Soiling on the camera lens can severely degrade the visual perception algorithms, and a camera cleaning system triggered by a soiling detection algorithm is increasingly being deployed. While adverse weather conditions, such as rain, are getting attention recently, there is only limited work on general soiling. The main reason is the difficulty in collecting a diverse dataset as it is a relatively rare event.We propose a novel GAN based algorithm for generating unseen patterns of soiled images. Additionally, the proposed method automatically provides the corresponding soiling masks eliminating the manual annotation cost. Augmentation of the generated soiled images for training improves the accuracy of soiling detection tasks significantly by 18% demonstrating its usefulness. The manually annotated soiling dataset and the generated augmentation dataset will be made public. We demonstrate the generalization of our fisheye trained GAN model on the Cityscapes dataset. We provide an empirical evaluation of the degradation of the semantic segmentation algorithm with the soiled data.
Cross Parallax Attention Network for Stereo Image Super-Resolution

Stereo super-resolution (SR) aims to enhance the spatial resolution of one camera view using additional information from the other. Previous deep-learning-based stereo SR methods indeed improved the SR performance effectively by employing additional information, but they are unable to super-resolve stereo images where there are large disparities, or different types of epipolar lines. Moreover, in these methods, one model can only super-solve images of a particular view, and for one specific scale factor. This paper proposes a cross parallax attention stereo super-resolution network (CPASSRnet) which can perform stereo SR of multiple scale factors for both views, with a single model. To overcome the difficulties of large disparity and different types of epipolar lines, a cross parallax attention module (CPAM) is presented, which captures the global correspondence of additional information for each view, relative to the other. CPAM allows the two views to exchange additional information with each other according to the generated attention maps. Quantitative and qualitative results compared with the state of the arts illustrate the superiority of CPASSRnet. Ablation experiments demonstrate that the proposed components are effective and noise tests verify the robustness of CPASSRnet.
A novel 3D laser scanning defect detection and measurement approach for automated fibre placement

Abstract Automated fibre placement (AFP) machine in-process layup inspection is essential for composite structure fabrication efficiency and quantity improvement. The laser profilometer scanning technique is one of the automated in-process inspection techniques. Laser profilometers can capture a large number of intensive high-resolution point clouds during the layup. To date, few studies have been published about processing point clouds at high speed for in-process layup defect detection and layup feature measurement. In this study, an algorithm called the cross-sectional line-processing algorithm was proposed, and a testbed was constructed to validate the algorithm. The algorithm processes each laser line captured by the laser profilometer and clusters the processed results together. Finally, the defect features and layup features can be segmented and recognized from the collected point cloud. Layup defect types can be recognized, and the dimensions of the layup features can be measured. The experimental results show that this approach can process at most 200 laser lines (about 160 000 points) per second, and the overall defect type recognition accuracy rate reaches about 78%, which meets basic AFP in-process inspection requirements.
Explainable artificial intelligence (XAI) interactively working with humans as a junior cyber analyst

Abstract There are many applications where artificial intelligence (AI) can add a benefit, but this benefit may not be fully realized, if the human cannot understand and interact with the output as required by their context. Allowing AI to explain its decisions can potentially mitigate this issue. To develop effective explainable AI methods to support this need, we need to understand both what the human needs for decision-making, as well as what information the AI has and can make available. This paper presents an example case of capturing those requirements. We explore how an operational planner (senior human analyst) for a cyber protection team could use a junior analyst virtual agent to scour, analyze, and present the data available on vulnerabilities and incidents on both the target systems as well as similar systems. We explore the interactions required to understand these outputs and to integrate additional knowledge held by the human. This is an exemplar case for integrating XAI into the real-world bi-directional workflow: the senior analyst needs to be able to understand the junior analysts results, particularly the assumptions and implications, in order to create a plan and brief it up the command chain. He or she may have further questions, or analysis needs to achieve this understanding. The application is the junior analyst agent and senior human analysts working together to create this understanding of threats, vulnerabilities, incidents, likely future attacks, and counteractions on the mission relevant cyber terrain that their unit has been assigned a mission on.
Blockchain for 5G and IoT: Opportunities and Challenges

Hitherto, the evolution of mobile networks have fulfilled the increasing demands for enhanced performance, availability, portability, elasticity, and energy efficiency posed by the ever growing network services. In line with the progression, 5G depicts the next generation of mobile networks that further promises remarkable performance improvements as well as creation of new value chain. In parallel to 5G, the Internet of Things (IoT) has emerged as another new paradigm for interconnection of massive communication-capable heterogeneous smart objects. 5G is envisaged to broaden IoT's scope and fields of applicability. However, since current mobile networks and also more general IoT systems are based on centralized models thus it is anticipated that they will face tremendous challenges to meet-up the requirements of future 5G-enabled-IoT use cases. To solve these inevitable issues Blockchain stands out as promising technology. Some of the offerings of Blockchain technology are immutability, non-repudiation, proof of provenance, integrity, privacy, etc. Blockchain's combination with 5G and IoT still requires essential insights with respect to concrete application domains, scalability, privacy issues, performance, and potential financial benefits. The paper aims to elaborate and emphasize the key aspects of the use of Blockchain for 5G and IoT.
Vehicle Detection and Disparity Estimation Using Blended Stereo Images

In this article, a new method is presented for simultaneous vehicle detection and disparity estimation from the blended multiple-view image. The multiple-view image is created by blending left and right images from a stereo camera. This blended image can implicitly encode the appearance and view disparity of an object in a single image, which provides clues for detection and disparity estimation. Since a single object may have multiple overlaid correspondence in a blended image, the detection task is formulated as finding a group of associated bounding boxes with similar appearances. The distances between the associated bounding boxes can be used to estimate the disparity of the object. The proposed method has following benefits: 1) The detection and disparity estimation tasks can be accomplished using a single input image; 2) Even the task is more complex than the popular detection task, the increase in computation cost is negligible; 3) Very competitive detection and disparity estimation accuracy is obtained, even the detection is performed on the blended image which is unfriendly to human vision. The experiment is conducted using KITTI stereo subset and very promising result has been observed.
Ellipse Detection and Localization with Applications to Knots in Sawn Lumber Images

While general object detection has seen tremendous progress, localization of elliptical objects has received little attention in the literature. Our motivating application is the detection of knots in sawn lumber images, which is an important problem since the number and types of knots are visual characteristics that adversely affect the quality of sawn lumber. We demonstrate how models can be tailored to the elliptical shape and thereby improve on general purpose detectors; more generally, elliptical defects are common in industrial production, such as enclosed air bubbles when casting glass or plastic. In this paper, we adapt the Faster R-CNN with its Region Proposal Network (RPN) to model elliptical objects with a Gaussian function, and extend the existing Gaussian Proposal Network (GPN) architecture by adding the region-of-interest pooling and regression branches, as well as using the Wasserstein distance as the loss function to predict the precise locations of elliptical objects. Our proposed method has promising results on the lumber knot dataset: knots are detected with an average intersection over union of 73.05%, compared to 63.63% for general purpose detectors. Specific to the lumber application, we also propose an algorithm to correct any misalignment in the raw lumber images during scanning, and contribute the first open-source lumber knot dataset by labeling the elliptical knots in the preprocessed images.
Exploiting 3D Hand Pose Estimation in Deep Learning-Based Sign Language Recognition from RGB Videos

In this paper, we investigate the benefit of 3D hand skeletal information to the task of sign language (SL) recognition from RGB videos, within a state-of-the-art, multiple-stream, deep-learning recognition system. As most SL datasets are available in traditional RGB-only video lacking depth information, we propose to infer 3D coordinates of the hand joints from RGB data via a powerful architecture that has been primarily introduced in the literature for the task of 3D human pose estimation. We then fuse these estimates with additional SL informative streams, namely 2D skeletal data, as well as convolutional neural network-based hand- and mouth-region representations, and employ an attention-based encoder-decoder for recognition. We evaluate our proposed approach on a corpus of isolated signs of Greek SL and a dataset of continuous finger-spelling in American SL, reporting significant gains by the inclusion of 3D hand pose information, while also outperforming the state-of-the-art on both databases. Further, we evaluate the 3D hand pose estimation technique as standalone.
A multiverse optimization based colour image segmentation using variational mode decomposition

Multilevel thresholding using the histogram is the most popular and accepted technique of image segmentation. The computational time of multilevel thresholding rises exponentially as the number of the thresholds increases. Histogram suffers from irregularities and sharp details which leads to stagnation. In this article, a newly developed multiverse optimization is combined with variational mode decomposition to overcome this problem. The histogram of an input image divided into several band-limit modes using VMD then reconstruct histogram using meaningful modes. The reconstructed histogram is free from the high-frequency fluctuation which causes local optima. The proposed method utilized two entropy function to develop image segmentation by determining the optimal threshold. The result of the proposed algorithm is analyzed with other evolutionary algorithms such as artificial bee colony, sine cosine algorithm, and salp swarm algorithm. Comparison is made based on comparative parameters such as peak signal to noise ratio, structural similarity index, feature similarity index, uniformity, normalized absolute error, quality index based on local variance, computational time, and mean square error. The test results validate that the proposed algorithm presents more reliable results than other existing techniques.