Reviewerly/exHarmony huggingface dataset:
Bridging Composite and Real: Towards End-to-End Deep Image Matting

Extracting accurate foregrounds from natural images benefits many downstream applications such as film production and augmented reality. However, the furry characteristics and various appearance of the foregrounds, e.g., animal and portrait, challenge existing matting methods, which usually require extra user inputs such as trimap or scribbles. To resolve these problems, we study the distinct roles of semantics and details for image matting and decompose the task into two parallel sub-tasks: high-level semantic segmentation and low-level details matting. Specifically, we propose a novel Glance and Focus Matting network (GFM), which employs a shared encoder and two separate decoders to learn both tasks in a collaborative manner for end-to-end natural image matting. Besides, due to the limitation of available natural images in the matting task, previous methods typically adopt composite images for training and evaluation, which result in limited generalization ability on real-world images. In this paper, we investigate the domain gap issue between composite images and real-world images systematically by conducting comprehensive analyses of various discrepancies between the foreground and background images. We find that a carefully designed composition route RSSN that aims to reduce the discrepancies can lead to a better model with remarkable generalization ability. Furthermore, we provide a benchmark containing 2,000 high-resolution real-world animal images and 10,000 portrait images along with their manually labeled alpha mattes to serve as a test bed for evaluating matting model’s generalization ability on real-world images. Comprehensive empirical studies have demonstrated that GFM outperforms state-of-the-art methods and effectively reduces the generalization error. The code and the datasets will be released at https://github.com/JizhiziLi/GFM .
Optical Rashba Effect in a Light‐Emitting Perovskite Metasurface

The Rashba effect, i.e., the splitting of electronic spin-polarized bands in the momentum space of a crystal with broken inversion symmetry, has enabled the realization of spin-orbitronic devices, in which spins are manipulated by spin-orbit coupling. In optics, where the helicity of light polarization represents the spin degree of freedom for spin-momentum coupling, the optical Rashba effect is manifested by the splitting of optical states with opposite chirality in the momentum space. Previous realizations of the optical Rashba effect relied on passive devices determining the surface plasmon or light propagation inside nanostructures, or the directional emission of chiral luminescence when hybridized with light-emitting media. An active device underpinned by the optical Rashba effect is demonstrated here, in which a monolithic halide perovskite metasurface emits highly directional chiral photoluminescence. An all-dielectric metasurface design with broken in-plane inversion symmetry is directly embossed into the high-refractive-index, light-emitting perovskite film, yielding a degree of circular polarization of photoluminescence of 60% at room temperature.
An integrated framework for blockchain-enabled supply chain trust management towards smart manufacturing

With the development of a new generation of information technology, smart manufacturing has put forward higher requirements for supply chain. It is necessary to ensure the synchronization of the supply chain operation and maintain the reliability of the supply chain management, therefore the trust evaluation for the supply chain becomes extremely important. Traditional supply chain management has problems such as information flow is easy to be tampered with, logistics is difficult to trace, and capital flow is not true, which leads to increased opportunity costs due to the lack of trust among transaction entities in the supply chain. The emergence of blockchain technology provides an opportunity to improve the supply chain ecosystem. In this paper, an integrated framework for blockchain-enabled supply chain trust management towards smart manufacturing is proposed to explain how to enhance trust management with the help of blockchain from the perspectives of information flow, logistics, and capital flow. An optimized trust management model is designed for better entities evaluation in supply chain. A coal mine equipment manufacturing industry scenario is presented to illustrate the effectiveness of the proposed framework.
VUDENC: Vulnerability Detection with Deep Learning on a Natural Codebase for Python

Identifying potential vulnerable code is important to improve the security of our software systems. However, the manual detection of software vulnerabilities requires expert knowledge and is time-consuming, and must be supported by automated techniques. Such automated vulnerability detection techniques should achieve a high accuracy, point developers directly to the vulnerable code fragments, scale to real-world software, generalize across the boundaries of a specific software project, and require no or only moderate setup or configuration effort. In this article, we present Vudenc (Vulnerability Detection with Deep Learning on a Natural Codebase), a deep learning-based vulnerability detection tool that automatically learns features of vulnerable code from a large and real-world Python codebase. Vudenc applies a word2vec model to identify semantically similar code tokens and to provide a vector representation. A network of long-short-term memory cells (LSTM) is then used to classify vulnerable code token sequences at a fine-grained level, highlight the specific areas in the source code that are likely to contain vulnerabilities, and provide confidence levels for its predictions. To evaluate Vudenc, we used 1,009 vulnerability-fixing commits from different GitHub repositories that contain seven different types of vulnerabilities (SQL injection, XSS, Command injection, XSRF, Remote code execution, Path disclosure, Open redirect) for training. In the experimental evaluation, Vudenc achieves a recall of 78%–87%, a precision of 82%–96%, and an F1 score of 80%–90%. Vudenc’s code, the datasets for the vulnerabilities, and the Python corpus for the word2vec model are available for reproduction. Our experimental results suggest that Vudenc is capable of outperforming most of its competitors in terms of vulnerably detection capabilities on real-world software. Comparable accuracy was only achieved on synthetic benchmarks, within single projects, or on a much coarser level of granularity such as entire source code files.
Basic Research on Ancient Bai Character Recognition Based on Mobile APP

Bai nationality has a long history and has its own language. Limited by the fact that there are fewer and fewer people who know the Bai language, the literature and culture of the Bai nationality begin to lose rapidly. In order to make the people who do not understand Bai characters can also read the ancient books of Bai nationality, this paper is based on the research of high-precision single character recognition model of Bai characters. First, with the help of Bai culture lovers and related scholars, we have constructed a data set of Bai characters, but limited by the need of expert knowledge, so the data set is limited in size. As a result, deep learning models with the nature of data hunger cannot get an ideal accuracy. In order to solve this issue, we propose to use the Chinese data set which also belongs to Sino-Tibetan language family to improve the recognition accuracy of Bai characters through transfer learning. In addition, we propose four transfer learning approaches: Direct Knowledge Transfer (DKT), Indirect Knowledge Transfer (IKT), Self-coding Knowledge Transfer (SCKT), and Self-supervised Knowledge Transfer (SSKT). Experiments show that our approaches greatly improve the recognition accuracy of Bai characters.
Model Inversion Attack by Integration of Deep Generative Models: Privacy-Sensitive Face Generation From a Face Recognition System

Cybersecurity in front of attacks to a face recognition system is an emerging issue in the cloud era, especially due to its strong bonds with the privacy of the users registered to the system. A possible attack is the model inversion attack (MIA) which aims to reveal the identity of a targeted user by generating the most proper datapoint input to the system with maximum corresponding confidence score at the output. The generated data of a registered user can be maliciously used as a serious invasion of the user privacy. In literature, MIA processes are categorized into white-box and black-box scenarios which are respectively with and without information about the system structure, parameters, and partially about the users. This research work assumes the MIA under semi-white box scenario of availability of system model structure and parameters but not any user data information, and verifies it as a severe threat even for a deep-learning-based face recognition system despite its complex structure and the diversity of registered user data. The alert state is promoted by Deep MIA which is the integration of deep generative models in MIA, and <inline-formula> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula>-GAN integrated MIA-initilized by a face based seed (<inline-formula> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula>-GAN-MIA-FS) is proposed. As a novel MIA search strategy, a pre-trained deep generative model with capability of generating a face image from a random feature vector is used for narrowing down the image search space to the feature vectors space, which has much lower dimensions. This allows the MIA process to efficiently search for a low-dimensional feature vector whose corresponding face image maximizes the confidence score. We have experimentally evaluated the proposed method by two objective criteria and three subjective criteria in comparison to <inline-formula> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula>-GAN-integrated MIA initialized with a random seed (<inline-formula> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula>-GAN-MIA-RS), DCGAN-integrated MIA (DCGAN-MIA), and the conventional MIA. The evaluation results approve the efficiency and superiority of the proposed technique in generating natural looking face clones with high recognizability as the targeted users.
Digital Twin Consensus for Blockchain-Enabled Intelligent Transportation Systems in Smart Cities

Digital Twin (DT) has become the key technology in the Intelligent Transportation Systems (ITS) in smart cities to keep the health and reliability of various DT requesters, such as private vehicles, public transportation, energy systems, etc. The combination of DT and ITS can further release the potential of participants in smart cities and guarantee their efficiency and reliability. Despite the advantages of DT-enabled ITS, not all requesters need the same level of DT service due to the highly dynamic nature of ITS. Safe and reliable matching between DT and ITS still needs to be resolved. To address these issues, we propose the blockchain-enabled Digital Twin as a Service (DTaaS) for ITS. First, we propose an on-demand DTaaS architecture to fully utilize the sensing capabilities of ITS and the macro perspective of DT. Second, a double-auction model and a price adjustment algorithm are proposed to realize the optimal DT matching for ITS requesters and ensure the benefits of participants. Third, a permissioned blockchain and a novel DT-DPoS consensus mechanism are established to enhance the security and efficiency of DTaaS. Simulation shows that the proposed DTaaS and double-auction can efficiently stimulate and facilitate DT transactions. The proposed DT-DPoS also has obvious advantages.
Transformer-Based Multistage Enhancement for Remote Sensing Image Super-Resolution

Convolutional neural networks have made a great breakthrough in recent remote sensing image super-resolution (SR) tasks. Most of these methods adopt upsampling layers at the end of the models to perform enlargement, which ignores feature extraction in the high-dimension space, and thus, limits SR performance. To address this problem, we propose a new SR framework for remote sensing images to enhance the high-dimensional feature representation after the upsampling layers. We name the proposed method as a transformer-based enhancement network (TransENet), where transformers are introduced to exploit features at different levels. The core of the TransENet is a transformer-based multistage enhancement structure, which can be combined with traditional SR frameworks to fuse multiscale high-/low-dimension features. Specifically, in this structure, the encoders aim to embed the multilevel features in the feature extraction part and the decoders are used to fuse these encoded embeddings. Experimental results demonstrate that our proposed TransENet can improve super-resolved results and obtain superior performance over several state-of-the-art methods.
Robust Image Forgery Detection Against Transmission Over Online Social Networks

The increasing abuse of image editing software causes the authenticity of digital images questionable. Meanwhile, the widespread availability of online social networks (OSNs) makes them the dominant channels for transmitting forged images to report fake news, propagate rumors, etc. Unfortunately, various lossy operations, e.g., compression and resizing, adopted by OSNs impose great challenges for implementing the robust image forgery detection. To fight against the OSN-shared forgeries, in this work, a novel robust training scheme is proposed. Firstly, we design a baseline detector, which won the top ranking in a recent certificate forgery detection competition. Then we conduct a thorough analysis of the noise introduced by OSNs, and decouple it into two parts, i.e., <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">predictable noise</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unseen noise</i> , which are modelled separately. The former simulates the noise introduced by the disclosed (known) operations of OSNs, while the latter is designed to not only complete the previous one, but also take into account the defects of the detector itself. We further incorporate the modelled noise into a robust training framework, significantly improving the robustness of the image forgery detector. Extensive experimental results are presented to validate the superiority of the proposed scheme compared with several state-of-the-art competitors, especially in the scenarios of detecting OSN-transmitted forgeries. Finally, to promote the future development of the image forgery detection, we build a public forgeries dataset based on four existing datasets through the uploading and downloading of four most popular OSNs. The data and code of this work are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/HighwayWu/ImageForensicsOSN</uri> .
PRIMA: general and precise neural network certification via scalable convex hull approximations

Formal verification of neural networks is critical for their safe adoption in real-world applications. However, designing a precise and scalable verifier which can handle different activation functions, realistic network architectures and relevant specifications remains an open and difficult challenge. In this paper, we take a major step forward in addressing this challenge and present a new verification framework, called PRIMA. PRIMA is both (i) general: it handles any non-linear activation function, and (ii) precise: it computes precise convex abstractions involving multiple neurons via novel convex hull approximation algorithms that leverage concepts from computational geometry. The algorithms have polynomial complexity, yield fewer constraints, and minimize precision loss. We evaluate the effectiveness of PRIMA on a variety of challenging tasks from prior work. Our results show that PRIMA is significantly more precise than the state-of-the-art, verifying robustness to input perturbations for up to 20%, 30%, and 34% more images than existing work on ReLU-, Sigmoid-, and Tanh-based networks, respectively. Further, PRIMA enables, for the first time, the precise verification of a realistic neural network for autonomous driving within a few minutes.
Single image depth estimation: An overview

We review solutions to the problem of depth estimation, arguably the most important subtask in scene understanding. We focus on the single image depth estimation problem. Due to its properties, the single image depth estimation problem is currently best tackled with machine learning methods, most successfully with convolutional neural networks. We provide an overview of the field by examining key works. We examine non-deep learning approaches that mostly predate deep learning and utilize hand-crafted features and assumptions, and more recent works that mostly use deep learning techniques. The single image depth estimation problem is tackled in a supervised fashion with absolute or relative depth information acquired from human or sensor-labeled data, or in an unsupervised way using unlabeled stereo images or video datasets. We also study multitask approaches that combine the depth estimation problem with related tasks such as semantic segmentation and surface normal estimation. Finally, we discuss investigations into the mechanisms, principles, and failure cases of contemporary solutions.
Blood cancer prediction using leukemia microarray gene data and hybrid logistic vector trees model

Blood cancer has been a growing concern during the last decade and requires early diagnosis to start proper treatment. The diagnosis process is costly and time-consuming involving medical experts and several tests. Thus, an automatic diagnosis system for its accurate prediction is of significant importance. Diagnosis of blood cancer using leukemia microarray gene data and machine learning approach has become an important medical research today. Despite research efforts, desired accuracy and efficiency necessitate further enhancements. This study proposes an approach for blood cancer disease prediction using the supervised machine learning approach. For the current study, the leukemia microarray gene dataset containing 22,283 genes, is used. ADASYN resampling and Chi-squared (Chi2) features selection techniques are used to resolve imbalanced and high-dimensional dataset problems. ADASYN generates artificial data to make the dataset balanced for each target class, and Chi2 selects the best features out of 22,283 to train learning models. For classification, a hybrid logistics vector trees classifier (LVTrees) is proposed which utilizes logistic regression, support vector classifier, and extra tree classifier. Besides extensive experiments on the datasets, performance comparison with the state-of-the-art methods has been made for determining the significance of the proposed approach. LVTrees outperform all other models with ADASYN and Chi2 techniques with a significant 100% accuracy. Further, a statistical significance T-test is also performed to show the efficacy of the proposed approach. Results using k-fold cross-validation prove the supremacy of the proposed model.
The Acceptance of Telepresence Robots in Higher Education

While telepresence robots have increasingly become accepted in diverse settings, the research on their acceptance in educational contexts has been underdeveloped. This study analyzed how the use intention of telepresence robots can be influenced by perceived usefulness, perceived ease of use, subjective norm, and perceived risk for students, faculty, and staff in higher education. Survey data were collected from 60 participants with direct operator experience with a variety of telepresence robots deployed in a large research university in the Midwest region of the United States. Path analysis results indicated that perceived usefulness was the only significant direct predictor of use intention of telepresence robots. Both perceived ease of use and subjective norm had a significant positive effect on perceived usefulness. Subjective norm also had a significant positive indirect effect on use intention, mediated by perceived usefulness. Perceived risk had a negative effect on perceived ease of use. These findings indicated that the usefulness of robots was central to operators' decisions to use telepresence robots. Therefore, design choice for telepresence robots should prioritize usefulness. Secondly, the design of telepresence robots should minimize complexity for the end user and minimize cognitive demand. Having nominal difficulty of use would also facilitate multiple embodiments by combining telepresence robots with other technologies to support more rich social interactions.
A new discrete chaotic map application in image encryption algorithm

Abstract In this work, a new discrete chaotic map is developed from the 1D ICMIC (Iterative Map with Infinite Collapses) to control the 2D Hénon map. The performances of the proposed discrete map are evaluated by using the phase diagram, Lyapunov exponent spectrum and complexity. It is found that the new chaotic map has larger value of complexity, and chaotic states are distributed in large parameter ranges. Thus the new chaotic map has better chaotic features and randomness, it is especially suitable for the chaotic secure communication. Then an image encryption algorithm is proposed based on the new chaotic map. In this algorithm, the sequences generated by the new chaotic map are used for pixel position scrambling and pixel value diffusion operations. The security features of the designed encryption scheme are evaluated by common attacks methods. The results confirmed that the proposed algorithm is effective to improve the security in image encryption.
An improved feature pyramid network for object detection

Object detection is one of the most important and challenging problems in the field of computer vision. In the current mainstream detection approaches, especially in the architectures of feature pyramid network (FPNs), feature fusion is a basic and essential method for all detectors. However, feature fusion does not fully consider the characteristics of the detection task for most detectors. To obtain suitable features for the detection task, in this paper, we propose two fusion methods: (1) For feature extraction, we propose an improved feature pyramid network (ImFPN) for superior representations. The most essential difference from FPNs is that the ImFPN includes a similarity-based fusion module, which can fuse different features to adapt to varying sizes of instances. (2) For specified tasks, since classification and regression tasks have different considerations in the same region, we build a new fusion mechanism between the dense and sparse heads in any two-stage detector based on an improved region proposal network (ImRPN). After adding these two modified architectures to Faster R-CNN with ResNet-101, the average precision (AP) improves from 39.7 to 41.4 on COCO test-dev. In addition, extensive experiments show the effectiveness of our methods on various models and datasets.