Reviewerly/exHarmony huggingface dataset:
Progressive Semantic-Aware Style Transformation for Blind Face Restoration

Face restoration is important in face image processing, and has been widely studied in recent years. However, previous works often fail to generate plausible high quality (HQ) results for real-world low quality (LQ) face images. In this paper, we propose a new progressive semantic-aware style transformation framework, named PSFR-GAN, for face restoration. Specifically, instead of using an encoder-decoder framework as previous methods, we formulate the restoration of LQ face images as a multi-scale progressive restoration procedure through semantic-aware style transformation. Given a pair of LQ face image and its corresponding parsing map, we first generate a multi-scale pyramid of the inputs, and then progressively modulate different scale features from coarse-to-fine in a semantic-aware style transfer way. Compared with previous networks, the proposed PSFR-GAN makes full use of the semantic (parsing maps) and pixel (LQ images) space information from different scales of input pairs. In addition, we further introduce a semantic aware style loss which calculates the feature style loss for each semantic region individually to improve the details of face textures. Finally, we pretrain a face parsing network which can generate decent parsing maps from real-world LQ face images. Experiment results show that our model trained with synthetic data can not only produce more realistic high-resolution results for synthetic LQ inputs but also generalize better to natural LQ face images compared with state-of-the-art methods.
Adaptive embedding: A novel meaningful image encryption scheme based on parallel compressive sensing and slant transform

A novel image encryption and adaptive embedding algorithm is proposed in this paper by combining 4D memristive hyperchaos, parallel compressive sensing (PCS) and slant transform (ST). This work dedicates to balancing the performance of existing visually meaningful image encryption algorithms which embed cipher image into host image in spatial or transform domains. First, the 2D discrete wavelet transform (DWT) is adopted to sparse the plain image. Then the sparse matrix after threshold processing is encrypted and compressed through Arnold scrambling and parallel compressive sensing to obtain the cipher data. Next the noise visibility function (NVF) is utilized to find the sub-images suitable for embedding in host image. After that the unquantified cipher data are randomly embedded in them by ST-based embedding method with block-wise manner and the final visually meaningful cipher image is obtained. To withstand the chosen-plaintext attacks (CPA) and the known-plaintext attacks (KPA), the frequency-domain information of plain image is used to control the 4D memristive hyperchaos to generate the scrambling matrix. Additionally, the experimental results and comprehensive analyses demonstrate that compared with the existing related algorithms, the proposed encryption algorithm has good visual security, decryption quality and superior robustness.
RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features

The two-stage methods for instance segmentation, e.g. Mask R-CNN, have achieved excellent performance recently. However, the segmented masks are still very coarse due to the downsampling operations in both the feature pyramid and the instance-wise pooling process, especially for large objects. In this work, we propose a new method called RefineMask for high-quality instance segmentation of objects and scenes, which incorporates fine-grained features during the instance-wise segmenting process in a multi-stage manner. Through fusing more detailed information stage by stage, RefineMask is able to refine high-quality masks consistently. RefineMask succeeds in segmenting hard cases such as bent parts of objects that are oversmoothed by most previous methods and outputs accurate boundaries. Without bells and whistles, RefineMask yields significant gains of 2.6, 3.4, 3.8 AP over Mask R-CNN on COCO, LVIS, and Cityscapes benchmarks respectively at a small amount of additional computational cost. Furthermore, our single-model result outperforms the winner of the LVIS Challenge 2020 by 1.3 points on the LVIS test-dev set and establishes a new state-of-the-art. Code will be available at https://github.com/zhanggang001/RefineMask.
Forecasting and uncertainty analysis of day-ahead photovoltaic power using a novel forecasting method

The primary means to promote grid-connected photovoltaic power generation is through accurately forecasting the power output from photovoltaic power stations. This paper proposes a method for day-ahead photovoltaic power forecasting (PPF) and uncertainty analysis using fuzzy c-means (FCM), whale optimization algorithm (WOA), least squares support vector machine (LSSVM), and non-parametric kernel density estimation (NPKDE). The FCM clustering algorithm was used to cluster historical data on numerical weather prediction and photovoltaic power stations, whereby daily data sharing similar meteorological information were clustered into one class. The rapid convergence speed and high convergence accuracy of the WOA were used to optimize the penalty factor and kernel function width of the LSSVM model; this was done to improve the calculation speed and forecasting accuracy of the LSSVM model. The WOA-LSSVM forecasting model was trained using the clustered numerical weather prediction and historical data of a photovoltaic power station. This was subsequently utilized to forecast day-ahead photovoltaic power. The NPKDE method was used to accurately calculate the probability density distribution of forecasting error and the confidence interval of the day-ahead PPF. The root mean square error (RMSE) values of the forecasting power of the WOA-LSSVM, PSO-LSSVM, LSSVM, LSTM and PSO-BP models are 2.55%, 3.00%, 5.60%, 6.03% and 3.18%, respectively, and the calculation results show that the forecasting accuracy of the WOA-LSSVM was higher relative to other models including PSO-LSSVM, LSSVM, LSTM and PSO-BP. Moreover, the NPKDE method was able to accurately describe the probability density distribution of the forecasting error.
MotionRNN: A Flexible Model for Video Prediction with Spacetime-Varying Motions

This paper tackles video prediction from a new dimension of predicting spacetime-varying motions that are incessantly changing across both space and time. Prior methods mainly capture the temporal state transitions but overlook the complex spatiotemporal variations of the motion itself, making them difficult to adapt to ever-changing motions. We observe that physical world motions can be decomposed into transient variation and motion trend, while the latter can be regarded as the accumulation of previous motions. Thus, simultaneously capturing the transient variation and the motion trend is the key to make spacetime-varying motions more predictable. Based on these observations, we propose the MotionRNN framework, which can capture the complex variations within motions and adapt to spacetime-varying scenarios. MotionRNN has two main contributions. The first is that we design the MotionGRU unit, which can model the transient variation and motion trend in a unified way. The second is that we apply the MotionGRU to RNN-based predictive models and indicate a new flexible video prediction architecture with a Motion Highway, which can significantly improve the ability to predict changeable motions and avoid motion vanishing for stacked multiple-layer predictive models. With high flexibility, this framework can adapt to a series of models for deterministic spatiotemporal prediction. Our MotionRNN can yield significant improvements on three challenging benchmarks for video prediction with spacetime-varying motions.
A level set method based on additive bias correction for image segmentation

Intensity inhomogeneity brings great difficulties to image segmentation. This problem is partly solved by the multiplicative bias field correction model. However, some other problems still exist, such as slow segmentation speed and narrow application field. In this paper, an additive bias correction (ABC) model based on intensity inhomogeneity is proposed. The model divides the observed image into three parts: additive bias function, reflection edge structure function and Gaussian noise. Firstly, the local area and local clustering criterion of intensity inhomogeneity are defined. Secondly, by introducing the level set function, the local clustering criterion is transformed into an energy function based on the level set model. Finally, the structure of the estimated bias field and the reflection edge is computed through the process of minimizing the energy function while the image is segmented. In order to improve the stability of the system, a de-parameterized regularization function and an adaptive data-driven term function are designed. Compared with the traditional multiplicative model, the addition model has faster calculation speed. The proposed model can obtain ideal segmentation effect for images with intensity inhomogeneity. Experiment results show that the proposed method is more robust, faster and more accurate than traditional piecewise and multiplicative models.
Artificial intelligence: The light and the darkness

That artificial intelligence (AI) has the potential to provide significant benefits is generally accepted by both practitioners and scholars. However, the dark side of AI is less discussed, and less understood. In this paper, the authors first classify the wellspring of AI benefits in both B2C and B2B settings. In B2C settings AI benefits are primarily via customized experiences, while B2B AI benefits are manifested via business efficiencies. Next, guided by the relationship marketing literature, the authors identify the drivers of the dark side of AI - lack of trust and power asymmetries, with lack of trust being a stronger factor in B2C settings and power asymmetries being a stronger factor in B2B settings. Finally, the authors provide an organizing framework for understanding both the bright side and the dark side of AI, in both B2C settings and B2B settings. This paper is differentiated from prior work by its focus on B2B settings (most focus on B2C settings), and by its focus on the dark side of AI (most focus on the bright side of AI).
A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions

Knowledge base question answering (KBQA) aims to answer a question over a knowledge base (KB). Recently, a large number of studies focus on semantically or syntactically complicated questions. In this paper, we elaborately summarize the typical challenges and solutions for complex KBQA. We begin with introducing the background about the KBQA task. Next, we present the two mainstream categories of methods for complex KBQA, namely semantic parsing-based (SP-based) methods and information retrieval-based (IR-based) methods. We then review the advanced methods comprehensively from the perspective of the two categories. Specifically, we explicate their solutions to the typical challenges. Finally, we conclude and discuss some promising directions for future research.
Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series Signals

Nowadays, multi-sensor technologies are applied in many fields, e.g., Health Care (HC), Human Activity Recognition (HAR), and Industrial Control System (ICS). These sensors can generate a substantial amount of multivariate time-series data. Unsupervised anomaly detection on multi-sensor time-series data has been proven critical in machine learning researches. The key challenge is to discover generalized normal patterns by capturing spatial-temporal correlation in multi-sensor data. Beyond this challenge, the noisy data is often intertwined with the training data, which is likely to mislead the model by making it hard to distinguish between the normal, abnormal, and noisy data. Few of previous researches can jointly address these two challenges. In this paper, we propose a novel deep learning-based anomaly detection algorithm called Deep Convolutional Autoencoding Memory network (CAE-M). We first build a Deep Convolutional Autoencoder to characterize spatial dependence of multi-sensor data with a Maximum Mean Discrepancy (MMD) to better distinguish between the noisy, normal, and abnormal data. Then, we construct a Memory Network consisting of linear (Autoregressive Model) and non-linear predictions (Bidirectional LSTM with Attention) to capture temporal dependence from time-series data. Finally, CAE-M jointly optimizes these two subnetworks. We empirically compare the proposed approach with several state-of-the-art anomaly detection methods on HAR and HC datasets. Experimental results demonstrate that our proposed model outperforms these existing methods.
An automated learning model for sentiment analysis and data classification of Twitter data using balanced CA-SVM

The modern society runs over the social media for their most time of every day. The web users spend their most time in social media and they share many details with their friends. Such information obtained from their chat has been used in several applications. The sentiment analysis is the one which has been applied with Twitter data set toward identifying the emotion of any user and based on those different problems can be solved. Primarily, the data as of the Twitter database is preprocessed. In this step, tokenization, stemming, stop word removal, and number removal are done. The proposed automated learning with CA-SVM based sentiment analysis model reads the Twitter data set. After that they have been processed to extract the features which yield set of terms. Using the terms, the tweets are clustered using TGS-K means clustering which measures Euclidean distance according to different features like semantic sentiment score (SSS), gazetteer and symbolic sentiment support (GSSS), and topical sentiment score (TSS). Further, the method classifies the tweets according to support vector machine (CA-SVM) which classifies the tweet according to the support value which is measured based on the above two measures. The attained results are validated utilizing k-fold cross-validation methodology. Then, the classification is performed by utilizing the Balanced CA-SVM (Deep Learning Modified Neural Network). The results are evaluated and compared with the existing works. The Proposed model achieved 92.48 % accuracy and 92.05% sentiment score contrasted with the existing works.
Federated Learning Empowered Computation Offloading and Resource Management in 6G-V2X

Humankind's urbanization and luxuriousneed increase the number of vehicles day by day. Alongside the advancement of technologies, autonomous vehicles have now come into a greater existence. Due to the resource constraints for obtaining a high processing rate in performing the intelligent tasks, Vehicular Edge Computing (VEC) has been introduced. However, a delay occurs in transferring the information due to the traits of automotive tasks. In this paper, a Federated Learning empowered computation Offloading and Resource management (FLOR) is proposed, which offloads and manages the resources with low latency. The FLOR framework operates over heterogeneous networks that include Vehicle-to-Everything (V2X) communication, Dedicated Short Range Communication (DSRC), 5G millimeter-wave (5G-mmWave), and 6G Vehicle-to-Vehicle (6G-V2V) communication. FLOR employs the stochastic network calculus mechanism for computing the upper bound delay of the heterogeneous communication and obtains the probability of offloading mechanism. Upon the detected probability value, the FLOR offloads the computation task to the optimal network. Further, to avoid the increase in resource cost and utilize the available resources efficiently, an effective Radio Resource Management (e-RRM) based on Federated Q-Learning is proposed in FLOR, for allocating and using the available resources optimally. The simulation result shows that the proposed FLOR framework effectively offloads the computation between the available heterogeneous networks with optimal resource allocation by 20.69% than the existing solutions.
Exploiting Dynamic Quantum Circuits in a Quantum Algorithm with Superconducting Qubits

To date, quantum computation on real, physical devices has largely been limited to simple, time-ordered sequences of unitary operations followed by a final projective measurement. As hardware platforms for quantum computing continue to mature in size and capability, it is imperative to enable quantum circuits beyond their conventional construction. Here we break into the realm of dynamic quantum circuits on a superconducting-based quantum system. Dynamic quantum circuits not only involve the evolution of the quantum state throughout the computation but also periodic measurements of qubits midcircuit and concurrent processing of the resulting classical information on timescales shorter than the execution times of the circuits. Using noisy quantum hardware, we explore one of the most fundamental quantum algorithms, quantum phase estimation, in its adaptive version, which exploits dynamic circuits, and compare the results to a nonadaptive implementation of the same algorithm. We demonstrate that the version of real-time quantum computing with dynamic circuits can yield results comparable to an approach involving classical asynchronous postprocessing, thus opening the door to a new realm of available algorithms on real quantum systems.
On-Site Construction Quality Inspection Using Blockchain and Smart Contracts

Typically, on-site construction quality inspection (OCQI) is manually conducted, and its effectiveness highly depends on the self-discipline of responsible personnel. Furthermore, construction quality information is still recorded on paper, with the risk of document loss and data manipulation, jeopardizing quality accountability. Blockchain has attracted increasing attention from the construction industry because it can record information in an immutable and transparent way and automate some businesses with smart contracts. These properties can tackle information fraud and improve the automation level of OCQI. Unfortunately, very few documented blockchain solutions for OCQI exist. To fill the theoretical knowledge gap, the design thinking approach was adopted to present a blockchain-based conceptual framework. The consortium network and Hyperledger Fabric (HLF) were selected as the suitable blockchain type and development platform for OCQI. The consensus process in HLF and smart contracts–based compliance checking are illustrated. Moreover, the benefits of integrating the Internet of Things (IoT) and ontology technology with blockchain are analyzed in detail, which can facilitate the coevaluation of these three technologies. Finally, a prototype system was developed to vividly show the theoretical feasibility. The proposed blockchain solution can be fine-tuned and evaluated in future research. Challenges hindering the practical implementation of blockchains are systematically discussed within the technology–organization–environment (TOE) theory, which can be reused in future investigations to identify construction stakeholders’ concerns on adopting blockchains.
Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation

Much of the recent progress in sequential and session-based recommendation has been driven by improvements in model architecture and pretraining techniques originating in the field of Natural Language Processing. Transformer architectures in particular have facilitated building higher-capacity models and provided data augmentation and training techniques which demonstrably improve the effectiveness of sequential recommendation. But with a thousandfold more research going on in NLP, the application of transformers for recommendation understandably lags behind. To remedy this we introduce Transformers4Rec, an open-source library built upon HuggingFace’s Transformers library with a similar goal of opening up the advances of NLP based Transformers to the recommender system community and making these advancements immediately accessible for the tasks of sequential and session-based recommendation. Like its core dependency, Transformers4Rec is designed to be extensible by researchers, simple for practitioners, and fast and robust in industrial deployments.
Content-Noise Complementary Learning for Medical Image Denoising

Medical imaging denoising faces great challenges, yet is in great demand. With its distinctive characteristics, medical imaging denoising in the image domain requires innovative deep learning strategies. In this study, we propose a simple yet effective strategy, the content-noise complementary learning (CNCL) strategy, in which two deep learning predictors are used to learn the respective content and noise of the image dataset complementarily. A medical image denoising pipeline based on the CNCL strategy is presented, and is implemented as a generative adversarial network, where various representative networks (including U-Net, DnCNN, and SRDenseNet) are investigated as the predictors. The performance of these implemented models has been validated on medical imaging datasets including CT, MR, and PET. The results show that this strategy outperforms state-of-the-art denoising algorithms in terms of visual quality and quantitative metrics, and the strategy demonstrates a robust generalization capability. These findings validate that this simple yet effective strategy demonstrates promising potential for medical image denoising tasks, which could exert a clinical impact in the future. Code is available at: https://github.com/gengmufeng/CNCL-denoising.
Vision-Language Transformer and Query Generation for Referring Segmentation

In this work, we address the challenging task of referring segmentation. The query expression in referring segmentation typically indicates the target object by describing its relationship with others. Therefore, to find the target one among all instances in the image, the model must have a holistic understanding of the whole image. To achieve this, we reformulate referring segmentation as a direct attention problem: finding the region in the image where the query language expression is most attended to. We introduce transformer and multi-head attention to build a network with an encoder-decoder attention mechanism architecture that "queries" the given image with the language expression. Furthermore, we propose a Query Generation Module, which produces multiple sets of queries with different attention weights that represent the diversified comprehensions of the language expression from different aspects. At the same time, to find the best way from these diversified comprehensions based on visual clues, we further propose a Query Balance Module to adaptively select the output features of these queries for a better mask generation. Without bells and whistles, our approach is light-weight and achieves new state-of-the-art performance consistently on three referring segmentation datasets, RefCOCO, RefCOCO+, and G-Ref. Our code is available at https://github.com/henghuiding/Vision-Language-Transformer.