Reviewerly/exHarmony huggingface dataset:
Optical image encryption based on quantum walks

In this paper, we propose a new optical encryption approach based on quantum walks and double random phase encoding technique. Quantum walks have been proven as an excellent tool for designing modern cryptographic mechanisms due to its ability to resist probable attacks from both digital and quantum computers. In the proposed approach, we use the alternate quantum walks (AQW) at two encryption stages. First, inner encryption and encoding by double random phase encoding (DRPE) which executed using permutation followed by substitution. Furthermore, AQW is adapted to generate two random masks for the DRPE process. Simulation-based validation of our proposed technique, including correlation, histogram, sensitivity, key space as well as noise and data loss analysis provide impetus to consider its use in applications in modern optical cryptosystems.
Context-Aware Multi-View Summarization Network for Image-Text Matching

Image-text matching is a vital yet challenging task in the field of multimedia analysis. Over the past decades, great efforts have been made to bridge the semantic gap between the visual and textual modalities. Despite the significance and value, most prior work is still confronted with a multi-view description challenge, i.e., how to align an image to multiple textual descriptions with semantic diversity. Toward this end, we present a novel context-aware multi-view summarization network to summarize context-enhanced visual region information from multiple views. To be more specific, we design an adaptive gating self-attention module to extract representations of visual regions and words. By controlling the internal information flow, we are able to adaptively capture context information. Afterwards, we introduce a summarization module with a diversity regularization to aggregate region-level features into image-level ones from different perspectives. Ultimately, we devise a multi-view matching scheme to match multi-view image features with corresponding text ones. To justify our work, we have conducted extensive experiments on two benchmark datasets, i.e., Flickr30K and MS-COCO, which demonstrates the superiority of our model as compared to several state-of-the-art baselines.
Virtual Reality with 360-Video Storytelling in Cultural Heritage: Study of Presence, Engagement, and Immersion

This paper presents a combined subjective and objective evaluation of an application mixing interactive virtual reality (VR) experience with 360° storytelling. The hypothesis that the modern immersive archaeological VR application presenting cultural heritage from a submerged site would sustain high levels of presence, immersion, and general engagement was leveraged in the investigation of the user experience with both the subjective (questionnaires) and the objective (neurophysiological recording of the brain signals using electroencephalography (EEG)) evaluation methods. Participants rated the VR experience positively in the questionnaire scales for presence, immersion, and subjective judgement. High positive rating concerned also the psychological states linked to the experience (engagement, emotions, and the state of flow), and the experience was mostly free from difficulties linked to the accustomization to the VR technology (technology adoption to the head-mounted display and controllers, VR sickness). EEG results are in line with past studies examining brain responses to virtual experiences, while new results in the beta band suggest that EEG is a viable tool for future studies of presence and immersion in VR.
TwinBERT

Pre-trained language models have achieved great success in a wide variety of natural language processing (NLP) tasks, while the superior performance comes with high demand in computational resources, which hinders the application in low-latency information retrieval (IR) systems. To address the problem, we present TwinBERT model, which has two improvements: 1) represent query and document separately using twin-structured encoders and 2) each encoder is a highly compressed BERT-like model with less than one third of the parameters. The former allows document embeddings to be pre-computed offline and cached in memory, which is different from BERT, where the two input sentences are concatenated and encoded together. The change saves large amount of computation time, however, it is still not sufficient for real-time retrieval considering the complexity of BERT model itself. To further reduce computational cost, a compressed multi-layer transformer encoder is proposed with special training strategies as a substitution of the original complex BERT encoder. Lastly, two versions of TwinBERT are developed to combine the query and keyword embeddings for retrieval and relevance tasks correspondingly. Both of them have met the real-time latency requirement and achieve close or on-par performance to BERT-Base model.
Memory-Efficient Incremental Learning Through Feature Adaptation

We introduce an approach for incremental learning that preserves feature descriptors of training images from previously learned classes, instead of the images themselves, unlike most existing work. Keeping the much lower-dimensional feature embeddings of images reduces the memory footprint significantly. We assume that the model is updated incrementally for new classes as new data becomes available sequentially. This requires adapting the previously stored feature vectors to the updated feature space without having access to the corresponding original training images. Feature adaptation is learned with a multi-layer perceptron, which is trained on feature pairs corresponding to the outputs of the original and updated network on a training image. We validate experimentally that such a transformation generalizes well to the features of the previous set of classes, and maps features to a discriminative subspace in the feature space. As a result, the classifier is optimized jointly over new and old classes without requiring old class images. Experimental results show that our method achieves state-of-the-art classification accuracy in incremental learning benchmarks, while having at least an order of magnitude lower memory footprint compared to image-preserving strategies.
Online Meta-learning for Multi-source and Semi-supervised Domain Adaptation

Domain adaptation (DA) is the topical problem of adapting models from labelled source datasets so that they perform well on target datasets where only unlabelled or partially labelled data is available. Many methods have been proposed to address this problem through different ways to minimise the domain shift between source and target datasets. In this paper we take an orthogonal perspective and propose a framework to further enhance performance by meta-learning the initial conditions of existing DA algorithms. This is challenging compared to the more widely considered setting of few-shot meta-learning, due to the length of the computation graph involved. Therefore we propose an online shortest-path meta-learning framework that is both computationally tractable and practically effective for improving DA performance. We present variants for both multi-source unsupervised domain adaptation (MSDA), and semi-supervised domain adaptation (SSDA). Importantly, our approach is agnostic to the base adaptation algorithm, and can be applied to improve many techniques. Experimentally, we demonstrate improvements on classic (DANN) and recent (MCD and MME) techniques for MSDA and SSDA, and ultimately achieve state of the art results on several DA benchmarks including the largest scale DomainNet.
Unpaired Learning of Deep Image Denoising

We investigate the task of learning blind image denoising networks from an unpaired set of clean and noisy images. Such problem setting generally is practical and valuable considering that it is feasible to collect unpaired noisy and clean images in most real-world applications. And we further assume that the noise can be signal dependent but is spatially uncorrelated. In order to facilitate unpaired learning of denoising network, this paper presents a two-stage scheme by incorporating self-supervised learning and knowledge distillation. For self-supervised learning, we suggest a dilated blind-spot network (D-BSN) to learn denoising solely from real noisy images. Due to the spatial independence of noise, we adopt a network by stacking $$1\times 1$$ convolution layers to estimate the noise level map for each image. Both the D-BSN and image-specific noise model ( $$\text {CNN}_{\text {est}}$$ ) can be jointly trained via maximizing the constrained log-likelihood. Given the output of D-BSN and estimated noise level map, improved denoising performance can be further obtained based on the Bayes’ rule. As for knowledge distillation, we first apply the learned noise models to clean images to synthesize a paired set of training images, and use the real noisy images and the corresponding denoising results in the first stage to form another paired set. Then, the ultimate denoising model can be distilled by training an existing denoising network using these two paired sets. Experiments show that our unpaired learning method performs favorably on both synthetic noisy images and real-world noisy photographs in terms of quantitative and qualitative evaluation. Code is available at https://github.com/XHWXD/DBSN .
Explanations of Model Predictions with live and breakDown Packages

Complex models are commonly used in predictive modeling. In this paper we present R packages that can be used to explain predictions from complex black box models and attribute parts of these predictions to input features. We introduce two new approaches and corresponding packages for such attribution, namely live and breakDown. We also compare their results with existing implementations of state of the art solutions, namely lime that implements Locally Interpretable Model-agnostic Explanations and ShapleyR that implements Shapley values.
Optimal Gaussian measurements for phase estimation in single-mode Gaussian metrology

The central issue in quantum parameter estimation is to find out the optimal measurement setup that leads to the ultimate lower bound of an estimation error. We address here a question of whether a Gaussian measurement scheme can achieve the ultimate bound for phase estimation in single-mode Gaussian metrology that exploits single-mode Gaussian probe states in a Gaussian environment. We identify three types of optimal Gaussian measurement setups yielding the maximal Fisher information depending on displacement, squeezing, and thermalization of the probe state. We show that the homodyne measurement attains the ultimate bound for both displaced thermal probe states and squeezed vacuum probe states, whereas for the other single-mode Gaussian probe states, the optimized Gaussian measurement cannot be the optimal setup, although they are sometimes nearly optimal. We then demonstrate that the measurement on the basis of the product quadrature operators XP+PX, i.e., a non-Gaussian measurement, is required to be fully optimal.
Quantum simulation of chemistry with sublinear scaling in basis size

We present a quantum algorithm for simulating quantum chemistry with gate complexity $\tilde{O}(N^{1/3} \eta^{8/3})$ where $\eta$ is the number of electrons and $N$ is the number of plane wave orbitals. In comparison, the most efficient prior algorithms for simulating electronic structure using plane waves (which are at least as efficient as algorithms using any other basis) have complexity $\tilde{O}(N^{8/3} /\eta^{2/3})$. We achieve our scaling in first quantization by performing simulation in the rotating frame of the kinetic operator using interaction picture techniques. Our algorithm is far more efficient than all prior approaches when $N \gg \eta$, as is needed to suppress discretization error when representing molecules in the plane wave basis, or when simulating without the Born-Oppenheimer approximation.
CLNet: A Compact Latent Network for Fast Adjusting Siamese Trackers

In this paper, we provide a deep analysis for Siamese-based trackers and find that the one core reason for their failure on challenging cases can be attributed to the problem of decisive samples missing during offline training. Furthermore, we notice that the samples given in the first frame can be viewed as the decisive samples for the sequence since they contain rich sequence-specific information. To make full use of these sequence-specific samples, we propose a compact latent network to quickly adjust the tracking model to adapt to new scenes. A statistic-based compact latent feature is proposed to efficiently capture the sequence-specific information for the fast adjustment. In addition, we design a new training approach based on a diverse sample mining strategy to further improve the discrimination ability of our compact latent network. To evaluate the effectiveness of our method, we apply it to adjust a recent state-of-the-art tracker, SiamRPN++. Extensive experimental results on five recent benchmarks demonstrate that the adjusted tracker achieves promising improvement in terms of tracking accuracy, with almost the same speed. The code and models are available at https://github.com/xingpingdong/CLNet-tracking .
PG-Net: Pixel to Global Matching Network for Visual Tracking

Siamese neural network has been well investigated by tracking frameworks due to its fast speed and high accuracy. However, very few efforts were spent on background-extraction by those approaches. In this paper, a Pixel to Global Matching Network (PG-Net) is proposed to suppress t+he influence of background in search image while achieving state-of-the-art tracking performance. To achieve this purpose, each pixel on search feature is utilized to calculate the similarity with global template feature. This calculation method can appropriately reduce the matching area, thus introducing less background interference. In addition, we propose a new tracking framework to perform correlation-shared tracking and multiple losses for training, which not only reduce the computational burden but also improve the performance. We conduct comparison experiments on various public tracking datasets, which obtains state-of-the-art performance while running with fast speed.
Scene Text Image Super-Resolution in the Wild

Low-resolution text images are often seen in natural scenes such as documents captured by mobile phones. Recognizing low-resolution text images is challenging because they lose detailed content information, leading to poor recognition accuracy. An intuitive solution is to introduce super-resolution (SR) techniques as pre-processing. However, previous single image super-resolution (SISR) methods are trained on synthetic low-resolution images (e.g. Bicubic down-sampling), which is simple and not suitable for real low-resolution text recognition. To this end, we propose a real scene text SR dataset, termed TextZoom. It contains paired real low-resolution and high-resolution images which are captured by cameras with different focal length in the wild. It is more authentic and challenging than synthetic data, as shown in Fig. 1. We argue improving the recognition accuracy is the ultimate goal for Scene Text SR. In this purpose, a new Text Super-Resolution Network, termed TSRN, with three novel modules is developed. (1) A sequential residual block is proposed to extract the sequential information of the text images. (2) A boundary-aware loss is designed to sharpen the character boundaries. (3) A central alignment module is proposed to relieve the misalignment problem in TextZoom. Extensive experiments on TextZoom demonstrate that our TSRN largely improves the recognition accuracy by over 13% of CRNN, and by nearly 9.0% of ASTER and MORAN compared to synthetic SR data. Furthermore, our TSRN clearly outperforms 7 state-of-the-art SR methods in boosting the recognition accuracy of LR images in TextZoom. For example, it outperforms LapSRN by over 5% and 8% on the recognition accuracy of ASTER and CRNN. Our results suggest that low-resolution text recognition in the wild is far from being solved, thus more research effort is needed. The codes and models will be released at: github.com/JasonBoy1/TextZoom
Improving Deep Video Compression by Resolution-Adaptive Flow Coding

In the learning based video compression approaches, it is an essential issue to compress pixel-level optical flow maps by developing new motion vector (MV) encoders. In this work, we propose a new framework called Resolution-adaptive Flow Coding (RaFC) to effectively compress the flow maps globally and locally, in which we use multi-resolution representations instead of single-resolution representations for both the input flow maps and the output motion features of the MV encoder. To handle complex or simple motion patterns globally, our frame-level scheme RaFC-frame automatically decides the optimal flow map resolution for each video frame. To cope different types of motion patterns locally, our block-level scheme called RaFC-block can also select the optimal resolution for each local block of motion features. In addition, the rate-distortion criterion is applied to both RaFC-frame and RaFC-block and select the optimal motion coding mode for effective flow coding. Comprehensive experiments on four benchmark datasets HEVC, VTL, UVG and MCL-JCV clearly demonstrate the effectiveness of our overall RaFC framework after combing RaFC-frame and RaFC-block for video compression.
Spatially Aware Multimodal Transformers for TextVQA

Textual cues are essential for everyday tasks like buying groceries and using public transport. To develop this assistive technology, we study the TextVQA task, i.e., reasoning about text in images to answer a question. Existing approaches are limited in their use of spatial relations and rely on fully-connected transformer-based architectures to implicitly learn the spatial structure of a scene. In contrast, we propose a novel spatially aware self-attention layer such that each visual entity only looks at neighboring entities defined by a spatial graph. Further, each head in our multi-head self-attention layer focuses on a different subset of relations. Our approach has two advantages: (1) each head considers local context instead of dispersing the attention amongst all visual entities; (2) we avoid learning redundant features. We show that our model improves the absolute accuracy of current state-of-the-art methods on TextVQA by 2.2% overall over an improved baseline, and 4.62% on questions that involve spatial reasoning and can be answered correctly using OCR tokens. Similarly on ST-VQA, we improve the absolute accuracy by 4.2%. We further show that spatially aware self-attention improves visual grounding.
Differential evolution algorithm with fitness and diversity ranking-based mutation operator

Differential evolution (DE) is a simple and efficient global optimization algorithm. Benefitting from its concise structure and strong search ability, DE has been widely used in various fields. Generally, the convergence performance of DE largely depends on its mutation operation. Meanwhile, individuals’ positions, which are selected as base vectors or making up difference vectors, are very important in mutation strategy. In this paper, we propose a differential evolution algorithm with both fitness and diversity ranking-based mutation operator (FDDE). Different from methods that use fitness as the only index to measure the quality of individuals, FDDE aims to assign suitable position for each individual in the mutation strategy by together considering both individuals’ fitness and their diversity contribution. Firstly, a new method of estimating the individual diversity by fitness values has been proposed. Then, each individual's fitness ranking and diversity contribution are considered together to calculate a newly defined individual's final ranking. Finally, the final ranking are used in the mutation strategy. The newly improved mutation operator could be integrated with any classical or advanced DE variants with little additional time or space complexity. The proposed FDDE is compared with some DE variants based on numerical experiments over the CEC (Congress on Evolutionary Computation) 2005 benchmark sets, CEC 2013 benchmark sets and CEC 2014 benchmark sets. Experimental results clearly indicate that FDDE performs better on most test functions and improves the convergence performance of its competitors of jDE, rank-jDE, advanced SHADE, rank-SHADE and L-SHADE in both low and high dimensional problems.
The future of value in digitalised higher education: why data privacy should not be our biggest concern

Universities around the world are increasingly digitalising all of their operations, with the current COVID-19 pandemic speeding up otherwise steady developments. This article focuses on the political economy of higher education (HE) digitalisation and suggests a new research programme. I foreground three principal arguments, which are empirically, theoretically, and politically crucial for HE scholars. First, most literature is examining the impacts of digitalisation on the HE sector and its subjects alone. I argue that current changes in digitalising HE cannot be studied in isolation from broader changes in the global economy. Specifically, HE digitalisation is embedded in the expansion of the digital economy, which is marked by new forms of value extraction and rentiership. Second, the emerging research on the intersection of marketisation and digitalisation in HE seems to follow the theories of marketisation qua production and commodification. I argue that we need theories with better explanatory power in analysing the current digitalisation dynamics. I propose to move from commodification to assetisation, and from prices to rents. Finally, universities are digitalising in the time when the practice is superseding policy, and there is no regulation beyond the question of data privacy. However, digital data property is already a reality, governed by 'terms of use', and protected by the intellectual property rights regime. The current pandemic has led to 'emergency pedagogy', which has intensified overall digitalisation in the sector and is bypassing concerns of data value redistribution. I argue that we urgently need public scrutiny and political action to address issues of value extraction and redistribution in HE.
A GIS-based multi-criteria decision making method for the potential assessment and suitable sites selection of PV and CSP plants

By combining geographic information systems (GIS) and multi-criteria decision making (MCDM) method, this paper presents a framework to evaluate the suitable location and technical potentials for large-scale solar photovoltaic (PV) and concentrated solar power (CSP) plants. With the assist of spatial tools, the potential locations are identified by extracting restrictive criteria (e.g. protected area, land cover, transport infrastructure, and orography). Using MCDM, the land suitability for PV and CSP plants is evaluated with the comprehensive consideration of different factors, such as climate, orography, water availability, and location, where factor weights are calculated by the analytic hierarchy process (AHP) method. The study framework is illustrated by Ningxia, China, a key energy project area along the Belt and Road with abundant solar radiation. The obtained results suggested that the middle area of Ningxia is recommended for the solar power plant deployment. Besides, due to the limitation of water resource availability, PV technology is more appropriate for the study area than CSP technology. The estimated total technical potentials of PV and CSP systems could be 443 TWh/year and 308 TWh/year, indicating great potential for future solar energy development to enhance the renewable energy supply and participate in energy cooperation with the other regions. Moreover, sensitivity analysis is conducted to investigate the impacts of system efficiency variation on energy output. The results can be used for the identification of the potential areas for solar power plant installation, as well as support for the synergic expansion of generation plants and the power grid.