Reviewerly/exHarmony huggingface dataset:
Understanding User Experience of COVID-19 Maps through Remote Elicitation Interviews

During the coronavirus pandemic, visualizations gained a new level of popularity and meaning for a wider audience.People were bombarded with a wide set of public health visualizations ranging from simple graphs to complex interactive dashboards.In a pandemic setting, where large amounts of the world population are socially distancing themselves, it becomes an urgent need to refine existing user experience evaluation methods for remote settings to understand how people make sense out of COVID-19 related visualizations.When evaluating visualizations aimed towards the general public with vastly different socio-demographic backgrounds and varying levels of technical savviness and data literacy, it is important to understand user feedback beyond aspects such as speed, task accuracy, or usability problems.As a part of this wider evaluation perspective, micro-phenomenology has been used to evaluate static and narrative visualizations to reveal the lived experience in a detailed way.Building upon these studies, we conducted a user study to understand how to employ Elicitation (aka Micro-phenomenological) interviews in remote settings.In a case study, we investigated what experiences the participants had with map-based interactive visualizations.Our findings reveal positive and negative aspects of conducting Elicitation interviews remotely.Our results can inform the process of planning and executing remote Elicitation interviews to evaluate interactive visualizations.In addition, we share recommendations regarding visualization techniques and interaction design about public health data.
Formosa Speech Recognition Challenge 2020 and Taiwanese Across Taiwan Corpus

Taiwanese (a.k.a. Taiwanese Hokkien, Hoklo, Taigi, Southern Min or Min-Nan) is an endangered language, because the domination of Mandarin, the number of Taiwanese speakers continues to drop, especially among the youth generations. In addressing this problem, a Taiwanese speech-enabled human-computer interface for supporting people's daily life is essential. Therefore, a Formosa Speech in the Wild (FSW) project was established to collect a large-scale Taiwanese speech across Taiwan (TAT) corpus to boost the development of Taiwanese speech recognition (TSR). A Formosa Speech Recognition Challenge 2020 (FSR-2020) was also hosted to promote the corpus as well as to evaluate the performance of state-of-the-art TSR systems. This paper briefly introduces TAT corpus and FSR-2020 challenge, presents the provided data profile, evaluation plan and reports experimental baseline results. A subset of TAT corpus, TAT-Vol1, is given away for free for all participants (non-commercial license), and its corresponding Kaldi baseline recipes have been published online. Experimental results have showed that the combination of TAT corpus and the baseline recipes is a good resource pack for TSR research and development.
Relation-aware Meta-learning for E-commerce Market Segment Demand Prediction with Limited Records

E-commerce business is revolutionizing our shopping experiences by providing convenient and straightforward services. One of the most fundamental problems is how to balance the demand and supply in market segments to build an efficient platform. While conventional machine learning models have achieved great success on data-sufficient segments, it may fail in a large-portion of segments in E-commerce platforms, where there are not sufficient records to learn well-trained models. In this paper, we tackle this problem in the context of market segment demand prediction. The goal is to facilitate the learning process in the target segments by leveraging the learned knowledge from data-sufficient source segments. Specifically, we propose a novel algorithm, RMLDP, to incorporate a multi-pattern fusion network (MPFN) with a meta-learning paradigm. The multi-pattern fusion network considers both local and seasonal temporal patterns for segment demand prediction. In the meta-learning paradigm, transferable knowledge is regarded as the model parameter initialization of MPFN, which are learned from diverse source segments. Furthermore, we capture the segment relations by combining data-driven segment representation and segment knowledge graph representation and tailor the segment-specific relations to customize transferable model parameter initialization. Thus, even with limited data, the target segment can quickly find the most relevant transferred knowledge and adapt to the optimal parameters. We conduct extensive experiments on two large-scale industrial datasets. The results justify that our RMLDP outperforms a set of state-of-the-art baselines. Besides, RMLDP has been deployed in Taobao, a real-world E-commerce platform. The online A/B testing results further demonstrate the practicality of RMLDP.
Video SAR Image Despeckling by Unsupervised Learning

It has long been recognized that synthetic aperture radar (SAR) images suffer in many applications from the speckle noise. Video SAR has a high frame rate of imaging and contains redundant information among frames. The temporal redundancy in video SAR images has been found useful for suppressing the speckle noise. However, the motion and the local differences between frames make it difficult to employ the temporal redundancy to suppress speckle noise. This article presents a video SAR image despeckling framework based on a new unsupervised training strategy referred to as DualNoise2Noise. This developed framework consists of a registration network and a denoising network. The registration network first compensates for the motion between two adjacent frames in video SAR in real time. After image registration, two adjacent frames with random speckle noise can be considered as the observations of the same region with local differences. The denoising network adopts the DualNoise2Noise training strategy to suppress speckle noise by using the temporal redundancy and to remove the negative impact caused by the local differences. The proposed approach has been used to process the real video SAR data, and the experimental results are convincing.
Generating unambiguous and diverse referring expressions

Neural Referring Expression Generation (REG) models have shown promising results in generating expressions which uniquely describe visual objects. However, current REG models still lack the ability to produce diverse and unambiguous referring expressions (REs). To address the lack of diversity, we propose generating a set of diverse REs, rather than one-shot REs. To reduce the ambiguity of referring expressions, we directly optimise non-differentiable test metrics using reinforcement learning (RL), and we show that our approaches achieve better results under multiple different settings. Specifically, we initially present a novel RL approach to REG training, which instead of drawing one sample per input, it averages over multiple samples to normalize the reward during RL training. Secondly, we present an innovative REG model that utilizes an object attention mechanism that explicitly incorporates information about the target object and is optimised using our proposed RL approach. Thirdly, we propose a novel transformer model optimised with RL that exploits different levels of visual information. Our human evaluation demonstrates the effectiveness of this model, where we improve the state-of-the-art results in RefCOCO testA and testB in terms of task success from 76.95% to 81.66% and from 78.10% to 83.33% respectively. While in RefCOCO+ testA we show improvements from 58.85% to 83.33%. Finally, we present a thorough comparison of diverse decoding strategies (sampling and maximisation-based) and how they control the trade-off between the quality and diversity.
Linear approximation of a vectorial Boolean function using quantum computing

A vectorial Boolean function takes multi-bit input and produces a multi-bit output. According to the input parameters, a vectorial Boolean function can be linear or non-linear. Non-linear vectorial Boolean functions have high importance in information theory and cryptology. Finding a linear approximation of a non-linear vectorial Boolean function is of high significance in these areas. Assuming that the quantum access to a vectorial Boolean function is provided, we present a quantum computing approach which can find the best linear approximation of the non-linear vectorial Boolean functions. The presented quantum approach is exponentially better than any known solution on the factors of querying the function, quantum circuit depth and memory requirements.
Deep Local and Global Spatiotemporal Feature Aggregation for Blind Video Quality Assessment

In recent years, deep learning has achieved promising success for multimedia quality assessment, especially for image quality assessment (IQA). However, since there exist more complex temporal characteristics in videos, very little work has been done on video quality assessment (VQA) by exploiting powerful deep convolutional neural networks (DCNNs). In this paper, we propose an efficient VQA method named Deep SpatioTemporal video Quality assessor (DeepSTQ) to predict the perceptual quality of various distorted videos in a no-reference manner. In the proposed DeepSTQ, we first extract local and global spatiotemporal features by pre-trained deep learning models without fine-tuning or training from scratch. The composited features consider distorted video frames as well as frame difference maps from both global and local views. Then, the feature aggregation is conducted by the regression model to predict the perceptual video quality. Finally, experimental results demonstrate that our proposed DeepSTQ outperforms state-of-the-art quality assessment algorithms.
CycleGAN-VC-GP: Improved CycleGAN-based Non-parallel Voice Conversion

Non-parallel voice conversion is an important but challenging task due to the lack of parallel data. Recently, Cycle-Consistent Generative Adversarial Networks based voice conversion (CycleGAN-VC) has achieved great success in non-parallel voice conversion. However, due to the instability of GAN training, there is still a large gap between the real target speech and the converted speech. To reduce the gap, in this paper, CycleGAN-VC-GP is proposed which incorporates two new techniques: (1) zero-centered gradient penalties are used to ensure the convergence of GAN; (2) the fundamental frequency is combined with the spectrum to improve prosody conversion. Both subjective and objective evaluation experiments showed that the proposed method obtained higher speaker similarity and comparable speech quality compared with the state-of-art method based on CycleGAN-VC.
Key-Leakage Resilient Encrypted Data Aggregation With Lightweight Verification in Fog-Assisted Smart Grids

In this article, we analyze the inherent characteristics of smart grids, and point out that some electricity consumption data are very sensitive and should be encrypted. However, once the corresponding private key is compromised, the content of encrypted data would be leaked, thereby violating users' privacy. Additionally, since a control center (CC) is always required to conduct accurate statistic analysis on these data for subsequent services, it is highly demanded for CC to check the integrity of encrypted data. To this end, based on a modified Boneh-Goh-Nissim (BGN) cryptosystem, we propose a key-leakage resilient encrypted data aggregation (KLR-EDA) scheme with lightweight verification in fog-assisted smart grids. KLR-EDA enables each fog node to aggregate first-level verifiable encrypted data from smart meters in the same grid area, and forward them to the cloud server (CS) for long-term storage. Upon receiving flexible challenging list of fog nodes from CC, CS produces second-level verifiable encrypted aggregated data and returns the results to CC. KLR-EDA enables CC to check the integrity of encrypted aggregated data efficiently, and further obtain the statistic analysis results on the aggregated data without learning any information of individual user. In particular, even the private key of CC is exposed or compromised, any adversary cannot break users' privacy. We provide security analysis of KLR-EDA, and conduct performance evaluation to demonstrate its lightweight statistical analysis and verification advantages on the CC side.
Knitted Sensors

Recent work has shown the feasibility of producing knitted capacitive touch sensors through digital fabrication with little human intervention in the textile production process. Such sensors can be designed and manufactured at scale and require only two connection points, regardless of the touch sensor form factor and size of the fabric, opening many possibilities for new designs and applications in textile sensors. To bring this technology closer to real-world use, we go beyond previous work on coarse touch discrimination to enable fine, accurate touch localization on a knitted sensor, using a recognition model able to capture the temporal behavior of the sensor. Moreover, signal acquisition and processing are performed in real-time, using swept frequency Bode analysis to quantify distortion from induced capacitance. After training our network model, we conducted a study with new users, and achieved a subject-independent accuracy of 66% in identifying the touch location on the 36-button sensor, while chance accuracy is approximately 3%. Additionally, we conducted a study demonstrating the viability of taking this solution closer to real-world scenarios by testing the sensor's resistance to potential deformation from everyday conditions. We also introduce several other knitted designs and related application prototypes to explore potential uses of the technology.
A quantum algorithm for spin chemistry: a Bayesian exchange coupling parameter calculator with broken-symmetry wave functions

The Heisenberg exchange coupling parameter J (H = -2JSi · Sj ) characterises the isotropic magnetic interaction between unpaired electrons, and it is one of the most important spin Hamiltonian parameters of multi-spin open shell systems. The J value is related to the energy difference between high-spin and low-spin states, and thus computing the energies of individual spin states are necessary to obtain the J values from quantum chemical calculations. Here, we propose a quantum algorithm, B̲ayesian ex̲change coupling parameter calculator with b̲roken-symmetry wave functions (BxB), which is capable of computing the J value directly, without calculating the energies of individual spin states. The BxB algorithm is composed of the quantum simulations of the time evolution of a broken-symmetry wave function under the Hamiltonian with an additional term jS2, the wave function overlap estimation with the SWAP test, and Bayesian optimisation of the parameter j. Numerical quantum circuit simulations for H2 under a covalent bond dissociation, C, O, Si, NH, OH+, CH2, NF, O2, and triple bond dissociated N2 molecule revealed that the BxB can compute the J value within 1 kcal mol-1 of errors with less computational costs than conventional quantum phase estimation-based approaches.
Change impact analysis: A systematic mapping study

Change Impact Analysis (CIA) is the process of exploring the tentative effects of a change in other parts of a system. CIA is considered beneficial in practice, since it reduces cost of maintenance and the risk of software development failures. In this paper, we present a systematic mapping study that covers a plethora of CIA methods (by exploring 111 papers), putting special emphasis on how the CIA phenomenon can be quantified: to be efficiently managed. The results of our study suggest that: (a) the practical benefits of CIA cover any type of maintenance request (e.g., feature additions, bug fixing) and can help in reducing relevant cost; (b) CIA quantification relies on four parameters (instability, amount of change, change proneness, and changeability), whose assessment is supported by various metrics and predictors; and (c) in this vast research field, there are still some viewpoints that remain unexplored (e.g., the negative consequences of highly change prone artifacts), whereas others are over-researched (e.g., quantification of instability based on metrics). Based on our results, we provide: (a) useful information for practitioners—i.e., the expected benefits of CIA, and a list of CIA-related metrics, emphasizing on the provision of a detailed interpretation of their relation to CIA; and (b) interesting future research directions—i.e., over- and under-researched sub-fields of CIA.
Introspective Failure Prediction for Semantic Image Segmentation

Semantic segmentation of images enables pixel-wise scene understanding which in turn is a critical component for tasks such as autonomous driving. While recent implementations of semantic image segmentation have achieved remarkable accuracy, misclassifications remain inevitable. For safety-critical tasks such as free-space computing, it is desirable to know when and where the segmentation will fail. We propose using the concept of introspection to predict the failures of a given semantic segmentation model. A separate introspective model is trained to predict the errors of a given model. This is accomplished by training the given model with the errors made on a set of previous inputs. By using the same architecture for the introspective model as for the semantic segmentation, the proposed model learns to predict pixel-wise failure probabilities. This allows to predict both when and where the semantic segmentation will fail. Sharing the feature encoder with the inspected model reduces training and inference time while improving performance. We evaluate our approach on the large-scale A2D2 driving data set. In a precision-recall analysis, the proposed method outperforms two state-of-the-art uncertainty estimation methods by 3.2% and 6.7% while requiring significantly less resources during inference. Additionally, combining introspection with a state-of-the-art method further increases the performance by up to 3.7%.
Deep Predictive Video Compression Using Mode-Selective Uni- and Bi-Directional Predictions Based on Multi-Frame Hypothesis

Recently, deep learning-based image compression has shown significant performance improvement in terms of coding efficiency and subjective quality. However, there has been relatively less effort on video compression based on deep neural networks. In this paper, we propose an end-to-end deep predictive video compression network, called DeepPVCnet, using mode-selective uni- and bi-directional predictions based on multi-frame hypothesis with a multi-scale structure and a temporal-context-adaptive entropy model. Our DeepPVCnet jointly compresses motion information and residual data that are generated from the multi-scale structure via the feature transformation layers. Recent deep learning-based video compression methods were proposed in a limited compression environment using only P-frame or B-frame. Learned from the lesson of the conventional video codecs, we firstly incorporate a mode-selective framework into our DeepPVCnet with uni- and bi-directional predictive modes in a rate-distortion minimization sense. Also, we propose a temporal-context-adaptive entropy model that utilizes the temporal context information of the reference frames for the current frame coding. The autoregressive entropy models for CNN-based image and video compression is difficult to compute with parallel processing. On the other hand, our temporal-context-adaptive entropy model utilizes temporally coherent context from the reference frames, so that the context information can be computed in parallel, which is computationally and architecturally advantageous. Extensive experiments show that our DeepPVCnet outperforms AVC/H.264, HEVC/H.265 and state-of-the-art methods in an MS-SSIM perspective.
Multi-frame Recurrent Adversarial Network for Moving Object Segmentation

Moving object segmentation (MOS) in different practical scenarios like weather degraded, dynamic background, etc. videos is a challenging and high demanding task for various computer vision applications. Existing supervised approaches achieve remarkable performance with complicated training or extensive fine-tuning or inappropriate training-testing data distribution. Also, the generalized effect of existing works with completely unseen data is difficult to identify. In this work, the recurrent feature sharing based generative adversarial network is proposed with unseen video analysis. The proposed network comprises of dilated convolution to extract the spatial features at multiple scales. Along with the temporally sampled multiple frames, previous frame output is considered as input to the network. As the motion is very minute between the two consecutive frames, the previous frame decoder features are shared with encoder features recurrently for current frame foreground segmentation. This recurrent feature sharing of different layers helps the encoder network to learn the hierarchical interactions between the motion and appearance-based features. Also, the learning of the proposed network is concentrated in different ways, like disjoint and global training-testing for MOS. An extensive experimental analysis of the proposed network is carried out on two benchmark video datasets with seen and unseen MOS video. Qualitative and quantitative experimental study shows that the proposed network outperforms the existing methods.
Biogeography based optimization for mining rules to assess credit risk

Summary Financial institutions, by and large, rely on the use of machine learning techniques to improve the classic credit risk assessment model for reduction of costs, delivery of faster decisions, guaranteed credit collections, and risk mitigations. As such, several data mining and machine learning approaches have been developed for computation of credit scores over the last few decades. Moreover, the existing rule‐based classification algorithms tend to generate a number of rules with a large number of conditions in the antecedent part. However, these algorithms fail to demonstrate high predictive accuracy while balancing coverage and simplicity. Thus, it becomes quite a challenging task for the researchers to generate an optimal rule set with high predictive accuracy. In this paper, we present an effective rule based classification technique for the prediction of credit risk using a novel Biogeography Based Optimization (BBO) method. The novel BBO in the context of rule mining is named as locally and globally tuned biogeography based rule‐miner (LGBBO‐RuleMiner). This is applied for discovering optimal rule set with high predictive accuracy from the dataset containing both the categorical and continuous attributes. The performance of the proposed algorithm is compared against a variety of rule‐miners such as OneR (1R), PART, JRip, Decision Table, Conjunctive Rule, J48, and Random Tree, along with some meta‐heuristic based rule mining techniques by considering two credit risk datasets obtained from University of California, Irvine (UCI) repository. It is found from the comparative study that the proposed rule miner in ten independent runs of ten‐fold cross validation outperforms all of the aforesaid algorithms in terms of predictive accuracy, coverage, and simplicity.
On Detecting Online Radicalization and Extremism Using Natural Language Processing

Due to the activity of terrorist propaganda on the Internet and social networks, as well as given the high dynamics of the emergence of new sites and accounts of extremist orientation, it is important to quickly detect content that demonstrates a tendency to extremism in the prevention of extremist and terrorist activities. This article is intended to explore the possibilities of automatic recognition of extremist content using machine learning from this point of view. This article is devoted to the application of machine learning methods for solving the problem of security, in part-countering terrorism and extremism using information from the Internet.