Reviewerly/exHarmony huggingface dataset:
The Influence of Image Search Intents on User Behavior and Satisfaction

Understanding search intents behind queries is of vital importance for improving search performance or designing better evaluation metrics. Although there exist many efforts in Web search user intent taxonomies and investigating how users' interaction behaviors vary with the intent types, only a few of them have been made specifically for the image search scenario. Different from previous works which investigate image search user behavior and task characteristics based on either lab studies or large scale log analysis, we conducted a field study which lasts one month and involves 2,040 search queries from 555 search tasks. By this means, we collected relatively large amount of practical search behavior data with extensive first-tier annotation from users. With this data set, we investigate how various image search intents affect users' search behavior, and try to adopt different signals to predict search satisfaction under the certain intent. Meanwhile, external assessors were also employed to categorize each search task using four orthogonal intent taxonomies. Based on the hypothesis that behavior is dependent of task type, we analyze user search behavior on the field study data, examining characteristics of the session, click and mouse patterns. We also link the search satisfaction prediction to image search intent, which shows that different types of signals play different roles in satisfaction prediction as intent varies. Our findings indicate the importance of considering search intent in user behavior analysis and satisfaction prediction in image search.
A Novel Steganography for Spatial Color Images Based on Pixel Vector Cost

Steganography aims to conceal secret data into common media, and steganalysis takes the adversarial position by trying to reveal embedding traces. Most of the existing steganographic schemes for spatial images are designed for gray-scale images. Their extensions to color images are often simply processed by treating each color channel as a single gray image and distributing payload uniformly to different color channels. However, color images are commonly used in daily life, and some effective steganalytic methods recently proposed for color images have improved detection performance with features designed by taking color correlations into consideration. In this paper, we propose a novel steganographic scheme for spatial color images by considering color pixel vectors (CPVs), which are composed of color components in the same spatial location, as embedding units. Embedding costs are directly defined on CPVs, and therefore, intricate relationships among color channels can be explicitly considered, and embedding payload is adaptively assigned to all three color channels. To further enhance the performance, an adapted clustering modification directions strategy is incorporated. The experimental results show that the proposed scheme can effectively resist the steganalytic methods, especially with color-rich model features.
bnclassify: Learning Bayesian Network Classifiers

The bnclassify package provides state-of-the art algorithms for learning Bayesian network classifiers from data.For structure learning it provides variants of the greedy hill-climbing search, a well-known adaptation of the Chow-Liu algorithm and averaged one-dependence estimators.It provides Bayesian and maximum likelihood parameter estimation, as well as three naive-Bayesspecific methods based on discriminative score optimization and Bayesian model averaging.The implementation is efficient enough to allow for time-consuming discriminative scores on mediumsized data sets.The bnclassify package provides utilities for model evaluation, such as cross-validated accuracy and penalized log-likelihood scores, and analysis of the underlying networks, including network plotting via the Rgraphviz package.It is extensively tested, with over 200 automated tests that give a code coverage of 94%.Here we present the main functionalities, illustrate them with a number of data sets, and comment on related software.
A Streaming Cloud Platform for Real-Time Video Processing on Embedded Devices

Real-time intelligent video processing on embedded devices with low power consumption can be useful for applications like drone surveillance, smart cars, and more. However, the limited resources of embedded devices is a challenging issue for effective embedded computing. Most of the existing work on this topic focuses on single device based solutions, without the use of cloud computing mechanisms for parallel processing to boost performance. In this paper, we propose a cloud platform for real-time video processing based on embedded devices. Eight NVIDIA Jetson TX1 and three Jetson TX2 GPUs are used to construct a streaming embedded cloud platform (SECP), on which Apache Storm is deployed as the cloud computing environment for deep learning algorithms (Convolutional Neural Networks - CNNs) to process video streams. Additionally, self-managing services are designed to ensure that this platform can run smoothly and stably, in the form of a metric sensor, a bottleneck detector and a scheduler. This platform is evaluated in terms of processing speed, power consumption, and network throughput by running various deep learning algorithms for object detection. The results show the proposed platform can run deep learning algorithms on embedded devices while meeting the high scalability and fault tolerance required for real-time video processing.
Sparse L1-norm two dimensional linear discriminant analysis via the generalized elastic net regularization

Linear discriminant analysis (LDA) and two dimensional LDA (2DLDA) are widely applied methods for dimensionality reduction. However, both of them lack of robustness and sparseness. Recent studies show that the elastic net and the L1-norm would improve the learning ability of dimensionality reduction. In this paper, we propose a generalized elastic net and apply it into the L1-norm based LDA and 2DLDA to extend LDA and 2DLDA with robustness and sparseness simultaneously (named LDAL1-S and 2DLDAL1-S, respectively). The generalized elastic net also helps LDAL1-S and 2DLDAL1-S avoiding the singularity problem. Moreover, the Lp-norm (0 < p ≤ 1) is used in the generalized elastic net, which makes LDAL1-S and 2DLDAL1-S realize the desired sparseness of the discriminant vectors by selecting proper p. Both LDAL1-S and 2DLDAL1-S are solved through a series of convex problems with equality constraints, with a closed solution for each single problem. Experimental results on three contaminated human face databases show the effectiveness of the proposed methods.
Multiobjective Evolutionary Optimization Based on Fuzzy Multicriteria Evaluation and Decomposition for Image Matting

Image matting is evolving for a wide range of applications including image/video editing. Sampling-based image matting aims to estimate the opacity of foreground objects by properly selecting a pair of foreground and background pixels for every unknown pixel. Sampling-based image matting is essentially an uncertain multicriteria optimization problem (UMCOP). It shows unique advantages in parallelization and handling spatially disconnected regions. However, sampling-based approaches encounter difficulty in accurately evaluating pixel pairs and efficiently optimizing the large-scale UMCOP. To address these two problems, a fuzzy multicriteria evaluation (FMCE) and a multiobjective evolutionary algorithm based on multicriteria decomposition (MOEA-MCD) are proposed. We model three fuzzy membership functions for three selection criteria and aggregate them by Einstein and averaging operators providing FMCE for pixel pairs. MOEA-MCD uses the heuristic information for each criterion by multicriteria decomposition that divides the single objective into multiple objectives and optimizes them simultaneously using a multiobjective optimizer with neighborhood grouping strategy. Experimental results show that FMCE accurately evaluates pixel pairs even in uncertain cases with low satisfaction degree of some evaluation criteria, and the heuristic information for each criterion enhances the population diversity of MOEA-MCD. MOEA-MCD outperforms state-of-the-art large-scale optimization approaches and sampling-based image matting approaches.
Visual assessment of multi-photon interference

Classical machine learning algorithms can provide insights on high-dimensional processes that are hardly accessible with conventional approaches. As a notable example, t-distributed Stochastic Neighbor Embedding (t-SNE) represents the state of the art for visualization of data sets of large dimensionality. An interesting question is then if this algorithm can provide useful information also in quantum experiments with very large Hilbert spaces. Leveraging these considerations, in this work we apply t-SNE to probe the spatial distribution of n-photon events in m-dimensional Hilbert spaces, showing that its findings can be beneficial for validating genuine quantum interference in boson sampling experiments. In particular, we find that nonlinear dimensionality reduction is capable to capture distinctive features in the spatial distribution of data related to multi-photon states with different evolutions. We envisage that this approach will inspire further theoretical investigations, for instance for a reliable assessment of quantum computational advantage.
Initializing a permutation-invariant quantum error-correction code

Recently, there has been growing interest in using quantum error correction in practical devices. A central issue in quantum error correction is the initialization of quantum data into a quantum error-correction code. Most studies have concentrated on generating quantum codes based on their encoding quantum circuits. However, this often leads to a large number of steps required in the initialization, and hence this process can be prone to errors. The purpose of this work is to demonstrate that permutation-invariant quantum error-correction codes can be created with high fidelity by exploiting their underlying symmetry. The code is initialized on multiple qubits that mutually interact or are themselves coupled to a quantum harmonic oscillator. We show that the so-called selective resonant interaction is derivable on such physical systems. By utilizing the selective resonant interaction, these highly symmetric codes may be rapidly generated with excellent fidelity. We also discuss the potential of initializing permutation-invariant quantum error-correction codes based on the state-of-art experimental techniques.
Multilayer Perceptron Neural Network-Based QoS-Aware, Content-Aware and Device-Aware QoE Prediction Model: A Proposed Prediction Model for Medical Ultrasound Streaming Over Small Cell Networks

This paper presents a QoS-aware, content-aware and device-aware nonintrusive medical QoE (m-QoE) prediction model over small cell networks. The proposed prediction model utilises a Multilayer Perceptron (MLP) neural network to predict m-QoE. It also acts as a platform to maintain and optimise the acceptable diagnostic quality through a device-aware adaptive video streaming mechanism. The proposed model is trained for an unseen dataset of input variables such as QoS, content features and display device characteristics, to produce an output value in the form of m-QoE (i.e. MOS). The efficiency of the proposed model is validated through subjective tests carried by medical experts. The prediction accuracy obtained via the correlation coefficient and Root Mean-Square-Error (RMSE) indicates that the proposed model succeeds in measuring m-QoE closer to the visual perception of the medical experts. Furthermore, we have addressed two main research questions: (1) How significant is ultrasound video content type in determining m-QoE? (2) How much of a role does the screen size and device resolution play in medical experts’ diagnostic experience? The former is answered through the content classification of ultrasound video sequences based on their spatiotemporal features, by including these features in the proposed prediction model, and validating their significance through medical experts’ subjective ratings. The latter is answered by conducting a novel subjective experiment of the ultrasound video sequences across multiple devices.
Image retrieval of wool fabric. Part I: Based on low-level texture features

With huge and ever-growing products in the factory, image retrieval can help the worker retrieve the same, or similar, existing products rapidly and accurately to guide production. In this paper, an effective method based on Fourier transform and local binary pattern is proposed to improve the retrieval efficiency of wool fabric. After capturing the fabric image, histogram equalization was implemented on the value of the Hue, Saturation, Value (HSV) mode to enhance the contrast. Subsequently, Fourier transform together with local binary pattern operator were performed to obtain the frequency spectrum and the local binary pattern, respectively. Each frequency spectrum was divided into 22 rings with the same width, and the standard deviation of the frequencies in each ring was calculated as a Fourier feature. Distinct output values of each local binary pattern were counted and normalized as local binary pattern features. Finally, Euclidean distance was adopted to measure the similarity based on the Fourier feature and local binary pattern feature. Twenty thousand wool fabric images were captured to demonstrate the efficacy of the proposed method. Experimental results indicate that the framework is effective and superior for image retrieval of wool fabric, providing referential assistance for the worker in the factory and improving retrieval efficiency.
Review Response Generation in E-Commerce Platforms with External Product Information

''User reviews” are becoming an essential component of e-commerce. When buyers write a negative or doubting review, ideally, the sellers need to quickly give a response to minimize the potential impact. When the number of reviews is growing at a frightening speed, there is an urgent need to build a response writing assistant for customer service providers. In order to generate high-quality responses, the algorithm needs to consume and understand the information from both the original review and the target product. The classical sequence-to-sequence (Seq2Seq) methods can hardly satisfy this requirement. In this study, we propose a novel deep neural network model based on the Seq2Seq framework for the review response generation task in e-commerce platforms, which can incorporate product information by a gated multi-source attention mechanism and a copy mechanism. Moreover, we employ a reinforcement learning technique to reduce the exposure bias problem. To evaluate the proposed model, we constructed a large-scale dataset from a popular e-commerce website, which contains product information. Empirical studies on both automatic evaluation metrics and human annotations show that the proposed model can generate informative and diverse responses, significantly outperforming state-of-the-art text generation models.
Dual Zero-Watermarking Scheme for Two-Dimensional Vector Map Based on Delaunay Triangle Mesh and Singular Value Decomposition

Digital watermarking is important for the copyright protection of electronic data, but embedding watermarks into vector maps could easily lead to changes in map precision. Zero-watermarking, a method that does not embed watermarks into maps, could avoid altering vector maps but often lack of robustness. This study proposes a dual zero-watermarking scheme that improves watermark robustness for two-dimensional (2D) vector maps. The proposed scheme first extracts the feature vertices and non-feature vertices of the vector map with the Douglas-Peucker algorithm and subsequently constructs the Delaunay Triangulation Mesh (DTM) to form a topological feature sequence of feature vertices as well as the Singular Value Decomposition (SVD) matrix to form intrinsic feature sequence of non-feature vertices. Next, zero-watermarks are obtained by executing exclusive disjunction (XOR) with the encrypted watermark image under the Arnold scramble algorithm. The experimental results show that the scheme that synthesizes both the feature and non-feature information improves the watermark capacity. Making use of complementary information between feature and non-feature vertices considerably improves the overall robustness of the watermarking scheme. The proposed dual zero-watermarking scheme combines the advantages of individual watermarking schemes and is robust against such attacks as geometric attacks, vertex attacks and object attacks.
Two-step random phase retrieval approach based on Gram-Schmidt orthonormalization and Lissajous ellipse fitting method

To overcome the phase shift error in phase shifting interferometry, a two-step random phase retrieval approach based on Gram-Schmidt (GS) orthonormalization and Lissajous ellipse fitting (LEF) method (GS&LEF) is proposed. It doesn't need pre-filtering, and can obtain relatively accurate phase distribution with only two phase shifted interferograms and less computational time. It is suitable for different background intensity, modulation amplitude distributions and noises. Last but not least, this method is effective for circular, straight or complex fringes. The simulations and experiments verify the correctness and feasibility of GS&LEF.
Real-time dual-sensitive shearography for simultaneous in-plane and out-of-plane strain measurements

A real-time, dual-sensitive shearography system using a single-wavelength laser was developed for simultaneous and dynamic in-plane and out-of-plane strain measurements. The shearography system is capable of measuring crack-tip deformation fields quantitatively. A spatial multiplexing technique based on Fourier transform is employed for simultaneous and dynamic multi-component phase retrieval. Two slit spatial filters and a common-path shearing interferometer are used to obtain an improved phase quality for crack-tip deformation measurements. Mode-I fracture experiments under three-point bending were conducted to validate the feasibility and the capability of this method.
Real-time Controlling Dynamics Sensing in Air Traffic System

In order to obtain real-time controlling dynamics in air traffic system, a framework is proposed to introduce and process air traffic control (ATC) speech via radiotelephony communication. An automatic speech recognition (ASR) and controlling instruction understanding (CIU)-based pipeline is designed to convert the ATC speech into ATC related elements, i.e., controlling intent and parameters. A correction procedure is also proposed to improve the reliability of the information obtained by the proposed framework. In the ASR model, acoustic model (AM), pronunciation model (PM), and phoneme- and word-based language model (LM) are proposed to unify multilingual ASR into one model. In this work, based on their tasks, the AM and PM are defined as speech recognition and machine translation problems respectively. Two-dimensional convolution and average-pooling layers are designed to solve special challenges of ASR in ATC. An encoder–decoder architecture-based neural network is proposed to translate phoneme labels into word labels, which achieves the purpose of ASR. In the CIU model, a recurrent neural network-based joint model is proposed to detect the controlling intent and label the controlling parameters, in which the two tasks are solved in one network to enhance the performance with each other based on ATC communication rules. The ATC speech is now converted into ATC related elements by the proposed ASR and CIU model. To further improve the accuracy of the sensing framework, a correction procedure is proposed to revise minor mistakes in ASR decoding results based on the flight information, such as flight plan, ADS-B. The proposed models are trained using real operating data and applied to a civil aviation airport in China to evaluate their performance. Experimental results show that the proposed framework can obtain real-time controlling dynamics with high performance, only 4% word-error rate. Meanwhile, the decoding efficiency can also meet the requirement of real-time applications, i.e., an average 0.147 real time factor. With the proposed framework and obtained traffic dynamics, current ATC applications can be accomplished with higher accuracy. In addition, the proposed ASR pipeline has high reusability, which allows us to apply it to other controlling scenes and languages with minor changes.
ChaboNet : Design of a deep CNN for prediction of visual saliency in natural video

Prediction of visual saliency in images and video is needed for video understanding, search and retrieval, coding, watermarking and other applications. The majority of prediction models are founded only on “bottom-up” features. Nevertheless, the “top-down” component of human visual attention becomes prevalent as human observers explore the visual scene. Visual saliency which is always a mix of bottom-up and top-down cues can be predicted on the basis of seen data. In this paper, a model of prediction of visual saliency in video on the basis of Deep convolutional neural networks (CNNs) is proposed. A Deep CNN architecture is designed. Various input channels for a CNN architecture are studied: using the known sensitivity of human visual system to residual motion, pixel colour values are completed with residual motion map. The latter is a normalized energy of residual motion in video frames with regard to the estimated global affine motion model. The experiments show that the choice of the input features for the Deep CNN depends on visual task: for highly dynamic content, the proposed model with residual motion is more efficient and gives decent results with relatively shallow Deep architecture.
Personalized Service Recommendation With Mashup Group Preference in Heterogeneous Information Network

In the service network, there exist various objects and rich relations among them.These various objects and rich relations naturally form a heterogeneous information network.Service recommendations can help users to locate their desired services.Previous service recommendation studies mainly aim at homogeneous networks or consider few kinds of relations rather than using the rich heterogeneous information.In this paper, we propose a mashup group preference-based service recommendation method in the heterogeneous information network for mashup creation.First of all, we analyze the historical invocation records between mashups and services and exploit the heterogeneous information to construct diverse meta paths with different semantic meanings.Then, we measure the similarity between the starting object and the ending object from different perspectives and integrate different similarity measures to obtain the hybrid similarity.Next, we introduce group preference to capture the rich interactions among mashups and apply a group preference-based Bayesian personalized ranking algorithm to learn the model.Finally, we recommend a list of personalized ranking services for mashup developers.A series of experiments conducted on a realworld dataset demonstrate the superiority of our proposed approach over other baseline approaches.