Reviewerly/exHarmony huggingface dataset:
Hierarchical Graph Semantic Pooling Network for Multi-modal Community Question Answer Matching

Nowadays, community question answering (CQA) systems have attracted millions of users to share their valuable knowledge. Matching relevant answers for a specific question is a core function of CQA systems. Previous interaction-based matching approaches show promising performance in CQA systems. However, they typically suffer from two limitations: (1) They usually model content as word sequences, which ignores the semantics provided by non-consecutive phrases, long-distance word dependency and visual information. (2) Word-level interactions focus on the distribution of similar words in terms of position, while being agnostic to the semantic-level interactions between questions and answers. To address these limitations, we propose aHierarchical Graph Semantic Pooling Network (HGSPN) to model the hierarchical semantic-level interactions in a unified framework for multi-modal CQA matching. Instead of viewing text content as word sequences, we convert them into graphs, which can model non-consecutive phrases and long-distance word dependency for better obtaining the composition of semantics. In addition, visual content is also modeled into the graphs to provide complementary semantics. A well-designed stacked graph pooling network is proposed to capture the hierarchical semantic-level interactions between questions and answers based on these graphs. A novel convolutional matching network is designed to infer the matching score by integrating the hierarchical semantic-level interaction features. Experimental results on two real-world datasets demonstrate that our model outperforms the state-of-the-art CQA matching models.
Single-shot Semantic Image Inpainting with Densely Connected Generative Networks

Semantic image inpainting - a task to speculate and fill in large missing areas of a natural image, has shown exciting progress with the introduction of generative adversarial networks (GANs). But due to lack of sufficient understanding of semantic and spatial context, existing methods easily generate blurred boundary and distorted structure, which are inconsistent with the surrounding area. In this paper, we propose a new end-to-end framework named Single-shot Densely Connected Generative Network (SSDCGN), which generates visually realistic and semantically distinct pixels for the missing content by a battery of symmetric encoder-decoder groups. To maximize semantic extraction and realize precise spatial context localization, we involve a deeper densely skip connection in our network. Extensive experiments on Paris StreetView and ImageNet datasets show the superiority of our method.
Entity-level stream classification: exploiting entity similarity to label the future observations referring to an entity

Stream classification algorithms traditionally treat arriving instances as independent. However, in many applications, the arriving examples may depend on the “entity” that generated them, e.g. in product reviews or in the interactions of users with an application server. In this study, we investigate the potential of this dependency by partitioning the original stream of instances/“observations” into entity-centric substreams and by incorporating entity-specific information into the learning model. We propose a k-nearest-neighbour-inspired stream classification approach, in which the label of an arriving observation is predicted by exploiting knowledge on the observations belonging to this entity and to entities similar to it. For the computation of entity similarity, we consider knowledge about the observations and knowledge about the entity, potentially from a domain/feature space different from that in which predictions are made. To distinguish between cases where this knowledge transfer is beneficial for stream classification and cases where the knowledge on the entities does not contribute to classifying the observations, we also propose a heuristic approach based on random sampling of substreams using k Random Entities (kRE). Our learning scenario is not fully supervised: after acquiring labels for the initial m observations of each entity, we assume that no additional labels arrive and attempt to predict the labels of near-future and far-future observations from that initial seed. We report on our findings from three datasets.
Tabulating Cybersecurity Solutions for Substations: Towards Pragmatic Design and Planning

Electric substations play an important role in the proper functioning of power grid systems. Recent incidents such as the Ukraine case have demonstrated the high risk of attacks against substations. A wealth of existing and emerging cyber defense mechanisms have been suggested for protecting substations, each with different defense objectives and using a variety of approaches that have their own strength and weakness. In this paper, we propose a pragmatic framework for reasoning about the different mechanisms by allowing critical comparisons of their features and the planning of their composition. The scope of our framework is general and can cover heterogeneous defense mechanisms, such as software integrity and physical zoning. We define specific criteria in our framework to compare those defense mechanisms in order to generate pragmatic deployment suggestions. We demonstrate the benefits of our framework by conducting a case study inspired by the Ukraine attack.
d-Simplexed: Adaptive Delaunay Triangulation for Performance Modeling and Prediction on Big Data Analytics

Big Data processing systems (e.g., Spark) have a number of resource configuration parameters, such as memory size, CPU allocation, and the number of running nodes. Regular users and even expert administrators struggle to understand the mutual relation between different parameter configurations and the overall performance of the system. In this paper, we address this challenge by proposing a performance prediction framework, called <inline-formula><tex-math notation="LaTeX">$d$</tex-math></inline-formula> -Simplexed, to build performance models with varied configurable parameters on Spark. We take inspiration from the field of Computational Geometry to construct a <inline-formula><tex-math notation="LaTeX">$d$</tex-math></inline-formula> -dimensional mesh using Delaunay Triangulation over a selected set of features. From this mesh, we predict execution time for various feature configurations. To minimize the time and resources in building a bootstrap model with a large number of configuration values, we propose an adaptive sampling technique to allow us to collect as few training points as required. Our evaluation on a cluster of computers using WordCount, PageRank, Kmeans, and Join workloads in HiBench benchmarking suites shows that we can achieve less than 5 percent error rate for estimation accuracy by sampling less than 1 percent of data.
Neural Storyboard Artist

A storyboard is a sequence of images to illustrate a story containing multiple sentences, which has been a key process to create different story products. In this paper, we tackle a new multimedia task of automatic storyboard creation to facilitate this process and inspire human artists. Inspired by the fact that our understanding of languages is based on our past experience, we propose a novel inspire-and-create framework with a story-to-image retriever that selects relevant cinematic images for inspiration and a storyboard creator that further refines and renders images to improve the relevancy and visual consistency. The proposed retriever dynamically employs contextual information in the story with hierarchical attentions and applies dense visual-semantic matching to accurately retrieve and ground images. The creator then employs three rendering steps to increase the flexibility of retrieved images, which include erasing irrelevant regions, unifying styles of images and substituting consistent characters. We carry out extensive experiments on both in-domain and out-of-domain visual story datasets. The proposed model achieves better quantitative performance than the state-of-the-art baselines for storyboard creation. Qualitative visualizations and user studies further verify that our approach can create high-quality storyboards even for stories in the wild.
Efficient and effective OCR engine training

Abstract We present an efficient and effective approach to train OCR engines using the Aletheia document analysis system. All components required for training are seamlessly integrated into Aletheia: training data preparation, the OCR engine’s training processes themselves, text recognition, and quantitative evaluation of the trained engine. Such a comprehensive training and evaluation system, guided through a GUI, allows for iterative incremental training to achieve best results. The widely used Tesseract OCR engine is used as a case study to demonstrate the efficiency and effectiveness of the proposed approach. Experimental results are presented validating the training approach with two different historical datasets, representative of recent significant digitisation projects. The impact of different training strategies and training data requirements is presented in detail.
Velocity refinement of PIV using global optical flow

In this study, we propose a method to enhance the particle image velocimetry (PIV) velocity resolution using global optical flow along with image warping. A global optical flow formula proposed by Brox et al. (High accuracy optical flow estimation based on a theory for warping. In: Proceedings of the 8th European conference on computer vision, vol 4, pp 25–36, 2004) is adopted to compensate the intensity changes of PIV image pairs, which depend on the set-up and synchronization of a laser and a camera. The proposed method is quantitatively evaluated and validated using synthetic particle image pairs generated for Rankine vortices and reference DNS-based velocity data. The proposed method outperforms the conventional PIV method in capturing small scale vortex and turbulent structures due to its enhanced spatial resolution. In addition, the proposed method shows good performance in large displacement fields and varying image intensity whereas optical flow is applicable to small displacement and susceptible to image intensity variation in general. Finally, the proposed method is applied to real PIV particle images of a multiple rectangular jet flow. The results show that the proposed method successfully works out high-resolution fluid mechanical structure and quantities while preserving the conventional PIV results.
Image classification using SLIC superpixel and FAAGKFCM image segmentation

Image classification is one of the popular fields for researchers in computer vision. This study highlights the use of simple linear iterative clustering (SLIC) superpixel in combination with fast and automatically adjustable Gaussian radial basis function kernel-based fuzzy C-means (FAAGKFCM) for image segmentation along with the deep learning techniques. Bag-of-feature with speeded up robust feature along with deep features are used for classification of 101 classes of the image and 256 classes of the image from Caltech 101, Caltech 256 and MIT 67 image datasets. The combination of SLIC superpixel with FAAGKFCM image segmentation acts as the pre-processing step for image classification, which in turn provides a better result in the classification of images. This method has achieved an accuracy of 94% in Caltech 101 dataset, 85% in Caltech 256 dataset and 84% in MIT 67 dataset.
A physics based generative adversarial network for single image defogging

In the field of single image defogging, there are two main methods. One is the image restoration method based on the atmospheric scattering theory which can recover the image texture details well. The other is the image enhancement method based on Retinex theory which can improve the image contrast well. In practice, however, the former can easily lead to low contrast images; the latter is prone to losing texture details. Therefore, how to effectively combine the advantages of both to remove fog is a key issue in the field. In this paper, we have developed a physics based generative adversarial network (PBGAN) to exploit the advantages between those two methods in parallel. To our knowledge, it is the first learning defogging framework that incorporates these two methods and to enable them to work together and complement each other. Our method has two generative adversarial modules, the Contrast Enhancement (CE) module and the Texture Restoration (TR) module. To improve contrast in the CE module, we introduced a novel inversion-adversarial loss and a novel inversion-cycle consistency loss for training the generator. To improve the texture in the TR module, we introduced two convolutional neural networks to learn the atmospheric light coefficient and the transmission map, respectively. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed approach performs better than several state-of-the-art methods quantitatively and qualitatively.
Content Authentication for Neural Imaging Pipelines: End-To-End Optimization of Photo Provenance in Complex Distribution Channels

Forensic analysis of digital photo provenance relies on intrinsic traces left in the photograph at the time of its acquisition. Such analysis becomes unreliable after heavy post-processing, such as down-sampling and re-compression applied upon distribution in the Web. This paper explores end-to-end optimization of the entire image acquisition and distribution workflow to facilitate reliable forensic analysis at the end of the distribution channel. We demonstrate that neural imaging pipelines can be trained to replace the internals of digital cameras, and jointly optimized for high-fidelity photo development and reliable provenance analysis. In our experiments, the proposed approach increased image manipulation detection accuracy from 45% to over 90%. The findings encourage further research towards building more reliable imaging pipelines with explicit provenance-guaranteeing properties.
Information Technology-Based Patent Retrieval Models

This chapter presents information technology ( information technology (IT) ) based patent retrieval models. It first compares and contrasts information retrieval ( information retrieval (IR) ) with patent retrieval, and highlights their key differences. For instance, IR can be considered as a precision-oriented retrieval, whereas patent retrieval can be considered as a recall-oriented retrieval. The chapter then describes the boolean retrieval model, which was designed for IR but can be used for patent retrieval. To facilitate effective patent retrieval, a basic patent retrieval model is presented. With this model, representative keyword terms are extracted from the user query and are ranked according to their importance so that top- $$k$$ relevant patents can be retrieved with irrelevant patents eliminated. Moreover, the chapter also presents some enhancements and extensions to the basic patent retrieval model, which include incorporation of relevance feedback, estimation of the importance of keyword terms, text preprocessing of patent documents, and handling of patent category frequency. In addition, two dynamic patent retrieval models are also described. These two models perform interactive patent retrieval via dispersion or accumulation to dynamically rank the patents. Experimental results with real-life datasets dataset real-life show that the models presented in this chapter outperformed many conventional search systems with respect to time and cost. While this chapter focuses on the theoretical aspects of IT based patent retrieval models which are of interest to IT specialists, practical illustrative examples in the chapter demonstrate the empirical aspects of patent retrieval models which are helpful to IT practitioners.
Versatile Multiple Choice Learning and Its Application to Vision Computing

Most existing ensemble methods aim to train the underlying embedded models independently and simply aggregate their final outputs via averaging or weighted voting. As many prediction tasks contain uncertainty, most of these ensemble methods just reduce variance of the predictions without considering the collaborations among the ensembles. Different from these ensemble methods, multiple choice learning (MCL) methods exploit the cooperation among all the embedded models to generate multiple diverse hypotheses. In this paper, a new MCL method, called vMCL (the abbreviation of versatile Multiple Choice Learning), is developed to extend the application scenarios of MCL methods by ensembling deep neural networks. Our vMCL method keeps the advantage of existing MCL methods while overcoming their major drawback, thus achieves better performance. The novelty of our vMCL lies in three aspects: (1) a choice network is designed to learn the confidence level of each specialist which can provide the best prediction base on multiple hypotheses; (2) a hinge loss is introduced to alleviate the overconfidence issue in MCL settings; (3) Easy to be implemented and can be trained in an end-to-end manner, which is a very attractive feature for many real-world applications. Experiments on image classification and image segmentation task show that vMCL outperforms the existing state-of-the-art MCL methods.
Monocular Viewing Protects Against Cybersickness Produced by Head Movements in the Oculus Rift

We compared the cybersickness produced when a virtual environment (VE) was viewed binocularly and monocularly through an Oculus Rift CV1 head-mounted display (HMD). During each exposure to the VE participants made continuous yaw head movements in time with a computer-generated metronome. Across trials we also varied their head movement frequency (0.5 or 1.0 Hz) and motion-to-photon delays (from ∼5 - ∼212 ms). We found that: 1) cybersickness severity increased with added display lag; and 2) monocular viewing appeared to protect against these increases in cybersickness. We conclude that active binocular viewing with this HMD introduced artifacts that increased the likelihood of more severe sickness.
Quantum Fabry-Perot Resonator: Extreme Angular Selectivity in Matter-Wave Tunneling

The simple quantum setup of the Fabry-Perot interferometer from optics is found to exhibit very strong directional preference for impinging particle beams. Two types of resonances are identified, the different mechanisms of which are demonstrated through the spatial distribution of wave-function magnitude and probability current. These findings offer additional degrees of freedom when designing architectures for a broad range of quantum engineering applications involving beam tunneling, from e-beam steering to matter-wave filtering and quantum sensing. They also provide a set of limits for the angular selectivity that can be achieved with the considered geometry.
Controller synthesis of service contracts with variability

Service contracts characterise the desired behavioural compliance of a composition of services. Compliance is typically defined by the fulfilment of all service requests through service offers, as dictated by a given Service-Level Agreement (SLA). Contract automata are a recently introduced formalism for specifying and composing service contracts. Based on the notion of synthesis of the most permissive controller from Supervisory Control Theory, a safe orchestration of contract automata can be computed that refines a composition into a compliant one. To model more fine-grained SLA and more adaptive service orchestrations, in this paper we endow contract automata with two orthogonal layers of variability: (i) at the structural level, constraints over service requests and offers define different configurations of a contract automaton, depending on which requests and offers are selected or discarded, and (ii) at the behavioural level, service requests of different levels of criticality can be declared, which induces the novel notion of semi-controllability. The synthesis of orchestrations is thus extended to respect both the structural and the behavioural variability constraints. Finally, we show how to efficiently compute the orchestration of all configurations from only a subset of these configurations. A prototypical tool supports the developed theory.
A Blockchain Based Framework for Blood Distribution

Today’s businesses have been evolving along with the technological developments, such as Industry 4.0, internet of things, and blockchain technology, in order to create not only fully digital and traceable but also transparent, reliable and secured environments. Blockchain technology serves for developing the transparency, reliability, and security characteristics of these environments even if it is frequently mentioned along with the monetary systems such as Bitcoin. The most common purposes to adopt the blockchain technology is to redesign and integrate processes, supply chains or financial arrangements. This chapter proposes such an Ethereum blockchain based framework called KanCoin concerning this potential in order to manage and adjust the processes for efficient distribution planning in blood delivery system from donors to distribution centers and patients at medical centers in a more effective way than the conventional procedures.
MOSAIC: Heterogeneity-, Communication-, and Constraint-Aware Model Slicing and Execution for Accurate and Efficient Inference

Heterogeneous embedded systems have surfaced as a promising solution for accurate and efficient deep-learning inference on mobile devices. Despite extensive prior works, it still remains unexplored to investigate the system-software support that efficiently executes inference workloads by judiciously considering their performance and energy heterogeneity, communication overheads, and constraints. To bridge this gap, we propose MOSAIC, heterogeneity-, communication-, and constraint-aware model slicing and execution for accurate and efficient inference on heterogeneous embedded systems. MOSAIC generates the efficient model slicing and execution plan for the target inference workload through dynamic programming. MOSAIC significantly reduces inference latency and energy, exhibits high estimation accuracy, and incurs small overheads.
Privacy Aspects and Subliminal Channels in Zcash

In this paper we analyze two privacy and security issues for the privacy-oriented cryptocurrency Zcash. First we study shielded transactions and show ways to fingerprint user transactions, including active attacks. We introduce two new attacks which we call Danaan-gift attack and Dust attack. Following the recent Sapling update of Zcash protocol we study the interaction between the new and the old zk-SNARK protocols and the effects of their interaction on transaction privacy. In the second part of the paper we check for the presence of subliminal channels in the zk-SNARK protocol and in Pedersen Commitments. We show presence of efficient 70-bit channels which could be used for tagging of shielded transactions which would allow the attacker (malicious transaction verifier) to link transactions issued by a maliciously modified zk-SNARK prover, while would be indistinguishable from regular transactions for the honest verifier/user. We discuss countermeasures against both of these privacy issues.