Reviewerly/exHarmony huggingface dataset:
Efficient Image Encryption Scheme Using Henon Map, Dynamic S-Boxes and Elliptic Curve Cryptography

Image encryption schemes can be vulnerable to a variety of cryptanalysis attacks. The use of key-dependent dynamic S-boxes has been shown to improve security. Threats of chosen-plaintext and chosen-ciphertext attacks still linger. In this paper, we present an efficient algorithm for constructing secure dynamic S-boxes derived from Henon map. We use the proposed dynamic S-box to construct an image encryption scheme that includes a novel combination of security features to resist chosen-plaintext and chosen-ciphertext attacks. Namely, a hash verification step at the end of the decryption procedure effectively thwarts chosen-ciphertext attacks. The hash also serves as an image dependent initialization for the keystream, which together with using an image dependent S-box resist known-plaintext attacks. Furthermore, encryption keys are protected against cryptanalysis using elliptic curve cryptography (ECC).Therefore, the recovery of secret keys is as hard as the elliptic curve discrete logarithm problem even in the unlikely case of the recovery of the temporary S-box or keystream. Our evaluation of the proposed image encryption scheme reveals that it achieves a higher security standard than existing techniques. Moreover, the proposed scheme is computationally efficient with encryption throughput approaching 60 MB/s.
Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering

This paper presents a novel method, termed Bridge to Answer, to infer correct answers for questions about a given video by leveraging adequate graph interactions of heterogeneous crossmodal graphs. To realize this, we learn question conditioned visual graphs by exploiting the relation between video and question to enable each visual node using question-to-visual interactions to encompass both visual and linguistic cues. In addition, we propose bridged visual-to-visual interactions to incorporate two complementary visual information on appearance and motion by placing the question graph as an intermediate bridge. This bridged architecture allows reliable message passing through compositional semantics of the question to generate an appropriate answer. As a result, our method can learn the question conditioned visual representations attributed to appearance and motion that show powerful capability for video question answering. Extensive experiments prove that the proposed method provides effective and superior performance than state-of-the-art methods on several benchmarks.
Bridging the Theoretical Bound and Deep Algorithms for Open Set Domain Adaptation

In the unsupervised open set domain adaptation (UOSDA), the target domain contains unknown classes that are not observed in the source domain. Researchers in this area aim to train a classifier to accurately: 1) recognize unknown target data (data with unknown classes) and 2) classify other target data. To achieve this aim, a previous study has proven an upper bound of the target-domain risk, and the open set difference, as an important term in the upper bound, is used to measure the risk on unknown target data. By minimizing the upper bound, a shallow classifier can be trained to achieve the aim. However, if the classifier is very flexible [e.g., deep neural networks (DNNs)], the open set difference will converge to a negative value when minimizing the upper bound, which causes an issue where most target data are recognized as unknown data. To address this issue, we propose a new upper bound of target-domain risk for UOSDA, which includes four terms: source-domain risk, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -open set difference ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Delta _\epsilon $ </tex-math></inline-formula> ), distributional discrepancy between domains, and a constant. Compared with the open set difference, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Delta _\epsilon $ </tex-math></inline-formula> is more robust against the issue when it is being minimized, and thus we are able to use very flexible classifiers (i.e., DNNs). Then, we propose a new principle-guided deep UOSDA method that trains DNNs via minimizing the new upper bound. Specifically, source-domain risk and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Delta _\epsilon $ </tex-math></inline-formula> are minimized by gradient descent, and the distributional discrepancy is minimized via a novel open set conditional adversarial training strategy. Finally, compared with the existing shallow and deep UOSDA methods, our method shows the state-of-the-art performance on several benchmark datasets, including digit recognition [modified National Institute of Standards and Technology database (MNIST), the Street View House Number (SVHN), U.S. Postal Service (USPS)], object recognition (Office-31, Office-Home), and face recognition [pose, illumination, and expression (PIE)].
A Process-Oriented Approach to Design Rationale

We propose an approach to design rationale (DR) that emphasizes supporting the design process in such a way that a trace of the rationale is captured with little disruption of the normal process. We describe a rhetorical method for design dialogue called IBIS (meaning &quot;issue-based information systems&quot;) and two implementations of this rhetorical method: a graphical hypertext tool for conducting IBIS discussions called gIBIS and a simple indented text notation. We describe a field trial in an industrial setting in which the &quot;low-tech&quot; indented text IBIS was used to capture more than 2,300 requirements and design decisions. We also explore the implications of this experience for the design of computer tools that, like gIBIS, seek to capture DR nonintrusively.
A Primer on PAC-Bayesian Learning

Generalised Bayesian learning algorithms are increasingly popular in machine learning, due to their PAC generalisation properties and flexibility. The present paper aims at providing a self-contained survey on the resulting PAC-Bayes framework and some of its main theoretical and algorithmic developments.
A PSO and BFO-Based Learning Strategy Applied to Faster R-CNN for Object Detection in Autonomous Driving

Recently outstanding object detection results are achieved by the faster region-based convolutional network (faster R-CNN). Particularly, high-quality detection proposals are obtained by the region proposal network (RPN). Nevertheless, part of the parameters in RPN is assigned by prior knowledge. Therefore the underfitting problem is likely to appear on the training model of RPN. In other words, the generalization ability of RPN is not enough. Increasing parameters is an effective solution to this problem. Thereupon a strengthened RPN (SRPN) is designed to expand the exploring space of RPN. Acquiring the optimal parameter values of SRPN is a non-deterministic polynomial-time hard problem, which can be solved by swarm intelligence algorithms. Thereafter a particle swarm optimization (PSO) and bacterial foraging optimization (BFO)-based learning strategy (PBLS) is introduced to optimize the classifier and loss function of SRPN. In SRPN, a novel multi-level extracting network is created to improve the feature sampling ability. Moreover, the mathematical model of the smooth L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> loss function is improved to boost the fitting ability. Additionally, support vector machine (SVM) method is applied to enhance the classifier learning capability. PBLS is applied to SRPN (PBLS_SRPN). The parameters of SVM and the improved loss function are optimized by the BFO and PSO methods, respectively. Then, the performance of SRPN is further promoted. The excellent results are obtained by our proposed methods on PASCAL VOC 2007, 2012, MS COCO, and KITTI data sets. Consequently, PBLS_SRPN is effective for object detection in autonomous driving.
Cryptocurrencies: Herding and the transfer currency

We contribute to the ongoing debate on the existence of herding behavior in the crypto market and provide statistically significant evidence thereof. This finding is in contrast to existing empirical evidence in this field, which is primarily due to previous studies suffering from a sample bias. By introducing the concept of beta herding to the debate, we provide further robustness for our results. Moreover, we propose the concept of Bitcoin as a ‘transfer currency’ and empirically show that herding measures centered around such a transfer currency provide a more precise representation of dispersion in investors beliefs on the crypto market.
Understanding the Origins of Bias in Word Embeddings

The power of machine learning systems not only promises great technical progress, but risks societal harm. As a recent example, researchers have shown that popular word embedding algorithms exhibit stereotypical biases, such as gender bias. The widespread use of these algorithms in machine learning systems, from automated translation services to curriculum vitae scanners, can amplify stereotypes in important contexts. Although methods have been developed to measure these biases and alter word embeddings to mitigate their biased representations, there is a lack of understanding in how word embedding bias depends on the training data. In this work, we develop a technique for understanding the origins of bias in word embeddings. Given a word embedding trained on a corpus, our method identifies how perturbing the corpus will affect the bias of the resulting embedding. This can be used to trace the origins of word embedding bias back to the original training documents. Using our method, one can investigate trends in the bias of the underlying corpus and identify subsets of documents whose removal would most reduce bias. We demonstrate our techniques on both a New York Times and Wikipedia corpus and find that our influence function-based approximations are very accurate.
Presenting hotels in virtual reality: does it influence the booking intention?

Purpose The purpose of this study is to investigate the effects of telepresence while using a smartphone-based virtual reality system (SBVR) to explore a hotel virtually and to determine the influence of this immersive experience on the booking intention of the potential customer. Design/methodology/approach Within the scope of this study, a conceptual research model was developed which covered utilitarian and hedonic aspects of the user experience of SBVRs and showed their relevance for the booking intention. A virtual reality application was programmed especially for the study, in which the test persons were able to virtually explore a hotel complex. A total of 569 people participated in the study. A questionnaire was used for the data collection. The structural equation modelling and hypothesis verification were carried out using the partial least squares method. Findings The immersive feeling of telepresence increases the perceived enjoyment and usefulness of the potential customer. In addition, the user's curiosity is aroused by the telepresence, which also significantly increases the perceived enjoyment as well as the perceived usefulness. The hedonic and utilitarian value of the virtual hotel experience increases the probability that the customer will book the travel accommodation. Research limitations/implications The virtual reality application developed for the study is based on static panoramic images and does not contain audio-visual elements (e.g. sound, video, animation). Audio-visual elements might increase the degree of immersion and could therefore be investigated in future research. Practical implications The results of the study show that the SBVR is a suitable marketing tool to present hotels in an informative and entertaining way, and can thereby increase sales and profits. Originality/value For the first time, this study investigates the potential of SBVRs for the virtual product presentation of hotels and provides empirical evidence that the availability of this innovative form of presentation leads to a higher booking intention.
Robustness to Adversarial Perturbations in Learning from Incomplete Data

What is the role of unlabeled data in an inference problem, when the presumed underlying distribution is adversarially perturbed? To provide a concrete answer to this question, this paper unifies two major learning frameworks: Semi-Supervised Learning (SSL) and Distributionally Robust Learning (DRL). We develop a generalization theory for our framework based on a number of novel complexity measures, such as an adversarial extension of Rademacher complexity and its semi-supervised analogue. Moreover, our analysis is able to quantify the role of unlabeled data in the generalization under a more general condition compared to the existing theoretical works in SSL. Based on our framework, we also present a hybrid of DRL and EM algorithms that has a guaranteed convergence rate. When implemented with deep neural networks, our method shows a comparable performance to those of the state-of-the-art on a number of real-world benchmark datasets.
High-Quality Self-Supervised Deep Image Denoising

We describe a novel method for training high-quality image denoising models based on unorganized collections of corrupted images. The training does not need access to clean reference images, or explicit pairs of corrupted images, and can thus be applied in situations where such data is unacceptably expensive or impossible to acquire. We build on a recent technique that removes the need for reference data by employing networks with a blind spot in the receptive field, and significantly improve two key aspects: image quality and training efficiency. Our result quality is on par with state-of-the-art neural network denoisers in the case of i.i.d. additive Gaussian noise, and not far behind with Poisson and impulse noise. We also successfully handle cases where parameters of the noise model are variable and/or unknown in both training and evaluation data.
Deep HDR Video from Sequences with Alternating Exposures

Abstract A practical way to generate a high dynamic range (HDR) video using off‐the‐shelf cameras is to capture a sequence with alternating exposures and reconstruct the missing content at each frame. Unfortunately, existing approaches are typically slow and are not able to handle challenging cases. In this paper, we propose a learning‐based approach to address this difficult problem. To do this, we use two sequential convolutional neural networks (CNN) to model the entire HDR video reconstruction process. In the first step, we align the neighboring frames to the current frame by estimating the flows between them using a network, which is specifically designed for this application. We then combine the aligned and current images using another CNN to produce the final HDR frame. We perform an end‐to‐end training by minimizing the error between the reconstructed and ground truth HDR images on a set of training scenes. We produce our training data synthetically from existing HDR video datasets and simulate the imperfections of standard digital cameras using a simple approach. Experimental results demonstrate that our approach produces high‐quality HDR videos and is an order of magnitude faster than the state‐of‐the‐art techniques for sequences with two and three alternating exposures.
Global Relational Models of Source Code

Models of code can learn distributed representations of a program's syntax and semantics to predict many non-trivial properties of a program. Recent state-of-the-art models leverage highly structured representations of programs, such as trees, graphs and paths therein (e.g. data-flow relations), which are precise and abundantly available for code. This provides a strong inductive bias towards semantically meaningful relations, yielding more generalizable representations than classical sequence-based models. Unfortunately, these models primarily rely on graph-based message passing to represent relations in code, which makes them de facto local due to the high cost of message-passing steps, quite in contrast to modern, global sequence-based models, such as the Transformer. In this work, we bridge this divide between global and structured models by introducing two new hybrid model families that are both global and incorporate structural bias: Graph Sandwiches, which wrap traditional (gated) graph message-passing layers in sequential message-passing layers; and Graph Relational Embedding Attention Transformers (GREAT for short), which bias traditional Transformers with relational information from graph edge types. By studying a popular, non-trivial program repair task, variable-misuse identification, we explore the relative merits of traditional and hybrid model families for code representation. Starting with a graph-based model that already improves upon the prior state-of-the-art for this task by 20%, we show that our proposed hybrid models improve an additional 10-15%, while training both faster and using fewer parameters.
Point-Voxel CNN for Efficient 3D Deep Learning

We present Point-Voxel CNN (PVCNN) for efficient, fast 3D deep learning. Previous work processes 3D data using either voxel-based or point-based NN models. However, both approaches are computationally inefficient. The computation cost and memory footprints of the voxel-based models grow cubically with the input resolution, making it memory-prohibitive to scale up the resolution. As for point-based networks, up to 80% of the time is wasted on dealing with the sparse data which have rather poor memory locality, not on the actual feature extraction. In this paper, we propose PVCNN that represents the 3D input data in points to reduce the memory consumption, while performing the convolutions in voxels to reduce the irregular, sparse data access and improve the locality. Our PVCNN model is both memory and computation efficient. Evaluated on semantic and part segmentation datasets, it achieves much higher accuracy than the voxel-based baseline with 10× GPU memory reduction; it also outperforms the state-of-the-art point-based models with 7× measured speedup on average. Remarkably, the narrower version of PVCNN achieves 2× speedup over PointNet (an extremely efficient model) on part and scene segmentation benchmarks with much higher accuracy. We validate the general effectiveness of PVCNN on 3D object detection: by replacing the primitives in Frustrum PointNet with PVConv, it outperforms Frustrum PointNet++ by 2.4% mAP on average with 1.5× measured speedup and GPU memory reduction.
A cross-modal adaptive gated fusion generative adversarial network for RGB-D salient object detection

Salient object detection in RGB-D images aims to identify the most attractive objects in a pair of color and depth images for the observer. As an important branch of salient object detection, it focuses on solving the following two major challenges: how to achieve cross-modal fusion that is efficient and beneficial for salient object detection; how to effectively extract the information of depth image with relatively poor quality. This paper proposes a cross-modal adaptive gated fusion generative adversarial network for RGB-D salient object detection by using color and depth images. Specifically, the generator network adopts double-stream encoder-decoder network and receives RGB and depth images at the same time. The proposed depthwise separable residual convolution module is used to deal with deep semantic information, and the processed feature is combined with side-output features of the encoder network progressively. In order to compensate for the shortcoming of poor quality of the depth image, the proposed method adds the cross-modal guidance from the side-output features of the RGB stream to the decoder network of depth stream. The discriminator network adaptively fuses the features of double streams using a gated fusion module, then sends the gated fusion saliency map to the discriminator to distinguish the similarity from ground-truth map. Adversarial learning forms the better generator network and discriminator network, and the gated fusion saliency map generated by the best generator network is served as final result. Experiments on five publicly RGB-D datasets demonstrate the effect of cross-modal fusion, depthwise separable residual convolution and adaptive gated fusion. Compared with the state-of-the-art methods, our method achieves the better performance.
Domain Knowledge Guided Deep Learning with Electronic Health Records

Due to their promising performance in clinical risk prediction with Electronic Health Records (EHRs), deep learning methods have attracted significant interest from healthcare researchers. However, there are 4 challenges: (i) Data insufficiency. Many methods require large amounts of training data to achieve satisfactory results. (ii) Interpretability. Results from many methods are hard to explain to clinicians (e.g., why the models make particular predictions and which events cause clinical outcomes). (iii) Domain knowledge integration. No existing method dynamically exploits complicated medical knowledge (e.g., relations such as cause and is-caused-by between clinical events). (iv) Time interval information. Most existing methods only consider the relative order of visits from EHRs, but ignore the irregular time intervals between neighboring visits. In the study, we propose a new model, Domain Knowledge Guided Recurrent Neural Networks (DG-RNN), by directly introducing domain knowledge from the medical knowledge graph into an RNN architecture, as well as taking the irregular time intervals into account. Experimental results on heart failure risk prediction tasks show that our model not only outperforms state-of-the-art deep-learning based risk prediction models, but also associates individual medical events with heart failure onset, thus paving the way for interpretable accurate clinical risk predictions.
End-to-End Learning Deep CRF Models for Multi-Object Tracking Deep CRF Models

By bundling multiple complex sub-problems into a unified framework, end-to-end deep learning frameworks reduce the need for hand engineering or tuning of parameters for each component, and optimize different modules jointly to ensure the generalization of the whole deep architecture. Despite tremendous success in numerous computer vision tasks, end-to-end learnings for multi-object tracking (MOT), especially for the assignment problem in data association, have been surprisingly less investigated mainly due to the lack of available training data. Furthermore, it is challenging to discriminate target objects under mutual occlusions or to reduce identity switches in crowded scenes. To tackle these challenges, this paper proposes learning deep conditional random field (CRF) networks, aiming to model the assignment costs as unary potentials and the long-term dependencies among detection results as pairwise potentials. Specifically, we use a bidirectional long short-term memory (LSTM) network to encode the long-term dependencies. We pose the CRF inference as a recurrent neural network learning process using the standard gradient descent algorithm, where unary and pairwise potentials are jointly optimized in an end-to-end manner. Extensive experiments are conducted on the challenging MOT datasets including MOT15, MOT16 and MOT17, and the results show that the proposed algorithm performs favorably against the state-of-the-art methods.