Reviewerly/exHarmony huggingface dataset:
Photo-Sketching: Inferring Contour Drawings From Images

Edges, boundaries and contours are important subjects of study in both computer graphics and computer vision. On one hand, they are the 2D elements that convey 3D shapes, on the other hand, they are indicative of occlusion events and thus separation of objects or semantic concepts. In this paper, we aim to generate contour drawings, boundary-like drawings that capture the outline of the visual scene. Prior art often cast this problem as boundary detection. However, the set of visual cues presented in the boundary detection output are different from the ones in contour drawings, and also the artistic style is ignored. We address these issues by collecting a new dataset of contour drawings and proposing a learning-based method that resolves diversity in the annotation and, unlike boundary detectors, can work with imperfect alignment of the annotation and the actual ground truth. Our method surpasses previous methods quantitatively and qualitatively. Surprisingly, when our model fine-tunes on BSDS500, we achieve the state-of-the-art performance in salient boundary detection, suggesting contour drawing might be a scalable alternative to boundary annotation, which at the same time is easier and more interesting for annotators to draw.
Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering

Text-based Question Generation (QG) aims at generating natural and relevant questions that can be answered by a given answer in some context.Existing QG models suffer from a "semantic drift" problem, i.e., the semantics of the model-generated question drifts away from the given context and answer.In this paper, we first propose two semantics-enhanced rewards obtained from downstream question paraphrasing and question answering tasks to regularize the QG model to generate semantically valid questions.Second, since the traditional evaluation metrics (e.g., BLEU) often fall short in evaluating the quality of generated questions, we propose a QA-based evaluation method which measures the QG model's ability to mimic human annotators in generating QA training data.Experiments show that our method achieves the new state-of-theart performance w.r.t.traditional metrics, and also performs best on our QA-based evaluation metrics.Further, we investigate how to use our QG model to augment QA datasets and enable semi-supervised QA.We propose two ways to generate synthetic QA pairs: generate new questions from existing articles or collect QA pairs from new articles.We also propose two empirically effective strategies, a data filter and mixing mini-batch training, to properly use the QG-generated data for QA.Experiments show that our method improves over both BiDAF and BERT QA baselines, even without introducing new articles. 1
Learning Sparse Sharing Architectures for Multiple Tasks

Most existing deep multi-task learning models are based on parameter sharing, such as hard sharing, hierarchical sharing, and soft sharing. How choosing a suitable sharing mechanism depends on the relations among the tasks, which is not easy since it is difficult to understand the underlying shared factors among these tasks. In this paper, we propose a novel parameter sharing mechanism, named Sparse Sharing. Given multiple tasks, our approach automatically finds a sparse sharing structure. We start with an over-parameterized base network, from which each task extracts a subnetwork. The subnetworks of multiple tasks are partially overlapped and trained in parallel. We show that both hard sharing and hierarchical sharing can be formulated as particular instances of the sparse sharing framework. We conduct extensive experiments on three sequence labeling tasks. Compared with single-task models and three typical multi-task learning baselines, our proposed approach achieves consistent improvement while requiring fewer parameters.
A combined multiple action recognition and summarization for surveillance video sequences

Abstract Human action recognition and video summarization represent challenging tasks for several computer vision applications including video surveillance, criminal investigations, and sports applications. For long videos, it is difficult to search within a video for a specific action and/or person. Usually, human action recognition approaches presented in the literature deal with videos that contain only a single person, and they are able to recognize his action. This paper proposes an effective approach to multiple human action detection, recognition, and summarization. The multiple action detection extracts human bodies’ silhouette, then generates a specific sequence for each one of them using motion detection and tracking method. Each of the extracted sequences is then divided into shots that represent homogeneous actions in the sequence using the similarity between each pair frames. Using the histogram of the oriented gradient (HOG) of the Temporal Difference Map (TDMap) of the frames of each shot, we recognize the action by performing a comparison between the generated HOG and the existed HOGs in the training phase which represents all the HOGs of many actions using a set of videos for training. Also, using the TDMap images we recognize the action using a proposed CNN model. Action summarization is performed for each detected person. The efficiency of the proposed approach is shown through the obtained results for mainly multi-action detection and recognition.
Proxy re-encryption enabled secure and anonymous IoT data sharing platform based on blockchain

Data is central to the Internet of Things (IoT) ecosystem. With billions of devices connected, most of the current IoT systems are using centralized cloud-based data sharing systems, which will be difficult to scale up to meet the demands of future IoT systems. The involvement of such a third-party service provider requires also trust from both the sensor owner and sensor data user. Moreover, fees need to be paid for their services. To tackle both the scalability and trust issues and to automatize the payments, this paper presents a blockchain-based marketplace for sharing of the IoT data. We also use a proxy re-encryption scheme for transferring the data securely and anonymously, from data producer to the consumer. The system stores the IoT data in cloud storage after encryption. To share the collected IoT data, the system establishes runtime dynamic smart contracts between the sensor and data consumer without the involvement of a trusted third-party. It also uses a very efficient proxy re-encryption scheme which allows that the data is only visible by the owner and the person present in the smart contract. This novel combination of smart contracts with proxy re-encryption provides an efficient, fast and secure platform for storing, trading and managing sensor data. The proposed system is implemented using off-the-shelf IoT sensors and computer devices. We also analyze the performance of our hybrid system by using the permission-less Ethereum blockchain and compare it to the IBM Hyperledger Fabric, a permissioned blockchain.
HRDNet: High-Resolution Detection Network for Small Objects

Small object detection is a very challenging yet practical vision task. With deep network-based methods, the contextual information of small objects may disappear when the network goes deeper. An intuitive solution to alleviate this issue is to increase the input resolution, however, it will aggravate the large variant of object scale and introduce unbearable computation cost. To leverage the benefits of high-resolution images without bringing up new problems, we propose a High-Resolution Detection Network (HRDNet) which takes multiple resolution inputs with multi-depth backbones. Meanwhile, we propose the Multi-Depth Image Pyramid Network (MD-IPN) and Multi-Scale Feature Pyramid Network (MS-FPN). The MD-IPN maintains multiple position information using multiple depth backbones. Specifically, high-resolution input will be fed into a shallow network to reserve more positional information and reduce computational costs, while low-resolution input will be fed into a deep network to extract more semantics. By extracting various features from high to low resolutions, the MD-IPN can improve the performance of small object detection and maintain the performance of middle and large objects. Additionally, MS-FPN is introduced to align and fuse multi-scale feature groups generated by MD-IPN to reduce the information imbalance. Extensive experiments are conducted on the COCO2017 and the typical small object dataset, VisDrone 2019. Notably, our HRDNet achieves the state-of-the-art on these two datasets with significant improvements on small objects.
Bridging Text Visualization and Mining: A Task-Driven Survey

Visual text analytics has recently emerged as one of the most prominent topics in both academic research and the commercial world. To provide an overview of the relevant techniques and analysis tasks, as well as the relationships between them, we comprehensively analyzed 263 visualization papers and 4,346 mining papers published between 1992-2017 in two fields: visualization and text mining. From the analysis, we derived around 300 concepts (visualization techniques, mining techniques, and analysis tasks) and built a taxonomy for each type of concept. The co-occurrence relationships between the concepts were also extracted. Our research can be used as a stepping-stone for other researchers to 1) understand a common set of concepts used in this research topic; 2) facilitate the exploration of the relationships between visualization techniques, mining techniques, and analysis tasks; 3) understand the current practice in developing visual text analytics tools; 4) seek potential research opportunities by narrowing the gulf between visualization and mining techniques based on the analysis tasks; and 5) analyze other interdisciplinary research areas in a similar way. We have also contributed a web-based visualization tool for analyzing and understanding research trends and opportunities in visual text analytics.
Multi attention module for visual tracking

We propose a new visual tracking algorithm leveraging multi-level visual attention to take full use of the information during tracking. Visual attention has been widely applied in many visual tasks, such as image captioning and question answering. However, most existing attention models only focus on one or two aspects, ignoring the other useful information in visual tracking. Here, we think there are four main attentional aspects in the tracking task and propose a unified network to leverage multi-level visual attention, which includes layer-wise attention, temporal attention, spatial attention and channel-wise attention. Considering that deep features of different levels may be suitable for different scenarios, we propose to train an attention network in the off-line stage to facilitate feature selection in online tracking. To better exploit the temporal consistency assumption of visual tracking, we implement the attention network with long short term memory (LSTM) units, which are capable of capturing the historical context information to perform more reliable inference at the current time step. Different from the image classification task, background clutter is more complicated in the tracking task. Thus, we purify the features by spatial attention and channel-wise attention to effectively suppress the background noise and highlight the target region. In addition, we also enforce deep feature sharing across target candidates using Region of Interest pooling, allowing the features of all candidates to be extracted in only one forward pass of the DNN. To further improve tracking accuracy, a promoting strategy for trackers with detection results of a generic object detector is proposed, reducing the risk of tracking drifts. The proposed tracking algorithm compares favorably against state-of-the-art methods on three popular benchmark datasets. Extensive experimental evaluations demonstrate the effectiveness of the proposed techniques.
Deep Knockoffs

This paper introduces a machine for sampling approximate model-X knockoffs for arbitrary and unspecified data distributions using deep generative models. The main idea is to iteratively refine a knockoff sampling mechanism until a criterion measuring the validity of the produced knockoffs is optimized; this criterion is inspired by the popular maximum mean discrepancy in machine learning and can be thought of as measuring the distance to pairwise exchangeability between original and knockoff features. By building upon the existing model-X framework, we thus obtain a flexible and model-free statistical tool to perform controlled variable selection. Extensive numerical experiments and quantitative tests confirm the generality, effectiveness, and power of our deep knockoff machines. Finally, we apply this new method to a real study of mutations linked to changes in drug resistance in the human immunodeficiency virus.
A Graph Embedding Framework for Maximum Mean Discrepancy-Based Domain Adaptation Algorithms

Domain adaptation aims to deal with learning problems in which the labeled training data and unlabeled testing data are differently distributed. Maximum mean discrepancy (MMD), as a distribution distance measure, is minimized in various domain adaptation algorithms for eliminating domain divergence. We analyze empirical MMD from the point of view of graph embedding. It is discovered from the MMD intrinsic graph that, when the empirical MMD is minimized, the compactness within each domain and each class is simultaneously reduced. Therefore, points from different classes may mutually overlap, leading to unsatisfactory classification results. To deal with this issue, we present a graph embedding framework with intrinsic and penalty graphs for MMD-based domain adaptation algorithms. In the framework, we revise the intrinsic graph of MMD-based algorithms such that the within-class scatter is minimized, and thus, the new features are discriminative. Two strategies are proposed. Based on the strategies, we instantiate the framework by exploiting four models. Each model has a penalty graph characterizing certain similarity property that should be avoided. Comprehensive experiments on visual cross-domain benchmark datasets demonstrate that the proposed models can greatly enhance the classification performance compared with the state-of-the-art methods.
Investigating Multilingual NMT Representations at Scale

Multilingual Neural Machine Translation (NMT) models have yielded large empirical success in transfer learning settings.However, these black-box representations are poorly understood, and their mode of transfer remains elusive.In this work, we attempt to understand massively multilingual NMT representations (with 103 languages) using Singular Value Canonical Correlation Analysis (SVCCA), a representation similarity framework that allows us to compare representations across different languages, layers and models.Our analysis validates several empirical results and long-standing intuitions, and unveils new observations regarding how representations evolve in a multilingual translation model.We draw three major conclusions from our analysis, with implications on cross-lingual transfer learning: (i) Encoder representations of different languages cluster based on linguistic similarity, (ii) Representations of a source language learned by the encoder are dependent on the target language, and vice-versa, and (iii) Representations of high resource and/or linguistically similar languages are more robust when fine-tuning on an arbitrary language pair, which is critical to determining how much cross-lingual transfer can be expected in a zero or few-shot setting.We further connect our findings with existing empirical observations in multilingual NMT and transfer learning.
Practical Obstacles to Deploying Active Learning

Active learning (AL) is a widely-used training strategy for maximizing predictive performance subject to a fixed annotation budget.In AL one iteratively selects training examples for annotation, often those for which the current model is most uncertain (by some measure).The hope is that active sampling leads to better performance than would be achieved under independent and identically distributed (i.i.d.) random samples.While AL has shown promise in retrospective evaluations, these studies often ignore practical obstacles to its use.In this paper we show that while AL may provide benefits when used with specific models and for particular domains, the benefits of current approaches do not generalize reliably across models and tasks.This is problematic because in practice one does not have the opportunity to explore and compare alternative AL strategies.Moreover, AL couples the training dataset with the model used to guide its acquisition.We find that subsequently training a successor model with an actively-acquired dataset does not consistently outperform training on i.i.d.sampled data.Our findings raise the question of whether the downsides inherent to AL are worth the modest and inconsistent performance gains it tends to afford.
Enhanced digital chaotic maps based on bit reversal with applications in random bit generators

Digital chaotic maps are becoming increasingly popular in the area of cryptography due to commonalities but have drawbacks which adversely effect security strength. Thus, enhancing digital chaotic maps in terms of their chaoticity and statistical properties contributes towards the improvement of chaos-based cryptography. This paper proposes a bit reversal approach to address these issues. The proposed method modifies chaotic state values (represented as fixed point numbers) by reversing the order of their fractional bits. Experimental verification indicates that chaotic maps modified by the proposed approach depict better chaotic performance, have higher complexity and larger chaotic parameter range. These results exceed those of existing digital chaotic maps and other chaotification methods. The simplicity of the proposed bit reversal approach and the use of fixed point representation makes it easy to implement on any computing platform. This approach is also highly flexible as it does not require any external inputs, making it a universal method for enhancing any digital chaotic map. As a proof-of-concept, a pseudorandom bit generator (PRBG) was designed based on cascading chaotic maps modified by the proposed method. Simulation and security analysis indicate that the proposed PRBG is statistically random, has a uniform data distribution and high key sensitivity.
Convolutional Neural Network-Based Radar Jamming Signal Classification With Sufficient and Limited Samples

Jamming is a big threat to radar system survival and anti-jamming is a part of the solution. The classification of radar jamming signal is the first step toward to anti-jamming. Recently, as an important part of deep learning, convolutional neural network (CNN) based methods have shown their capability in discriminant feature extraction and accurate classification. In this study, in order to harness the powerfulness of deep learning, CNN based methods are proposed to classify radar jamming signal acting on pulse compression radar. Specifically, a 1D-CNN is designed for radar jamming signal classification under the condition of sufficient training samples. Furthermore, due to the fact that the collection of sufficient training samples is time-consuming and expensive, a CNN-based siamese network is proposed for radar jamming signal classification to deal with the issue of limited training samples. The experimental results with sufficient and limited training samples show that the CNN-based classification methods obtain good classification performance in terms of classification accuracy and show a huge potential for radar jamming signal classification.
Whole Slide Imaging: Technology and Applications

Pathology has benefited from advanced innovation with novel technology to implement a digital solution. Whole slide imaging is a disruptive technology where glass slides are scanned to produce digital images. There have been significant advances in whole slide scanning hardware and software that have allowed for ready access of whole slide images. The digital images, or whole slide images, can be viewed comparable to glass slides in a microscope, as digital files. Whole slide imaging has increased in adoption among pathologists, pathology departments, and scientists for clinical, educational, and research initiatives. Worldwide usage of whole slide imaging has grown significantly. Pathology regulatory organizations (ie, College of American Pathologists) have put forth guidelines for clinical validation, and the US Food and Drug Administration have also approved whole slide imaging for primary diagnosis. This article will review the digital pathology ecosystem and discuss clinical and nonclinical applications of its use.
Intensive Care Unit Mortality Prediction: An Improved Patient-Specific Stacking Ensemble Model

The intensive care unit (ICU) admits the most seriously ill patients requiring extensive monitoring.Early ICU mortality prediction is crucial for identifying patients who are at great risk of dying and for providing suitable interventions to save their lives.Accordingly, early prediction of patients at high mortality risk will enable their provision of appropriate and timely medical services.Although various severity scores and machine-learning models have recently been developed for early mortality prediction, such prediction remains challenging.This paper proposes a novel stacking ensemble approach to predict the mortality of ICU patients.Our approach is more accurate and medically intuitive compared to the literature work.Data were prepared and feature selection was processed under the supervision of the ICU domain expert.The data were split into six modalities based on the expert's decisions.For the prediction process, a separate classifier was selected for each modality based on the performance of the classifiers.We utilized the most popular and diverse classifiers in the literature, including linear discriminant analysis, decision tree (DT), multilayer perceptron, k-nearest neighbor, and logistic regression (LR).Then, a stacking ensemble classifier was constructed and optimized based on the fusion of these five classifier decisions.
Experimental multiphase estimation on a chip

Multiparameter estimation is a general problem that aims at measuring unknown physical quantities, obtaining high precision in the process.In this context, the adoption of quantum resources promises a substantial boost in the achievable performances with respect to the classical case.However, several open problems remain to be addressed in the multiparameter scenario.A crucial requirement is the identification of suitable platforms to develop and experimentally test novel efficient methodologies that can be employed in this general framework.We report the experimental implementation of a reconfigurable integrated multimode interferometer designed for the simultaneous estimation of two optical phases.We verify the high-fidelity operation of the implemented device, and demonstrate quantum-enhanced performances in two-phase estimation with respect to the best classical case, post-selected to the number of detected coincidences.This device can be employed to test general adaptive multiphase protocols due to its high reconfigurability level, and represents a powerful platform to investigate the multiparameter estimation scenario.