Reviewerly/exHarmony huggingface dataset:
Learning to Detect Pedestrian Flow in Traffic Intersections from Synthetic Data

Detecting the pedestrian flow from different directions at a traffic intersection has always been a challenging task. Challenges include different crowd densities, occlusions, lack of available data and so on. Emergence of deep learning and computer vision methods has shown potentials to deal with this problem. Most of the recent works focus on detecting combined pedestrian flow or counting the total number of pedestrians. In this work, we propose to detect not only combined pedestrian flow but also pedestrian flows from different directions on a crosswalk at a traffic intersection. Our main contributions are summarized as follows: 1) we introduce a synthetic dataset that we create using computer game and a real-world dataset we collect from the street; 2) we propose a Pedestrian Counting Network (PCNet) to count pedestrians from different directions along a crosswalk and then propose a Pedestrian Flow Inference Model (PFIM) to infer the pedestrian flow parameters of volume and density; 3) we design a structure-aware domain adaptation for learning from synthetic data to real data. The experimental results show the effectiveness and accuracy of the proposed method.
Optimal Decision Making for Automated Vehicles Using Homotopy Generation and Nonlinear Model Predictive Control

To navigate complex driving scenarios, automated vehicles must be able to make decisions that reflect higher-level goals such as safety and efficiency, leveraging the vehicle's full capabilities if necessary. We introduce an architecture that is capable of handling combinatorial decision making and control with a high fidelity vehicle model. This is accomplished by solving a nonlinear model predictive control optimization for each maneuver variant, or homotopy, identified in the drivable space. These locally optimal solutions are then evaluated on a criterion that reflects high-level objectives. Experimental results on a full-scale vehicle demonstrate this architecture's effectiveness in an overtaking scenario with oncoming traffic that requires the ego vehicle to decide whether to pass before or after the oncoming traffic passes.
Focal Attention Networks: Optimising Attention for Biomedical Image Segmentation

In recent years, there has been a rising interest to incorporate attention into deep learning architectures for biomedical image segmentation. The modular design of attention mechanisms enable flexible integration into convolutional neural network architectures such as the U-Net. Whether attention is appropriate to use, what type of attention to use, and where in the network to incorporate attention modules, are all important considerations that are currently overlooked. In this paper, we investigate the role of the Focal parameter in modulating attention, revealing a link between attention in loss functions and networks. By incorporating a Focal distance penalty term, we extend the Unified Focal loss framework to include boundary-based losses. Furthermore, we develop a simple and interpretable, dataset and model-specific heuristic to integrate the Focal parameter into the Squeeze-and-Excitation block and Attention Gate, achieving the best results with fewer number of attention modules on three well-validated biomedical imaging datasets, suggesting judicious use of attention modules results in better performance and efficiency. The source code is available at: https://github.com/mlyg/focal-attention-networks.
Multilabel Emotion Tagging for Domain-Specific Texts

In this article, we propose a novel approach to bootstrap a general seed emotion lexicon with words found in a domain-specific corpus. The approach divulges the contextual similarity between two words in the corpus via lexical-, dictionary-, and topic-based features, thus revealing the emotion labels of domain-specific words. As unfolding the recursive structure of language is an irreducible component of how humans understand a sentence, in this study, a propagation mechanism is designed that takes advantage of a shallow parser to derive the emotions associated with the words and their parent phrases. This mechanism pushes beyond the limits of most word co-occurrence approaches and facilitates the multilabel emotion tagging of a sentence in a manner reflecting human cognition. Evaluations on two benchmark corpora support the validity of the propagation mechanism. Further evaluation of a financial corpus indicates that our system outperforms the traditional bag-of-words approach. Our approach provides better modeling of compositional emotions by considering the emotion-bearing words, shifters, intensifiers, and overall sentence structure.
Novel Ensemble Diversification Methods for Open-Set Scenarios

We revisit existing ensemble diversification approaches and present two novel diversification methods tailored for open-set scenarios. The first method uses a new loss, designed to encourage models disagreement on outliers only, thus alleviating the intrinsic accuracy-diversity trade-off. The second method achieves diversity via automated feature engineering, by training each model to disregard input features learned by previously trained ensemble models. We conduct an extensive evaluation and analysis of the proposed techniques on seven datasets that cover image classification, re-identification and recognition domains. We compare to and demonstrate accuracy improvements over the existing state-of-the-art ensemble diversification methods.
CSDQA: Diagram Question Answering in Computer Science

Visual Question Answering (VQA) has been a research focus of the computer vision community for recent years. Most of them are accomplished and verified on images of natural scenes. However, Diagram Question Answering (DQA) which is the task of answering natural language questions based on diagram is rarely noticed. Diagram is a more abstract carrier of knowledge and important resource composition in the multi-modal knowledge graph, research on it is of great significance for understanding the cognitive behavior of learners. In order to fill the scarcity of such data, this paper proposes the Computer Science Diagram Question Answering (CSDQA) dataset, which is the first geometric type of diagram dataset in this field. This dataset contains 1,294 diagrams with rich fine-grained annotations and 3,494 question-answer pairs, including multiple choice and true-or-false questions with two levels of difficulty. We have open sourced all the data in http://zscl.xjtudlc.com:888/CSDQA, hoping to provide convenience for researchers and make it the high-quality data foundation of DQA.
Generalized entanglement measure for continuous-variable systems

Concurrence introduced by Hill and Wootters [Phys. Rev. Lett. 78, 5022 (1997)], provides an important measure of entanglement for a general pair of qubits that is strictly positive entangled states and vanishes for all separable states. We present an extension of entanglement measure to general continuous variable states of multiple degrees of freedom by generalizing the Lagrange's identity and wedge product framework proposed by Bhaskara et al. [Quantum Inf. Process. 16, 118 (2017)] for pure discrete variable systems in arbitrary dimensions. A family of faithful entanglement measures is constructed that admit necessary and sufficient conditions for separability across arbitrary bipartitions presented by Vedral et al. [Phys. Rev. Lett. 16 2275 (1997)]. The computed entanglement measure in the present approach for general Gaussian states, pair coherent states and non Gaussian continuous variable Bell states, matches with known results. We also quantify entanglement of phase randomized squeezed states and superposition of squeezed states. Our results also simplify several results in quantum entanglement theory.
Explanation and answers to critiques on: Blockchain-based Privacy-Preserving Record Linkage

The “Blockchain-based Privacy-Preserving Record Linkage—Enhancing Data Privacy in an Untrusted Environment” (BC-PPRL) uses Blockchain technology to provide accountability to the computation performed during the comparison step of PPRL. The BC-PPRL utilizes small fragments (splits) of the encoded records to iterative compute the similarity of the records and classify them into matches and non-matches, without sharing the complete information of the encoded records. Christen et al. propose a novel attack that leverages the exchanged information by the BC-PPRL. In this work, we acknowledge the Christen et al. findings and provide a detailed explanation of how the privacy of BB-PPRL could be compromised. We also make available a simplified version of the BC-PPRL, the datasets, and version (ported to python 3) of the attack that could be executed in the google cloud environment at: https://github.com/thiagonobrega/bcpprl-simplified.
Uncertainty-Based Dynamic Graph Neighborhoods for Medical Segmentation

In recent years, deep learning based methods have shown success in essential medical image analysis tasks such as segmentation. Post-processing and refining the results of segmentation is a common practice to decrease the misclassifications originating from the segmentation network. In addition to widely used methods like Conditional Random Fields (CRFs) which focus on the structure of the segmented volume/area, a graph-based recent approach makes use of certain and uncertain points in a graph and refines the segmentation according to a small graph convolutional network (GCN). However, there are two drawbacks of the approach: most of the edges in the graph are assigned randomly and the GCN is trained independently from the segmentation network. To address these issues, we define a new neighbor-selection mechanism according to feature distances and combine the two networks in the training procedure. According to the experimental results on pancreas segmentation from Computed Tomography (CT) images, we demonstrate improvement in the quantitative measures. Also, examining the dynamic neighbors created by our method, edges between semantically similar image parts are observed. The proposed method also shows qualitative enhancements in the segmentation maps, as demonstrated in the visual results.
Image Processing on Fuzzy Near Sets

Fuzzy image processing involves various fuzzy approaches. These fuzzy approaches help in understanding and processing of images as fuzzy sets. Mathematical morphology is used for analysing the shapes and forms of the objects in images. In morphological methods structuring elements are applied to an input image which creates an output image of the same size. The size and shape of the neighbourhood is chosen in such a way that a morphological operation relating to different shapes in the input image is constructed. In this paper the pixel values of an image are converted to fuzzy near membership values and the corresponding image is found. Using the fuzzy near membership image, gray and binary images are formed using MATLAB. Mathematical morphology operations dilation, erosion, opening and closing for different structuring elements such as line, square, diamond, sphere, disk, cube, neighbourhood, octagon and rectangle are performed for fuzzy membership and binary images. Taking dilation and opening as upper approximations, erosion and closing as lower approximations, the exact measures are calculated for different structuring elements and the corresponding images formed. It is found that the exact measure for the structuring element neighbourhood of fuzzy near membership and binary images are exactly equal. The corresponding images are also exactly similar.
How Realistic Should Synthetic Images Be for Training Crowd Counting Models?

Using synthetic images has been proposed to avoid collecting and manually annotating a sufficiently large and representative training set for several computer vision tasks, including crowd counting. While existing methods for crowd counting are based on generating realistic images, we start investigating how crowd counting accuracy is affected by increasing the realism of synthetic training images. Preliminary experiments on state-of-the-art CNN-based methods, focused on image background and pedestrian appearance, show that realism in both of them is beneficial to a different extent, depending on the kind of model (regression- or detection-based) and on pedestrian size in the images.
Unsupervised Anomaly Approach to Pedestrian Age Classification from Surveillance Cameras Using an Adversarial Model with Skip-Connections

Anomaly detection is an active research area within the machine learning and scene understanding fields. Despite the ambiguous definition, anomaly detection is considered an outlier detection in a given data based on normality constraints. The biggest problem in real-world anomaly detection applications is the high bias of the available data due to the class imbalance, meaning a limited amount of all possible anomalous and normal samples, thus making supervised learning model use difficult. This paper introduces an unsupervised and adversarially trained anomaly model with a unique encoder–decoder structure to address this issue. The proposed model distinguishes different age groups of people—namely child, adult, and elderly—from surveillance camera data in Busan, Republic of Korea. The proposed model has three major parts: a parallel-pipeline encoder with a conventional convolutional neural network and a dilated-convolutional neural network. The latent space vectors created at the end of both networks are concatenated. While the convolutional pipeline extracts local features, the dilated convolutional pipeline extracts the global features from the same input image. Concatenation of these features is sent as the input into the decoder, which has partial skip-connection elements from both pipelines. This, along with the concatenated feature vector, improves feature diversity. The input image is reconstructed from the feature vector through the stacked transpose convolution layers. Afterward, both the original input image and the corresponding reconstructed image are sent into the discriminator and are distinguished as real or fake. The image reconstruction loss and its corresponding latent space loss are considered for the training of the model and the adversarial Wasserstein loss. Only normal-designated class images are used during the training. The hypothesis is that if the model is trained with normal class images, then during the inference, the construction loss will be minimal. On the other hand, if the untrained anomalous class images are input through the model, the reconstruction loss value will be very high. This method is applied to distinguish different age clusters of people using unsupervised training. The proposed model outperforms the benchmark models in both the qualitative and the quantitative measurements.
Quantum Cryptanalysis Landscape of Shor’s Algorithm for Elliptic Curve Discrete Logarithm Problem

Shor’s algorithm is recognized as one of the most influential algorithms that shape the research interest in quantum computation and quantum cryptanalysis today. In particular, the algorithm is one of the firsts to show clear applicability of quantum computation to solve classically intractable problems underpinning the widely used public-key cryptosystems (i.e., RSA and ECC), igniting an ever-growing research interest in the respective fields. However, of the two algorithms introduced by Peter Shor in 1994, i.e., Shor’s quantum factoring algorithm and Shor’s discrete logarithm problem, the latter is not discussed as often. In this paper, we discuss the works done on Shor’s algorithm to solve the elliptic curve discrete logarithm problem. In particular, we introduce the methods and optimizations proposed by researchers from the early era to the most recent ones and map them to obtain a comprehensive overview of the existing works. From our mapping, we analyze some of the trends in optimizations that help carve the research direction in Shor’s algorithm for the elliptic curve discrete logarithm problem.
Comparative Analysis of Lightning's Routing Clients

Lightning Network, the off-chain solution to Bitcoin, uses a combination of source routing, layered encryption and conditional commitments to perform anonymous and atomic multi-hop off-chain payments. The three predominant routing clients used in Lightning - LND, c- Lightning and Eclair - differ in their cost functions for adding channels to a payment path and hence result in differ paths. At the moment, there is no detailed study on how the different path selections affect the quality of the payment in terms of effectiveness and efficiency. Such an evaluation is needed to i) allow users to make informed choices of which client to use, and ii) enable developers of new routing algorithms to take lessons learned from the current designs into account. In this work, we analyze and compare the three predominant routing clients used by Lightning in terms of the path length, maximal delays a payment might experience, fee borne by the sender and success rate of multi-hop transactions. Based on our experiments and results, we recommend LND for users desiring payments with short paths and high success rates, c- Lightning for users desiring paths with low maximal latencies, and Eclair for those looking for cheap paths in terms of fees paid to the nodes that route payments as intermediaries. We further give guidelines for the design of future routing algorithms based on our insights gained from Lightning's current routing.
Hybrid Environment Reconstruction Improving User Experience and Workload in Augmented Virtuality Teleoperation

Concurrent with autonomous robots, teleoperation gains importance in industrial applications. This includes remote intervention scenarios and human-robot-cooperation during complex or harmful tasks operated from a distance. For an efficient teleoperation, an interface achieving a good user experience and relieving workload of the operators is decisive. Whereas virtual reality allows a spatial integration, environment reconstruction is often based on point clouds in combination with pre-known object models. Challenges exist in spatial representation, through bandwidth limitations and sensor interferences. Furthermore, the potential of knowledge augmentation for workload improvement is often not fully utilized. To overcome these challenges, we present a hybrid environment reconstruction method, which is adaptive regarding the integration of different data sources as well as their derived semantic information levels. For evaluation, a user study is performed. Thereby known procedures to measure usability, immersion, user experience and operator workload serve for evaluation and comparison.
Fact-Finding or Exploration: Characterizing Reading Strategies in User’s Search Activities

Adaptive information retrieval systems could provide users individual support for their current search activity. For that a clear detection and understanding of such activities are a necessity. In this paper we analyze reading strategies, such as Scanning, Skimming and ’hard’ Reading, given two search activities, namely Fact-Finding and Exploratory search activities. We analyzed the eye tracking data of a lab experiment and identified a positive correlated trend of ’hard’ Reading towards Exploratory searches and Skimming towards Fact-Finding searches. Using the interpretation of these reading strategies, we argue that we are able to draw conclusions about the possible search intent during individual search activities.
TrUMAn: Trope Understanding in Movies and Animations

Understanding and comprehending video content is crucial for many real-world applications such as search and recommendation systems. While recent progress of deep learning has boosted performance on various tasks using visual cues, deep cognition to reason intentions, motivation, or causality remains challenging. Existing datasets that aim to examine video reasoning capability focus on visual signals such as actions, objects, relations, or could be answered utilizing text bias. Observing this, we propose a novel task, along with a new dataset: Trope Understanding in Movies and Animations (TrUMAn), with 2423 videos associated with 132 tropes, intending to evaluate and develop learning systems beyond visual signals. Tropes are frequently used storytelling devices for creative works. By coping with the trope understanding task and enabling the deep cognition skills of machines, data mining applications and algorithms could be taken to the next level. To tackle the challenging TrUMAn dataset, we present a Trope Understanding and Storytelling (TrUSt) with a new Conceptual Storyteller module, which guides the video encoder by performing video storytelling on a latent space. Experimental results demonstrate that state-of-the-art learning systems on existing tasks reach only 12.01% of accuracy with raw input signals. Also, even in the oracle case with human-annotated descriptions, BERT contextual embedding achieves at most 28% of accuracy. Our proposed TrUSt boosts the model performance and reaches 13.94% performance. We also provide detailed analysis to pave the way for future research. TrUMAn is publicly available at:https://www.cmlab.csie.ntu.edu.tw/project/trope
Wasserstein Based EmoGANs+

Nowadays, numerous generative models are powerful and becoming popular for image synthesis because their generated images are more and more similar to the actual images, especially, Generative Adversarial Networks (GANs) and their variants. Such generative models are helpful to cover the shortage of datasets in various areas with their impressive realistic looking generated images. In this paper, we proposed the EmoGANs+ model to create the compound facial expressions images with stable adversarial training by using Wasserstein loss. The proposed methodology consists of three steps: preprocessing, image generation with proposed EmoGANs+, and lastly, evaluation. Our experiments are conducted on the Multimedia Understanding Group (MUG) facial expression dataset, Extended Cohn-Kanade dataset (CK+), and Japanese Female Facial Expressions (JAFFE) dataset. Our proposed model provides high feature similarity scores between the features of generated images and ground-truth compound facial expressions images.
New alternative derivations of Cramér–Rao bound for equality‐constrained estimation

This paper presents new alternative derivations of the well-known constrained Cram e ́ r–Rao bound for estimating parameters that satisfy a set of deterministic and differentiable equality constraints. Specifically, for unbiased parameter estimation, the equality constraints are treated as pseudo-measurements corrupted by independent zero-mean Gaussian noise with infinitely small covariance. In this way, the desired constrained Cram e ́ r–Rao bound can be established via utilising the property that the Fisher information matrices for independent measurements are additive. This paper also provides a new simple way for deriving the constrained Cram e ́ r–Rao bound when the parameter estimate has a specified bias through exploring that the estimation error lies in the null space of the gradient matrix of the constraints and invoking the Cauchy–Schwarz inequality.