Reviewerly/exHarmony huggingface dataset:
Identity and access management using distributed ledger technology: A survey

Summary As the basic building block of any information security system, identity and access management (IAM) solutions play vital role in enterprise's security programmes. Providing centric solutions for IAM is inefficient in terms of having single point of failure, high cost, duplication and complexity to the users. Recently, emerging the distributed ledger technology (DLT) has attracted significant scientific interests in research areas like identity management, authentication and access control processes. In these contexts, Blockchain can offer greater data and rule confidentiality and integrity, as well as increasing the availability of the system by removing the single point of failure in the procedure. In this paper, we provide a comprehensive overview of the IAM solutions based on their basic components including identity management, authentication and access control. In the identity concept, we discuss about self‐sovereign identity which enhances privacy and security of distributed digital identities by providing individual's consolidated digital identity and verified attributes for enabling them to utilize their ownership. To offer a clearer understanding of the state of the art, we propose taxonomy to categorize them based on their features. For the conclusion of the paper, we compare the existing methods based on proposed taxonomy. Also, considering the advantages and disadvantages of existing methods, we discussed about the possible future directions.
Multi-Label Classification Neural Networks with Hard Logical Constraints

Multi-label classification (MC) is a standard machine learning problem in which a data point can be associated with a set of classes. A more challenging scenario is given by hierarchical multi-label classification (HMC) problems, in which every prediction must satisfy a given set of hard constraints expressing subclass relationships between classes. In this paper, we propose C-HMCNN(h), a novel approach for solving HMC problems, which, given a network h for the underlying MC problem, exploits the hierarchy information in order to produce predictions coherent with the constraints and to improve performance. Furthermore, we extend the logic used to express HMC constraints in order to be able to specify more complex relations among the classes and propose a new model CCN(h), which extends C-HMCNN(h) and is again able to satisfy and exploit the constraints to improve performance. We conduct an extensive experimental analysis showing the superior performance of both C-HMCNN(h) and CCN(h) when compared to state-of-the-art models in both the HMC and the general MC setting with hard logical constraints.
Removing Foreground Occlusions in Light Field using Micro-lens Dynamic Filter

Foreground occlusion removal task aims to automatically detect and remove foreground occlusions and recover background objects. Since for Light Fields (LFs), background objects occluded in some views may be seen in other views, the foreground occlusion removal task for LFs is easy to achieve. In this paper, we propose a learning-based method combining ‘seeking’ and ‘generating’ to recover occluded background. Specifically, the micro-lens dynamic filters are proposed to ‘seek’ occluded background points in shifted micro-lens images and remove occlusions using angular information. The shifted images are then combined to further ‘generate’ background regions to supplement more background details using spatial information. By fully exploring the angular and spatial information in LFs, the dense and complex occlusions can be easily removed. Quantitative and qualitative experimental results show that our method outperforms other state-of-the-arts methods by a large margin.
Application of different imaging techniques for the characterization of damage in fiber reinforced polymer

The heterogeneous and anisotropic nature of fiber reinforced polymers induces the initiation of various damage mechanisms at different scales. Even though full-field surface measurements may successfully track damage development, such approaches do not provide insights into the material bulk. The present work aims at comparing surface and bulk full-field measurements (via Digital Image/Volume Correlation) for a comprehensive analysis of polymer composite damage. Bulk measurements revealed that damage initiated at the microscale, thus contributing to surface localization phenomena. Full-field surface measurements could not capture the local strain states that the constituents were subjected to, which led to final fracture.
VITALITY: Promoting Serendipitous Discovery of Academic Literature with Transformers &amp; Visual Analytics

There are a few prominent practices for conducting reviews of academic literature, including searching for specific keywords on Google Scholar or checking citations from some initial seed paper(s). These approaches serve a critical purpose for academic literature reviews, yet there remain challenges in identifying relevant literature when similar work may utilize different terminology (e.g., mixed-initiative visual analytics papers may not use the same terminology as papers on model-steering, yet the two topics are relevant to one another). In this paper, we introduce a system, VITALITY, intended to complement existing practices. In particular, VITALITY promotes serendipitous discovery of relevant literature using transformer language models, allowing users to find semantically similar papers in a word embedding space given (1) a list of input paper(s) or (2) a working abstract. VITALITY visualizes this document-level embedding space in an interactive 2-D scatterplot using dimension reduction. VITALITY also summarizes meta information about the document corpus or search query, including keywords and co-authors, and allows users to save and export papers for use in a literature review. We present qualitative findings from an evaluation of VITALITY, suggesting it can be a promising complementary technique for conducting academic literature reviews. Furthermore, we contribute data from 38 popular data visualization publication venues in VITALITY, and we provide scrapers for the open-source community to continue to grow the list of supported venues.
A Study on Sensor System Latency in VR Motion Sickness

One of the most frequent technical factors affecting Virtual Reality (VR) performance and causing motion sickness is system latency. In this paper, we adopted predictive algorithms (i.e., Dead Reckoning, Kalman Filtering, and Deep Learning algorithms) to reduce the system latency. Cubic, quadratic, and linear functions are used to predict and curve fitting for the Dead Reckoning and Kalman Filtering algorithms. We propose a time series-based LSTM (long short-term memory), Bidirectional LSTM, and Convolutional LSTM to predict the head and body motion and reduce the motion to photon latency in VR devices. The error between the predicted data and the actual data is compared for statistical methods and deep learning techniques. The Kalman Filtering method is suitable for predicting since it is quicker to predict; however, the error is relatively high. However, the error property is good for the Dead Reckoning algorithm, even though the curve fitting is not satisfactory compared to Kalman Filtering. To overcome this poor performance, we adopted deep-learning-based LSTM for prediction. The LSTM showed improved performance when compared to the Dead Reckoning and Kalman Filtering algorithm. The simulation results suggest that the deep learning techniques outperformed the statistical methods in terms of error comparison. Overall, Convolutional LSTM outperformed the other deep learning techniques (much better than LSTM and Bidirectional LSTM) in terms of error.
Speech2Talking-Face: Inferring and Driving a Face with Synchronized Audio-Visual Representation

What can we picture solely from a clip of speech? Previous research has shown the possibility of directly inferring the appearance of a person's face by listening to a voice. However, within human speech lies not only the biometric identity signal but also the identity-irrelevant information such as the talking content. Our goal is to extract as much information from a clip of speech as possible. In particular, we aim at not only inferring the face of a person but also animating it. Our key insight is to synchronize audio and visual representations from two perspectives in a style-based generative framework. Specifically, contrastive learning is leveraged to map both the identity and speech content information within the speech to visual representation spaces. Furthermore, the identity space is strengthened with class centroids. Through curriculum learning, the style-based generator is capable of automatically balancing the information from the two latent spaces. Extensive experiments show that our approach encourages better speech-identity correlation learning while generating vivid faces whose identities are consistent with given speech samples. Moreover, by leveraging the same model, these inferred faces can be driven to talk by the audio.
A Non-Interactive Attribute-Based Access Control Scheme by Blockchain for IoT

As an important method of protecting data confidentiality in the Internet of Things (IoT), access control has been widely concerned. Because attribute-based access control mechanisms are dynamic, it is not only suitable to solve the dynamic access problem in IoT, but also to deal with the dynamic caused by node movement and access data change. The traditional centralized attribute-based access control mechanism has some problems: due to the large number of devices in IoT, the central trusted entity may become the bottleneck of the whole system. Moreover, when a central trusted entity is under distributed denial-of-service (DDoS) attack, the entire system may crash. Blockchain is a good way to solve the above problems. Therefore, we developed a non-interactive, attribute-based access control scheme that applies blockchain technology in IoT scenarios by using PSI technology. In addition, the attributes of data user and data holder are hidden, which protects the privacy of both parties’ attributes and access policy. Furthermore, the experimental results indicate that our scheme has high efficiency.
UniMF: A Unified Framework to Incorporate Multimodal Knowledge Bases intoEnd-to-End Task-Oriented Dialogue Systems

Knowledge bases (KBs) are usually essential for building practical dialogue systems. Recently we have seen rapidly growing interest in integrating knowledge bases into dialogue systems. However, existing approaches mostly deal with knowledge bases of a single modality, typically textual information. As today's knowledge bases become abundant with multimodal information such as images, audios and videos, the limitation of existing approaches greatly hinders the development of dialogue systems. In this paper, we focus on task-oriented dialogue systems and address this limitation by proposing a novel model that integrates external multimodal KB reasoning with pre-trained language models. We further enhance the model via a novel multi-granularity fusion mechanism to capture multi-grained semantics in the dialogue history. To validate the effectiveness of the proposed model, we collect a new large-scale (14K) dialogue dataset MMDialKB, built upon multimodal KB. Both automatic and human evaluation results on MMDialKB demonstrate the superiority of our proposed framework over strong baselines.
Feature Space Targeted Attacks by Statistic Alignment

By adding human-imperceptible perturbations to images, DNNs can be easily fooled. As one of the mainstream methods, feature space targeted attacks perturb images by modulating their intermediate feature maps, for the discrepancy between the intermediate source and target features is minimized. However, the current choice of pixel-wise Euclidean Distance to measure the discrepancy is questionable because it unreasonably imposes a spatial-consistency constraint on the source and target features. Intuitively, an image can be categorized as "cat'' no matter the cat is on the left or right of the image. To address this issue, we propose to measure this discrepancy using statistic alignment. Specifically, we design two novel approaches called Pair-wise Alignment Attack and Global-wise Alignment Attack, which attempt to measure similarities between feature maps by high-order statistics with translation invariance. Furthermore, we systematically analyze the layer-wise transferability with varied difficulties to obtain highly reliable attacks. Extensive experiments verify the effectiveness of our proposed method, and it outperforms the state-of-the-art algorithms by a large margin. Our code is publicly available at https://github.com/yaya-cheng/PAA-GAA.
The Traffic Scene Understanding and Prediction Based on Image Captioning

The traffic scene understanding is the core technology in Intelligent Transportation Systems (ITS) and Advanced Driver Assistance System (ADAS), and it is becoming increasingly important for smart or autonomous vehicles.The recent methods for traffic scene understanding, such as Traffic Sign Recognition (TSR), Pedestrian Detection, and Vehicle Detection, have three major shortcomings.First, most models are customized for recognizing a specific category of traffic target instead of general traffic targets.Second, as for these recognition modules, the task of traffic scene understanding is to recognize objects rather than make driving suggestions or strategies.Third, numerous independent recognition modules disadvantage to fusing multi-modal information to make a comprehensive decision for driving operation in accordance with complicated traffic scenes.In this paper, we first introduce the image captioning model to alleviate the aforementioned shortcomings.Different from existing methods, our primary idea is to accurately identify all categories of traffic objects and understand traffic scenes by making full use of all information, and making the suggestions or strategy for driving operation in natural language by using Long Short Term Memory network (LSTM) rather than keywords.The proposed solution naturally solves the problems of feature fusion, general object recognition, and low-level semantic understanding.We tested the solution on our created traffic scene image dataset for evaluation of image captioning.Extensive experiments including quantitative and qualitative comparisons demonstrate that the proposed solution can identify more objects and produce higher-level semantic information than the state-of-the-arts.
Automatic Fairness Testing of Neural Classifiers Through Adversarial Sampling

Although deep learning has demonstrated astonishing performance in many applications, there are still concerns about its dependability.One desirable property of deep learning applications with societal impact is fairness (i.e., non-discrimination).Unfortunately, discrimination might be intrinsically embedded into the models due to the discrimination in the training data.As a countermeasure, fairness testing systemically identifies discriminatory samples, which can be used to retrain the model and improve the model's fairness.Existing fairness testing approaches however have two major limitations.Firstly, they only work well on traditional machine learning models and have poor performance (e.g., effectiveness and efficiency) on deep learning models.Secondly, they only work on simple structured (e.g., tabular) data and are not applicable for domains such as text.In this work, we bridge the gap by proposing a scalable and effective approach for systematically searching for discriminatory samples while extending existing fairness testing approaches to address a more challenging domain, i.e., text classification.Compared with state-of-the-art methods, our approach only employs lightweight procedures like gradient computation and clustering, which is significantly more scalable and effective.Experimental results show that on average, our approach explores the search space much more effectively (9.62 and 2.38 times more than the state-of-the-art methods respectively on tabular and text datasets) and generates much more discriminatory samples (24.95 and 2.68 times) within a same reasonable time.Moreover, the retrained models reduce discrimination by 57.2% and 60.2% respectively on average.
Bipartite Matching for Crowd Counting with Point Supervision

For crowd counting task, it has been demonstrated that imposing Gaussians to point annotations hurts generalization performance. Several methods attempt to utilize point annotations as supervision directly. And they have made significant improvement compared with density-map based methods. However, these point based methods ignore the inevitable annotation noises and still suffer from low robustness to noisy annotations. To address the problem, we propose a bipartite matching based method for crowd counting with only point supervision (BM-Count). In BM-Count, we select a subset of most similar pixels from the predicted density map to match annotated pixels via bipartite matching. Then loss functions can be defined based on the matching pairs to alleviate the bad effect caused by those annotated dots with incorrect positions. Under the noisy annotations, our method reduces MAE and RMSE by 9% and 11.2% respectively. Moreover, we propose a novel ranking distribution learning framework to address the imbalanced distribution problem of head counts, which encodes the head counts as classification distribution in the ranking domain and refines the estimated count map in the continuous domain. Extensive experiments on four datasets show that our method achieves state-of-the-art performance and performs better crowd localization.
Label-free imaging and classification of live P. falciparum enables high performance parasitemia quantification without fixation or staining

Manual microscopic inspection of fixed and stained blood smears has remained the gold standard for Plasmodium parasitemia analysis for over a century. Unfortunately, smear preparation consumes time and reagents, while manual microscopy is skill-dependent and labor-intensive. Here, we demonstrate that deep learning enables both life stage classification and accurate parasitemia quantification of ordinary brightfield microscopy images of live, unstained red blood cells. We tested our method using both a standard light microscope equipped with visible and near-ultraviolet (UV) illumination, and a custom-built microscope employing deep-UV illumination. While using deep-UV light achieved an overall four-category classification of Plasmodium falciparum blood stages of greater than 99% and a recall of 89.8% for ring-stage parasites, imaging with near-UV light on a standard microscope resulted in 96.8% overall accuracy and over 90% recall for ring-stage parasites. Both imaging systems were tested extrinsically by parasitemia titration, revealing superior performance over manually-scored Giemsa-stained smears, and a limit of detection below 0.1%. Our results establish that label-free parasitemia analysis of live cells is possible in a biomedical laboratory setting without the need for complex optical instrumentation. We anticipate future extensions of this work could enable label-free clinical diagnostic measurements, one day eliminating the need for conventional blood smear analysis.
RGB<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si62.svg"><mml:mrow><mml:mo>×</mml:mo></mml:mrow></mml:math>D: Learning depth-weighted RGB patches for RGB-D indoor semantic segmentation

Significant advances have been made in designing CNNs for RGB semantic segmentation. However, these CNNs are not widely adopted for RGB-D segmentation, due to the asymmetry between the RGB and depth modalities. Instead, dedicated architectures are designed to fuse them for effective RGB-D segmentation, wherein complex structures are often employed, resulting in much increased computational cost. In this paper, we propose a novel way to learn the fusion of RGB and depth information in an early stage. This enables our method to easily adopt existing RGB segmentation networks with minimal modification. Our method is simple yet effective to build a bridge between RGB and RGBD semantic segmentation, so as to avoid designing a far more complex network structure for RGBD segmentation. The proposed method treats RGB and depth information in an inherently asymmetric manner, and to the best of our knowledge, this is the first approach that learns to fuse them in a multiplicative manner for RGB-D segmentation; thus, we call it RGB×D. Extensive experiments and ablation studies on the challenging NYUDv2, SUN RGB-D and Cityscapes semantic segmentation benchmarks show that the proposed RGB×D offers a consistent improvement over several baselines.
Head-Mounted Display with Increased Downward Field of View Improves Presence and Sense of Self-Location

Common existing head-mounted displays (HMDs) for virtual reality (VR) provide users with a high presence and embodiment. However, the field of view (FoV) of a typical HMD for VR is about 90 to 110 [deg] in the diagonal direction and about 70 to 90 [deg] in the vertical direction, which is narrower than that of humans. Specifically, the downward FoV of conventional HMDs is too narrow to present the user avatar's body and feet. To address this problem, we have developed a novel HMD with a pair of additional display units to increase the downward FoV by approximately 60 ( 10+50) [deg]. We comprehensively investigated the effects of the increased downward FoV on the sense of immersion that includes presence, sense of self-location (SoSL), sense of agency (SoA), and sense of body ownership (SoBO) during VR experience and on patterns of head movements and cybersickness as its secondary effects. As a result, it was clarified that the HMD with an increased FoV improved presence and SoSL. Also, it was confirmed that the user could see the object below with a head movement pattern close to the real behavior, and did not suffer from cybersickness. Moreover, the effect of the increased downward FoV on SoBO and SoA was limited since it was easier to perceive the misalignment between the real and virtual bodies.
Full multipartite steering inseparability, genuine multipartite steering, and monogamy for continuous-variable systems

We derive inequalities sufficient to detect the genuine $N$-partite steering of $N$ distinct systems. Here, we are careful to distinguish between the concepts of full $N$-partite steering inseparability (where steering is confirmed individually for all bipartitions of the $N$ systems, thus negating the bilocal hidden state model for each bipartition) and genuine $N$-partite steering (which excludes all convex combinations of the bilocal hidden state models). Other definitions of multipartite steering are possible and we derive inequalities to detect a stricter genuine $N$-partite steering based on only one trusted site. The inequalities are expressed as variances of quadrature phase amplitudes and thus apply to continuous variable systems. We show how genuine $N$-partite steerable states can be created and detected for the nodes of a network formed from a single-mode squeezed state passed through a sequence of $N-1$ beam splitters. A stronger genuine $N$-partite steering is created, if one uses two squeezed inputs, or $N$ squeezed inputs. We are able to confirm that genuine tripartite steering (by the above definition and the stricter definition) has been realised experimentally. Finally, we analyze how bipartite steering and entanglement is distributed among the systems in the tripartite case, illustrating with monogamy inequalities. While we use Gaussian states to benchmark the criteria, the inequalities derived in this paper are not based on the assumption of Gaussian states, which gives advantage for quantum communication protocols.