Reviewerly/exHarmony huggingface dataset:
Region interaction and attribute embedding for zero-shot learning

Zero-shot learning (ZSL) aims to recognize unseen class images only by using seen class images for training. In ZSL tasks, unseen classes share local attribute semantics with seen classes. However, most ZSL methods directly learn an embedding from global feature space to semantic space, which may fail to discover local attribute semantics and cause the strong bias problem. In this paper, a novel method called region interaction and attribute embedding (RIAE) is proposed. RIAE consists of two modules: the region graph network (RGN) and the attribute feature embedding (AFE). RGN constructs a region graph where each region (image patch) is regarded as a graph node for region interaction. That is, through the graph-convolution operations of RGN, the node features can aggregate the information from their neighboring node features and update themselves. In order to learn the compatibility between the updated node features and local attribute semantics, AFE is designed to assign high attention weights to the node features which have large compatibilities with attribute semantic vectors. Relying on attribute semantic vectors rather than human annotations, RIAE can adaptively extract and update local attribute features, and further learn an embedding from attribute feature space to semantic space. Extensive experiments on the benchmark datasets show that RIAE gets state-of-the-art or competitive performances.
Saliency detection for surface defects of ceramic tile

Considering the lack of techniques for automatically detecting the surface defects of ceramic tiles, owing to the numerous types and complex surface textures of ceramic tiles, a salient region detection (SRD) algorithm is proposed. After the superpixel segmentation of the surface image of ceramic tiles, colour and structure saliency maps were constructed by taking the differences between the defects and backgrounds, in terms of colour and structure, as the salient values. The final saliency map for the surface defects was then obtained through the fusion of these two maps. The salient regions in the map were determined using the connected domain method and were then marked using minimum circumscribed rectangles for detecting the surface defects. The feasibility and effectiveness of the proposed SRD algorithm was verified through comparative experiments. The experimental results show that the detection accuracy of the algorithm for unknown types of ceramic tiles is 92.60%, which was 5.40% higher than that of the ResNet-50 algorithm, which was the highest for known types of ceramic tiles.
Unsupervised dimensionality reduction by jointing dynamic hypergraph and low-rank embedding for classification and clustering

Hypergraph learning is largely used to handle dimensional disaster problem of high-dimensional data in view of its flexible ability to model complex data correlations. However, the following three problems limit the performance of unsupervised hypergraph-based dimensionality reduction methods: (1) most of them are unable to obtain consistently good performance for different data sets; (2) noise or outliers in the data can lead to unreliability of the fixed hypergraph; and (3) the simple K-nearest neighbor strategy generates hypergraph with a fixed number of vertices in each hyperedge. To solve the above problems, an unsupervised dimensionality reduction method named joint dynamic hypergraph and low-rank embedding (DHLRE) is proposed. Specifically, DHLRE unifies hypergraph learning and low-rank learning into a single objective function. By doing so, both global and higher-order local structural information of the data can be extracted by DHLRE, and the two structural information complement each other to make the projection matrix have stronger discriminative power. In addition, the weight of hyperedge and hypergraph in DHLRE are dynamically generated in dimension-reduced space, which suppresses the ability of noise or outlier to affect the projection matrix. To divide hyperedge in a flexible way, a new approach without neighborhood parameter is proposed. By using this approach, the number of vertices of the hyperedge can be adaptively decided for different data sets. Further, an iterative algorithm is designed to solve the objective function of DHLRE. Finally, experimental results on several real-world data sets show that DHLRE outperforms others related methods for both classification and clustering tasks.
Deep Dynamic Patient Similarity Analysis: Model Development and Validation in ICU

Personalized medicine requires the patient similarity analysis for providing specific treatments tailed for each patient. However, the patient similarity analysis in personalized clinical scenarios encounters challenges, which are twofold. First, heterogeneous and multi-type data are usually recorded to Electronic Health Records (EHRs) during the course of admissions, which makes it difficult to measure the patient similarity. Second, disease progression manifests diverse disease states at different times, which brings sequential complexity to dynamically retrieve similar patients' sequences. To overcome the above-mentioned challenges, we propose a novel dynamic patient similarity analysis model based on deep learning. Specifically, the proposed model embeds the multi-type and heterogeneous data into hidden representations with a specially designed embedding and attention module. Thereafter, the proposed model retrieves similar patients' sequences based on these hidden representations in a dynamic manner. More importantly, we adopt two clinical tasks, i.e., diagnosis prediction and medication recommendation, to validate the effectiveness of the proposed model. It is worth noticing that the proposed model integrates a drug-drug interaction (DDI) knowledge graph in the medication recommendation task to reduce adverse reactions caused by combinational treatments, such that a more rational strategy can be realized. We evaluate our proposed model using the critical care database MIMIC-III, which includes 5,430 patients covering 14,096 clinical visits. The proposed model outperforms several state-of-the-art methods. For diagnosis prediction, the average PR-AUC score of the proposed model reaches 0.6200, which is significantly higher than that of the baseline models (0.2497∼0.5407). Meanwhile, for medication recommendation, the average PR-AUC of the proposed model is 0.6682 (Jaccard: 0.4070; F1: 0.5672; Recall: 0.7832) whereas the K-nearest model can only reach 0.3805 (Jaccard: 0.3911; F1: 0.5465; Recall: 0.5705). In addition, our proposed model achieves a lower DDI rate. We propose a novel dynamic patient similarity analysis model, which can be implemented into a decision support system for clinical tasks including diagnosis prediction, surgical procedure selection, medication recommendation, etc. Also, the proposed model serves as an explainable protocol in clinical practice thanks to its analogy to real clinical reasoning where a doctor diagnoses diseases and prescribes medications according to the previous cured patients empirically.
Discriminative feature mining hashing for fine-grained image retrieval

• We propose the DFMH framework to solve the fine-grained image retrieval problem by mining discriminative features and constructing hash codes. • We design DEFM to locate the discriminative part of an object and selectively enhance the details of more important local parts by simple but effective attention drop and re-sampling operations. • We design SHCM that can learn hash coding using classification loss, while also introducing quantization loss and bit-balancing loss to suppress coding bias. • Experimental results on three datasets achieve state-of-the-art results. With the development of multimedia technology, fine-grained image retrieval has gradually become a new hot topic in computer vision, while its accuracy and speed are limited due to the low discriminative high-dimensional real-valued embedding. To solve this problem, we propose an end-to-end framework named DFMH (Discriminative Feature Mining Hashing), which consists of the DFEM (Discriminative Feature Extracting Module) and SHCM (Semantic Hash Coding Module). Specifically, DFEM explores more discriminative local regions by attention drop and obtains finer local feature expression by attention re-sample. SHCM generates high-quality hash codes by combining the quantization loss and bit balance loss. Validated by extensive experiments and ablation studies, our method consistently outperforms both the state-of-the-art generic retrieval methods as well as fine-grained retrieval methods on three datasets, including CUB Birds, Stanford Dogs and Stanford Cars.
CPP: A lightweight memory page management extension to prevent code pointer leakage

Protecting code pointers (e.g., return address, function pointer) from leakage is desirable from a security perspective. Isolation mechanisms have been the favored candidate to protect code pointers. However, these mechanisms result in significant performance overhead as they need to instrument extra instructions for frequent permission switching or bound checking. In this paper, we propose CPP, a novel Code Pointer-only Memory Page Management to restrict attack-critical operations for code pointers by hardware. Our hardware–software co-design allows CPP mark code pointers at page granularity that requires minor hardware modification. CPP checks the legality of their operations in parallel with instruction execution. We implement a prototype system and our evaluation shows CPP can effectively mitigate the code pointer leakage attacks with less than 2.1% performance overhead.
Multi-view clustering with adaptive procrustes on Grassmann manifold

Multi-view clustering plays an important role in a wide spectrum of applications. In this article, we propose a multi-view clustering approach with adaptive Procrustes on Grassmann manifold (MC-APGM) to overcome the three demerits in existing graph-based multi-view clustering methods, namely, insufficient mining of subspace information of views, a requirement for post-processing, and high computational complexity. Specifically, in the proposed model, the indicator matrix is directly learned from multiple orthogonal spectral embeddings, avoiding the random clustering results caused by post-processing; The orthogonal form of the indicator matrix approximates multiple orthogonal spectral embeddings on the Grassmann manifold, fully uncovering subspace information of views and thus improving clustering performance; Both implicitly and explicitly weighted learning mechanisms are established to capture inconsistencies among different views. Moreover, an efficient algorithm with rigorous convergence guarantee is derived to optimize the proposed model. Finally, experimental results on both toy and real-world datasets demonstrate the effectiveness and efficiency of this proposed method.
How to achieve adaptive security for asynchronous BFT?

We consider how to build practical asynchronous BFT protocols with adaptive security. In particular, we build two protocols in both the computational setting (where the adversary is limited to polynomial-time) and the stronger information-theoretic model (where the adversary is unbounded). In the computational model, we provide EPIC using adaptively secure key generation and common coin protocols. In the information-theoretical model, we provide HALE leveraging the classic local coin protocol of Bracha. HALE is more robust than EPIC and does not need distributed key generation. Via a five-continent deployment on Amazon EC2, we show EPIC is slightly slower for small and medium-sized networks than the most efficient asynchronous BFT protocols with static security. As the number of replicas is smaller than 46, EPIC's throughput is stable, achieving peak throughput of 8,000–12,500 tx/sec with a transaction size of 250 bytes. When the network size grows larger, EPIC is not as efficient as asynchronous BFT protocols with static security, with throughput of 4,000–6,300 tx/sec. We also show while HALE is in general less efficient than EPIC, HALE is reasonably fast, achieving 42,000 tx/sec and 3,400 tx/sec for the 4-server setting in the LAN/WAN environments, respectively. Remarkably, HALE outperforms EPIC in LANs when the number of replicas is smaller than 16.
Hierarchical binding in convolutional neural networks: Making adversarial attacks geometrically challenging

We approach the issue of robust machine vision by presenting a novel deep-learning architecture, inspired by work in theoretical neuroscience on how the primate brain performs visual feature binding. Feature binding describes how separately represented features are encoded in a relationally meaningful way, such as an edge composing part of the larger contour of an object. We propose that the absence of such representations from current models might partly explain their vulnerability to small, often humanly-imperceptible distortions known as adversarial examples. It has been proposed that adversarial examples are a result of 'off-manifold' perturbations of images. Our novel architecture is designed to approximate hierarchical feature binding, providing explicit representations in these otherwise vulnerable directions. Having introduced these representations into convolutional neural networks, we provide empirical evidence of enhanced robustness against a broad range of L0, L2 and L∞ attacks, particularly in the black-box setting. While we eventually report that the model remains vulnerable to a sufficiently powerful attacker (i.e. the defense can be broken), we demonstrate that our main results cannot be accounted for by trivial, false robustness (gradient masking). Analysis of the representational geometry of our architectures shows a positive relationship between hierarchical binding, expanded manifolds, and robustness. Through hyperparameter manipulation, we find evidence that robustness emerges through the preservation of general low-level information alongside more abstract features, rather than by capturing which specific low-level features drove the abstract representation. Finally, we propose how hierarchical binding relates to the observation that, under appropriate viewing conditions, humans show sensitivity to adversarial examples.
Intent-Aware Long Short-Term Memory for Intelligent Training of Clinical Handover

Clinical handover is a crucial yet high-risk communication event in the provision of safe patient care. However, training standardized clinical handover in real-world scenarios often requires huge labor cost. To tackle with this issue, we propose a computer-aided method for delivering intelligent training of clinical handover at a low labor cost. Specifically, we formulate it as a continuous intent detection task that provides timely feedback during a simulated clinical handover conversation. Towards this goal, we collaborate with experts from a local hospital to collect a clinical handover dataset on real-world handover scenarios. According to the sequential nature of the handover conversation, we further propose the Intent-Aware Long Short-Term Memory (IA-LSTM) model that yields superior performance to baseline methods. Our work shows promise for the computer-aided training of clinical handover in hospitals and can encourage researchers in natural language processing to develop methods on standardized communication.
On Sample Based Explanation Methods for Sequence-to-Sequence Applications

As deep learning models continue to develop more and more complex, the scale of natural language processing(NLP) datasets is also increasing, which challenges the ability of sample based explanation methods in terms of their interpretability, faithfulness, etc. In this work, we propose a matching influence function TracInS by selecting representative sequence-to-sequence applications that require high interpretability according to the needs of people who want to understand model behavior. Thereafter, we design enhancement based on TracInS, using arbitrary spans as fine-grained explanation units to achieve interpretability for sequence-to-sequence applications on valid datasets. Finally, we design targeted influence function evaluations, which are semantic-based and retraining-based evaluation methods, to verify that the influence function is effective. At the same time, the improvement of the experimental effect proves that the enhancement is more interpretable.
Unsupervised Cross Domain Learning for Noise Removal from a Single Image

Most supervised deep learning-based frameworks designed for image noise removal usually exhibit robust denoising performance depending on large scale clean/noisy paired training images. However, paired training data may be unavailable in several real scenarios. In recent, cross domain transferring has been applied to unsupervised learning for image restoration. To avoid possible domain shift problem in existing cross domain learning frameworks, in this paper, an unsupervised cross domain deep learning framework is proposed for noise removal from a single image. Our goal is to learn an image generator without paired training data to directly learn invariant feature representations from noisy images and generate the corresponding clean images. In our framework, we aim at learning two image generators to transfer noisy images to clean images as well as clean images to noisy images, respectively, denoted by noise-to-clean and clean-to-noise generators, based on unpaired training images. To train this cross domain learning model, we propose a generative adversarial network (GAN)-based network architecture with different types of discriminators and loss functions. As a result, both generators can be efficiently trained, and the learned noise-to-clean generator can robustly and effectively perform feature learning from input noisy images and produce the corresponding denoised images.
Illumination-Aware Image Segmentation for Real-Time Moving Cast Shadow Suppression

One of the main challenges facing foreground detection methods is the performance deterioration due to shadows cast by moving objects. In this paper, a new real-time method is proposed that integrates various cues for region-wise classification to deal with achromaticity and camouflage issues in suppressing cast shadows. Specifically, after background subtraction, a locally near-invariant illumination feature is used as input for watershed segmentation approach to extract a number of superpixels. The superpixels are further merged according to three illumination criteria with the purpose of constructing segments that are locally homogeneous in terms of illumination variations. These segments are then classified according to the number of potential shadow candidates, gradient direction correlation, and the number of external boundary points. The potential shadow candidates are extracted by establishing a set of chromatic criteria in the HSV color-space. The gradient correlation is considered due to the fact that shadows do not impose considerable variations in the gradient directions. On the other hand, shadow segments contain a notable number of extrinsic boundary points which is used as an additional cue. Final shadow detection is achieved by integrating the outputs of the previous steps. The experimental results using publicly available videos from ATON dataset show the feasibility of our proposed method for real-time applications. The code is publicly available at: http://github.com/hadi-ghnd/ShadowDetection.
cryptolib: Comparing and selecting cryptography libraries

Selecting a library out of numerous candidates can be a laborious and resource-intensive task. We present the cryptolib index, a tool for decision-makers to choose the best fitting cryptography library for a given context. To define our index, 15 library attributes were synthesized from findings based on a literature review and interviews with decision-makers. These attributes were afterwards validated and weighted via an online survey. In order to create the index value for a given library, the individual attributes are assessed using given evaluation criteria associated with the respective attribute. As a proof of concept and to give a practical usage example, the derivation of the cryptolib values for the libraries Bouncy Castle and Tink are shown in detail. Overall, by tailoring the weighting of the cryptolib attributes to their current use case, decision-makers are enabled to systematically select a cryptography library fitting best to their software project at hand in a guided, repeatable and reliable way.
Frequency-Centroid Features for Word Recognition of Non-Native English Speakers

The objective of this work is to investigate complementary features which can aid the quintessential Mel frequency cepstral coefficients (MFCCs) in the task of closed, limited set word recognition for non-native English speakers of different mother-tongues. Unlike the MFCCs, which are derived from the spectral energy of the speech signal, the proposed frequency-centroids (FCs) encapsulate the spectral centres of the different bands of the speech spectrum, with the bands defined by the Mel filterbank. These features, in combination with the MFCCs, are observed to provide relative performance improvement in English word recognition, particularly under varied noisy conditions. A two-stage Convolution Neural Network (CNN) is used to model the features of the English words uttered with Arabic, French and Spanish accents.
Two sepedi-english code-switched speech corpora

We report on the development of two reference corpora for the analysis of Sepedi-English code-switched speech in the context of automatic speech recognition. For the first corpus, possible English events were obtained from an existing corpus of transcribed Sepedi-English speech. The second corpus is based on the analysis of radio broadcasts: actual instances of code switching were transcribed and reproduced by a number of native Sepedi speakers. We describe the process to develop and verify both corpora and perform an initial analysis of the newly produced data sets. We find that, in naturally occurring speech, the frequency of code switching is unexpectedly high for this language pair, and that the continuum of code switching (from unmodified embedded words to loanwords absorbed into the matrix language) makes this a particularly challenging task for speech recognition systems.
Application of the User Centered Design Framework – Case Study Using the Example of an AR Feature for a Mobile Shopping App

Augmented reality (AR) has become increasingly important in recent years and is already being used in many areas. AR is also increasingly penetrating the retail sector as a functionality of mobile apps. At the same time, companies must meet user requirements when developing AR features. This case study exemplifies how the User Centered Design framework (UCD) can be used to develop an AR feature for a mobile shopping app that can be used to virtually color walls in a room. To begin, the UCD will be introduced and compared to the New Product Development framework and the Information System Research framework. The four phases of the UCD and possible methods to be applied are discussed. One method per phase is then applied as an example. The results are shown and discussed.