Reviewerly/exHarmony huggingface dataset:
Defocus Blur Detection via Boosting Diversity of Deep Ensemble Networks

Existing defocus blur detection (DBD) methods usually explore multi-scale and multi-level features to improve performance. However, defocus blur regions normally have incomplete semantic information, which will reduce DBD's performance if it can't be used properly. In this paper, we address the above problem by exploring deep ensemble networks, where we boost diversity of defocus blur detectors to force the network to generate diverse results that some rely more on high-level semantic information while some ones rely more on low-level information. Then, diverse result ensemble makes detection errors cancel out each other. Specifically, we propose two deep ensemble networks (e.g., adaptive ensemble network (AENet) and encoder-feature ensemble network (EFENet)), which focus on boosting diversity while costing less computation. AENet constructs different light-weight sequential adapters for one backbone network to generate diverse results without introducing too many parameters and computation. AENet is optimized only by the self- negative correlation loss. On the other hand, we propose EFENet by exploring the diversity of multiple encoded features and ensemble strategies of features (e.g., group-channel uniformly weighted average ensemble and self-gate weighted ensemble). Diversity is represented by encoded features with less parameters, and a simple mean squared error loss can achieve the superior performance. Experimental results demonstrate the superiority over the state-of-the-arts in terms of accuracy and speed. Codes and models are available at: https://github.com/wdzhao123/DENets.
Collaborative manufacturing design: a mixed reality and cloud-based framework for part design

Computer Aided Design (CAD) and Computer Aided Manufacturing (CAM) can be considered as the cornerstones of the lifecycle of a manufacturing asset. Since the above-mentioned processes often involve multiple engineers, from different departments even from different companies, it is crucial to ensure the flawless communication between the different individuals as well as to make the design process more intuitive. In the era of digitalization and Internet of Things, digital technologies such as Mixed Reality (MR) are utilized from engineers in order to leverage the capabilities of existing computer aided tools (CAx). Therefore, in this research work, the design and development of a Cloud-based and Mixed Reality-based framework is presented. The purpose of the framework is to facilitate engineers during the design and manufacturing phase of new components by providing a mobile application and common design tools via MR graphical user interfaces (GUI). Further to that, taking into consideration the advanced capabilities of MR devices in terms of user interaction, special emphasis is given on the interaction of engineers with the holograms of the components via simple hand gestures. The resulting 3D geometries can be imported in CAD suites either for further development or for manufacturing the designed component. The applicability of the proposed framework is tested and validated in a laboratory-based machine shop.
Head-Mounted Augmented Reality Platform for Markerless Orthopaedic Navigation

Visual augmented reality (AR) has the potential to improve the accuracy, efficiency and reproducibility of computer-assisted orthopaedic surgery (CAOS). AR Head-mounted displays (HMDs) further allow non-eye-shift target observation and egocentric view. Recently, a markerless tracking and registration (MTR) algorithm was proposed to avoid the artificial markers that are conventionally pinned into the target anatomy for tracking, as their use prolongs surgical workflow, introduces human-induced errors, and necessitates additional surgical invasion in patients. However, such an MTR-based method has neither been explored for surgical applications nor integrated into current AR HMDs, making the ergonomic HMD-based markerless AR CAOS navigation hard to achieve. To these aims, we present a versatile, device-agnostic and accurate HMD-based AR platform. Our software platform, supporting both video see-through (VST) and optical see-through (OST) modes, integrates two proposed fast calibration procedures using a specially designed calibration tool. According to the camera-based evaluation, our AR platform achieves a display error of 6.31 ± 2.55 arcmin for VST and 7.72 ± 3.73 arcmin for OST. A proof-of-concept markerless surgical navigation system to assist in femoral bone drilling was then developed based on the platform and Microsoft HoloLens 1. According to the user study, both VST and OST markerless navigation systems are reliable, with the OST system providing the best usability. The measured navigation error is 4.90 ± 1.04 mm, 5.96 ± 2.22 ° for the VST system, and 4.36 ± 0.80 mm, 5.65 ± 1.42 ° for the OST system.
Connectivity status of fuzzy graphs

Network science is a widely studied subject and graph theory has a major role in it. This paper studies connectivity status of vertices in a fuzzy graph. This paper adopt connectivity status to build up the status sequence related to a fuzzy graph. Results on connectivity status and status sequence of different structures are obtained. On the basis of connectivity status, vertices of a fuzzy graph can be classified into connectivity status enhancing vertices, connectivity status reducing vertices and connectivity status neutral vertices. Connectivity status analysis of vertices is also carried out. Algorithms related to these concepts are provided and an application related to bandwidth allocation problem in networking is proposed.
OAA-SVM-MS: A fast and efficient multi-class classification algorithm

This paper introduces the idea of learning uniformly ergodic Markov chain for one-against-all support vector machine (OAA-SVM) algorithm. We first obtain the generalization error of OAA-SVM with fast learning rate for uniformly ergodic Markov samples. We also propose a new OAA-SVM method with Markov sampling (OAA-SVM-MS). The experimental researches for benchmark repository confirm that the OAA-SVM-MS algorithm has significantly better performance in sampling and training total time, classification accuracy and the obtained classifier’s sparsity compared to the classical OAA-SVM algorithm and other multi-class SVM algorithms.
IsGAN: Identity-sensitive generative adversarial network for face photo-sketch synthesis

Face photo-sketch synthesis aims to generate face sketches from real photos and vice versa. It can be abstracted as a constrained quantization problem. Although many efforts have been dedicated to this problem, it is still a challenging task to synthesize detail-preserving photos or sketches due to the significant differences between face sketch (drawn by people) and photo (taken by cameras) domains. In this paper, we propose a novel Identity-sensitive Generative Adversarial Network (IsGAN) to address it. Our key insight is to formalize face photo-sketch synthesis as a special case of image-to-image translation and propose to embed identity information through adversarial learning. In particular, an adversarial architecture is used to capture the differences between the two domains, and a new network loss, namely, identity recognition loss is introduced to preserve the detailed identifiable information, which is crucial for photo-sketch synthesis. In addition, to enforce structural consistency during generation, a cyclic-synthesized loss is applied between the generated image of one domain and cycled image of another. The experiments on the CUFS and CUFSF datasets suggest that our model achieves state-of-the-art performance in both qualitative and quantitative measures.
On the Difficulty of Membership Inference Attacks

Recent studies propose membership inference (MI) attacks on deep models, where the goal is to infer if a sample has been used in the training process. Despite their apparent success, these studies only report accuracy, precision, and recall of the positive class (member class). Hence, the performance of these attacks have not been clearly reported on negative class (non-member class). In this paper, we show that the way the MI attack performance has been reported is often misleading because they suffer from high false positive rate or false alarm rate (FAR) that has not been reported. FAR shows how often the attack model mislabel non-training samples (non-member) as training (member) ones. The high FAR makes MI attacks fundamentally impractical, which is particularly more significant for tasks such as membership inference where the majority of samples in reality belong to the negative (non-training) class. Moreover, we show that the current MI attack models can only identify the membership of misclassified samples with mediocre accuracy at best, which only constitute a very small portion of training samples.We analyze several new features that have not been comprehensively explored for membership inference before, including distance to the decision boundary and gradient norms, and conclude that deep models’ responses are mostly similar among train and non-train samples. We conduct several experiments on image classification tasks, including MNIST, CIFAR-10, CIFAR-100, and ImageNet, using various model architecture, including LeNet, AlexNet, ResNet, etc. We show that the current state-of-the-art MI attacks cannot achieve high accuracy and low FAR at the same time, even when the attacker is given several advantages. The source code is available at https://github.com/shrezaei/MI-Attack.
Contour Moments Based Manipulation of Composite Rigid-Deformable Objects With Finite Time Model Estimation and Shape/Position Control

The robotic manipulation of composite rigid-deformable objects (i.e., those with mixed nonhomogeneous stiffness properties) is a challenging problem with clear practical applications that, despite the recent progress in the field, it has not been sufficiently studied in the literature. To deal with this issue, in this article, we propose a new visual servoing method that has the capability to manipulate this broad class of objects (which varies from soft to rigid) with the same adaptive strategy. To quantify the object’s infinite-dimensional configuration, our new approach computes a compact feedback vector of 2-D contour moments features. A sliding mode control scheme is then designed to simultaneously ensure the finite-time convergence of both the feedback shape error and the model estimation error. The stability of the proposed framework (including the boundedness of all the signals) is rigorously proved with Lyapunov theory. Detailed simulations and experiments are presented to validate the effectiveness of the proposed approach. To the best of the author’s knowledge, this is the first time that contour moments along with finite-time control have been used to solve this difficult manipulation problem.
Can BERT Dig It? Named Entity Recognition for Information Retrieval in the Archaeology Domain

The amount of archaeological literature is growing rapidly. Until recently, these data were only accessible through metadata search. We implemented a text retrieval engine for a large archaeological text collection (~658 million words). In archaeological IR, domain-specific entities such as locations, time periods and artefacts play a central role. This motivated the development of a named entity recognition (NER) model to annotate the full collection with archaeological named entities. In this article, we present ArcheoBERTje, a BERT (Bidirectional Encoder Representations from Transformers) model pre-trained on Dutch archaeological texts. We compare the model’s quality and output on an NER task to a generic multilingual model and a generic Dutch model. We also investigate ensemble methods for combining multiple BERT models, and combining the best BERT model with a domain thesaurus using conditional random fields. We find that ArcheoBERTje outperforms both the multilingual and Dutch model significantly with a smaller standard deviation between runs, reaching an average F1 score of 0.735. The model also outperforms ensemble methods combining the three models. Combining ArcheoBERTje predictions and explicit domain knowledge from the thesaurus did not increase the F1 score. We quantitatively and qualitatively analyse the differences between the vocabulary and output of the BERT models on the full collection and provide some valuable insights in the effect of fine-tuning for specific domains. Our results indicate that for a highly specific text domain such as archaeology, further pre-training on domain-specific data increases the model’s quality on NER by a much larger margin than shown for other domains in the literature, and that domain-specific pre-training makes the addition of domain knowledge from a thesaurus unnecessary.
Deep neural network compression through interpretability-based filter pruning

This paper proposes a method to compress deep neural networks (DNNs) based on interpretability. For a trained DNN model, the activation maximization technique is first used to visualize every filter of the DNN model. Then, a single-layer filter pruning approach is introduced from what is learned by visualization. The entire DNN model is compressed layer by layer by using the single-layer filter pruning method in which the compression of the current layer is based on the compression of the preceding layers. Importantly, in addition to effective compression, the proposed method renders a better interpretation of the deep learning process. With a 60% compression rate of the VGG-16, our method achieves 0.8429 Top-1 accuracy under CIFAR-10, with a slight accuracy drop of only 0.0322, and the storage space of the model can be compressed to 9.42 Mb. For a modern DNN model such as ResNet50, our visualization-based filter pruning method is significantly better than other pruning strategies in different convolutional layers under different compression rates and the larger ImageNet dataset. After pruning, the computation cost and storage requirement of the DNN can be significantly reduced, which means that complex DNN models can be easily implemented in small mobile devices, thus enabling the efficient use of DNNs in the Internet of Things technologies.
Quantum-assisted associative adversarial network: applying quantum annealing in deep learning

We present an algorithm for learning a latent variable generative model via generative adversarial learning where the canonical uniform noise input is replaced by samples from a graphical model. This graphical model is learned by a Boltzmann machine which learns low-dimensional feature representation of data extracted by the discriminator. A quantum annealer, the D-Wave 2000Q, is used to sample from this model. This algorithm joins a growing family of algorithms that use a quantum annealing subroutine in deep learning, and provides a framework to test the advantages of quantum-assisted learning in GANs. Fully connected, symmetric bipartite and Chimera graph topologies are compared on a reduced stochastically binarized MNIST dataset, for both classical and quantum annealing sampling methods. The quantum-assisted associative adversarial network successfully learns a generative model of the MNIST dataset for all topologies, and is also applied to the LSUN dataset bedrooms class for the Chimera topology. Evaluated using the Fr\'{e}chet inception distance and inception score, the quantum and classical versions of the algorithm are found to have equivalent performance for learning an implicit generative model of the MNIST dataset.
A framework for the prediction of earthquake using federated learning

Earthquakes are a natural phenomenon which may cause significant loss of life and infrastructure. Researchers have applied multiple artificial intelligence based techniques to predict earthquakes, but high accuracies could not be achieved due to the huge size of multidimensional data, communication delays, transmission latency, limited processing capacity and data privacy issues. Federated learning (FL) is a machine learning (ML) technique that provides an opportunity to collect and process data onsite without compromising on data privacy and preventing data transmission to the central server. The federated concept of obtaining a global data model by aggregation of local data models inherently ensures data security, data privacy, and data heterogeneity. In this article, a novel earthquake prediction framework using FL has been proposed. The proposed FL framework has given better performance over already developed ML based earthquake predicting models in terms of efficiency, reliability, and precision. We have analyzed three different local datasets to generate multiple ML based local data models. These local data models have been aggregated to generate global data model on the central FL server using FedQuake algorithm. Meta classifier has been trained at the FL server on global data model to generate more accurate earthquake predictions. We have tested the proposed framework by analyzing multidimensional seismic data within 100 km radial area from 34.708° N, 72.5478° E in Western Himalayas. The results of the proposed framework have been validated against instrumentally recorded regional seismic data of last thirty-five years, and 88.87% prediction accuracy has been recorded. These results obtained by the proposed framework can serve as a useful component in the development of earthquake early warning systems.
Self-attention based Text Knowledge Mining for Text Detection

Pre-trained models play an important role in deep learning based text detectors. However, most methods ignore the gap between natural images and scene text images and directly apply ImageNet for pre-training. To address such a problem, some of them firstly pre-train the model using a large amount of synthetic data and then fine-tune it on target datasets, which is task-specific and has limited generalization capability. In this paper, we focus on providing general pre-trained models for text detectors. Considering the importance of exploring text contents for text detection, we propose STKM (Self-attention based Text Knowledge Mining), which consists of a CNN Encoder and a Self-attention Decoder, to learn general prior knowledge for text detection from SynthText. Given only image level text labels, Self-attention Decoder directly decodes features extracted from CNN Encoder to texts without requirement of detection, which guides the CNN backbone to explicitly learn discriminative semantic representations ignored by previous approaches. After that, the text knowledge learned by the backbone can be transferred to various text detectors to significantly improve their detection performance (e.g., 5.89% higher F-measure for EAST on ICDAR15 dataset) without bells and whistles. Pre-trained model is available at: https://github.com/CVI-SZU/STKM
Web question answering with neurosymbolic program synthesis

In this paper, we propose a new technique based on program synthesis for extracting information from webpages. Given a natural language query and a few labeled webpages, our method synthesizes a program that can be used to extract similar types of information from other unlabeled webpages. To handle websites with diverse structure, our approach employs a neurosymbolic DSL that incorporates both neural NLP models as well as standard language constructs for tree navigation and string manipulation. We also propose an optimal synthesis algorithm that generates all DSL programs that achieve optimal F1 score on the training examples. Our synthesis technique is compositional, prunes the search space by exploiting a monotonicity property of the DSL, and uses transductive learning to select programs with good generalization power. We have implemented these ideas in a new tool called WebQA and evaluate it on 25 different tasks across multiple domains. Our experiments show that WebQA significantly outperforms existing tools such as state-of-the-art question answering models and wrapper induction systems.
Asymmetric Leaky Private Information Retrieval

Information-theoretic formulations of the private information retrieval (PIR) problem have been investigated under a variety of scenarios. Symmetric private information retrieval (SPIR) is a variant where a user is able to privately retrieve one out of K messages from N non-colluding replicated databases without learning anything about the remaining K-1 messages. However, the goal of perfect privacy can be too taxing for certain applications. In this paper, we investigate if the information-theoretic capacity of SPIR (equivalently, the inverse of the minimum download cost) can be increased by relaxing both user and DB privacy definitions. Such relaxation is relevant in applications where privacy can be traded for communication efficiency. We introduce and investigate the Asymmetric Leaky PIR (AL-PIR) model with different privacy leakage budgets in each direction. For user privacy leakage, we bound the probability ratios between all possible realizations of DB queries by a function of a non-negative constant ϵ. For DB privacy, we bound the mutual information between the undesired messages, the queries, and the answers, by a function of a non-negative constant δ. We propose a general AL-PIR scheme that achieves an upper bound on the optimal download cost for arbitrary ϵ and δ. We show that the optimal download cost of AL-PIR is upper-bounded as D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> (ϵ,δ) ≤ 1+\frac 1N-1-\frac δe <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ϵ</sup> N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K-1</sup> -1. Second, we obtain an information-theoretic lower bound on the download cost as D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> (ϵ,δ) ≥ 1+\frac 1Ne <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ϵ</sup> -1-\frac δ(Ne <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ϵ</sup> ) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K-1</sup> -1. The gap analysis between the two bounds shows that our AL-PIR scheme is optimal when ϵ = 0, i.e., under perfect user privacy and it is optimal within a maximum multiplicative gap of \frac N-e <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-ϵ</sup> N-1 for any ϵ > 0 and δ > 0.
Human Behaviour Based Optimization Supported With Self-Organizing Maps for Solving the S-Box Design Problem

The cryptanalytic resistance of modern block and stream encryption systems mainly depends on the substitution box (S-box).In this context, the problem is thus to create an S-box with higher value of nonlinearity because this property can provide some degree of protection against linear and differential cryptanalysis attacks.In this paper, we design a scheme built on a human behavior-based optimization algorithm, supported with Self-Organizing Maps to prevent premature convergence and improve the nonlinearity property in order to obtain strong 8 × 8 substitution boxes.The experiments are compared with S-boxes obtained using other metaheuristic algorithms such as Ant Colony Optimization, Genetic Algorithm and an approach based on chaotic functions and show that the obtained S-boxes have good cryptographic properties.The obtained S-box is investigated against standard tests such as bijectivity, nonlinearity, strict avalanche criterion, bit independence criterion, linear probability and differential probability, proving that the proposed scheme is proficient to discover a strong nonlinear component of encryption systems.