Reviewerly/exHarmony huggingface dataset:
Ontology Building for Cyber–Physical Systems: Application in the Manufacturing Domain

Cyber-physical systems (CPSs) in the manufacturing domain can be deployed to support monitoring and analysis of production systems of a factory in order to improve, support, or automate processes, such as maintenance or scheduling. When a network of CPS is subject to frequent changes, the semantic interoperability between the CPSs is of special interest in order to avoid manual, tedious, and error-prone information model alignments at runtime. Ontologies are a suitable technology to enable semantic interoperability, as they allow the building of information models that lank machine-readable meaning to information, thus enabling CPSs to mutually understand the shared information. The contribution of this article is twofold. First, we present an ontology building method that is tailored toward the needs of CPSs in the manufacturing domain. For this purpose, we introduce the requirements regarding this method and discuss related research concerning ontology building. The method itself is designed to begin with ontological requirements and to yield a formal ontology. As the reuse of ontologies and other information resources (IRs) is crucial to the success of ontology building projects, we put special emphasis on how to reuse IRs in the CPS domain. Second, we present a reusable set of ontology design patterns that have been developed with the aforementioned method in an industrial use case and illustrate their application in the considered industrial environment. The contribution of this article extends the method introduced, as a postconference paper, by a detailed industrial application. Note to Practitioners-With growing digitalization in industry, the exchange and use of manufacturing-related data are becoming increasingly important to improve, support, or automate processes. Thus, it is necessary to combine information from different data sources that have been designed by different vendors and may, therefore, be heterogeneous in structure and semantics. A system that plans a maintenance worker's daily schedule, for instance, requires information about the status of machines, production plans, and inventory, which resides in other systems, such as programmable logic controllers (PLCs) or databases. When creating such information systems, accessing, searching, and understanding the different data sources is a time-intensive and error-prone procedure due to the heterogeneities of the data sources. Even worse, this procedure has to be repeated for every newly built system and for every newly introduced data source. To allow for eased access, searching, and understanding of these heterogeneous data sources, ontology can be used to integrate all heterogeneous data sources in one schema. This article contributes a method for building such ontologies in the manufacturing domain. Furthermore, a set of ontology design patterns is presented, which can be reused when building ontologies for a domain.
Price-aware Recommendation with Graph Convolutional Networks

In recent years, much research effort on recommendation has been devoted to mining user behaviors, i.e., collaborative filtering, along with the general information which describes users or items, e.g., textual attributes, categorical demographics, product images, and so on. Price, an important factor in marketing - which determines whether a user will make the final purchase decision on an item - surprisingly, has received relatively little scrutiny. In this work, we aim at developing an effective method to predict user purchase intention with the focus on the price factor in recommender systems. The main difficulties are twofold: 1) the preference and sensitivity of a user on item price are unknown, which are only implicitly reflected in the items that the user has purchased, and 2) how the item price affects a user's intention depends largely on the product category, that is, the perception and affordability of a user on item price could vary significantly across categories. Towards the first difficulty, we propose to model the transitive relationship between user-to-item and item-to-price, taking the inspiration from the recently developed Graph Convolution Networks (GCN). The key idea is to propagate the influence of price on users with items as the bridge, so as to make the learned user representations be price-aware. For the second difficulty, we further integrate item categories into the propagation progress and model the possible pairwise interactions for predicting user-item interactions. We conduct extensive experiments on two real-world datasets, demonstrating the effectiveness of our GCN-based method in learning the price-aware preference of users. Further analysis reveals that modeling the price awareness is particularly useful for predicting user preference on items of unexplored categories.
A Transactional Perspective on Execute-order-validate Blockchains

Smart contracts have enabled blockchain systems to evolve from simple cryptocurrency platforms to general transactional systems. A new architecture called execute-order-validate has been proposed in Hyperledger Fabric to support parallel transactions. However, this architecture might render many invalid transactions when serializing them. This problem is further exaggerated as the block formation rate is inherently limited due to other factors beside data processing, such as cryptography and consensus. Inspired by optimistic concurrency control in modern databases, we propose a novel method to enhance the execute-order-validate architecture, by reordering transactions to reduce the abort rate. In contrast to existing blockchains that adopt database's preventive approaches which might over-abort serializable transactions, our method is theoretically more fine-grained: unserializable transactions are aborted before reordering and the rest are guaranteed to be serializable. We implement our method in two blockchains respectively, FabricSharp on top of Hyperledger Fabric, and FastFabricSharp on top of FastFabric. We compare the performance of FabricSharp with vanilla Fabric and three related systems, two of which are respectively implemented with one standard and one state-of-the-art concurrency control techniques from databases. The results demonstrate that FabricSharp achieves 25% higher throughput compared to the other systems in nearly all experimental scenarios. Moreover, the FastFabricSharp's improvement on FastFabric is up to 66%.
Advanced orthogonal moth flame optimization with Broyden–Fletcher–Goldfarb–Shanno algorithm: Framework and real-world problems

As a typical emergent swarm intelligence algorithm, Moth-Flame Optimization (MFO) has been created to deal with global optimization problems. Since the introduction, it has been applied to various optimization problems. However, MFO may have the trouble of getting into the local best, and the convergence rate cannot be satisfying when handling the high-dimensional and some multimodal problems. In this work, an enhanced MFO integrated with orthogonal learning (OL) and Broyden-Fletcher-Goldfarb-Shanno (BFGS), which we called BFGSOLMFO, is proposed to alleviate the stagnation shortcomings and accelerate the performance of well-regarded MFO. In the BFGSOLMFO, OL is used to construct a better candidate solution for each moth and then guide the whole population to a reasonable potential area. Meanwhile, in each iteration, after the evolution of population finished and the global optima are obtainable, BFGS is employed to further excavate the potential of the global best moth in the current population. With the aim of evaluating the efficacy of the BFGSOLMFO, first of all, the IEEE CEC2014 benchmark set is utilized to measure the performance in solving function optimizations with high-dimensional and multimodal characteristics. Both sets of the IEEE CEC2011 real-world benchmark problems and the three constrained engineering optimization problems are adopted to estimate the performance of BFGSOLMFO in tackling practical scenarios. In all the experiments, the developed BFGSOLMFO is compared with state-of-the-art advanced algorithms. Experimental results and statistical tests demonstrate that the proposed method outperforms the basic MFO and a comprehensive set of advanced algorithms.
"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans

To support human decision making with machine learning models, we often need to elucidate patterns embedded in the models that are unsalient, unknown, or counterintuitive to humans. While existing approaches focus on explaining machine predictions with real-time assistance, we explore model-driven tutorials to help humans understand these patterns in a train- ing phase. We consider both tutorials with guidelines from scientific papers, analogous to current practices of science communication, and automatically selected examples from training data with explanations. We use deceptive review detection as a testbed and conduct large-scale, randomized human-subject experiments to examine the effectiveness of such tutorials. We find that tutorials indeed improve human performance, with and without real-time assistance. In particular, although deep learning provides superior predictive performance than simple models, tutorials and explanations from simple models are more useful to humans. Our work suggests future directions for human-centered tutorials and explanations towards a synergy between humans and AI.
An Efficient Deep Learning Approach for Collaborative Filtering Recommender System

Owing to the enormous growth in information over the past few decades, the world has become a global village. The recommendation system remains the most widely used type of commercial websites. The personalized recommender system is of paramount importance in modeling user’s preference on items based on their past interactions (e.g., ratings and clicks), known as collaborative filtering (CF) technique. Although CF is very important among the algorithms used in recommendation systems, it suffers some setbacks such as the sparsity of matrix ratings, scalability, and integrals nature of data. Several research studies have shown that the above-mentioned obstacle could be tackled with the help of matrix factorization (MF) techniques. In spite of the fact that the technique is likely to suffer from lack of some meaningful signals by using a low ranked approximation as well as lack of sparsity in times of denser singular vectors. Recently, deep learning techniques have proven to learn good representation in natural language processing, image classification, and so on. In this work, we propose a deep learning method of collaborative recommender systems (DLCRS). We have made a comparative study of the proposed method and existing methods. Experimental results demonstrate that our approach gives improved results compared to already existing methods. We empirically evaluate DLCRS on two famous datasets: 100K and 1M Movielens.
Zero-shot Handwritten Chinese Character Recognition with hierarchical decomposition embedding

Handwritten Chinese Character Recognition (HCCR) is a challenging topic in the field of pattern recognition due to large-scale character vocabulary, complex hierarchical structure, various writing styles, and scarce training samples. In this paper, we explored the hierarchical knowledge of Chinese characters and presented a novel zero-shot HCCR method. First, we handled the relations between the characters and their primitives, such as radicals and structures, to obtain a tree layout of primitives. Then, we presented a novel zero-shot hierarchical decomposition embedding method to encode the tree layout into a semantic vector. Next, we devised a Convolutional Neural Network (CNN) based framework to learn both radicals and structures of characters via the semantic vector. As different Chinese characters share some common radicals and structures, our method is able to recognize new categories without any labeled samples from them. Moreover, our method is effective in both traditional HCCR and zero-shot HCCR tasks. It achieves competitive performance on the traditional experiment setting and significantly surpasses the state-of-the-art methods on the zero-shot experiment setting.
Cross-chain interoperability among blockchain-based systems using transactions

Abstract The blockchain is an emerging technology which has the potential to improve many information systems. In this regard, the applications and the platform they are built on must be able to connect and communicate with each other. However, the current blockchain platforms have several limitations, such as lack of interoperability among different systems. The existing platforms of blockchain applications operate only within their own networks. Even though the underlying technology is similar, it relies on centralized third-party mediators to exchange or retrieve information from other blockchain networks. The current third-party intermediaries establish trust and security by preserving a centralized ledger to track ‘account balances’ and vouch for a transaction’s authenticity. The inability for independent blockchains to communicate with one another is an inherent problem in the decentralized systems. Lack of appropriate inter-blockchain communication puts a strain on the mainstream adoption of blockchain. It is evident that blockchain technology has the potential to become a suitable solution for some systems if it can scale and is able to cross communicate with other systems. For the multisystem blockchain concept to become a reality, a mechanism is required that would connect and communicate with multiple entities’ blockchain systems in a distributed fashion (without any intermediary), while maintaining the property of trust and integrity built by individual blockchains. In this article, we propose a mechanism that provides cross-chain interoperability using transactions.
Can GAN Generated Morphs Threaten Face Recognition Systems Equally as Landmark Based Morphs? - Vulnerability and Detection

The primary objective of face morphing is to com-bine face images of different data subjects (e.g. an malicious actor and an accomplice) to generate a face image that can be equally verified for both contributing data subjects. In this paper, we propose a new framework for generating face morphs using a newer Generative Adversarial Network (GAN) - StyleGAN. In contrast to earlier works, we generate realistic morphs of both high-quality and high resolution of 1024 × 1024 pixels. With the newly created morphing dataset of 2500 morphed face images, we pose a critical question in this work. (i) Can GAN generated morphs threaten Face Recognition Systems (FRS) equally as Landmark based morphs? Seeking an answer, we benchmark the vulnerability of a Commercial-Off-The-Shelf FRS (COTS) and a deep learning-based FRS (ArcFace). This work also benchmarks the detection approaches for both GAN generated morphs against the landmark based morphs using established Morphing Attack Detection (MAD) schemes.
A Comparative Evaluation of a Virtual Reality Table and a HoloLens-Based Augmented Reality System for Anatomy Training

Anatomy training with real cadavers poses many practical problems for which new training and educational solutions have been developed making use of technologies based on real-time 3-D graphics. Although virtual reality (VR) and augmented reality (AR) have been previously used in the medical field, it is not easy to select the right 3-D technology or setup for each particular problem. For this reason, this article presents a comprehensive comparative study with 82 participants between two different 3-D interactive setups: an optical-based AR setup, implemented with a Microsoft HoloLens device, and a semi-immersive setup based on a VR Table. Both setups are tested using an anatomy training software application. Our primary hypothesis is that there would be statistically significant differences between the use of the AR application and the use of the VR Table. Our secondary hypothesis is that user preference and recommendation for the VR setup would be higher than for the HoloLens-based system. After completing two different tasks with both setups, the participants filled two questionnaires about the use of the anatomy training application. Three objective measures are also recorded (time, number of movements, and a score). The results of the experiments show that more than two-thirds of the users prefer, recommend, and find more useful the VR setup. The results also show that there are statistically significant differences in the use of both systems in favor of the VR Table.
Counting Out Time: Class Agnostic Video Repetition Counting in the Wild

We present an approach for estimating the period with which an action is repeated in a video. The crux of the approach lies in constraining the period prediction module to use temporal self-similarity as an intermediate representation bottleneck that allows generalization to unseen repetitions in videos in the wild. We train this model, called RepNet, with a synthetic dataset that is generated from a large unlabeled video collection by sampling short clips of varying lengths and repeating them with different periods and counts. This combination of synthetic data and a powerful yet constrained model, allows us to predict periods in a class-agnostic fashion. Our model substantially exceeds the state of the art performance on existing periodicity (PERTUBE) and repetition counting (QUVA) benchmarks. We also collect a new challenging dataset called Countix (~90 times larger than existing datasets) which captures the challenges of repetition counting in real-world videos. Project webpage: https://sites.google.com/view/repnet .
Geometric Structure Based and Regularized Depth Estimation From 360 Indoor Imagery

Motivated by the correlation between the depth and the geometric structure of a 360 indoor image, we propose a novel learning-based depth estimation framework that leverages the geometric structure of a scene to conduct depth estimation. Specifically, we represent the geometric structure of an indoor scene as a collection of corners, boundaries and planes. On the one hand, once a depth map is estimated, this geometric structure can be inferred from the estimated depth map; thus, the geometric structure functions as a regularizer for depth estimation. On the other hand, this estimation also benefits from the geometric structure of a scene estimated from an image where the structure functions as a prior. However, furniture in indoor scenes makes it challenging to infer geometric structure from depth or image data. An attention map is inferred to facilitate both depth estimation from features of the geometric structure and also geometric inferences from the estimated depth map. To validate the effectiveness of each component in our framework under controlled conditions, we render a synthetic dataset, Shanghaitech-Kujiale Indoor 360 dataset with 3550 360 indoor images. Extensive experiments on popular datasets validate the effectiveness of our solution. We also demonstrate that our method can also be applied to counterfactual depth.
Cross-Domain Semantic Segmentation via Domain-Invariant Interactive Relation Transfer

Exploiting photo-realistic synthetic data to train semantic segmentation models has received increasing attention over the past years. However, the domain mismatch between synthetic and real images will cause a significant performance drop when the model trained with synthetic images is directly applied to real-world scenarios. In this paper, we propose a new domain adaptation approach, called Pivot Interaction Transfer (PIT). Our method mainly focuses on constructing pivot information that is common knowledge shared across domains as a bridge to promote the adaptation of semantic segmentation model from synthetic domains to real-world domains. Specifically, we first infer the image-level category information about the target images, which is then utilized to facilitate pixel-level transfer for semantic segmentation, with the assumption that the interactive relation between the image-level category information and the pixel-level semantic information is invariant across domains. To this end, we propose a novel multi-level region expansion mechanism that aligns both the image-level and pixel-level information. Comprehensive experiments on the adaptation from both GTAV and SYNTHIA to Cityscapes clearly demonstrate the superiority of our method.
On the Design of Secure and Efficient Three-Factor Authentication Protocol Using Honey List for Wireless Sensor Networks

The Internet of Thing (IoT) is useful for connecting and collecting variable data of objects through the Internet, which makes to generate useful data for humanity. An indispensable enabler of IoT is the wireless sensor networks (WSNs). Many environments, such as smart healthcare, smart transportation and smart grid, have adopted WSN. Nonetheless, WSNs remain vulnerable to variety of attacks because they send and receive data over public channels. Moreover, the performance of IoT enabled sensor devices has limitations since the sensors are lightweight devices and are resource constrained. To overcome these problems, many security authentication protocols for WSNs have been proposed. However, many researchers have pointed out that preventing smartcard stolen and off-line guessing attacks is an important security issue, and guessing identity and password at the same time is still possible. To address these weaknesses, this paper presents a secure and efficient authentication protocol based on three-factor authentication by taking advantage of biometrics. Meanwhile, the proposed protocol uses a honey_list technique to protect against brute force and stolen smartcard attacks. By using the honey_list technique and three factors, the proposed protocol can provide security even if two of the three factors are compromised. Considering the limited performance of the sensors, we propose an efficient protocol using only hash functions excluding the public key based elliptic curve cryptography. For security evaluation of the proposed authentication protocol, we perform informal security analysis, and Real-Or-Random (ROR) model-based and Burrows Abadi Needham (BAN) logic based formal security analysis. We also perform the formal verification using the widely-used Automated Validation of Internet Security Protocols and Applications (AVISPA) simulation software. Besides, compared to previous researches, we demonstrate that our proposed authentication protocol for WSNs systems is more suitable and secure than others.
RoutedFusion: Learning Real-Time Depth Map Fusion

The efficient fusion of depth maps is a key part of most state-of-the-art 3D reconstruction methods. Besides requiring high accuracy, these depth fusion methods need to be scalable and real-time capable. To this end, we present a novel real-time capable machine learning-based method for depth map fusion. Similar to the seminal depth map fusion approach by Curless and Levoy, we only update a local group of voxels to ensure real-time capability. Instead of a simple linear fusion of depth information, we propose a neural network that predicts non-linear updates to better account for typical fusion errors. Our network is composed of a 2D depth routing network and a 3D depth fusion network which efficiently handle sensor-specific noise and outliers. This is especially useful for surface edges and thin objects for which the original approach suffers from thickening artifacts. Our method outperforms the traditional fusion approach and related learned approaches on both synthetic and real data. We demonstrate the performance of our method in reconstructing fine geometric details from noise and outlier contaminated data on various scenes.