Reviewerly/exHarmony huggingface dataset:
Cross-Modal Relationship Inference for Grounding Referring Expressions

Grounding referring expressions is a fundamental yet challenging task facilitating human-machine communication in the physical world. It locates the target object in an image on the basis of the comprehension of the relationships between referring natural language expressions and the image. A feasible solution for grounding referring expressions not only needs to extract all the necessary information (i.e. objects and the relationships among them) in both the image and referring expressions, but also compute and represent multimodal contexts from the extracted information. Unfortunately, existing work on grounding referring expressions cannot extract multi-order relationships from the referring expressions accurately and the contexts they obtain have discrepancies with the contexts described by referring expressions. In this paper, we propose a Cross-Modal Relationship Extractor (CMRE) to adaptively highlight objects and relationships, that have connections with a given expression, with a cross-modal attention mechanism, and represent the extracted information as a language-guided visual relation graph. In addition, we propose a Gated Graph Convolutional Network (GGCN) to compute multimodal semantic contexts by fusing information from different modes and propagating multimodal information in the structured relation graph. Experiments on various common benchmark datasets show that our Cross-Modal Relationship Inference Network, which consists of CMRE and GGCN, outperforms all existing state-of-the-art methods.
Augmented Reality in Educational Inclusion. A Systematic Review on the Last Decade

The use of Augmented Reality (AR) to achieve educational inclusion has been not deeply explored. This systematic review describes the current state of using AR as an educational technology that takes into consideration the needs of all students including those with a disability. It is done through the analysis of factors, such as the advantages of AR, its limitations, uses, challenges, its scope in the educational field, the attended population and the positive or negative effects of its use in learning scenarios that involve students with diverse educational needs. A total of 50 studies between 2008 and 2018 were analyzed through searching in three interdisciplinary databases: Scopus, Web of Science, and Springer link. For this, the methodological stages considered were planning the review, search, analysis of literature and results report. After analyzing the results, it was possible to demonstrate that the use of AR for inclusive education in the field of sciences is where more studies have been conducted. In regard to the population with disabilities, among the most representative advantages reported were the motivation, interaction and generating interest on the part of the student. At the same time, an important methodological limitation identified was the size of the sample; some investigations were done with two or three subjects, some studies Single Subject Designs were found. In terms of the population attended, the studies generally included students with different impairments (hearing, visual, motor or cognitive), minorities (ethnic, vulnerable), leaving aside other groups excluded as exceptional talents and immigrants, which could be explored in the future. Despite different problems to be addressed, few frameworks to the diversity attention in education were reported, and there was no model and methodology in inclusive education considered in the studies. Finally, from this review we have identified open issues that could give rise to new research in the subject of using AR to favor the creation of inclusive learning scenarios.
Real-time Credit Card Fraud Detection Using Machine Learning

Credit card fraud events take place frequently and then result in huge financial losses [1]. The number of online transactions has grown in large quantities and online credit card transactions holds a huge share of these transactions. Therefore, banks and financial institutions offer credit card fraud detection applications much value and demand. Fraudulent transactions can occur in various ways and can be put into different categories. This paper focuses on four main fraud occasions in real-world transactions. Each fraud is addressed using a series of machine learning models and the best method is selected via an evaluation. This evaluation provides a comprehensive guide to selecting an optimal algorithm with respect to the type of the frauds and we illustrate the evaluation with an appropriate performance measure. Another major key area that we address in our project is real-time credit card fraud detection. For this, we take the use of predictive analytics done by the implemented machine learning models and an API module to decide if a particular transaction is genuine or fraudulent. We also assess a novel strategy that effectively addresses the skewed distribution of data. The data used in our experiments come from a financial institution according to a confidential disclosure agreement.
Local Relationship Learning With Person-Specific Shape Regularization for Facial Action Unit Detection

Encoding individual facial expressions via action units (AUs) coded by the Facial Action Coding System (FACS) has been found to be an effective approach in resolving the ambiguity issue among different expressions. While a number of methods have been proposed for AU detection, robust AU detection in the wild remains a challenging problem because of the diverse baseline AU intensities across individual subjects, and the weakness of appearance signal of AUs. To resolve these issues, in this work, we propose a novel AU detection method by utilizing local information and the relationship of individual local face regions. Through such a local relationship learning, we expect to utilize rich local information to improve the AU detection robustness against the potential perceptual inconsistency of individual local regions. In addition, considering the diversity in the baseline AU intensities of individual subjects, we further regularize local relationship learning via person-specific face shape information, i.e., reducing the influence of person-specific shape information, and obtaining more AU discriminative features. The proposed approach outperforms the state-of-the-art methods on two widely used AU detection datasets in the public domain (BP4D and DISFA).
An image-text consistency driven multimodal sentiment analysis approach for social media

Social media users are increasingly using both images and text to express their opinions and share their experiences, instead of only using text in the conventional social media. Consequently, the conventional text-based sentiment analysis has evolved into more complicated studies of multimodal sentiment analysis. To tackle the challenge of how to effectively exploit the information from both visual content and textual content from image-text posts, this paper proposes a new image-text consistency driven multimodal sentiment analysis approach. The proposed approach explores the correlation between the image and the text, followed by a multimodal adaptive sentiment analysis method. To be more specific, the mid-level visual features extracted by the conventional SentiBank approach are used to represent visual concepts, with the integration of other features, including textual, visual and social features, to develop a machine learning sentiment analysis approach. Extensive experiments are conducted to demonstrate the superior performance of the proposed approach.
Segmentation and Classification of Cervical Cells Using Deep Learning

Cervical cancer is the fourth most prevalent disease in women. Accurate and timely cancer detection can save lives. Automatic and reliable cervical cancer detection methods can be devised through the accurate segmentation and classification of Pap smear cell images. This paper presents an approach to whole cervical cell segmentation using a mask regional convolutional neural network (Mask R-CNN) and classifies this using a smaller Visual Geometry Group-like Network (VGG-like Net). ResNet10 is used to make full use of spatial information and prior knowledge as the backbone of the Mask R-CNN. We evaluate our proposed method on the Herlev Pap Smear dataset. In the segmentation phase, when Mask R-CNN is applied on the whole cell, it outperforms the previous segmentation method in precision (0.92±0.06), recall (0.91±0.05) and ZSI (0.91±0.04). In the classification phase, VGG-like Net is applied on the whole segmented cell and yields a sensitivity score of more than 96% with low standard deviation (±2.8%) for the binary classification problem and yields a higher result of more than 95% with low standard deviation (maximum 4.2% in accuracy measurement) for the 7-class problem in terms of sensitivity, specificity, accuracy, h-mean, and F1 score.
Optical and Electromagnetic Tracking Systems for Biomedical Applications: A Critical Review on Potentialities and Limitations

Optical and electromagnetic tracking systems represent the two main technologies integrated into commercially-available surgical navigators for computer-assisted image-guided surgery so far. Optical Tracking Systems (OTSs) work within the optical spectrum to track the position and orientation, i.e., pose of target surgical instruments. OTSs are characterized by high accuracy and robustness to environmental conditions. The main limitation of OTSs is the need of a direct line-of-sight between the optical markers and the camera sensor, rigidly fixed into the operating theatre. Electromagnetic Tracking Systems (EMTSs) use electromagnetic field generator to detect the pose of electromagnetic sensors. EMTSs do not require such a direct line-of-sight, however the presence of metal or ferromagnetic sources in the operating workspace can significantly affect the measurement accuracy. The aim of the proposed review is to provide a complete and detailed overview of optical and electromagnetic tracking systems, including working principles, source of error and validation protocols. Moreover, commercial and research-oriented solutions, as well as clinical applications, are described for both technologies. Finally, a critical comparative analysis of the state of the art which highlights the potentialities and the limitations of each tracking system for a medical use is provided.
BloHosT: Blockchain Enabled Smart Tourism and Hospitality Management

In the era of Industry 4.0, e-tourism uses bulk of digital payments through applications supported by heterogeneous payment gateways. These heterogeneous payment gateways open the doors for the attackers to perform malicious activities such as-hacking of wallet accounts, identity theft, attacks on payment clearance cycles. In e-tourism, financial data is maintained in a centralized cloud server, which can lead to payment failures during peak traffic. The aforementioned issues can be addressed by the usage of a decentralized mechanism such as-blockchain, which enables trust and reputation management among various stakeholders such as-banks, travel agencies, airports, railways, cruises, hotels, restaurants, and local taxis. Motivated by the above discussion, we propose a framework named as BloHosT (Blockchain Enabled Smart Tourism and Hospitality Management), which allows tourists to interact with various stakeholders through a single wallet identifier linked with a cryptocurrency server to initiate payments. BloHosT uses an immutable ledger, where no proofs are required during travel that provides a hassle-free experience to tourists. Also, a Tourism enabled Deep-Learning (TeDL) framework is presented as a part of BloHosT framework, which is trained on experience of previous visited travelers. It provides rating scores to prospective travelers about the recently visited locations by previous travelers. Finally, through case studies, we demonstrate that BloHosT achieves a high Return of Investment (ROI) in tourism sector as compared to traditional frameworks.
Cloud/Edge Computing Service Management in Blockchain Networks: Multi-leader Multi-follower Game-based ADMM for Pricing

The mining process in public blockchains with the Nakamoto consensus protocol requires solving a computational puzzle, i.e., proof-of-work, which is resource expensive to implement in lightweight devices with limited computing resources and energy. Thus, renting mining service from cloud providers becomes a reasonable solution, which is called cloud mining. This enables users who want to mine, i.e., miners, to purchase and lease an amount of hashing power from the cloud/edge providers without any hassle of managing the infrastructure. In this paper, we study the interactions among the cloud/edge providers and miners in blockchain using a multi-leader multi-follower game-theoretic approach, in order to support proof-of-work based blockchains application. Due to the inherent complexity of the formulated game, we employ the Alternating Direction Method of Multipliers (ADMM) algorithm to investigate the optimum solution. Utilizing the decomposition characteristics and fast convergence of ADMM, we obtain the optimum results in a distributed manner. Simulation results demonstrate that with the proposed solutions, the optimization of the utilities of miners and the profits of providers can be jointly achieved.
Blockchain Mutability: Challenges and Proposed Solutions

Blockchain's evolution during the past decade is astonishing: from bitcoin to over 2.000 altcoins, and from decentralised electronic payments to transactions programmable by smart contracts and complex tokens governed by decentralised organisations. While the new generation of blockchain applications is still evolving, blockchain's technical characteristics are also advancing. Yet, immutability, a hitherto indisputable and highly advertised property according to which blockchain data cannot be edited nor deleted, remains the cornerstone of blockchain's security. Nevertheless, blockchain's immutability is being called into question lately in the light of the new erasing requirements imposed by the GDPR's “ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Right to be Forgotten (RtbF)</i> ” provision. As the RtbF obliges blockchain data to be editable in order restricted content redactions, modifications or deletions to be applied when requested, blockchains compliance with the regulation is indeed challenging, if not impracticable. Towards resolving this contradiction, various methods and techniques for mutable blockchains have been proposed to satisfy regulatory erasing requirements while preserving blockchains’ security. To this end, this work aims to provide a comprehensive review on the state-of-the-art research approaches, technical workarounds and advanced cryptographic techniques that have been put forward to resolve this conflict and to discuss their potentials, constraints and limitations when applied in the wild to either permissioned or permissionless blockchains.
Experimental Implementation of Universal Nonadiabatic Geometric Quantum Gates in a Superconducting Circuit

Using geometric phases to realize noise-resilient quantum computing is an important method to enhance the control fidelity. In this work, we experimentally realize a universal nonadiabatic geometric quantum gate set in a superconducting qubit chain. We characterize the realized single- and two-qubit geometric gates with both quantum process tomography and randomized benchmarking methods. The measured average fidelities for the single-qubit rotation gates and two-qubit controlled-Z gate are 0.9977(1) and 0.977(9), respectively. Besides, we also experimentally demonstrate the noise-resilient feature of the realized single-qubit geometric gates by comparing their performance with the conventional dynamical gates with different types of errors in the control field. Thus, our experiment proves a way to achieve high-fidelity geometric quantum gates for robust quantum computation.
Perception-based model for analyzing the impact of enterprise blockchain adoption on SCM in the Indian service industry

Permissioned blockchain utilized by any organization is commonly known as Enterprise Blockchain (EBC). EBC represents a popular investment option as enterprise system in the service industry. This paper aims at analyzing the perception of EBC among practitioners in service industry. As significant hype prevails about EBC usefulness in service industry, we examine if the perception of the usefulness of EBC among practitioners is borne out of knowledge of theoretical benefits of EBC, or is it borne out of the hype. A scale for measuring EBC benefits is developed. In order to test the hypothesized relationships, structural equation modelling (SEM) is used. We extend the concept of perceived usefulness from technology acceptance model for service supply chain management (SCM) and measure the perceived usefulness of EBC for six supply chain dimensions in the service industry. We examine if the service industry managers perceive EBC to be profitable based on the practical usefulness of EBC in various dimensions of service SCM. Moderating effects of 5 factors are analysed on relationships of our research model. This paper provides insights about the dimensions of service SCM, in which service industry practitioners perceive EBC to be useful and may contribute to increment in organization’s profitability.
Edge-based differential privacy computing for sensor–cloud systems

In sensor–cloud systems, with more personal data being hosted in cloud, privacy leakage is becoming one of the most serious concerns. Privacy computing is emerging as a paradigm to systematically enhance privacy protection. In other words, the new paradigm requests us to improve the computing model to provide a general privacy protection service. In this paper, we propose an edge-based model for data collection, in which the raw data from wireless sensor networks (WSNs) is differentially processed by algorithms on edge servers for privacy computing. A small quantity of the core data is stored on edge and local servers while the rest is transmitted to cloud for storage. In this way, the benefits are twofold. First, the data privacy is preserved since the original data cannot be retrieved even if the data stored in the cloud is leaked. Second, implemented by a differential storage method, compared to the state of the art, the edge-based model sends less data to the cloud and reduces the cost of communication and storage. Both theoretical analyses and extensive experiments validate our proposed method.
Access Control for Electronic Health Records with Hybrid Blockchain-Edge Architecture

The global Electronic Health Record (EHR) market is growing dramatically and expected to reach $39.7 billions by 2022. To safe-guard security and privacy of EHR, access control is an essential mechanism for managing EHR data. This paper proposes a hybrid architecture to facilitate access control of EHR data by using both blockchain and edge node. Within the architecture, a blockchain-based controller manages identity and access control policies and serves as a tamper-proof log of access events. In addition, off-chain edge nodes store the EHR data and apply policies specified in Abbreviated Language For Authorization (ALFA) to enforce attribute-based access control on EHR data in collaboration with the blockchain-based access control logs. We evaluate the proposed hybrid architecture by utilizing Hyperledger Composer Fabric blockchain to measure the performance of executing smart contracts and ACL policies in terms of transaction processing time and response time against unauthorized data retrieval.
Crossref: The sustainable source of community-owned scholarly metadata

This paper describes the scholarly metadata collected and made available by Crossref, as well as its importance in the scholarly research ecosystem. Containing over 106 million records and expanding at an average rate of 11% a year, Crossref’s metadata has become one of the major sources of scholarly data for publishers, authors, librarians, funders, and researchers. The metadata set consists of 13 content types, including not only traditional types, such as journals and conference papers, but also data sets, reports, preprints, peer reviews, and grants. The metadata is not limited to basic publication metadata, but can also include abstracts and links to full text, funding and license information, citation links, and the information about corrections, updates, retractions, etc. This scale and breadth make Crossref a valuable source for research in scientometrics, including measuring the growth and impact of science and understanding new trends in scholarly communications. The metadata is available through a number of APIs, including REST API and OAI-PMH. In this paper, we describe the kind of metadata that Crossref provides and how it is collected and curated. We also look at Crossref’s role in the research ecosystem and trends in metadata curation over the years, including the evolution of its citation data provision. We summarize the research used in Crossref’s metadata and describe plans that will improve metadata quality and retrieval in the future.
Big data mining for the estimation of hourly rooftop photovoltaic potential and its uncertainty

The large-scale deployment of photovoltaics (PV) on building rooftops can play a significant role in the transition to a low-carbon energy system. To date, the lack of high-resolution building and environmental data and the large uncertainties related to existing processing methods impede the accurate estimation of large-scale rooftop PV potentials. To address this gap, we developed a methodology that combines Machine Learning algorithms, Geographic Information Systems and physical models to estimate the technical PV potential for individual roof surfaces at hourly temporal resolution. We further estimate the uncertainties related to each step of the potential assessment and combine them to quantify the uncertainty on the final PV potential. The methodology is applied to 9.6 million rooftops in Switzerland and can be transferred to any large region or country with sufficient available data. Our results suggest that 55% of the total Swiss roof surface is available for the installation of PV panels, yielding an annual technical rooftop PV potential of 24±9TWh. This could meet more than 40% of Switzerland’s current annual electricity demand. The presented method for an hourly rooftop PV potential and uncertainty estimation can be applied to the large-scale assessment of future energy systems with decentralised electricity grids. The results can be used to propose effective policies for the integration of rooftop photovoltaics in the built environment.
An Effective Optimization Method for Machine Learning Based on ADAM

A machine is taught by finding the minimum value of the cost function which is induced by learning data. Unfortunately, as the amount of learning increases, the non-liner activation function in the artificial neural network (ANN), the complexity of the artificial intelligence structures, and the cost function’s non-convex complexity all increase. We know that a non-convex function has local minimums, and that the first derivative of the cost function is zero at a local minimum. Therefore, the methods based on a gradient descent optimization do not undergo further change when they fall to a local minimum because they are based on the first derivative of the cost function. This paper introduces a novel optimization method to make machine learning more efficient. In other words, we construct an effective optimization method for non-convex cost function. The proposed method solves the problem of falling into a local minimum by adding the cost function in the parameter update rule of the ADAM method. We prove the convergence of the sequences generated from the proposed method and the superiority of the proposed method by numerical comparison with gradient descent (GD, ADAM, and AdaMax).