Reviewerly/exHarmony huggingface dataset:
Image Captioning as an Assistive Technology: Lessons Learned from VizWiz 2020 Challenge

Image captioning has recently demonstrated impressive progress largely owing to the introduction of neural network algorithms trained on curated dataset like MS-COCO. Often work in this field is motivated by the promise of deployment of captioning systems in practical applications. However, the scarcity of data and contexts in many competition datasets renders the utility of systems trained on these datasets limited as an assistive technology in real-world settings, such as helping visually impaired people navigate and accomplish everyday tasks. This gap motivated the introduction of the novel VizWiz dataset, which consists of images taken by the visually impaired and captions that have useful, task-oriented information. In an attempt to help the machine learning computer vision field realize its promise of producing technologies that have positive social impact, the curators of the VizWiz dataset host several competitions, including one for image captioning. This work details the theory and engineering from our winning submission to the 2020 captioning competition. Our work provides a step towards improved assistive image captioning systems.&#x0D; This article appears in the special track on AI &amp; Society.
Cloze Distillation: Improving Neural Language Models with Human Next-Word Prediction

Contemporary autoregressive language models (LMs) trained purely on corpus data have been shown to capture numerous features of human incremental processing.However, past work has also suggested dissociations between corpus probabilities and human next-word predictions.Here we evaluate several state-of-theart language models for their match to human next-word predictions and to reading time behavior from eye movements.We then propose a novel method for distilling the linguistic information implicit in human linguistic predictions into pre-trained LMs: Cloze Distillation.We apply this method to a baseline neural LM and show potential improvement in reading time prediction and generalization to held-out human cloze data.
A Person-to-Person and Person-to-Place COVID-19 Contact Tracing System Based on OGC IndoorGML

With the wide availability of low-cost proximity sensors, a large body of research focuses on digital person-to-person contact tracing applications that use proximity sensors. In most contact tracing applications, the impact of SARS-CoV-2 spread through touching contaminated surfaces in enclosed places is overlooked. This study is focused on tracing human contact within indoor places using the open OGC IndoorGML standard. This paper proposes a graph-based data model that considers the semantics of indoor locations, time, and users’ contexts in a hierarchical structure. The functionality of the proposed data model is evaluated for a COVID-19 contact tracing application with scalable system architecture. Indoor trajectory preprocessing is enabled by spatial topology to detect and remove semantically invalid real-world trajectory points. Results show that 91.18% percent of semantically invalid indoor trajectory data points are filtered out. Moreover, indoor trajectory data analysis is innovatively empowered by semantic user contexts (e.g., disinfecting activities) extracted from user profiles. In an enhanced contact tracing scenario, considering the disinfecting activities and sequential order of visiting common places outperformed contact tracing results by filtering out unnecessary potential contacts by 44.98 percent. However, the average execution time of person-to-place contact tracing is increased by 58.3%.
Automatic fuzzy-DBSCAN algorithm for morphological and overlapping datasets

Clustering is one of the unsupervised learning problems. It is a procedure which partitions data objects into groups. Many algorithms could not overcome the problems of morphology, overlapping and the large number of clusters at the same time. Many scientific communities have used the clustering algorithm from the perspective of density, which is one of the best methods in clustering. This study proposes a density-based spatial clustering of applications with noise (DBSCAN) algorithm based on the selected high-density areas by automatic fuzzy-DBSCAN (AFD) which works with the initialization of two parameters. AFD, by using fuzzy and DBSCAN features, is modeled by the selection of high-density areas and generates two parameters for merging and separating automatically. The two generated parameters provide a state of sub-cluster rules in the Cartesian coordinate system for the dataset. The model overcomes the problems of clustering such as morphology, overlapping, and the number of clusters in a dataset simultaneously. In the experiments, all algorithms are performed on eight data sets with 30 times of running. Three of them are related to overlapping real datasets and the rest are morphologic and synthetic datasets. It is demonstrated that the AFD algorithm outperforms other recently developed clustering algorithms.
CxGBERT: BERT meets Construction Grammar

While lexico-semantic elements no doubt capture a large amount of linguistic information, it has been argued that they do not capture all information contained in text.This assumption is central to constructionist approaches to language which argue that language consists of constructions, learned pairings of a form and a function or meaning that are either frequent or have a meaning that cannot be predicted from its component parts.BERT's training objectives give it access to a tremendous amount of lexico-semantic information, and while BERTology has shown that BERT captures certain important linguistic dimensions, there have been no studies exploring the extent to which BERT might have access to constructional information.In this work we design several probes and conduct extensive experiments to answer this question.Our results allow us to conclude that BERT does indeed have access to a significant amount of information, much of which linguists typically call constructional information.The impact of this observation is potentially far-reaching as it provides insights into what deep learning methods learn from text, while also showing that information contained in constructions is redundantly encoded in lexicosemantics.
Image-to-Image Retrieval by Learning Similarity between Scene Graphs

As a scene graph compactly summarizes the high-level content of an image in a structured and symbolic manner, the similarity between scene graphs of two images reflects the relevance of their contents. Based on this idea, we propose a novel approach for image-to-image retrieval using scene graph similarity measured by graph neural networks. In our approach, graph neural networks are trained to predict the proxy image relevance measure, computed from human-annotated captions using a pre-trained sentence similarity model. We collect and publish the dataset for image relevance measured by human annotators to evaluate retrieval algorithms. The collected dataset shows that our method agrees well with the human perception of image similarity than other competitive baselines.
Mobile Assisted Language Learning (MALL): Trends from 2010 to 2020 Using Text Analysis Techniques

Throughout the decade of 2010-2020, the widespread use of mobile devices of any type (smartphones, tablets) has encouraged and strengthened their use in different learning processes and in different ways. Latest improvements in devices’ processing power, in storage capacity, in memory allocation, in wireless connectivity, in GPS and in Bluetooth capabilities, has led to their wider integration and smoother use in the field of learning and in the field of language learning as well. This study examined the trend of a large number of academic studies concerning Mobile Assisted Language Learning (MALL) using text analysis techniques and tools, published in the decade 2010-2020. Over three hundred and forty (340) publications such as journal articles, conferences proceedings papers, books chapters and books were retrieved and analyzed. The preliminary analysis presents the main characteristics and the research trends of this decade and discuss how the field of mobile assisted language learning has evolved in these years.
Knowledge Graph Embeddings in Geometric Algebras

Knowledge graph (KG) embedding aims at embedding entities and relations in a KG into a low dimensional latent representation space.Existing KG embedding approaches model entities and relations in a KG by utilizing real-valued , complex-valued, or hypercomplex-valued (Quaternion or Octonion) representations, all of which are subsumed into a geometric algebra.In this work, we introduce a novel geometric algebra-based KG embedding framework, GeomE, which utilizes multivector representations and the geometric product to model entities and relations.Our framework subsumes several state-of-the-art KG embedding approaches and is advantageous with its ability of modeling various key relation patterns, including (anti-)symmetry, inversion and composition, rich expressiveness with higher degree of freedom as well as good generalization capacity.Experimental results on multiple benchmark knowledge graphs show that the proposed approach outperforms existing state-of-the-art models for link prediction.
Who's Hosting the Block Party? Studying Third-Party Blockage of CSP and SRI

The Web has grown into the most widely used application platform for our daily lives. First-party Web applications thrive due to many different third parties they rely on to provide auxiliary functionality, like maps or ads, to their sites. In this paper, we set out to understand to what extent this outsourcing has adverse effects on two key security mechanisms, namely Content Security Policy (CSP; to mitigate XSS) and Subresource Integrity (SRI; to mitigate third-party compromises) by conducting a longitudinal study over 12 weeks on 10,000 top sites. Under the assumption that a first party wants to deploy CSP and SRI and is able to make their code base compliant with these mechanisms, we assess how many sites could fully deploy the mechanisms without cooperation from their third parties. For those unable to do so without cooperation, we also measure how many third parties would jointly have to make their code compliant to enable first-party usage of CSP and SRI.To more accurately depict trust relations, we rely on holistic views into inclusion chains within all pages of the investigated sites. In addition, based on a combination of heuristics and manual validation, we identify different eTLD+1s belonging to the same business entity, allowing us to more accurately discerning parties from each other. Doing so, we show that the vast majority of sites includes third-party code which necessitates the use of unsafe-inline (75%) or unsafe-eval (61%), or makes deployment of strict-dynamic impossible (76%) without breakage of functionality. For SRI, based on the analysis of a single snapshot (within less than 12 hours), we also show that more than half of all sites cannot fully rely on SRI to protect them from third-party compromise due to randomized third-party content.
Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale

Automatic evaluation of language generation systems is a well-studied problem in Natural Language Processing.While novel metrics are proposed every year, a few popular metrics remain as the de facto metrics to evaluate tasks such as image captioning and machine translation, despite their known limitations.This is partly due to ease of use, and partly because researchers expect to see them and know how to interpret them.In this paper, we urge the community for more careful consideration of how they automatically evaluate their models by demonstrating important failure cases on multiple datasets, language pairs and tasks.Our experiments show that metrics (i) usually prefer system outputs to human-authored texts, (ii) can be insensitive to correct translations of rare words, (iii) can yield surprisingly high scores when given a single sentence as system output for the entire test set.
A Novel Method to Enhance Sustainable Systems Security in Cloud Computing Based on the Combination of Encryption and Data Mining

Due to the increasing growth of technologies and the diversity of user needs in the field of information technology, the position of cloud computing is becoming more apparent. The development of computing infrastructure in any organization requires spending a lot of money, time, and manpower, which sometimes does not fit into the operational capacity of an organization. Therefore, organizations tend to use such technologies to advance their goals. A fully open and distributed structure in cloud computing and its services makes it an attractive target for attackers. This structure includes multiple service-oriented and distributed paradigms, multiple leases, multiple domains, and multi-user autonomous management structures that are more prone to security threats and vulnerabilities. In this paper, the basic concepts of the cloud computing and its applications have been investigated regarding to the importance of security issues. The proposed algorithm improves the level of security in the cloud computing platform through data mining and decision tree algorithm. Low computational burden and client numbers independency help to effectively implement the proposed algorithm in reality.
Use of augmented reality (AR) to aid bioscience education and enrich student experience

In recent years, development of new technologies designed to enhance user experience have accelerated, often being used in modern media such as in films and games. Specifically, immersive experiences, such as virtual reality (VR) and augmented reality (AR), have redefined how digital media can be delivered, encouraging us to interact with and explore our environment. Reciprocally, as the power of these technologies has advanced, the associated costs to implement them have decreased, making them more cost-effective and feasible to deliver in a variety of settings. Despite the cost reduction, several issues remain with accessibility due to the knowledge base required to generate, optimise and deliver three-dimensional (3D)-digital content in both AR and VR. Here, we sought to integrate an AR-based experience into a level-4 biochemistry module in order to support the delivery of university lectures on protein structure and function. Traditionally, this topic would comprise two-dimensional still images of complex 3D structures. By combining a breadth of subject-specific and technological expertise from across the university, we developed an AR-enhanced learning experience hosted on the Zapworks AR platform. AR enabled full illustration of the complexity of these 3D structures, while promoting collaboration through a shared user experience. Assessing the impact of the AR experience via a formative test and survey revealed that despite only a modest increase in test performance, students overwhelmingly reported positively on the engaging nature and interactivity of AR. Critically, expanding our repertoire of content delivery formats will support the forward-thinking blended learning environments adopted across the higher education sector.
Solving the same-different task with convolutional neural networks

Deep learning demonstrated major abilities in solving many kinds of different real-world problems in computer vision literature. However, they are still strained by simple reasoning tasks that humans consider easy to solve. In this work, we probe current state-of-the-art convolutional neural networks on a difficult set of tasks known as the same-different problems. All the problems require the same prerequisite to be solved correctly: understanding if two random shapes inside the same image are the same or not. With the experiments carried out in this work, we demonstrate that residual connections, and more generally the skip connections, seem to have only a marginal impact on the learning of the proposed problems. In particular, we experiment with DenseNets, and we examine the contribution of residual and recurrent connections in already tested architectures, ResNet-18, and CorNet-S respectively. Our experiments show that older feed-forward networks, AlexNet and VGG, are almost unable to learn the proposed problems, except in some specific scenarios. We show that recently introduced architectures can converge even in the cases where the important parts of their architecture are removed. We finally carry out some zero-shot generalization tests, and we discover that in these scenarios residual and recurrent connections can have a stronger impact on the overall test accuracy. On four difficult problems from the SVRT dataset, we can reach state-of-the-art results with respect to the previous approaches, obtaining super-human performances on three of the four problems.
Online rating system development using blockchain-based distributed ledger technology

In most websites, the online rating system provides the ratings of products and services to users. Lack of trust in data integrity and its manipulation has hindered fulfilling user satisfaction. Since existing online rating systems deal with a central server, all rating data is stored on the central server. Therefore, all rating data can be removed, modified, and manipulated by the system manager to change the ratings in favor of the service or product provider. In this paper, an online rating system using distributed ledger technologies has been presented as the proposed system to solve all the weaknesses of current systems. Distributed ledger technologies are completely decentralized and there is no centralization on them by any institution. Distributed ledger technologies have different variants. Among distributed ledger technologies, blockchain technology has been used in the proposed rating system because of its support for smart contracts. In the proposed online rating system, the Ethereum platform has been chosen from different blockchain platforms that have a public permission network. In this system, the raters cannot rate unless they submit a request to the system and be authorized to take part in the online product rating process. The important feature of the Ethereum platform is its support for smart contracts, which can be used to write the rating contract in the Solidity language. Also, using Proof of Authority consensus mechanisms, all rating transactions are approved by the surveyors. Since in the real Ethereum system, each rating transaction is sent to the network by the raters, some gas must be paid for each rating transaction. However, since this method is expensive, TestNet blockchain can be used in the rating system. Finally, the proposed rating system was used for rating the restaurants of a website and its features were tested.
Single- and Cross-Modality Near Duplicate Image Pairs Detection via Spatial Transformer Comparing CNN

Recently, both single modality and cross modality near-duplicate image detection tasks have received wide attention in the community of pattern recognition and computer vision. Existing deep neural networks-based methods have achieved remarkable performance in this task. However, most of the methods mainly focus on the learning of each image from the image pair, thus leading to less use of the information between the near duplicate image pairs to some extent. In this paper, to make more use of the correlations between image pairs, we propose a spatial transformer comparing convolutional neural network (CNN) model to compare near-duplicate image pairs. Specifically, we firstly propose a comparing CNN framework, which is equipped with a cross-stream to fully learn the correlation information between image pairs, while considering the features of each image. Furthermore, to deal with the local deformations led by cropping, translation, scaling, and non-rigid transformations, we additionally introduce a spatial transformer comparing CNN model by incorporating a spatial transformer module to the comparing CNN architecture. To demonstrate the effectiveness of the proposed method on both the single-modality and cross-modality (Optical-InfraRed) near-duplicate image pair detection tasks, we conduct extensive experiments on three popular benchmark datasets, namely CaliforniaND (ND means near duplicate), Mir-Flickr Near Duplicate, and TNO Multi-band Image Data Collection. The experimental results show that the proposed method can achieve superior performance compared with many state-of-the-art methods on both tasks.
Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection

Real-life applications, heavily relying on machine learning, such as dialog systems, demand for out-of-domain detection methods. Intent classification models should be equipped with a mechanism to distinguish seen intents from unseen ones so that the dialog agent is capable of rejecting the latter and avoiding undesired behavior. However, despite increasing attention paid to the task, the best practices for out-of-domain intent detection have not yet been fully established. This paper conducts a thorough comparison of out-of-domain intent detection methods. We prioritize the methods, not requiring access to out-of-domain data during training, gathering of which is extremely time- and labor-consuming due to lexical and stylistic variation of user utterances. We evaluate multiple contextual encoders and methods, proven to be efficient, on three common datasets for intent classification, expanded with out-of-domain utterances. Our main findings show that fine-tuning Transformer-based encoders on in-domain data leads to superior results. Mahalanobis distance, together with utterance representations, derived from Transformer-based encoders, outperform other methods by a wide margin(1-5% in terms of AUROC) and establish new state-of-the-art results for all datasets. The broader analysis shows that the reason for success lies in the fact that the fine-tuned Transformer is capable of constructing homogeneous representations of in-domain utterances, revealing geometrical disparity to out of domain utterances. In turn, the Mahalanobis distance captures this disparity easily.
Economic Development in Mexico and the Role of Blockchain

This paper studies recent economic and political developments in Mexico and the potential role of blockchain technology in improving governance and fighting corruption in the country.Mexico badly needs innovative solutions to fight the persistent problems of corruption and inefficient governance.At the same time, blockchain, being a decentralized technology, promises to reduce administrative costs and ensure transparency in the public services.The paper takes stock of the current state of blockchain implementation in the economy and in the public administration of Mexico as well as studies the advantages and disadvantages related to the further outspread of the technology in the country.Moreover, the work aims at drawing policy recommendations as to the future role of blockchain in the public management and economic development in Mexico.