Reviewerly/exHarmony huggingface dataset:
Item response theory in AI: Analysing machine learning classifiers at the instance level

AI systems are usually evaluated on a range of problem instances and compared to other AI systems that use different strategies. These instances are rarely independent. Machine learning, and supervised learning in particular, is a very good example of this. Given a machine learning model, its behaviour for a single instance cannot be understood in isolation but rather in relation to the rest of the data distribution or dataset. In a dual way, the results of one machine learning model for an instance can be analysed in comparison to other models. While this analysis is relative to a population or distribution of models, it can give much more insight than an isolated analysis. Item response theory (IRT) combines this duality between items and respondents to extract latent variables of the items (such as discrimination or difficulty) and the respondents (such as ability). IRT can be adapted to the analysis of machine learning experiments (and by extension to any other artificial intelligence experiments). In this paper, we see that IRT suits classification tasks perfectly, where instances correspond to items and classifiers correspond to respondents. We perform a series of experiments with a range of datasets and classification methods to fully understand what the IRT parameters such as discrimination, difficulty and guessing mean for classification instances (and their relation to instance hardness measures) and how the estimated classifier ability can be used to compare classifier performance in a different way through classifier characteristic curves.
SECTOR: A Neural Model for Coherent Topic Segmentation and Classification

When searching for information, a human reader first glances over a document, spots relevant sections, and then focuses on a few sentences for resolving her intention. However, the high variance of document structure complicates the identification of the salient topic of a given section at a glance. To tackle this challenge, we present SECTOR, a model to support machine reading systems by segmenting documents into coherent sections and assigning topic labels to each section. Our deep neural network architecture learns a latent topic embedding over the course of a document. This can be leveraged to classify local topics from plain text and segment a document at topic shifts. In addition, we contribute WikiSection, a publicly available data set with 242k labeled sections in English and German from two distinct domains: diseases and cities. From our extensive evaluation of 20 architectures, we report a highest score of 71.6% F1 for the segmentation and classification of 30 topics from the English city domain, scored by our SECTOR long short-term memory model with Bloom filter embeddings and bidirectional segmentation. This is a significant improvement of 29.5 points F1 over state-of-the-art CNN classifiers with baseline segmentation.
DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

Radial distortion, which severely hinders object detection and semantic recognition, frequently exists in images captured using a wide-angle lens. Correction of this distortion of images is crucial in many computer vision applications. In this paper, we present distortion rectification generative adversarial network (DR-GAN), a conditional generative adversarial network (GAN) for automatic radial DR. To the best of our knowledge, this is the first end-to-end trainable adversarial framework for radial distortion rectification. The DR-GAN trained using the proposed low-to-high perceptual loss learns the mapping relation between different structural images rather than estimating multifarious distortion parameters, while also realizing label-free training and one-stage rectification. As a benefit of one-stage rectification, the proposed method is extremely fast with the completion of rectification in real time. This is approximately 22 times faster than the state-of-the-art methods. The experimental results show that the DR-GAN achieves an excellent performance in both quantitative measure (PSNR and SSIM) and visual qualitative appearance.
Automatically Assessing Code Understandability

Understanding software is an inherent requirement for many maintenance and evolution tasks. Without a thorough understanding of the code, developers would not be able to fix bugs or add new features timely. Measuring code understandability might be useful to guide developers in writing better code, and could also help in estimating the effort required to modify code components. Unfortunately, there are no metrics designed to assess the understandability of code snippets. In this work, we perform an extensive evaluation of 121 existing as well as new code-related, documentation-related, and developer-related metrics. We try to (i) correlate each metric with understandability and (ii) build models combining metrics to assess understandability. To do this, we use 444 human evaluations from 63 developers and we obtained a bold negative result: none of the 121 experimented metrics is able to capture code understandability, not even the ones assumed to assess quality attributes apparently related, such as code readability and complexity. While we observed some improvements while combining metrics in models, their effectiveness is still far from making them suitable for practical applications. Finally, we conducted interviews with five professional developers to understand the factors that influence their ability to understand code snippets, aiming at identifying possible new metrics.
Classification of Melanocytic Lesions in Selected and Whole-Slide Images via Convolutional Neural Networks

Whole-slide images (WSIs) are a rich new source of biomedical imaging data. The use of automated systems to classify and segment WSIs has recently come to forefront of the pathology research community. While digital slides have obvious educational and clinical uses, their most exciting potential lies in the application of quantitative computational tools to automate search tasks, assist in classic diagnostic classification tasks, and improve prognosis and theranostics. An essential step in enabling these advancements is to apply advances in machine learning and artificial intelligence from other fields to previously inaccessible pathology datasets, thereby enabling the application of new technologies to solve persistent diagnostic challenges in pathology. Here, we applied convolutional neural networks to differentiate between two forms of melanocytic lesions (Spitz and conventional). Classification accuracy at the patch level was 99.0%-2% when applied to WSI. Importantly, when the model was trained without careful image curation by a pathologist, the training took significantly longer and had lower overall performance. These results highlight the utility of augmented human intelligence in digital pathology applications, and the critical role pathologists will play in the evolution of computational pathology algorithms.
Mobile Augmented Reality for Cultural Heritage: Following the Footsteps of Ovid among Different Locations in Europe

There are many examples of cultural events that distinguish people nationally. Celebrating this can bring people closer, as inhabitants of different countries share similar cultural values. This study investigates a sustainable way to enhance these types of events. On the occasion of the 2000-year anniversary of the death of the Roman poet Ovid, we propose a mobile augmented reality (MAR) application that contains historical information related to his life. As Ovid often stated in his last poems, he feared his work would be forgotten after his exile from Rome. This paper focuses on assessing whether this is the case, while also disseminating factual, historic data to people who tested the application. Experiments were conducted in Italy and Romania, in three different cities: Sulmona, Rome, and Constanta. Based on the results collected, four constructs were investigated: comprehensibility, manipulability, enjoyment, and usefulness. The results revealed that the usability of the MAR application, and the interaction metaphor, are appropriate for the general public. The MAR application provided a positive experience, and thus, increased the extent of the occasion.
The effectiveness of intelligent tutoring systems on K‐12 students' reading comprehension: A meta‐analysis

Abstract This meta‐analysis examined the effectiveness of improving reading comprehension for students in K‐12 classrooms using intelligent tutoring systems (ITSs), a computer‐based learning environment that provides customizable and immediate feedback to the learner. Nineteen studies from 13 publications incorporating approximately 10 000 students were included in the final analysis; using robust variance estimation to account for statistical dependencies, the 19 studies yielded 88 effect size estimates. The meta‐analysis indicated that the overall random effect size of ITSs on reading comprehension was 0.60 (using a mix of standardized and researcher‐designed measures) with a 95% confidence interval 0.36 to 0.85 ( p &lt; 0.001). This review confirms previous studies comparing ITSs to human tutoring: ITSs produced a small effect size when compared to human tutoring (0.20, 0.02–0.38, p = 0.036, n = 21). All comparisons to human tutoring used standardized measures. This review also found that ITSs produced a larger effect size on reading comprehension when compared to traditional instruction (0.86) for mixed measures and (0.26) for standardized measures. These findings may be of interest to practitioners and policy makers seeking to improve reading comprehension using consistent and accessible ITSs. Recommendations for researchers include conducting studies to understand the difference between traditional and updated versions of ITSs and employing valid and reliable standardized tests and researcher‐designed measures.
Analyzing the Bitcoin Ponzi Scheme Ecosystem

This paper analyzes the supply and demand for Bitcoin-based Ponzi schemes. There are a variety of these types of scams: from long cons such as Bitcoin Savings & Trust to overnight doubling schemes that do not take off. We investigate what makes some Ponzi schemes successful and others less so. By scouring 11 424 threads on bitcointalk.org , we identify 1 780 distinct scams. Of these, half lasted a week or less. Using survival analysis, we identify factors that affect scam persistence. One approach that appears to elongate the life of the scam is when the scammer interacts a lot with their victims, such as by posting more than a quarter of the comments in the related thread. By contrast, we also find that scams are shorter-lived when the scammers register their account on the same day that they post about their scam. Surprisingly, more daily posts by victims is associated with the scam ending sooner.
Similarity Measures for Closed General Type-2 Fuzzy Sets: Overview, Comparisons, and a Geometric Approach

The similarity between two fuzzy sets (FSs) is an important concept in fuzzy logic. As the research interest on general type-2 (GT2) FSs has increased recently, many similarity measures for them have also been proposed. This paper gives a comprehensive overview of existing similarity measures for GT2 FSs, points out their limitations, and, by using an intuitive geometric explanation, proposes a Jaccard similarity measure for GT2 FSs that is an extension of the popular Jaccard similarity measure for type-1 and interval type-2 FSs. The fundamental difference between the proposed Jaccard similarity measure for GT2 FSs and all existing similarity measures is that the Jaccard similarity measure considers the overall geometries of two GT2 FSs and does not depend on a specific representation of the GT2 FSs, whereas all existing similarity measures for GT2 FSs depend either on the vertical slice representation or the α-plane representation. We show that the Jaccard similarity measure for GT2 FSs satisfies four properties of a similarity measure and demonstrate its reasonableness using two examples.
Scaling Up Dynamic Optimization Problems: A Divide-and-Conquer Approach

Scalability is a crucial aspect of designing efficient algorithms. Despite their prevalence, large-scale dynamic optimization problems are not well studied in the literature. This paper is concerned with designing benchmarks and frameworks for the study of large-scale dynamic optimization problems. We start by a formal analysis of the moving peaks benchmark (MPB) and show its nonseparable nature irrespective of its number of peaks. We then propose a composite MPB suite with exploitable modularity covering a wide range of scalable partially separable functions suitable for the study of large-scale dynamic optimization problems. The benchmark exhibits modularity, heterogeneity, and imbalance features to resemble real-world problems. To deal with the intricacies of large-scale dynamic optimization problems, we propose a decomposition-based coevolutionary framework which breaks a large-scale dynamic optimization problem into a set of lower-dimensional components. A novel aspect of the framework is its efficient bi-level resource allocation mechanism which controls the budget assignment to components and the populations responsible for tracking multiple moving optima. Based on a comprehensive empirical study on a wide range of large-scale dynamic optimization problems with up to 200-D, we show the crucial role of problem decomposition and resource allocation in dealing with these problems. The experimental results clearly show the superiority of the proposed framework over three other approaches in solving large-scale dynamic optimization problems.
Solving Traveling Salesman Problem by Using Combinatorial Artificial Bee Colony Algorithms

Artificial bee colony (ABC) is a quite popular optimization approach that has been used in many fields, with its not only standard form but also improved versions. In this paper, new versions of ABC algorithm to solve TSP are introduced and described in detail. One of these is the combinatorial version of standard ABC, called combinatorial ABC (CABC) algorithm. The other one is an improved version of CABC algorithm, called quick CABC (qCABC) algorithm. In order to see the efficiency of the new versions, 15 different TSP benchmarks are considered and the results generated are compared with different well-known optimization methods. The simulation results show that, both CABC and qCABC algorithms demonstrate good performance for TSP and also the new definition in quick ABC (qABC) improves the convergence performance of CABC on TSP.
Enhancing perception for the visually impaired with deep learning techniques and low-cost wearable sensors

As estimated by the World Health Organization, there are millions of people who lives with some form of vision impairment. As a consequence, some of them present mobility problems in outdoor environments. With the aim of helping them, we propose in this work a system which is capable of delivering the position of potential obstacles in outdoor scenarios. Our approach is based on non-intrusive wearable devices and focuses also on being low-cost. First, a depth map of the scene is estimated from a color image, which provides 3D information of the environment. Then, an urban object detector is in charge of detecting the semantics of the objects in the scene. Finally, the three-dimensional and semantic data is summarized in a simpler representation of the potential obstacles the users have in front of them. This information is transmitted to the user through spoken or haptic feedback. Our system is able to run at about 3.8 fps and achieved a 87.99% mean accuracy in obstacle presence detection. Finally, we deployed our system in a pilot test which involved an actual person with vision impairment, who validated the effectiveness of our proposal for improving its navigation capabilities in outdoors.
Human factor in intelligent manufacturing systems - knowledge acquisition and motivation

People play a central role in intelligent manufacturing systems because of two reasons: their knowledge is indispensable to create and improve intelligent manufacturing systems; and their motivation is very important to identify and solve causes of the problems which may occur in order to prevent them in the future. Therefore, adequate learning methods are required to accomplish these two goals: empower and motivate people. In this paper innovative methods such as learning by doing, simulations and virtual reality will be presented as the ways to transfer the knowledge about intelligent manufacturing systems and to increase motivation concerning their improvements.
Learning QoE of Mobile Video Transmission With Deep Neural Network: A Data-Driven Approach

Quality of experience (QoE) serves as a direct evaluation of users' experiences in mobile video transmission and thus essential for network management, such as network optimization. In this paper, we propose a deep learning-based QoE prediction approach with a large-scale QoE dataset for mobile video transmission. Specifically, we develop a mobile phone application for collecting user QoE data when viewing videos transmitted over the mobile internet in a practical environment. Then, we construct a large-scale dataset by collecting over 80000 piece of data with four kinds of subjective scores and 89 network parameters. Each QoE metric is related to only some of the 89 network parameters. Therefore, we apply the feature selection method to find the feature parameters related to user scores. Additionally, the boxplot method is used to clean the raw data by removing outliers. Finally, a deep neural network (DNN) is developed to learn the relationships between the network parameters and the subjective QoE scores. The proposed DNN can also be seen as a data-driven objective QoE prediction approach for mobile video transmission, which can be used to predict the user QoE scores. The experimental results show that the proposed approach can effectively remove most features irrelevant to QoE prediction. Moreover, the performance of QoE prediction by the proposed model outperforms other state-of-the-art approaches.
Remote Sensing Image Registration Based on Modified SIFT and Feature Slope Grouping

In feature-based remote sensing image registration, the scale-invariant feature transform (SIFT) algorithm has been one of the most popular solutions. However, it is still a challenge to possess an appropriate amount of correct matches while eliminating mismatches. In this letter, inspired by SIFT, an accurate and robust feature matching framework based on feature slope grouping (FSG) for remote sensing image registration is proposed. Our FSG-SIFT algorithm consists of four major phases: modified SIFT, feature slope computation, feature point grouping, and outlier removal and transformation. Specifically, the random sample consensus is adopted to refine the matches followed by the affine transform. The proposed remote sensing image registration algorithm has been validated on a wide variety of high-resolution orthoimagery data. Experimental results with multispectral and multitemporal images suggested that this new image registration algorithm well improved the feature matching accuracy with better registration performance over five state-of-the-art methods.
Whale Optimization Algorithm: Theory, Literature Review, and Application in Designing Photonic Crystal Filters

This chapter presents and analyzes the Whale Optimization Algorithm. The inspiration of this algorithm is first discussed in details, which is the bubble-net foraging behaviour of humpback whales in nature. The mathematical models of this algorithm is then discussed. Due to the large number of applications, a brief literature review of WOA is provided including recent works on the algorithms itself and its applications. The chapter also tests the performance of WOA on several test functions and a real case study in the field of photonic crystal filter. The qualitative and quantitative results show that merits of this algorithm for solving a wide range of challenging problems.
Differential evolution with adaptive mechanism of population size according to current population diversity

A new mechanism for the adaptation of the population size in differential evolution (DE) is described. The adaptive mechanism is based on linear reduction of the population diversity and enables to decrease or increase the population size during the search. The efficiency of DE variants with and without this adaptive mechanism is experimentally compared on the CEC 2014 test suite. Moreover, this new mechanism is also compared with the linear reduction of the population size proposed in L-SHADE and used in its successful successors. The results of the comparison demonstrate the benefit from the new adaptive mechanism to the algorithm efficiency in more than a half of test problems while the deterioration of its efficiency occurs rarely. As it was expected, the use of the proposed mechanism is considerably beneficial, especially in more complex problems like multimodal, hybrid, or composition objective functions.
Analogues of mental simulation and imagination in deep learning

Mental simulation — the capacity to imagine what will or what could be — is a salient feature of human cognition, playing a key role in a wide range of cognitive abilities. In artificial intelligence, the last few years have seen the development of methods which are analogous to mental models and mental simulation. This paper outlines recent methods in deep learning for constructing such models from data and learning to use them via reinforcement learning, and compares such approaches to human mental simulation. Model-based methods in deep learning can serve as powerful tools for building and scaling cognitive models. However, a number of challenges remain in matching the capacity of human mental simulation for efficiency, compositionality, generalization, and creativity.
An IOT-Oriented Privacy-Preserving Publish/Subscribe Model Over Blockchains

In the Internet of Things (IoT), a cyber physical system (CPS) has achieved great success in a wide range of distributed integration environments. In the cyber physical system (CPS), interconnected sensing devices collect data in the surrounding environment and send data to all interested nodes through the network, thereby sharing all nodes data. This process can be implemented by using a publish/subscribe (pub/sub for short) system. Providing the basic security mechanisms such as authorization and confidentiality is a challenge due to the loose coupling of subscribers and publishers in such a pub/sub system. At the meanwhile, the existing IoT ecosystem mostly relies on a centralized server and thus faces the problem of single point failure. Thus, it is interesting to realize a brokerless or decentralized pub/sub model. Inspired by this motivation, this paper mainly proposes a privacy-preserving publish/subscribe model by using the blockchain technique, which evades the centralized trustroot setting and the problem of single point failure. Another key point of our proposal is that the primitive of public key encryption with equality test (PKEwET) is employed to enable all the required authorization, communication and topics matching can be finished in a confidential manner. Finally, a lightweight prototype of our proposal is implemented by using web3j, and the security and efficiency analysis are also presented.