Reviewerly/exHarmony huggingface dataset:
Blockchain and K-Means Algorithm for Edge AI Computing

The current development of blockchain, technically speaking, still faces many key problems such as efficiency and scalability issues, and any distributed system faces the problem of how to balance consistency, availability, and fault tolerance need to be solved urgently. The advantage of blockchain is decentralization, and the most important thing in a decentralized system is how to make nodes reach a consensus quickly. This research mainly discusses the blockchain and K-means algorithm for edge AI computing. The natural pan-central distributed trustworthiness of blockchain provides new ideas for designing the framework and paradigm of edge AI computing. In edge AI computing, multiple devices running AI algorithms are scattered across the edge network. When it comes to decentralized management, blockchain is the underlying technology of the Bitcoin system. Due to its characteristics of immutability, traceability, and consensus mechanism of transaction data storage, it has recently received extensive attention. Blockchain technology is essentially a public ledger. This is done by recording data related to trust management to this ledger. To collaboratively complete artificial intelligence computing tasks or jointly make intelligent group decisions, frequent communication is required between these devices. By integrating idle computing resources in an area, a distributed edge computing platform is formed. Users obtain benefits by sharing their computing resources, and nodes in need complete computing tasks through the shared platform. In view of the identity security problems faced in the sharing process, this article introduces blockchain technology to realize the trust between users. All participants must register a secure identity in the blockchain network and conduct transactions in this security system. A K-means algorithm suitable for edge environments is proposed to identify different degradation stages of equipment operation reflected by multiple types of data. Based on the prediction of the fault state for a single type of data, the algorithm uses the historical data of multiple types of data together with the prediction data to predict the fault stage. During the research process, the average optimization energy consumption of K-means algorithm is 14.6% lower than that of GA. On the basis of designing a resource allocation scheme based on blockchain, the problem of how the participants can realize reliable resource use according to the recorded data on the chain is studied. The article implements the verification of the legality of the use of blockchain resources. In addition, a control node is introduced to master the global real-time information of the network to provide data support for the user's choice.
Identification of Illicit Activities &amp; Scream Detection using Computer Vision &amp; Deep Learning

Self-reliant technologies as well as fully automated technologies are the few technology behemoths that propel entire disciplines forward by various bounds. Significant advances in technology can be tagged to activities which are better handled and are more visible by means of automation. Machine learning is currently being used to make these computer algorithms usable to be smart enough to carry out tasks with right decision making, causing a massive reduce in human interference within redundant processes. Our proposal adheres to the following ideals: creating a solution that reduces manual work (physically and emotionally) for jobs that can be smoothly automated and processed and at the same time addressing the core statement at hand. At times like this, surveillance cameras serve an important role in ensuring people’s safety, but they are only multimedia entities with no intelligence. Because of the increased volume of data generated by security footage, automated video streaming have become necessary for automatically recognising abnormal occurrences. Along with this, audio data of these surveillances are hardly put to use. The project’s main goal is to promote safety by techniques incorporating deep learning to optimise the recording and evaluating crimes from sensory Closed-Circuit Television (CCTV), delegating the liability of recognizing illegal conduct to something like a workflow that can recognize trends and classify them for scanning. Along with this, it aims to detect screams from input audios in cases of failure that can correspond to crimes like rape. Scream Detection Model’s accuracy comes up till 0.95 in differentiating three categories.
ILMART: Interpretable Ranking with Constrained LambdaMART

Interpretable Learning to Rank (LtR) is an emerging field within the research area of explainable AI, aiming at developing intelligible and accurate predictive models. While most of the previous research efforts focus on creating post-hoc explanations, in this paper we investigate how to train effective and intrinsically-interpretable ranking models. Developing these models is particularly challenging and it also requires finding a trade-off between ranking quality and model complexity. State-of-the-art rankers, made of either large ensembles of trees or several neural layers, exploit in fact an unlimited number of feature interactions making them black boxes. Previous approaches on intrinsically-interpretable ranking models address this issue by avoiding interactions between features thus paying a significant performance drop with respect to full-complexity models. Conversely, ILMART, our novel and interpretable LtR solution based on LambdaMART, is able to train effective and intelligible models by exploiting a limited and controlled number of pairwise feature interactions. Exhaustive and reproducible experiments conducted on three publicly-available LtR datasets show that ILMART outperforms the current state-of-the-art solution for interpretable ranking of a large margin with a gain of nDCG of up to 8%.
Face Biometric Authentication System for ATM using Deep Learning

Automated Teller Machines (ATMs) are frequently used in recent years. The ATM has become insecure as the number of crooks and their actions has increased dramatically. This study presents a security architecture for automated teller machines that uses a Deep Convolutional electronic facial recognition. In the financial sector, there is a persistent need to improve security. For identity verification, ATM systems now just require an access card and a PIN. Recent advances in biometric authentication systems, such as fingerprinting, retina scanning, and facial recognition, have gone a long way in resolving the ATM’s dangerous concern. The proposed work is designed with a security paradigm for automated teller machines that uses a Deep Convolutional Neural Network to integrate an access card physically and facial recognition electronically. Faces, as well as their accounts, would be secured if this technique became widely used. For remote certification, a verification portal for face identification is developed and provided to the end user to validate the authorization of an illegal user using certain full-fledged agents of artificial intelligent. It is a challenge to identify the man’s biometric traits which cannot be duplicated. Thus the system will go a long way towards solving the account security paradigm by allowing the actual account to use.
Towards efficient calibration for webcam eye-tracking in online experiments

Calibration is performed in eye-tracking studies to map raw model outputs to gaze-points on the screen and improve accuracy of gaze predictions. Calibration parameters, such as user-screen distance, camera intrinsic properties, and position of the screen with respect to the camera can be easily calculated in controlled offline setups, however, their estimation is non-trivial in unrestricted, online, experimental settings. Here, we propose the application of deep learning models for eye-tracking in online experiments, providing suitable strategies to estimate calibration parameters and perform personal gaze calibration. Focusing on fixation accuracy, we compare results with respect to calibration frequency, the time point of calibration during data collection (beginning, middle, end), and calibration procedure (fixation-point or smooth pursuit-based). Calibration using fixation and smooth pursuit tasks, pooled over three collection time-points, resulted in the best fixation accuracy. By combining device calibration, gaze calibration, and the best-performing deep-learning model, we achieve an accuracy of 2.580−a considerable improvement over reported accuracies in previous online eye-tracking studies.
Decisions are not all equal—Introducing a utility metric based on case-wise raters’ perceptions

Background and Objective Evaluation of AI-based decision support systems (AI-DSS) is of critical importance in practical applications, nonetheless common evaluation metrics fail to properly consider relevant and contextual information. In this article we discuss a novel utility metric, the weighted Utility (wU), for the evaluation of AI-DSS, which is based on the raters' perceptions of their annotation hesitation and of the relevance of the training cases. Methods We discuss the relationship between the proposed metric and other previous proposals; and we describe the application of the proposed metric for both model evaluation and optimization, through three realistic case studies. Results We show that our metric generalizes the well-known Net Benefit, as well as other common error-based and utility-based metrics. Through the empirical studies, we show that our metric can provide a more flexible tool for the evaluation of AI models. We also show that, compared to other optimization metrics, model optimization based on the wU can provide significantly better performance (AUC 0.862 vs 0.895, p-value <0.05), especially on cases judged to be more complex by the human annotators (AUC 0.85 vs 0.92, p-value <0.05). Conclusions We make the point for having utility as a primary concern in the evaluation and optimization of machine learning models in critical domains, like the medical one; and for the importance of a human-centred approach to assess the potential impact of AI models on human decision making also on the basis of further information that can be collected during the ground-truthing process.
Automatically Capturing Quality-Related Concerns in Bug Report Descriptions for Efficient Bug Triaging

In the early phases of a project, software architects and developers design solutions to satisfy quality concerns. However, as a byproduct of the long-term maintenance effort, qualities tend to erode, causing quality-related bugs to surface across the codebase. In principle, quality-related concerns not only can be expensive and difficult to detect, but they can have a detrimental effect on the system operating as intended. Moreover, quality-related concerns can directly affect users’ experiences at large. To address this problem, we build a quality-based bug classifier that leverages several feature selection techniques, TF-IDF, Chi-square (χ2), Mutual Information, and Extra Randomized Trees, including the incorporation of various machine learning algorithms. Our results indicate that Random Forest with the (TF-IDF+χ2) configuration achieved the best results for detecting six-quality related types, achieving a precision of 76%, recall of 70%, and F1 of 70%. However, the same approach returned low precision of 48%, recall of 15%, and F1 of 23% for detecting functional-related bugs. We argue that such low performance has resulted in an aftermath of overlapping content caused by functional and quality-related information which opens another challenging topic that we aim to expand in future work.
Image Classification and Recognition Based on Deep Learning and Random Forest Algorithm

Convolutional neural network (CNN) is an important way to solve the problems of image classification and recognition. It can realize effective feature representation and make continuous breakthroughs in the field of image recognition, but it needs a lot of time in the training process. At the same time, random forest (RF) has the advantages of fast training speed and high classification accuracy. Aiming at the problem of image classification and recognition, this paper proposes a hybrid model based on CNN, which inputs the features extracted by CNN into RF for classification. Since the random weight network can also obtain valid results, the gradient algorithm is not used to adjust the network parameters to avoid consuming a lot of time. Finally, experiments are conducted on MNIST dataset and rotated MNIST dataset, and the results show that the classification accuracy of the hybrid model has improved more than RF, and also, the generalization ability has been improved.
EssayGAN: Essay Data Augmentation Based on Generative Adversarial Networks for Automated Essay Scoring

In large-scale testing and e-learning environments, automated essay scoring (AES) can relieve the burden upon human raters by replacing human grading with machine grading. However, building AES systems based on deep learning requires a training dataset consisting of essays that are manually rated with scores. In this study, we introduce EssayGAN, an automatic essay generator based on generative adversarial networks (GANs). To generate essays rated with scores, EssayGAN has multiple generators for each score range and one discriminator. Each generator is dedicated to a specific score and can generate an essay rated with that score. Therefore, the generators can focus only on generating a realistic-looking essay that can fool the discriminator without considering the target score. Although ordinary-text GANs generate text on a word basis, EssayGAN generates essays on a sentence basis. Therefore, EssayGAN can compose not only a long essay by predicting a sentence instead of a word at each step, but can also compose a score-rated essay by adopting multiple generators dedicated to the target score. Since EssayGAN can generate score-rated essays, the generated essays can be used in the supervised learning process for AES systems. Experimental results show that data augmentation using augmented essays helps to improve the performance of AES systems. We conclude that EssayGAN can generate essays not only consisting of multiple sentences but also maintaining coherence between sentences in essays.
Hieroglyphs Language Translator using deep learning techniques (Scriba)

Egyptian hieroglyphs are symbols that are part of the ancient Egyptian writing system. Hieroglyphs do not have a fixed writing direction, and a single glyph can have multiple definitions, making them difficult to translate into spoken language. Deep learning techniques, such as EfficientNet, MobileNet, and ShuffleNet, have been tested on two hieroglyph datasets (with and without data augmentation). Current experiments revealed that MobileNet and EfficientNet produced the most accurate results Ranging from 99% to 100%. Scriba is a flutter-based mobile application that shall use one of the mentioned algorithms to provide a precise translation of hieroglyphs. This paper thoroughly discusses the proposed development approach, experiments that were carried out (and their results), and the implementation of the mobile application.
Data-driven Visual Query Interfaces for Graphs: Past, Present, and (Near) Future

Visual graph query interfaces (VQI) widen the reach of graph querying frameworks across a variety of end users by enabling non-programmers to use them. Several industrial and academic frameworks for querying graphs expose such visual interfaces. In this tutorial, we survey recent developments in the emerging area of data-driven visual query interface that is grounded on the principles of human-computer interaction (HCI) and cognitive psychology to enhance usability of graph querying frameworks. A data-driven VQI has many benefits such as reducing the cost in constructing and maintaining an interface, superior support for query formulation, and increased portability of the interface. We discuss the notion of making VQIs data-driven and compare it with its classical manual counterpart, and review techniques for automatic construction and maintenance of these interfaces. In addition, the tutorial suggests open problems and new research directions. In summary, in this tutorial, we review and summarize the research thus far into data-driven visual graph query interface management, giving researchers a snapshot of the current state of the art in this topic, and future research directions.
MedicalSeg: A Medical GUI Application for Image Segmentation Management

In the field of medical imaging, the division of an image into meaningful structures using image segmentation is an essential step for pre-processing analysis. Many studies have been carried out to solve the general problem of the evaluation of image segmentation results. One of the main focuses in the computer vision field is based on artificial intelligence algorithms for segmentation and classification, including machine learning and deep learning approaches. The main drawback of supervised segmentation approaches is that a large dataset of ground truth validated by medical experts is required. In this sense, many research groups have developed their segmentation approaches according to their specific needs. However, a generalised application aimed at visualizing, assessing and comparing the results of different methods facilitating the generation of a ground-truth repository is not found in recent literature. In this paper, a new graphical user interface application (MedicalSeg) for the management of medical imaging based on pre-processing and segmentation is presented. The objective is twofold, first to create a test platform for comparing segmentation approaches, and secondly to generate segmented images to create ground truths that can then be used for future purposes as artificial intelligence tools. An experimental demonstration and performance analysis discussion are presented in this paper.
An Improved U-Net Segmentation Model That Integrates a Dual Attention Mechanism and a Residual Network for Transformer Oil Leakage Detection

Accurately detecting oil leakage from a power transformer is important to maintain its normal operation. Deep learning (DL) methods have achieved satisfactory performance in automatic oil detection, but challenges remain due to the small amount of training data and oil targets with large variations in position, shape, and scale. To manage these issues, we propose a dual attention residual U-net (DAttRes-Unet) within a U-net architecture that extensively uses a residual network as well as spatial and channel-wise attention modules. To overcome the vanishing gradient problem due to deeper layers and a small amount of training data, a residual module from ResNet18 is used to construct the encoder path in the U-net framework. Meanwhile, to overcome the issue of training difficulty for the network, inspired by the advantage of transfer learning, initial network parameters in the encoder are obtained from the pre-trained ResNet18 on the ImageNet dataset. Further, in the decoder path, spatial attention and channel attention are integrated to highlight oil-stained regions while suppressing the background or irrelevant parts/channels. To facilitate the acquisition of the fluorescence images of the transformer, we designed a portable acquisition device integrating an ultraviolet light source and a digital camera. The proposed network is trained on the amount of fluorescence images after data augmentation is used and tested on actual fluorescence images. The experiment results show that the proposed DAttRes-Unet network can recognize oil-stained regions with a high accuracy of 98.49% for various shapes and scales of oil leakage.
An Elliptic Centerness for Object Instance Segmentation in Aerial Images

Instance segmentation in aerial images is an important and challenging task. Most of the existing methods have adapted instance segmentation algorithms developed for natural images to aerial images. However, these methods easily suffer from performance degradation in aerial images, due to the scale variations, large aspect ratios, and arbitrary orientations of instances caused by the bird’s-eye view of aerial images. To address this issue, we propose an elliptic centerness (EC) for instance segmentation in aerial images, which can assign the proper centerness values to the intricate aerial instances and thus mitigate the performance degradation. Specifically, we introduce ellipses to fit the various contours of aerial instances and measure these fitted ellipses by two-dimensional anisotropic Gaussian distribution. Armed with EC, we develop a one-stage aerial instance segmentation network. Extensive experiments on a commonly used dataset, the instance segmentation in aerial images dataset (iSAID), demonstrate that our proposed method can achieve a remarkable performance of instance segmentation while introducing negligible computational cost.
MFEN: Lightweight multi-scale feature extraction super-resolution network in embedded system

Deep convolutional neural networks (CNN) have achieved remarkable performance in super-resolution (SR) recently. However, deep CNN-based methods are difficult to be utilized in embedded portable device due to their heavy computation and memory consumption. To solve the above problem, we propose an effective lightweight multi-scale feature extraction super-resolution network (MFEN) by constructing multi-scale feature extraction blocks (MFEB), which progressively obtains multi-scale and hierarchical information. In addition, we also propose an efficient progressive feature fusion (PFF) strategy to aggregate multi-scale informative features. Qualitative and quantitative evaluation results on the benchmark datasets show that our designed method can obtain better performance than most state-of-the-art methods. Moreover, the computation complexity and running time of the MFEN are significantly reduced to provide convenience in real-time image processing technology for embedded devices.
Multilingual Offline Signature Verification Based on Improved Inverse Discriminator Network

To further improve the accuracy of multilingual off-line handwritten signature verification, this paper studies the off-line handwritten signature verification of monolingual and multilingual mixture and proposes an improved verification network (IDN), which adopts user-independent (WI) handwritten signature verification, to determine the true signature or false signature. The IDN model contains four neural network streams with shared weights, of which two receiving the original signature images are the discriminative streams, and the other two streams are the reverse stream of the gray inversion image. The enhanced spatial attention models connect the discriminative streams and reverse flow to realize message propagation. The IDN model uses the channel attention mechanism (SE) and the improved spatial attention module (ESA) to propose the effective feature information of signature verification. Since there is no suitable multilingual signature data set, this paper collects two language data sets (Chinese and Uyghur), including 100,000 signatures of 200 people. Our method is tested on the self-built data set and the public data sets of Bengali (BHsig-B) and Hindi (BHsig-H). The method proposed in this paper has the highest discrimination rate of FRR of 10.5%, FAR of 2.06%, and ACC of 96.33% for the mixture of two languages.