Reviewerly/exHarmony huggingface dataset:
Efficient and Effective Sparse LSTM on FPGA with Bank-Balanced Sparsity

Neural networks based on Long Short-Term Memory (LSTM) are widely deployed in latency-sensitive language and speech applications. To speed up LSTM inference, previous research proposes weight pruning techniques to reduce computational cost. Unfortunately, irregular computation and memory accesses in unrestricted sparse LSTM limit the realizable parallelism, especially when implemented on FPGA. To address this issue, some researchers propose block-based sparsity patterns to increase the regularity of sparse weight matrices, but these approaches suffer from deteriorated prediction accuracy. This work presents Bank-Balanced Sparsity (BBS), a novel sparsity pattern that can maintain model accuracy at a high sparsity level while still enable an efficient FPGA implementation. BBS partitions each weight matrix row into banks for parallel computing, while adopts fine-grained pruning inside each bank to maintain model accuracy. We develop a 3-step software-hardware co-optimization approach to apply BBS in real FPGA hardware. First, we propose a bank-balanced pruning method to induce the BBS pattern on weight matrices. Then we introduce a decoding-free sparse matrix format, Compressed Sparse Banks (CSB), that transparently exposes inter-bank parallelism in BBS to hardware. Finally, we design an FPGA accelerator that takes advantage of BBS to eliminate irregular computation and memory accesses. Implemented on Intel Arria-10 FPGA, the BBS accelerator can achieve 750.9 GOPs on sparse LSTM networks with a batch size of 1. Compared to state-of-the-art FPGA accelerators for LSTM with different compression techniques, the BBS accelerator achieves 2.3 ~ 3.7x improvement on energy efficiency and 7.0 ~ 34.4x reduction on latency with negligible loss of model accuracy.
A novel image steganography technique based on quantum substitution boxes

Substitution boxes play an essential role in designing secure cryptosystems. With the evolution of quantum technologies, current data security mechanisms may be broken due to their construction based on mathematical computation. Quantum walks, a universal quantum computational model, play an essential role in designing quantum algorithms. We utilize the benefits of quantum walks to present a novel technique for constructing substitution boxes (S-boxes) based on quantum walks (QWs). The performance of the presented QWs S-box technique is evaluated by S-box evaluation criteria, and our results prove that the constructed S-box has vital qualities for viable applications in security purposes. Furthermore, a new technique for image steganography is constructed. The proposed technique is an integrated mechanism between classical data hiding and quantum walks to achieve better security for the embedded data. The embedding and extraction procedures are controlled by QWs S-box. The inclusion of cryptographic QWs S-box ensures the security of both embedding and extraction phases. At the extraction phase, only the stego image and the secret values are needed for constructing the secret data. Experimental results demonstrate that the presented technique has a high-security, high embedding capacity and good visual quality.
A Call to Unify Definitions of Virtual Reality

Virtual reality (VR) will be widely adopted by nursing within the next five years as a simulation method. The confusion generated by the various emerging definitions of VR led to the authors to review various definitions and to make a recommendation for the use of the concepts immersion and presence as a way to define VR.
Intelligent computing for numerical treatment of nonlinear prey–predator models

In this study, a new computing paradigm is presented for evaluation of dynamics of nonlinear prey–predator mathematical model by exploiting the strengths of integrated intelligent mechanism through artificial neural networks, genetic algorithms and interior-point algorithm. In the scheme, artificial neural network based differential equation models of the system are constructed and optimization of the networks is performed with effective global search ability of genetic algorithm and its hybridization with interior-point algorithm for rapid local search. The proposed technique is applied to variants of nonlinear prey–predator models by taking different rating factors and comparison with Adams numerical solver certify the correctness for each scenario. The statistical studies have been conducted to authenticate the accuracy and convergence of the design methodology in terms of mean absolute error, root mean squared error and Nash-Sutcliffe efficiency performance indices.
A Modified LSTM Model for Continuous Sign Language Recognition Using Leap Motion

Sign language facilitates communication between hearing impaired peoples and the rest of the society. A number of sign language recognition (SLR) systems have been developed by researchers, but they are limited to isolated sign gestures only. In this paper, we propose a modified long short-term memory (LSTM) model for continuous sequences of gestures or continuous SLR that recognizes a sequence of connected gestures. It is based on splitting of continuous signs into sub-units and modeling them with neural networks. Thus, the consideration of a different combination of sub-units is not required during training. The proposed system has been tested with 942 signed sentences of Indian Sign Language (ISL). These sign sentences are recognized using 35 different sign words. The average accuracy of 72.3% and 89.5% has been recorded on signed sentences and isolated sign words, respectively.
VerifAI: A Toolkit for the Formal Design and Analysis of Artificial Intelligence-Based Systems

We present VerifAI, a software toolkit for the formal design and analysis of systems that include artificial intelligence (AI) and machine learning (ML) components. VerifAI particularly addresses challenges with applying formal methods to ML components such as perception systems based on deep neural networks, as well as systems containing them, and to model and analyze system behavior in the presence of environment uncertainty. We describe the initial version of VerifAI, which centers on simulation-based verification and synthesis, guided by formal models and specifications. We give examples of several use cases, including temporal-logic falsification, model-based systematic fuzz testing, parameter synthesis, counterexample analysis, and data set augmentation.
How Will Blockchain Technology Impact Auditing and Accounting: Permissionless versus Permissioned Blockchain

SUMMARY Blockchain offers a drastically new way to record, process, and store financial transactions and information, and has the potential to fundamentally change the landscape of the accounting profession and reshape the business ecosystem. In this article, we introduce two types (i.e., permissionless and permissioned) of blockchain and lay out their technological features. We further discuss implications of blockchain to auditing and elaborate on opportunities and challenges of the two types of blockchain to auditors. We conclude by making specific recommendations for auditors to adapt, adjust, and elevate themselves to the role of strategic partners in blockchain implementation. JEL Classifications: M15; M41; M42; O14; O33; O55.
Multimodal Representation Learning for Recommendation in Internet of Things

The recommender system has recently drawn a lot of attention to the communities of information services and mobile applications. Many deep learning-based recommendation models have been proposed to learn the feature representations from items. However, in Internet of Things (IoT), items' description information are typically heterogeneous and multimodal, posing a challenge to items' representation learning of recommendation models. To address this challenge and to improve the recommendation effectiveness in IoT, a novel multimodal representation learning-based model (MRLM) has been proposed. In MRLM, two closely related modules were trained simultaneously; they are global feature representation learning and multimodal feature representation learning. The former was designed to learn to accurately represent the global features of items and users through simultaneous training on three tasks: 1) triplet metric learning; 2) softmax classification; and 3) microscopic verification. The latter was proposed to refine items' global features and to generate the final multimodal features by using items' multimodal description information. After MRLM converged, items' multimodal features and users' global features could be used to calculate users' preferences on items via cosine similarity. Through extensive experiments on two real-world datasets, MRLM remarkably improved the recommendation effectiveness in IoT.
Grouped Spatial-Temporal Aggregation for Efficient Action Recognition

Temporal reasoning is an important aspect of video analysis. 3D CNN shows good performance by exploring spatial-temporal features jointly in an unconstrained way, but it also increases the computational cost a lot. Previous works try to reduce the complexity by decoupling the spatial and temporal filters. In this paper, we propose a novel decomposition method that decomposes the feature channels into spatial and temporal groups in parallel. This decomposition can make two groups focus on static and dynamic cues separately. We call this grouped spatial-temporal aggregation (GST). This decomposition is more parameter-efficient and enables us to quantitatively analyze the contributions of spatial and temporal features in different layers. We verify our model on several action recognition tasks that require temporal reasoning and show its effectiveness.
Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge

This paper presents our method for enabling a UAV quadrotor, equipped with a monocular camera, to autonomously avoid collisions with obstacles in unstructured and unknown indoor environments. When compared to obstacle avoidance in ground vehicular robots, UAV navigation brings in additional challenges because the UAV motion is no more constrained to a well-defined indoor ground or street environment. Unlike ground vehicular robots, a UAV has to navigate across more types of obstacles - for e.g., objects like decorative items, furnishings, ceiling fans, sign-boards, tree branches, etc., are also potential obstacles for a UAV. Thus, methods of obstacle avoidance developed for ground robots are clearly inadequate for UAV navigation. Current control methods using monocular images for UAV obstacle avoidance are heavily dependent on environment information. These controllers do not fully retain and utilize the extensively available information about the ambient environment for decision making. We propose a deep reinforcement learning based method for UAV obstacle avoidance (OA) which is capable of doing exactly the same. The crucial idea in our method is the concept of partial observability and how UAVs can retain relevant information about the environment structure to make better future navigation decisions. Our OA technique uses recurrent neural networks with temporal attention and provides better results compared to prior works in terms of distance covered without collisions. In addition, our technique has a high inference rate and reduces power wastage as it minimizes oscillatory motion of UAV.
Blockchain technology, improvement suggestions, security challenges on smart grid and its application in healthcare for sustainable development

Blockchain technology has gained considerable attention for different types of stakeholders due to its stable implementation in the field of digital currency like Bitcoin. Some users use Bitcoin for payment exchanges against any business while others use the Bitcoin network for earning Bitcoins itself, and there is also another type of user who called hackers those flood different types of attacks to illegally earn some Bitcoins or collapsing overall network. There are also numerous uses of blockchain technology, e.g. health, automation industry, energy sector, security and authentication in smart grids. In this study, we have elaborated on different critical aspects of Blockchain technology like its style of working mechanism, possible improvement suggestions by using Proof-of-Stake, and other custom variations, attempting seven types of challenges by different novel techniques. Moreover, we have also explained the current state-of-the-artwork in blockchain’s non-financial applications like Healthcare in which contribution of four-layered custom blockchain models related to precision medicine and the clinical trial was notable. Moreover, a mobile app model called HDG for the automation of medical records without compromising privacy was also a prominent contribution.
An efficient Harris hawks-inspired image segmentation method

Segmentation is a crucial phase in image processing because it simplifies the representation of an image and facilitates its analysis. The multilevel thresholding method is more efficient for segmenting digital mammograms compared to the classic bi-level thresholding since it uses a higher number of intensities to represent different regions in the image. In the literature, there are different techniques for multilevel segmentation; however, most of these approaches do not obtain good segmented images. In addition, they are computationally expensive. Recently, statistical criteria such as Otsu, Kapur, and cross-entropy have been utilized in combination with evolutionary and swarm-based strategies to investigate the optimal threshold values for multilevel segmentation. In this paper, an efficient methodology for multilevel segmentation is proposed using the Harris Hawks Optimization (HHO) algorithm and the minimum cross-entropy as a fitness function. To substantiate the results and effectiveness of the HHO-based method, it has been tested over a benchmark set of reference images, with the Berkeley segmentation database, and with medical images of digital mammography. The proposed HHO-based solver is verified based on other comparable optimizers and two machine learning algorithms K-means and the Fuzzy IterAg. The comparisons were performed based on three groups. This first one is to provide evidence of the optimization capabilities of the HHO using the Wilcoxon test, and the second is to verify segmented image quality using the PSNR, SSIM, and FSIM metrics. Then, the third way is to verify the segmented image comparing it with the ground-truth through the metrics PRI, GCE, and VoI. The experimental results, which are validated by statistical analysis, show that the introduced method produces efficient and reliable results in terms of quality, consistency, and accuracy in comparison with the other methods. This HHO-based method presents an improvement over other segmentation approaches that are currently used in the literature.
Iterative Answer Prediction With Pointer-Augmented Multimodal Transformers for TextVQA

Many visual scenes contain text that carries crucial information, and it is thus essential to understand text in images for downstream reasoning tasks. For example, a deep water label on a warning sign warns people about the danger in the scene. Recent work has explored the TextVQA task that requires reading and understanding text in images to answer a question. However, existing approaches for TextVQA are mostly based on custom pairwise fusion mechanisms between a pair of two modalities and are restricted to a single prediction step by casting TextVQA as a classification task. In this work, we propose a novel model for the TextVQA task based on a multimodal transformer architecture accompanied by a rich representation for text in images. Our model naturally fuses different modalities homogeneously by embedding them into a common semantic space where self-attention is applied to model inter- and intra- modality context. Furthermore, it enables iterative answer decoding with a dynamic pointer network, allowing the model to form an answer through multi-step prediction instead of one-step classification. Our model outperforms existing approaches on three benchmark datasets for the TextVQA task by a large margin.
Does learning require memorization? a short tale about a long tail

State-of-the-art results on image recognition tasks are achieved using over-parameterized learning algorithms that (nearly) perfectly fit the training set and are known to fit well even random labels. This tendency to memorize seemingly useless training data labels is not explained by existing theoretical analyses. Memorization of the training data also presents significant privacy risks when the training data contains sensitive personal information and thus it is important to understand whether such memorization is necessary for accurate learning.
From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality

Blind or no-reference (NR) perceptual picture quality prediction is a difficult, unsolved problem of great consequence to the social and streaming media industries that impacts billions of viewers daily. Unfortunately, popular NR prediction models perform poorly on real-world distorted pictures. To advance progress on this problem, we introduce the largest (by far) subjective picture quality database, containing about 40, 000 real-world distorted pictures and 120, 000 patches, on which we collected about 4M human judgments of picture quality. Using these picture and patch quality labels, we built deep region-based architectures that learn to produce state-of-the-art global picture quality predictions as well as useful local picture quality maps. Our innovations include picture quality prediction architectures that produce global-to-local inferences as well as local-to-global inferences (via feedback). The dataset and source code are available at https: //live.ece.utexas.edu/research.php.
Smart Secure Sensing for IoT-Based Agriculture: Blockchain Perspective

Agriculture is a vital area for the sustenance of mankind engulfing manufacturing, security, traceability, and sustainable resource management. With the resources receding expeditiously, it is of utmost significance to innovate techniques that help in the subsistence of agriculture. The growth of Internet of Things (IoT) and Blockchain technology as two rapidly emerging fields can ameliorate the state of food chain today. This paper provides a rigorous literature review to inspect the state-of-the-art development of the schemes that provide information security using blockchain technology. After identifying the core requirements in smart agriculture, a generalized blockchain-based security architecture has been proposed. A detailed cost analysis has been conducted on the studied schemes. A meticulous comparative analysis uncovered the drawbacks in existing research. Furthermore, detailed analysis of the literature has also revealed the security goals towards which the research has been directed and helped to identify new avenues for future research using artificial intelligence.
Industry 4.0 and the supply chain digitalisation: a blockchain diffusion perspective

The emergence of Industry 4.0 has brought in its wake an important number of challenges and opportunities for organisations across the globe. To cope with such a fast-changing environment, organisations have been steadily implementing different types of technologies, and at different stages. One of the most disruptive and promising technology is blockchain, and its potential to transform various aspects of organisations’ business and operations, including the supply chain relationships, is tremendous. In line with the global research trend in this domain, this paper proposes a multi-stage model of adoption (intention, adoption, and routinisation stages), for a better understanding of blockchain diffusion across supply chains. We drew on the diffusion of innovations theory, the resource-based view, dynamic capability, the technology adoption model, and the institutional theory to propose a multi-stage model. We validated the model using PLS-SEM, which was applied on data collected in India and the U.S. Our results showed that, from one country to another, there are essential differences in the variables that determine blockchain innovation and in the stage of diffusion. Additionally, our proposed model provided a good explanation at all stages of blockchain diffusion. This study offers significant and valuable contributions in terms of theory and management.
Rediscovering the use of chatbots in education: A systematic literature review

Abstract Chatbots have been around for years and have been used in many areas such as medicine or commerce. Our focus is on the development and current uses of chatbots in the field of education, where they can function as service assistants or as educational agents. In this research paper, we attempt to make a systematic review of the literature on educational chatbots that address various issues. From 485 sources, 80 studies on chatbots and their application in education were selected through a step‐by‐step procedure based on the guidelines of the PRISMA framework, using a set of predefined criteria. The results obtained demonstrate the existence of different types of educational chatbots currently in use that affect student learning or improve services in various areas. This paper also examines the type of technology used to unravel the learning outcome that can be obtained from each type of chatbots. Finally, our results identify instances where a chatbot can assist in learning under conditions similar to those of a human tutor, while exploring other possibilities and techniques for assessing the quality of chatbots. Our analysis details these findings and can provide a solid framework for research and development of chatbots for the educational field.
A comprehensive review of deep learning in colon cancer

Deep learning has emerged as a leading machine learning tool in object detection and has attracted attention with its achievements in progressing medical image analysis. Convolutional Neural Networks (CNNs) are the most preferred method of deep learning algorithms for this purpose and they have an essential role in the detection and potential early diagnosis of colon cancer. In this article, we hope to bring a perspective to progress in this area by reviewing deep learning practices for colon cancer analysis. This study first presents an overview of popular deep learning architectures used in colon cancer analysis. After that, all studies related to colon cancer analysis are collected under the field of colon cancer and deep learning, then they are divided into five categories that are detection, classification, segmentation, survival prediction, and inflammatory bowel diseases. Then, the studies collected under each category are summarized in detail and listed. We conclude our work with a summary of recent deep learning practices for colon cancer analysis, a critical discussion of the challenges faced, and suggestions for future research. This study differs from other studies by including 135 recent academic papers, separating colon cancer into five different classes, and providing a comprehensive structure. We hope that this study is beneficial to researchers interested in using deep learning techniques for the diagnosis of colon cancer.
DLFix

Automated Program Repair (APR) is very useful in helping developers in the process of software development and maintenance. Despite recent advances in deep learning (DL), the DL-based APR approaches still have limitations in learning bug-fixing code changes and the context of the surrounding source code of the bug-fixing code changes. These limitations lead to incorrect fixing locations or fixes. In this paper, we introduce DLFix, a two-tier DL model that treats APR as code transformation learning from the prior bug fixes and the surrounding code contexts of the fixes. The first layer is a tree-based RNN model that learns the contexts of bug fixes and its result is used as an additional weighting input for the second layer designed to learn the bug-fixing code transformations.