Reviewerly/exHarmony huggingface dataset:
Understanding the role of individual units in a deep neural network

Deep neural networks excel at finding hierarchical representations that solve complex tasks over large data sets. How can we humans understand these learned representations? In this work, we present network dissection, an analytic framework to systematically identify the semantics of individual hidden units within image classification and image generation networks. First, we analyze a convolutional neural network (CNN) trained on scene classification and discover units that match a diverse set of object concepts. We find evidence that the network has learned many object classes that play crucial roles in classifying scene classes. Second, we use a similar analytic method to analyze a generative adversarial network (GAN) model trained to generate scenes. By analyzing changes made when small sets of units are activated or deactivated, we find that objects can be added and removed from the output scenes while adapting to the context. Finally, we apply our analytic framework to understanding adversarial attacks and to semantic image editing.
Blockchain-Enhanced Data Sharing With Traceable and Direct Revocation in IIoT

The industrial Internet of Things (IIoT) supports recent developments in data management and information services, as well as services for smart factories. Nowadays, many mature IIoT cloud platforms are available to serve smart factories. However, due to the semicredibility nature of the IIoT cloud platforms, how to achieve secure storage, access control, information update and deletion for smart factory data, as well as the tracking and revocation of malicious users has become an urgent problem. To solve these problems, in this article, a blockchain-enhanced security access control scheme that supports traceability and revocability has been proposed in IIoT for smart factories. The blockchain first performs unified identity authentication, and stores all public keys, user attribute sets, and revocation list. The system administrator then generates system parameters and issues private keys to users. The domain administrator is responsible for formulating domain security and privacy-protection policies, and performing encryption operations. If the attributes meet the access policies and the user's ID is not in the revocation list, they can obtain the intermediate decryption parameters from the edge/cloud servers. Malicious users can be tracked and revoked during all stages if needed, which ensures the system security under the Decisional Bilinear Diffie-Hellman (DBDH) assumption and can resist multiple attacks. The evaluation has shown that the size of the public/private keys is smaller compared to other schemes, and the overhead time is less for public key generation, data encryption, and data decryption stages.
Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning

We address the challenging problem of whole slide image (WSI) classification. WSIs have very high resolutions and usually lack localized annotations. WSI classification can be cast as a multiple instance learning (MIL) problem when only slide-level labels are available. We propose a MIL-based method for WSI classification and tumor detection that does not require localized annotations. Our method has three major components. First, we introduce a novel MIL aggregator that models the relations of the instances in a dual-stream architecture with trainable distance measurement. Second, since WSIs can produce large or unbalanced bags that hinder the training of MIL models, we propose to use self-supervised contrastive learning to extract good representations for MIL and alleviate the issue of prohibitive memory cost for large bags. Third, we adopt a pyramidal fusion mechanism for multiscale WSI features, and further improve the accuracy of classification and localization. Our model is evaluated on two representative WSI datasets. The classification accuracy of our model compares favorably to fully-supervised methods, with less than 2% accuracy gap across datasets. Our results also outperform all previous MIL-based methods. Additional benchmark results on standard MIL datasets further demonstrate the superior performance of our MIL aggregator on general MIL problems.
Emotion and sentiment analysis from Twitter text

Online social networks have emerged as new platform that provide an arena for people to share their views and perspectives on different issues and subjects with their friends, family, relatives, etc. We can share our thoughts, mental state, moments, stand on specific social, national, international issues through text, photos, audio and video messages and posts. Indeed, despite the availability of other forms of communication, text is still one of the most common ways of communication in a social network. The target of the work described in this paper is to detect and analyze sentiment and emotion expressed by people from text in their twitter posts and use them for generating recommendations. We collected tweets and replies on few specific topics and created a dataset with text, user, emotion, sentiment information, etc. We used the dataset to detect sentiment and emotion from tweets and their replies and measured the influence scores of users based on various user-based and tweet-based parameters. Finally, we used the latter information to generate generalized and personalized recommendations for users based on their twitter activity. The method we used in this paper includes some interesting novelties such as, (i) including replies to tweets in the dataset and measurements, (ii) introducing agreement score, sentiment score and emotion score of replies in influence score calculation, (iii) generating general and personalized recommendation containing list of users who agreed on the same topic and expressed similar emotions and sentiments towards that particular topic.
CONV-SRAM: An Energy-Efficient SRAM With In-Memory Dot-Product Computation for Low-Power Convolutional Neural Networks

This paper presents an energy-efficient static random access memory (SRAM) with embedded dot-product computation capability, for binary-weight convolutional neural networks. A 10T bit-cell-based SRAM array is used to store the 1-b filter weights. The array implements dot-product as a weighted average of the bitline voltages, which are proportional to the digital input values. Local integrating analog-to-digital converters compute the digital convolution outputs, corresponding to each filter. We have successfully demonstrated functionality (>98% accuracy) with the 10 000 test images in the MNIST hand-written digit recognition data set, using 6-b inputs/outputs. Compared to conventional full-digital implementations using small bitwidths, we achieve similar or better energy efficiency, by reducing data transfer, due to the highly parallel in-memory analog computations.
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making

Machine learning (ML) is increasingly being used in image retrieval systems for medical decision making. One application of ML is to retrieve visually similar medical images from past patients (e.g. tissue from biopsies) to reference when making a medical decision with a new patient. However, no algorithm can perfectly capture an expert's ideal notion of similarity for every case: an image that is algorithmically determined to be similar may not be medically relevant to a doctor's specific diagnostic needs. In this paper, we identified the needs of pathologists when searching for similar images retrieved using a deep learning algorithm, and developed tools that empower users to cope with the search algorithm on-the-fly, communicating what types of similarity are most important at different moments in time. In two evaluations with pathologists, we found that these tools increased the diagnostic utility of images found and increased user trust in the algorithm. The tools were preferred over a traditional interface, without a loss in diagnostic accuracy. We also observed that users adopted new strategies when using refinement tools, re-purposing them to test and understand the underlying algorithm and to disambiguate ML errors from their own errors. Taken together, these findings inform future human-ML collaborative systems for expert decision-making.
Review of Artificial Intelligence Adversarial Attack and Defense Technologies

In recent years, artificial intelligence technologies have been widely used in computer vision, natural language processing, automatic driving, and other fields. However, artificial intelligence systems are vulnerable to adversarial attacks, which limit the applications of artificial intelligence (AI) technologies in key security fields. Therefore, improving the robustness of AI systems against adversarial attacks has played an increasingly important role in the further development of AI. This paper aims to comprehensively summarize the latest research progress on adversarial attack and defense technologies in deep learning. According to the target model’s different stages where the adversarial attack occurred, this paper expounds the adversarial attack methods in the training stage and testing stage respectively. Then, we sort out the applications of adversarial attack technologies in computer vision, natural language processing, cyberspace security, and the physical world. Finally, we describe the existing adversarial defense methods respectively in three main categories, i.e., modifying data, modifying models and using auxiliary tools.
Quantum Natural Gradient

A quantum generalization of Natural Gradient Descent is presented as part of a general-purpose optimization framework for variational quantum circuits. The optimization dynamics is interpreted as moving in the steepest descent direction with respect to the Quantum Information Geometry, corresponding to the real part of the Quantum Geometric Tensor (QGT), also known as the Fubini-Study metric tensor. An efficient algorithm is presented for computing a block-diagonal approximation to the Fubini-Study metric tensor for parametrized quantum circuits, which may be of independent interest.
Vision-based human activity recognition: a survey

Abstract Human activity recognition (HAR) systems attempt to automatically identify and analyze human activities using acquired information from various types of sensors. Although several extensive review papers have already been published in the general HAR topics, the growing technologies in the field as well as the multi-disciplinary nature of HAR prompt the need for constant updates in the field. In this respect, this paper attempts to review and summarize the progress of HAR systems from the computer vision perspective. Indeed, most computer vision applications such as human computer interaction, virtual reality, security, video surveillance and home monitoring are highly correlated to HAR tasks. This establishes new trend and milestone in the development cycle of HAR systems. Therefore, the current survey aims to provide the reader with an up to date analysis of vision-based HAR related literature and recent progress in the field. At the same time, it will highlight the main challenges and future directions.
Golden eagle optimizer: A nature-inspired metaheuristic algorithm

This paper proposes a nature-inspired swarm-based metaheuristic for solving global optimization problems called Golden Eagle Optimizer (GEO). The core inspiration of GEO is the intelligence of golden eagles in tuning speed at different stages of their spiral trajectory for hunting. They show more propensity to cruise around and search for prey in the initial stages of hunting and more propensity to attack in the final stages. A golden eagle adjusts these two components to catch the best possible prey in feasible region the shortest possible time. This behavior is mathematically modeled to highlight exploration and exploitation for a global optimization method. The performance of the proposed algorithm is tested and confirmed using 33 benchmark test functions and a scalability test. Results were compared to that of six other well-known algorithms, which revealed GEO’s superiority, which indicates that it can find the global optimum and avoid local optima effectively. The Multi-Objective Golden Eagle Optimizer (MOGEO) is also proposed to solve multi-objective problems. The performance of MOGEO is also tested and verified on ten multi-objective benchmark functions. Results were compared to that of two other multi-objective algorithms, which showed that it can approximate true Pareto optimal solutions better than the other two algorithms. The software (toolbox) and source code for GEO and MOGEO are also provided, which are publicly available.
Highlighting photonics: looking into the next decade

Abstract Let there be light –to change the world we want to be! Over the past several decades, and ever since the birth of the first laser, mankind has witnessed the development of the science of light, as light-based technologies have revolutionarily changed our lives. Needless to say, photonics has now penetrated into many aspects of science and technology, turning into an important and dynamically changing field of increasing interdisciplinary interest. In this inaugural issue of eLight , we highlight a few emerging trends in photonics that we think are likely to have major impact at least in the upcoming decade, spanning from integrated quantum photonics and quantum computing, through topological/non-Hermitian photonics and topological insulator lasers, to AI-empowered nanophotonics and photonic machine learning. This Perspective is by no means an attempt to summarize all the latest advances in photonics, yet we wish our subjective vision could fuel inspiration and foster excitement in scientific research especially for young researchers who love the science of light .
Particle Swarm Optimization Algorithm and Its Applications: A Systematic Review

Abstract Throughout the centuries, nature has been a source of inspiration, with much still to learn from and discover about. Among many others, Swarm Intelligence (SI), a substantial branch of Artificial Intelligence, is built on the intelligent collective behavior of social swarms in nature. One of the most popular SI paradigms, the Particle Swarm Optimization algorithm (PSO), is presented in this work. Many changes have been made to PSO since its inception in the mid 1990s. Since their learning about the technique, researchers and practitioners have developed new applications, derived new versions, and published theoretical studies on the potential influence of various parameters and aspects of the algorithm. Various perspectives are surveyed in this paper on existing and ongoing research, including algorithm methods, diverse application domains, open issues, and future perspectives, based on the Systematic Review (SR) process. More specifically, this paper analyzes the existing research on methods and applications published between 2017 and 2019 in a technical taxonomy of the picked content, including hybridization, improvement, and variants of PSO, as well as real-world applications of the algorithm categorized into: health-care, environmental, industrial, commercial, smart city, and general aspects applications. Some technical characteristics, including accuracy, evaluation environments, and proposed case study are involved to investigate the effectiveness of different PSO methods and applications. Each addressed study has some valuable advantages and unavoidable drawbacks which are discussed and has accordingly yielded some hints presented for addressing the weaknesses of those studies and highlighting the open issues and future research perspectives on the algorithm.
Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances

Emotion is intrinsic to humans and consequently, emotion understanding is a key part of human-like artificial intelligence (AI).Emotion recognition in conversation (ERC) is becoming increasingly popular as a new research frontier in natural language processing (NLP) due to its ability to mine opinions from the plethora of publicly available conversational data on platforms such as Facebook, Youtube, Reddit, Twitter, and others.Moreover, it has potential applications in health-care systems (as a tool for psychological analysis), education (understanding student frustration), and more.In Addition, ERC is also extremely important for generating emotion-aware dialogues that require an understanding of the user's emotions.Catering to these needs calls for effective and scalable conversational emotion-recognition algorithms.However, it is a difficult problem to solve because of several research challenges.In this paper, we discuss these challenges and shed light on recent research in this field.We also describe the drawbacks of these approaches and discuss the reasons why they fail to successfully overcome the research challenges in ERC. INDEX TERMS Emotion recognition
An Efficient Methodology for Mapping Quantum Circuits to the IBM QX Architectures

In the past years, quantum computers more and more have evolved from an academic idea to an upcoming reality. IBM's project IBM Q can be seen as evidence of this progress. Launched in March 2017 with the goal to provide access to quantum computers for a broad audience, this allowed users to conduct quantum experiments on a 5-qubit and, since June 2017, also on a 16-qubit quantum computer (called IBM QX2 and IBM QX3, respectively). Revised versions of these 5- and 16-qubit quantum computers (named IBM QX4 and IBM QX5, respectively) are available since September 2017. In order to use these, the desired quantum functionality (e.g., provided in terms of a quantum circuit) has to be properly mapped so that the underlying physical constraints are satisfied-a complex task. This demands solutions to automatically and efficiently conduct this mapping process. In this paper, we propose a methodology which addresses this problem, i.e., maps the given quantum functionality to a realization which satisfies all constraints given by the architecture and, at the same time, keeps the overhead in terms of additionally required quantum gates minimal. The proposed methodology is generic, can easily be configured for similar future architectures, and is fully integrated into IBM's SDK. Experimental evaluations show that the proposed approach clearly outperforms IBM's own mapping solution. In fact, for many quantum circuits, the proposed approach determines a mapping to the IBM architecture within minutes, while IBM's solution suffers from long runtimes and runs into a timeout of 1 h in several cases. As an additional benefit, the proposed approach yields mapped circuits with smaller costs (i.e., fewer additional gates are required). All implementations of the proposed methodology are publicly available at http://iic.jku.at/eda/research/ibm_qx_mapping.
Optimization of Big Data Scheduling in Social Networks

In social network big data scheduling, it is easy for target data to conflict in the same data node. Of the different kinds of entropy measures, this paper focuses on the optimization of target entropy. Therefore, this paper presents an optimized method for the scheduling of big data in social networks and also takes into account each task’s amount of data communication during target data transmission to construct a big data scheduling model. Firstly, the task scheduling model is constructed to solve the problem of conflicting target data in the same data node. Next, the necessary conditions for the scheduling of tasks are analyzed. Then, the a periodic task distribution function is calculated. Finally, tasks are scheduled based on the minimum product of the corresponding resource level and the minimum execution time of each task is calculated. Experimental results show that our optimized scheduling model quickly optimizes the scheduling of social network data and solves the problem of strong data collision.
Trading Private Range Counting over Big IoT Data

Data privacy arises as one of the most important concerns, facing the pervasive commoditization of big data statistic analysis in Internet of Things (IoT). Current solutions are incapable to thoroughly solve the privacy issues on data pricing and guarantee the utility of statistic outputs. Therefore, this paper studies the problem of trading private statistic results for IoT data, by considering three factors. Specifically, a novel framework for trading range counting results is proposed. The framework applies a sampling-based method to generate approximated counting results, which are further perturbed for privacy concerns and then released. The results are theoretically proved to achieve unbiasedness, bounded variance, and strengthened privacy guarantee under differential privacy. Moreover, a pricing approach is proposed for the traded results, which is proved to be immune against arbitrage attacks. The framework is evaluated by estimating the air pollution levels with different ranges on 2014 CityPulse Smart City datasets. The analysis and evaluation results demonstrate that our framework greatly reduces the error of range counting approximation; and the optimal perturbation approach enables that the private counting satisfies the specified approximation degree while providing strong privacy guarantee.
System architecture for blockchain based transparency of supply chain social sustainability

Social sustainability is a major concern in global supply chains for protecting workers from exploitation and for providing a safe working environment. Although there are stipulated standards to govern supply chain social sustainability, it is not uncommon to hear of businesses being reported for noncompliance issues. Even reputable firms such as Unilever have been criticized for production labor exploitation. Consumers now increasingly expect sellers to disclose information on social sustainability, but sellers are confronted with the challenge of traceability in their multi-tier global supply chains. Blockchain offers a promising future to achieve instant traceability in supply chain social sustainability. This study develops a system architecture that integrates the use of blockchain, internet-of-things (IoT) and big data analytics to allow sellers to monitor their supply chain social sustainability efficiently and effectively. System implementation cost and potential challenges are analyzed before the research is concluded.
SIGMA: A Sparse and Irregular GEMM Accelerator with Flexible Interconnects for DNN Training

The advent of Deep Learning (DL) has radically transformed the computing industry across the entire spectrum from algorithms to circuits. As myriad application domains embrace DL, it has become synonymous with a genre of workloads across vision, speech, language, recommendations, robotics, and games. The key compute kernel within most DL workloads is general matrix-matrix multiplications (GEMMs), which appears frequently during both the forward pass (inference and training) and backward pass (training). GEMMs are a natural choice for hardware acceleration to speed up training, and have led to 2D systolic architectures like NVIDIA tensor cores and Google Tensor Processing Unit (TPU). Unfortunately, emerging GEMMs in DL are highly irregular and sparse, which lead to poor data mappings on systolic architectures. This paper proposes SIGMA, a flexible and scalable architecture that offers high utilization of all its processing elements (PEs) regardless of kernel shape and sparsity. Within SIGMA includes a novel reduction tree microarchitecture named Forwarding Adder Network (FAN). SIGMA performs 5.7x better than systolic array architectures for irregular sparse matrices, and roughly 3x better than state-of-the-art sparse accelerators. We demonstrate an instance of SIGMA operating at 10.8 TFLOPS efficiency across arbitrary levels of sparsity, with a 65.10 mm^2 and 22.33 W footprint on a 28 nm process.