Reviewerly/exHarmony huggingface dataset:
Computational Emotion Analysis From Images: Recent Advances and Future Directions

Emotions are usually evoked in humans by images. Recently, extensive research efforts have been dedicated to understanding the emotions of images. In this chapter, we aim to introduce image emotion analysis (IEA) from a computational perspective with the focus on summarizing recent advances and suggesting future directions. We begin with commonly used emotion representation models from psychology. We then define the key computational problems that the researchers have been trying to solve and provide supervised frameworks that are generally used for different IEA tasks. After the introduction of major challenges in IEA, we present some representative methods on emotion feature extraction, supervised classifier learning, and domain adaptation. Furthermore, we introduce available datasets for evaluation and summarize some main results. Finally, we discuss some open questions and future directions that researchers can pursue.
Attributed Graph Clustering with Dual Redundancy Reduction

Attributed graph clustering is a basic yet essential method for graph data exploration. Recent efforts over graph contrastive learning have achieved impressive clustering performance. However, we observe that the commonly adopted InfoMax operation tends to capture redundant information, limiting the downstream clustering performance. To this end, we develop a novel method termed attributed graph clustering with dual redundancy reduction (AGC-DRR) to reduce the information redundancy in both input space and latent feature space. Specifically, for the input space redundancy reduction, we introduce an adversarial learning mechanism to adaptively learn a redundant edge-dropping matrix to ensure the diversity of the compared sample pairs. To reduce the redundancy in the latent space, we force the correlation matrix of the cross-augmentation sample embedding to approximate an identity matrix. Consequently, the learned network is forced to be robust against perturbation while discriminative against different samples. Extensive experiments have demonstrated that AGC-DRR outperforms the state-of-the-art clustering methods on most of our benchmarks. The corresponding code is available at https://github.com/gongleii/AGC-DRR.
Imperceptible Backdoor Attack: From Input Space to Feature Representation

Backdoor attacks are rapidly emerging threats to deep neural networks (DNNs). In the backdoor attack scenario, attackers usually implant the backdoor into the target model by manipulating the training dataset or training process. Then, the compromised model behaves normally for benign input yet makes mistakes when the pre-defined trigger appears. In this paper, we analyze the drawbacks of existing attack approaches and propose a novel imperceptible backdoor attack. We treat the trigger pattern as a special kind of noise following a multinomial distribution. A U-net-based network is employed to generate concrete parameters of multinomial distribution for each benign input. This elaborated trigger ensures that our approach is invisible to both humans and statistical detection. Besides the design of the trigger, we also consider the robustness of our approach against model diagnose-based defences. We force the feature representation of malicious input stamped with the trigger to be entangled with the benign one. We demonstrate the effectiveness and robustness against multiple state-of-the-art defences through extensive datasets and networks. Our trigger only modifies less than 1\% pixels of a benign image while the modification magnitude is 1. Our source code is available at https://github.com/Ekko-zn/IJCAI2022-Backdoor.
Personalized Federated Learning With a Graph

Knowledge sharing and model personalization are two key components in the conceptual framework of personalized federated learning (PFL). Existing PFL methods focus on proposing new model personalization mechanisms while simply implementing knowledge sharing by aggregating models from all clients, regardless of their relation graph. This paper aims to enhance the knowledge-sharing process in PFL by leveraging the graph-based structural information among clients. We propose a novel structured federated learning (SFL) framework to learn both the global and personalized models simultaneously using client-wise relation graphs and clients' private data. We cast SFL with graph into a novel optimization problem that can model the client-wise complex relations and graph-based structural topology by a unified framework. Moreover, in addition to using an existing relation graph, SFL could be expanded to learn the hidden relations among clients. Experiments on traffic and image benchmark datasets can demonstrate the effectiveness of the proposed method.
Digital transformation as the main condition for the development of modern higher education

The article considers the potential and limitations that arise in the higher education system as a result of the e-technology use. E-learning has become an integral part of the 21st century education system. The pace of dissemination and implementation of e-technologies in all spheres of public life, in science and education in particular, is constantly growing. There are different views in the scientific community and the public on the digital transformation of education. However, with the entry into our lives of COVID-19, the use of e-technologies in education ceases to be a matter of innovation, and becomes a vital necessity for the functioning of the educational system and the activities of educational institutions. Analysis of more than 50 concepts that characterize the modern education system, educational technologies and the process of digital transformation of education allowed the authors to draw conclusions about the importance of information technology in education and the main trends in their development. The article also summarizes the different views of scientists on the digital transformation of higher education, the use of e-technology in the educational process. The advantages that e-technologies provide to the educational process and the challenges of their use that need to be addressed are outlined. The study of the experience of the leading universities of Ukraine allowed the authors to draw conclusions about the state of digital transformation of higher education in Ukraine. Elaboration of scientific literature, own observations and research have allowed to define the tendencies of development of higher education in the epoch of digital transformation.
M-Theory and F-Theory over Theoretical Analysis on Cosmic Strings and Calabi-Yau Manifolds Subject to Conifold Singularity with Randall-Sundrum Model

String theory always comes with heavy mathematical rigor as it questions the most significant and impossible attempt to make a scale-invariant phenomenology between general relativity and quantum theory. Thus, steps have been taken to simplify the theory a bit thereby making it accessible to general yet enthusiastic readers of physics. However, as there is numerous mathematics involved in the modeling of this theory, thus, any chance to make a purely non-mathematical approach towards strings would prove vacuous and intimidating making the pathway of this marvelous theory chocked with unnecessary assumptions resulting in false analogies (or hopes) relating to this theory. Thus, where it’s almost impossible to proceed without any equations, we have given a few just to wipe out some logical confusion arising to the first readers of strings. Few necessary diagrams are included along with intense theory and least mathematics for making this significant approach of theoretical physicists accessible to general learners or readers.&#x0D; Topics: Bosonic string theory, supersymmetric string theory, M – theory, F – theory, dualities and interconnectedness, viability, Randall – Sundrum model for tackling the hierarchy problem of particle physics, conifold singularities, Branes, Bulks, Extremal black holes, Ekpyrotic cosmology, topological aspects of Calabi – Yau (CY) manifolds, A and B models, Mirror Symmetries; AdS/CFT, cosmic strings, all in a way accessible to every reader.&#x0D; Methods: Theoretical analogies, deductions, principles behind the origin, development along with the probable conclusion of this theory, the roots of its origin, the necessary difficulty for detecting those strings, and approaches done by theorists to work out the pathway of achieving Einstein’s dream of unification irrespective of several hindrances.&#x0D; Results: String theory itself is not a complete theory. Rather it’s in the process of further development through the increment of time resulting in more applications of mathematics by developing or incorporating them in due needs. Thus, without stating any concrete results, the theory has been tackled in this paper with a viable hypothesis based on the current understanding, and previous attempts are stated to have been made for its success.
IBGS: A Wearable Smart System to Assist Visually Challenged

Traditional blind guide devices are large and high cost. In this paper, a kind of intelligent blind guide system (IBGS) is introduced. GD32 is used as the main control chip, it cooperates with various functional modules to realize traffic light recognition, obstacle avoidance, payment, and navigation functions on the basis of speech recognition. At the same time, IBGS uses WIFI instead of Bluetooth to get rid of the dependence on smart phones. In addition, a cloud database is built, and the Internet of Things technology is used to realize the information interconnection between the IBGS, the database, and the guardian&#x2019;s mobile terminal. In order to better realize the function of speech recognition, this paper proposes a Conv-Transformer Transducer (ConvT-T) speech recognition framework based on Weak-Attention Suppression (WAS), which improves the efficiency of multi-head attention through WAS. The proposed method achieves word error rate (WER) of 3.2% in test-clean and 7.9% in test-other on the LibriSpeech ASR corpus with only 73M parameters, which reduces the WER by 0.3% and 0.4% compared to ConvT-T respectively, indicating that WAS can effectively play the role of suppressing non-critical attention on the ConvT-T framework. At the same time, the IBGS has been tested in a comprehensive application scenario. In outdoor traffic light recognition, speech recognition, and obstacle avoidance tests, the accuracy rates of 92.33%, 90.33%, and 96.67% were achieved respectively. The results show that the IBGS can effectively deal with various challenges encountered by the blind in their daily outdoor walking.
Nyströmformer: A Nystöm-based Algorithm for Approximating Self-Attention.

Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the self-attention mechanism that encodes the influence or dependence of other tokens on each specific token. While beneficial, the quadratic complexity of self-attention on the input sequence length has limited its application to longer sequences - a topic being actively studied in the community. To address this limitation, we propose Nyströmformer - a model that exhibits favorable scalability as a function of sequence length. Our idea is based on adapting the Nyström method to approximate standard self-attention with O(n) complexity. The scalability of Nyströmformer enables application to longer sequences with thousands of tokens. We perform evaluations on multiple downstream tasks on the GLUE benchmark and IMDB reviews with standard sequence length, and find that our Nyströmformer performs comparably, or in a few cases, even slightly better, than standard self-attention. On longer sequence tasks in the Long Range Arena (LRA) benchmark, Nyströmformer performs favorably relative to other efficient self-attention methods. Our code is available at https://github.com/mlpen/Nystromformer.
Meta Learning for Natural Language Processing: A Survey

Deep learning has been the mainstream technique in natural language processing (NLP) area.However, the techniques require many labeled data and are less generalizable across domains.Meta-learning is an arising field in machine learning studying approaches to learn better learning algorithms.Approaches aim at improving algorithms in various aspects, including data efficiency and generalizability.Efficacy of approaches has been shown in many NLP tasks, but there is no systematic survey of these approaches in NLP, which hinders more researchers from joining the field.Our goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation.This paper first introduces the general concepts of meta-learning and the common approaches.Then we summarize task construction settings and application of meta-learning for various NLP problems and review the development of meta-learning in NLP community.
Why Is Multiclass Classification Hard?

In classification problems, as the number of classes increases, correctly classifying a new instance into one of them is assumed to be more challenging than making the same decision in the presence of fewer classes. The essence of the problem is that using the learning algorithm on each decision boundary individually is better than using the same learning algorithm on several of them simultaneously. However, why and when it happens is still not well-understood today. This work&#x2019;s main contribution is to introduce the concept of heterogeneity of decision boundaries as an explanation of this phenomenon. Based on the definition of heterogeneity of decision boundaries, we analyze and explain the differences in the performance of state of the art approaches to solve multi-class classification. We demonstrate that as the heterogeneity increases, the performances of all approaches, except one-vs-one, decrease. We show that by correctly encoding the knowledge of the heterogeneity of decision boundaries in a decomposition of the multi-class problem, we can obtain better results than state of the art decompositions. The benefits can be an increase in classification performance or a decrease in the time it takes to train and evaluate the models. We first provide intuitions and illustrate the effects of the heterogeneity of decision boundaries using synthetic datasets and a simplistic classifier. Then, we demonstrate how a real dataset exhibits these same principles, also under realistic learning algorithms. In this setting, we devise a method to quantify the heterogeneity of different decision boundaries, and use it to decompose the multi-class problem. The results show significant improvements over state-of-the-art decompositions that do not take the heterogeneity of decision boundaries into account.
Contrastive Data and Learning for Natural Language Processing

Current NLP models heavily rely on effective representation learning algorithms.Contrastive learning is one such technique to learn an embedding space such that similar data sample pairs have close representations while dissimilar samples stay far apart from each other.It can be used in supervised or unsupervised settings using different loss functions to produce task-specific or general-purpose representations.While it has originally enabled the success for vision tasks, recent years have seen a growing number of publications in contrastive NLP as shown in Figure 1.This first line of works not only delivers promising performance improvements in various NLP tasks, but also provides desired characteristics such as task-agnostic sentence representation, faithful text generation, data-efficient learning in zero-shot and few-shot settings, interpretability and explainability.In this tutorial, we aim to provide a gentle introduction to the fundamentals of contrastive learning approaches and the theory behind them.We then survey the benefits and the best practices of contrastive learning for various downstream NLP applications including Text Classification, Question Answering, Summarization, Text Generation, Interpretability and Explainability, Commonsense Knowledge and Reasoning, Vision-and-Language.This tutorial intends to help researchers in the NLP and computational linguistics community to understand this emerging topic and promote future research directions of using contrastive learning for NLP applications. 1
SparseTrain

Our community has greatly improved the efficiency of deep learning applications, including by exploiting sparsity in inputs. Most of that work, though, is for inference, where weight sparsity is known statically, and/or for specialized hardware. We propose a scheme to leverage dynamic sparsity during training. In particular, we exploit zeros introduced by the ReLU activation function to both feature maps and their gradients. This is challenging because the sparsity degree is moderate and the locations of zeros change over time. We also rely purely on software. We identify zeros in a dense data representation without transforming the data and performs conventional vectorized computation. Variations of the scheme are applicable to all major components of training: forward propagation, backward propagation by inputs, and backward propagation by weights. Our method significantly outperforms a highly-optimized dense direct convolution on several popular deep neural networks. At realistic sparsity, we speed up the training of the non-initial convolutional layers in VGG16, ResNet-34, ResNet-50, and Fixup ResNet-50 by 2.19x, 1.37x, 1.31x, and 1.51x respectively on an Intel Skylake-X CPU.
Self-Supervised Keypoint Detection and Cross-Fusion Matching Networks for Multimodal Remote Sensing Image Registration

Remote sensing image matching is the basis upon which to obtain integrated observations and complementary information representation of the same scene from multiple source sensors, which is a prerequisite for remote sensing tasks such as remote sensing image fusion and change detection. However, the intricate geometric and radiometric differences between the multimodal images render the registration quite challenging. Although multimodal remote sensing image matching methods have been developed in recent decades, most classical and deep learning based techniques cannot effectively extract high repeatable keypoints and discriminative descriptors for multimodal images. Therefore, we propose a two-step “detection + matching” framework in this paper, where each step consists of a deep neural network. A self-supervised detection network is first designed to generate similar keypoint feature maps between multimodal images, which is used to detect highly repeatable keypoints. We then propose a cross-fusion matching network, which aims to exploit global optimization and fusion information for cross-modal feature descriptors and matching. The experiments show that the proposed method has superior feature detection and matching performance compared with current state-of-the-art methods. Specifically, the keypoint repetition rate of the detection network and the NN mAP of the matching network are 0.435 and 0.712 on test datasets, respectively. The proposed whole pipeline framework is evaluated, which achieves an average M.S. and RMSE of 0.298 and 3.41, respectively. This provides a novel solution for the joint use of multimodal remote sensing images for observation and localization.
Improvement of energy conservation using blockchain-enabled cognitive wireless networks for smart cities

In Smart Cities' applications, Multi-node cooperative spectrum sensing (CSS) can boost spectrum sensing efficiency in cognitive wireless networks (CWN), although there is a non-linear interaction among number of nodes and sensing efficiency. Cooperative sensing by nodes with low computational cost is not favorable to improving sensing reliability and diminishes spectrum sensing energy efficiency, which poses obstacles to the regular operation of CWN. To enhance the evaluation and interpretation of nodes and resolves the difficulty of sensor selection in cognitive sensor networks for energy-efficient spectrum sensing. We examined reducing energy usage in smart cities while substantially boosting spectrum detecting accuracy. In optimizing energy effectiveness in spectrum sensing while minimizing complexity, we use the energy detection for spectrum sensing and describe the challenge of sensor selection. This article proposed the algorithm for choosing the sensing nodes while reducing the energy utilization and improving the sensing efficiency. All the information regarding nodes is saved in the fusion center (FC) through which blockchain encrypts the information of nodes ensuring that a node's trust value conforms to its own without any ambiguity, CWN-FC pick high-performance nodes to engage in CSS. The performance evaluation and computation results shows the comparison between various algorithms with the proposed approach which achieves 10% sensing efficiency in finding the solution for identification and triggering possibilities with the value of [Formula: see text] and [Formula: see text] with the varying number of nodes.
A taxonomy of prompt modifiers for text-to-image generation

Text-guided synthesis of images has become enormously popular and online communities dedicated to text-to-image generation and art generated with Artificial Intelligence (AI) have emerged. While deep generative models can synthesise high-quality images and artworks from simple descriptive text prompts, practitioners of text-to-image generation typically seek to control the generative model’s output by adding short key phrases (‘modifiers’) to the prompt. This paper identifies six types of prompt modifiers used by practitioners in the online text-to-image community based on a 3-month ethnographic study. The novel taxonomy of prompt modifiers provides researchers a conceptual starting point for investigating the practice of text-to-image generation, but may also help practitioners of AI generated art improve their images. We further outline how prompt modifiers are applied in the practice of ‘prompt engineering.’ and discuss research opportunities of this novel creative practice in the field of Human–Computer Interaction (HCI). The paper concludes with a discussion of broader implications of prompt engineering from the perspective of Human-AI Interaction (HAI) in future applications beyond the use case of text-to-image generation and AI generated art.
Adaptive construction of shallower quantum circuits with quantum spin projection for fermionic systems

Quantum computing is a promising approach to harnessing strong correlation in molecular systems; however, current devices only allow for hybrid quantum-classical algorithms with a shallow circuit depth, such as the variational quantum eigensolver (VQE). In this paper, we report the importance of the Hamiltonian symmetry in constructing VQE circuits adaptively (ADAPT-VQE). This treatment often violates symmetry, thereby deteriorating the convergence of fidelity to the exact solution and ultimately resulting in deeper circuits. We demonstrate that spin-symmetry projection can provide a simple yet effective solution to this problem, by keeping the quantum state in the correct symmetry space, to reduce the overall gate operations. To further investigate the role of spin-symmetry in computing molecular properties with ADAPT-VQE, we have derived the analytical derivative of symmetry-projected VQE energy. Our illustrative calculations reveal the significance of preserving symmetry in providing accurate dipole moments and geometries with variational approximations.