Reviewerly/exHarmony huggingface dataset:
Asynchronous Digital Participation in Urban Design Processes: Qualitative Data Exploration and Analysis With Natural Language Processing

This paper aims to improve the usability of qualitative urban big data sources by utilizing Natural Language Processing (NLP) as a promising AI-based technique.In this research, we designed a digital participation experiment by deploying an open-source and customizable asynchronous participation tool, "Consul Project", with 47 participants in the campus transformation process of the Singapore University of Technology and Design (SUTD).At the end of the data collection process with several debate topics and proposals, we analysed the qualitative data in entry scale, topic scale, and module scale.We investigated the impact of sentiment scores of each entry on the overall discussion and the sentiment scores of each introduction text on the ongoing discussions to trace the interaction and engagement.Furthermore, we used Latent Dirichlet Allocation (LDA) topic modelling to visualize the abstract topics that occurred in the participation experiment.The results revealed the links between different debates and proposals, which allow designers and decision makers to identify the most interacted arguments and engaging topics throughout participation processes.Eventually, this research presented the potentials of qualitative data while highlighting the necessity of adopting new methods and techniques, e.g., NLP, sentiment analysis, LDA topic modelling, to analyse and represent the collected qualitative data in asynchronous digital participation processes.
Influence of the Urban Built Environment on Physical and Mental Health of the Elderly under the Background of Big Data

With the advent of the information technology revolution and the Internet era, information technology is gradually occupying an important position and becoming an important strategic factor in economic development. As an emerging technology that has been developing continuously in recent years, big data is becoming an important industry to improve the innovation and development of the urban economy. Like AI technology, cloud computing, and the Internet, big data has become an important application technology for economic growth and economic efficiency improvement in today’s world. It is an effective means of progress and development in a region and an important strategic resource. As a new technology, big data has attracted more and more attention from all walks of life. Many companies have turned their attention to developing big data for economic benefits. “Enjoy your old age” is the yearning of every old man and his family. In recent years, the national level has been committed to “creating an urban built environment for the elderly to achieve healthy aging.” From the perspective of promoting the physical and mental health of the elderly, this paper analyzes the impact of the urban built environment on the physical and mental health of the elderly based on the needs of the elderly and puts forward countermeasures and suggestions based on the current status and existing problems of the urban built environment for the elderly. Based on the combined data analysis method and technology in big data, this paper conducted a field questionnaire survey on a total of 4,000 elderly people in urban and rural areas by means of the questionnaire survey. It is found that the existing problems of the built environment in the old cities include scattered content, one-sided understanding, and rigid design. According to the problems, the solutions of building consensus, paying attention to planning, combining urban characteristics, and the joint efforts of all sectors of society are put forward. And programming tools are used to combine formulas and analyze related data in detail. The analysis results show that the physical and mental health index of the elderly is highly correlated with factors such as changes in the consensus degree of the urban built environment, urban built environment planning, urban built environment policy support, and multiparty efforts in the urban built environment. Changes show a positive change.
AHRNN: Attention‐Based Hybrid Robust Neural Network for emotion recognition

In order to solve the problem that the existing methods cannot effectively capture the semantic emotion of the sentence when faced with the lack of cross-language corpus, it is difficult to effectively perform cross-language sentiment analysis, we propose a neural network architecture called the Attention-Based Hybrid Robust Neural Network. The proposed architecture includes pre-trained word embedding with fine-tuning training to obtain prior semantic information, two sub-networks and attention mechanism to capture the global semantic emotional information in the text, and a fully connected layer and softmax function to jointly perform final emotional classification. The Convolutional Neural Networks sub-network captures the local semantic emotional information of the text, the BiLSTM sub-network captures the contextual semantic emotional information of the text, and the attention mechanism dynamically integrates the semantic emotional information to obtain key emotional information. We conduct experiments on Chinese (International Conference on Natural Language Processing and Chinese Computing) and English (SST) datasets. The experiment is divided into three subtasks to evaluate the superiority of our method. It improves the recognition accuracy of single sentence positive/negative classification from 79% to 86% in the single-language emotion recognition task. The recognition performance of fine-grained emotional tags is also improved by 9.6%. The recognition accuracy of cross-language emotion recognition tasks has also been improved by 1.5%. Even in the face of faulty data, the performance of our model is not significantly reduced when the error rate is less than 20%. These experimental results prove the superiority of our method.
TEXTUAL EMOTION DETECTION APPROACHES: A SURVEY

Over the past decades, social media attracted individuals to express their feelings on any topic or item, resulting in an incremental growth in the size of created data. These feelings and unstructured data paved the path for business organizations to gather information and build statistical analysis. Various machine learning and natural language processing-based approaches are used for sentiment and emotion analysis. Moreover, deep learning-based approaches recently gained popularity due to their remarkable performance in text analysis. This paper provides a comprehensive overview of the prominent machine learning models applied in emotion analysis. It explores various emotion analysis taxonomies, in addition to the constraints of prevalent deep learning architectures. The paper also reviews some of the previously presented contributions in emotion analysis with a focus on deep learning methodologies as well as the most common datasets. It presents a comprehensive comparison between several emotion analysis models. This paper demonstrates the effectiveness of learning-based techniques in tackling emotion analysis challenges.
Neural Link Prediction with Walk Pooling

Graph neural networks achieve high accuracy in link prediction by jointly leveraging graph topology and node attributes. Topology, however, is represented indirectly; state-of-the-art methods based on subgraph classification label nodes with distance to the target link, so that, although topological information is present, it is tempered by pooling. This makes it challenging to leverage features like loops and motifs associated with network formation mechanisms. We propose a link prediction algorithm based on a new pooling scheme called WalkPool. WalkPool combines the expressivity of topological heuristics with the feature-learning ability of neural networks. It summarizes a putative link by random walk probabilities of adjacent paths. Instead of extracting transition probabilities from the original graph, it computes the transition matrix of a "predictive" latent graph by applying attention to learned features; this may be interpreted as feature-sensitive topology fingerprinting. WalkPool can leverage unsupervised node features or be combined with GNNs and trained end-to-end. It outperforms state-of-the-art methods on all common link prediction benchmarks, both homophilic and heterophilic, with and without node attributes. Applying WalkPool to a set of unsupervised GNNs significantly improves prediction accuracy, suggesting that it may be used as a general-purpose graph pooling scheme.
Federated Learning for Open Banking

Open banking enables individual customers to own their banking data, which provides fundamental support for the boosting of a new ecosystem of data marketplaces and financial services. In the near future, it is foreseeable to have decentralized data ownership in the finance sector using federated learning. This is a just-in-time technology that can learn intelligent models in a decentralized training manner. The most attractive aspect of federated learning is its ability to decompose model training into a centralized server and distributed nodes without collecting private data. This kind of decomposed learning framework has great potential to protect users' privacy and sensitive data. Therefore, federated learning combines naturally with an open banking data marketplaces. This chapter will discuss the possible challenges for applying federated learning in the context of open banking, and the corresponding solutions have been explored as well.
An Online Graphical User Interface Application to Remove Barriers in the Process of Learning Neural Networks and Deep Learning Concepts Using Tensorflow

Over the years, neural networks have become increasingly important and complex due to the rising popularity of artificial intelligence technologies. It allows for complex decision prediction making, and is an essential part in the modern AI industry. However, due to the complex nature of neural networks, a lot of complex math and logic has to be well understood along with a proficiency in programming in order for one to make anything practical with this technology. This is unfortunate, however, that many do not have the required high level math skill, or the proficiency in coding, blocking a lot of people from reaching and experimenting with this technology. My method attempts to eliminate the complexity that developing neural networks bring, and bring a clearer picture of what the user may be creating and working with. With the help of modern web technologies such as JavaScript and tensorflow.js, I was able to create a GUI program that can create, train, and test a neural network right on a browser, and without writing any code with a comparable result [13].
Deep Learning-based Face Super-Resolution: A Survey

Face super-resolution (FSR), also known as face hallucination, which is aimed at enhancing the resolution of low-resolution (LR) face images to generate high-resolution (HR) face images, is a domain-specific image super-resolution problem. Recently, FSR has received considerable attention and witnessed dazzling advances with the development of deep learning techniques. To date, few summaries of the studies on the deep learning-based FSR are available. In this survey, we present a comprehensive review of deep learning-based FSR methods in a systematic manner. First, we summarize the problem formulation of FSR and introduce popular assessment metrics and loss functions. Second, we elaborate on the facial characteristics and popular datasets used in FSR. Third, we roughly categorize existing methods according to the utilization of facial characteristics. In each category, we start with a general description of design principles, then present an overview of representative approaches, and then discuss the pros and cons among them. Fourth, we evaluate the performance of some state-of-the-art methods. Fifth, joint FSR and other tasks, and FSR-related applications are roughly introduced. Finally, we envision the prospects of further technological advancement in this field. A curated list of papers and resources to face super-resolution are available at \url{https://github.com/junjun-jiang/Face-Hallucination-Benchmark}
When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale  Sparse Logistic Regression and Applications in Risk Control

Logistic Regression (LR) is the most widely used machine learning model in industry for its efficiency, robustness, and interpretability. Due to the problem of data isolation and the requirement of high model performance, many applications in industry call for building a secure and efficient LR model for multiple parties. Most existing work uses either Homomorphic Encryption (HE) or Secret Sharing (SS) to build secure LR. HE based methods can deal with high-dimensional sparse features, but they incur potential security risks. SS based methods have provable security, but they have efficiency issue under high-dimensional sparse features. In this paper, we first present CAESAR, which combines HE and SS to build secure large-scale sparse logistic regression model and achieves both efficiency and security. We then present the distributed implementation of CAESAR for scalability requirement. We have deployed CAESAR in a risk control task and conducted comprehensive experiments. Our experimental results show that CAESAR improves the state-of-the-art model by around 130 times.
Evaluation of Text Generation: A Survey

The paper surveys evaluation methods of natural language generation (NLG) systems that have been developed in the last few years. We group NLG evaluation methods into three categories: (1) human-centric evaluation metrics, (2) automatic metrics that require no training, and (3) machine-learned metrics. For each category, we discuss the progress that has been made and the challenges still being faced, with a focus on the evaluation of recently proposed NLG tasks and neural NLG models. We then present two examples for task-specific NLG evaluations for automatic text summarization and long text generation, and conclude the paper by proposing future research directions.
Learning Diverse and Discriminative Representations via the Principle of  Maximal Coding Rate Reduction

To learn intrinsic low-dimensional structures from high-dimensional data that most discriminate between classes, we propose the principle of Maximal Coding Rate Reduction ($\text{MCR}^2$), an information-theoretic measure that maximizes the coding rate difference between the whole dataset and the sum of each individual class. We clarify its relationships with most existing frameworks such as cross-entropy, information bottleneck, information gain, contractive and contrastive learning, and provide theoretical guarantees for learning diverse and discriminative features. The coding rate can be accurately computed from finite samples of degenerate subspace-like distributions and can learn intrinsic representations in supervised, self-supervised, and unsupervised settings in a unified manner. Empirically, the representations learned using this principle alone are significantly more robust to label corruptions in classification than those using cross-entropy, and can lead to state-of-the-art results in clustering mixed data from self-learned invariant features.
Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for  Offensive Language Detection

Nowadays, offensive content in social media has become a serious problem, and automatically detecting offensive language is an essential task. In this paper, we build an offensive language detection system, which combines multi-task learning with BERT-based models. Using a pre-trained language model such as BERT, we can effectively learn the representations for noisy text in social media. Besides, to boost the performance of offensive language detection, we leverage the supervision signals from other related tasks. In the OffensEval-2020 competition, our model achieves 91.51% F1 score in English Sub-task A, which is comparable to the first place (92.23%F1). An empirical analysis is provided to explain the effectiveness of our approaches.
Prognostic Diagnosis for Breast Cancer Patients Using Probabilistic Bayesian Classification

The diagnosis and treatment of patients in the healthcare industry are greatly aided by data analytics. Massive amounts of data should be handled using machine learning approaches to provide tools for prediction and categorization to support practitioner decision-making. Based on the kind of tumor, disorders like breast cancer can be categorized. The difficulties associated with evaluating vast amounts of data should be overcome by discovering an efficient method for categorization. Based on the Bayesian method, we analyzed the influence of clinic pathological indicators on the prognosis and survival rate of breast cancer patients and compared the local resection value directly using the lymph node ratio (LNR) and the overall value using the LNR differences in effect between estimates. Logistic regression was used to estimate the overall LNR of patients. After that, a probabilistic Bayesian classifier-based dynamic regression model for prognosis analysis is built to capture the dynamic effect of multiple clinic pathological markers on patient prognosis. The dynamic regression model employing the total estimated value of LNR had the best fitting impact on the data, according to the simulation findings. In comparison to other models, this model has the greatest overall survival forecast accuracy. These prognostic techniques shed light on the nodal survival and status particular to the patient. Additionally, the framework is flexible and may be used with various cancer types and datasets.
Gaussian Mixture Variational Autoencoder with Whitening Score for Multimodal Time Series Anomaly Detection

Time series anomaly detection has attracted great attention due to its widespread existence in real life. With the increasing development and advancement of deep learning, many unsupervised deep learning methods have been proposed for time series anomaly detection since labeling time series is prohibitively expensive. In this paper, we propose an unsupervised anomaly detection method: Gaussian Mixture Variational Autoencoder with Whitening Distance Anomaly Score (WGVAE). Concretely, we employ an LSTM-based variational autoencoder to capture the long-term dependence of time series and learn the low-dimensional feature representation and distribution, in which the Gaussian mixture prior are used to characterize multimodal time series. Further, whitening distance anomaly scores are used to make the multidimensional time series independently and identically distributed among each dimension, which combines the distribution characteristics of the samples to measure the degree of outliers. When the anomaly score is below the threshold, the sample is detected as anomalous. Finally, comprehensive experiments are given to verify the effectiveness of our method.
KroneckerBERT: Significant Compression of Pre-trained Language Models Through Kronecker Decomposition and Knowledge Distillation

The development of over-parameterized pretrained language models has made a significant contribution toward the success of natural language processing.While over-parameterization of these models is the key to their generalization power, it makes them unsuitable for deployment on low-capacity devices.We push the limits of state-of-the-art Transformer-based pre-trained language model compression using Kronecker decomposition.We present our KroneckerBERT, a compressed version of the BERT BASE model obtained by compressing the embedding layer and the linear mappings in the multi-head attention, and the feed-forward network modules in the Transformer layers.Our KroneckerBERT is trained via a very efficient two-stage knowledge distillation scheme using far fewer data samples than state-of-the-art models like MobileBERT and TinyBERT.We evaluate the performance of KroneckerBERT on well-known NLP benchmarks.We show that our KroneckerBERT with compression factors of 7.7× and 21× outperforms state-of-theart compression methods on the GLUE and SQuAD benchmarks.In particular, using only 13% of the teacher model parameters, it retain more than 99% of the accuracy on the majority of GLUE tasks.
Using Item Response Theory to Measure Gender and Racial Bias of a BERT-based Automated English Speech Assessment System

Recent advances in natural language processing and transformer-based models have made it easier to implement accurate, automated English speech assessments.Yet, without careful examination, applications of these models may exacerbate social prejudices based on gender and race.This study addresses the need to examine potential biases of transformer-based models in the context of automated English speech assessment.For this purpose, we developed a BERT-based automated speech assessment system and investigated gender and racial bias of examinees' automated scores.Gender and racial bias was measured by examining differential item functioning (DIF) using an item response theory framework.Preliminary results, which focused on a single verbal-response item, showed no statistically significant DIF based on gender or race for automated scores.
MarSan at SemEval-2022 Task 11: Multilingual complex named entity recognition using T5 and transformer encoder

The multilingual complex named entity recognition task of SemEval2020 required participants to detect semantically ambiguous and complex entities in 11 languages.In order to participate in this competition, a deep learning model is being used with the T5 text-to-text language model and its multilingual version, MT5, along with the transformer's encoder module.The subtoken check has also been introduced, resulting in a 4% increase in the model F1score in English.We also examined the use of the BPEmb model for converting input tokens to representation vectors in this research.A performance evaluation of the proposed entity detection model is presented at the end of this paper.Six different scenarios were defined, and the proposed model was evaluated in each scenario within the English development set.Our model is also evaluated in other languages.* Equal contribution.Listing order is random.
McRock at SemEval-2022 Task 4: Patronizing and Condescending Language Detection using Multi-Channel CNN, Hybrid LSTM, DistilBERT and XLNet

In this paper we propose four deep learning models for the task of detecting and classifying Patronizing and Condescending Language (PCL) using a corpus of over 13,000 annotated paragraphs in English.The task, hosted at SemEval-2022, consists of two different subtasks.The Subtask 1 is a binary classification problem.Namely, given a paragraph, a system must predict whether or not it contains any form of PCL.The Subtask 2 is a multi-label classification task.Given a paragraph, a system must identify which PCL categories express the condescension.A paragraph might contain one or more categories of PCL.To face with the first subtask we propose a multi-channel Convolutional Neural Network (CNN) and an Hybrid LSTM.Using the multi-channel CNN we explore the impact of parallel word emebeddings and convolutional layers involving different kernel sizes.With Hybrid LSTM we focus on extracting features in advance, thanks to a convolutional layer followed by two bidirectional LSTM layers.For the second subtask a Transformer BERT-based model (i.e.Dis-tilBERT) and an XLNet-based model are proposed.The multi-channel CNN model is able to reach an F1 score of 0.2928, the Hybrid LSTM model is able to reach an F1 score of 0.2815, the DistilBERT-based one an average F1 of 0.2165 and the XLNet an average F1 of 0.2296.In this paper, in addition to system descriptions, we also provide further analysis of the results, highlighting strengths and limitations.We make all the code publicly available and reusable on GitHub 1 .