Reviewerly/exHarmony huggingface dataset:
Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation

In recent years, machine learning models have rapidly become better at generating clinical consultation notes; yet, there is little work on how to properly evaluate the generated consultation notes to understand the impact they may have on both the clinician using them and the patient's clinical safety.To address this we present an extensive human evaluation study of consultation notes where 5 clinicians (i) listen to 57 mock consultations, (ii) write their own notes, (iii) post-edit a number of automatically generated notes, and (iv) extract all the errors, both quantitative and qualitative.We then carry out a correlation study with 18 automatic quality metrics and the human judgements.We find that a simple, character-based Levenshtein distance metric performs on par if not better than common model-based metrics like BertScore.All our findings and annotations are open-sourced.
Modeling Persuasive Discourse to Adaptively Support Students’ Argumentative Writing

We introduce an argumentation annotation approach to model the structure of argumentative discourse in student-written business model pitches.Additionally, the annotation scheme captures a series of persuasiveness scores such as the specificity, strength, evidence, and relevance of the pitch and the individual components.Based on this scheme, we annotated a corpus of 200 business model pitches in German.Moreover, we trained predictive models to detect argumentative discourse structures and embedded them in an adaptive writing support system for students that provides them with individual argumentation feedback independent of an instructor, time, and location.We evaluated our tool in a real-world writing exercise and found promising results for the measured self-efficacy and perceived ease-ofuse.Finally, we present our freely available corpus of persuasive business model pitches with 3,207 annotated sentences in German language and our annotation guidelines.
Toward More Meaningful Resources for Lower-resourced Languages

In this position paper, we describe our perspective on how meaningful resources for lowerresourced languages should be developed in connection with the speakers of those languages.Before advancing that position, we first examine two massively multilingual resources used in language technology development, identifying shortcomings that limit their usefulness.We explore the contents of the names stored in Wikidata for a few lowerresourced languages and find that many of them are not in fact in the languages they claim to be, requiring non-trivial effort to correct.We discuss quality issues present in WikiAnn and evaluate whether it is a useful supplement to handannotated data.We then discuss the importance of creating annotations for lower-resourced languages in a thoughtful and ethical way that includes the language speakers as part of the development process.We conclude with recommended guidelines for resource development.
STUDY OF BLOCKCHAIN TECHNOLOGY, AI AND DIGITAL NETWORKING IN METAVERSE

The phrase "Metaverse" has grabbed the imagination of younger generations. The Metaverse seamlessly merges the physical and virtual worlds, allowing avatars to engage in a variety of activities such as creation, exhibition, entertainment, social networking, and commerce. The metaverse will be enabled, populated, and sustained by artificial intelligence (AI). The Metaverse will be developed with augmented and virtual reality (AR/VR), as well as AI, digital networks, and blockchains. Meta is well-known for its work in artificial intelligence and algorithmic development. Artificial intelligence and blockchain technologies are poised to play important roles in the ever-expanding metaverse. Metaverse, for example, uses artificial intelligence, a digital network, and blockchain to build a digital virtual world in which anybody may participate securely and freely activities that are social and economic in nature but do not take place in the physical world.
CogTaskonomy: Cognitively Inspired Task Taxonomy Is Beneficial to Transfer Learning in NLP

Is there a principle to guide transfer learning across tasks in natural language processing (NLP)?Taxonomy (Zamir et al., 2018) finds that a structure exists among visual tasks, as a principle underlying transfer learning for them.In this paper, we propose a cognitively inspired framework, CogTaskonomy, to learn taxonomy for NLP tasks.The framework consists of Cognitive Representation Analytics (CRA) and Cognitive-Neural Mapping (CNM).The former employs Representational Similarity Analysis, which is commonly used in computational neuroscience to find a correlation between brainactivity measurement and computational modeling, to estimate task similarity with taskspecific sentence representations.The latter learns to detect task relations by projecting neural representations from NLP models to cognitive signals (i.e., fMRI voxels).Experiments on 12 NLP tasks, where BERT/TinyBERT are used as the underlying models for transfer learning, demonstrate that the proposed Cog-Taskonomy is able to guide transfer learning, achieving performance competitive to the Analytic Hierarchy Process (Saaty, 1987) used in visual Taskonomy (Zamir et al., 2018) but without requiring exhaustive pairwise O(m 2 ) task transferring.Analyses further discover that CNM is capable of learning modelagnostic task taxonomy.The source code is available at https://github.com/tjunlp-lab/CogTaskonomy.git.
Efficient Hierarchical Storage Management Empowered by Reinforcement Learning

With the rapid development of big data and cloud computing, data management has become increasingly challenging. Over the years, a number of frameworks for data management have become available. Most of them are highly efficient, but ultimately create data silos. It becomes difficult to move and work coherently with data as new requirements emerge. A possible solution is to use an intelligent hierarchical (multi-tier) storage system (HSS). A HSS is a meta solution that consists of different storage frameworks organized as a jointly constructed storage pool. A built-in data migration policy that determines the optimal placement of the datasets in the hierarchy is essential. Placement decisions is a non-trivial task since it should be made according to the characteristics of the dataset, the tier status in a hierarchy, and access patterns. This paper presents an open-source hierarchical storage framework with a dynamic migration policy based on reinforcement learning (RL). We present a mathematical model, a software architecture, and implementations based on both simulations and a live cloud-based environment. We compare the proposed RL-based strategy to a baseline of three rule-based policies, showing that the RL-based policy achieves significantly higher efficiency and optimal data distribution in different scenarios.
Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold

The prototypical NLP experiment trains a standard architecture on labeled English data and optimizes for accuracy, without accounting for other dimensions such as fairness, interpretability, or computational efficiency.We show through a manual classification of recent NLP research papers that this is indeed the case and refer to it as the square one experimental setup.We observe that NLP research often goes beyond the square one setup, e.g, focusing not only on accuracy, but also on fairness or interpretability, but typically only along a single dimension.Most work targeting multilinguality, for example, considers only accuracy; most work on fairness or interpretability considers only English; and so on.Such one-dimensionality of most research means we are only exploring a fraction of the NLP research search space.We provide historical and recent examples of how the square one bias has led researchers to draw false conclusions or make unwise choices, point to promising yet unexplored directions on the research manifold, and make practical recommendations to enable more multi-dimensional research.We open-source the results of our annotations to enable further analysis.
Multimodal fusion via cortical network inspired losses

Information integration from different modalities is an active area of research.Human beings and, in general, biological neural systems are quite adept at using a multitude of signals from different sensory perceptive fields to interact with the environment and each other.Recent work in deep fusion models via neural networks has led to substantial improvements over unimodal approaches in areas like speech recognition, emotion recognition and analysis, captioning and image description.However, such research has mostly focused on architectural changes allowing for fusion of different modalities while keeping the model complexity manageable.Inspired by neuroscientific ideas about multisensory integration and processing, we investigate the effect of introducing neural dependencies in the loss functions.Experiments on multimodal sentiment analysis tasks with different models show that our approach provides a consistent performance boost.
Developing Data Services Skills in Academic Libraries

Research data services are increasingly offered by academic libraries. As a result, librarians may need to upskill to provide data services and build capacity. This study measures the current level of data services skills of academic librarians and explores preferred methods of continuing education. An online survey was circulated asking respondents to self-assess data skills in four categories. The results capture a baseline of self-assessed data skills and show statistical significance between the percentage of time a librarian provides data services and higher levels of technical skill sets. The findings support the hiring of data librarians in academic libraries offering data services and providing training for librarians who provide any level of data services.
Boundary-Focused Generative Adversarial Networks for Imbalanced and Multimodal Time Series

Class imbalance problems have been reported as a major issue in various applications. Classification becomes further complicated when an imbalance occurs in time series data sets. To address time series data, it is necessary to consider their characteristics (i.e., high dimensionality, high correlations, and multimodality). Oversampling is a well-known approach for addressing this problem; however, such an approach does not appropriately consider the characteristics of time series data. This paper addresses these limitations by presenting a model-based oversampling approach, a boundary-focused generative adversarial network (BFGAN). The proposed BFGAN employs a specifically designed additional label for reflecting the importance of a sample's position in data space. Furthermore, the BFGAN generates artificial samples after taking into consideration a sample's multimodality and importance by using a suitable modified GAN structure. We present empirical results that reveal a significant improvement in the quality of the generated data when the proposed BFGAN is used as an oversampling algorithm for an imbalanced multimodal time series data set.
IsoScore: Measuring the Uniformity of Embedding Space Utilization

The recent success of distributed word representations has led to an increased interest in analyzing the properties of their spatial distribution.Several studies have suggested that contextualized word embedding models do not isotropically project tokens into vector space.However, current methods designed to measure isotropy, such as average random cosine similarity and the partition score, have not been thoroughly analyzed and are not appropriate for measuring isotropy.We propose IsoScore: a novel tool that quantifies the degree to which a point cloud uniformly utilizes the ambient vector space.Using rigorously designed tests, we demonstrate that IsoScore is the only tool available in the literature that accurately measures how uniformly distributed variance is across dimensions in vector space.Additionally, we use IsoScore to challenge a number of recent conclusions in the NLP literature that have been derived using brittle metrics of isotropy.We caution future studies from using existing tools to measure isotropy in contextualized embedding space as resulting conclusions will be misleading or altogether inaccurate.
Multiplex Anti-Asian Sentiment before and during the Pandemic: Introducing New Datasets from Twitter Mining

COVID-19 has disproportionately threatened minority communities in the U.S, not only in health but also in societal impact.However, social scientists and policymakers lack critical data to capture the dynamics of the anti-Asian hate trend and to evaluate its scale and scope.We introduce new datasets from Twitter related to anti-Asian hate sentiment before and during the pandemic.Relying on Twitter's academic API, we retrieve hateful and counter-hate tweets from the Twitter Historical Database.To build contextual understanding and collect related racial cues, we also collect instances of heated arguments, often political, but not necessarily hateful, discussing Chinese issues.We then use the state-of-the-art hate speech classifiers to discern whether these tweets express hatred.These datasets can be used to study hate speech, general anti-Asian or Chinese sentiment, and hate linguistics by social scientists as well as to evaluate and build hate speech or sentiment analysis classifiers by computational scholars.
CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues

This paper addresses the problem of dialogue reasoning with contextualized commonsense inference.We curate CICERO, a dataset of dyadic conversations with five types of utterance-level reasoning-based inferences: cause, subsequent event, prerequisite, motivation, and emotional reaction.The dataset contains 53,105 of such inferences from 5,672 dialogues.We use this dataset to solve relevant generative and discriminative tasks: generation of cause and subsequent event; generation of prerequisite, motivation, and listener's emotional reaction; and selection of plausible alternatives.Our results ascertain the value of such dialogue-centric commonsense knowledge datasets.It is our hope that CI-CERO will open new research avenues into commonsense-based dialogue reasoning.A: Can I help you?B: Yes, please.I'd like some oranges.A: Do you want Florida or California oranges
QuickGraph: A Rapid Annotation Tool for Knowledge Graph Extraction from Technical Text

Acquiring high-quality annotated corpora for complex multi-task information extraction (MT-IE) is an arduous and costly process for humanannotators.Adoption of unsupervised techniques for automated annotation have thus become popular.However, these techniques rely heavily on dictionaries, gazetteers, and knowledge bases.While such resources are abundant for general domains, they are scarce for specialised technical domains.To tackle this challenge, we present QuickGraph 1 , the first collaborative MT-IE annotation tool built with indirect weak supervision and clustering to maximise annotator productivity.QuickGraph's main contribution is a set of novel features that enable knowledge graph extraction through rapid and consistent complex multi-task entity and relation annotation.In this paper, we discuss these key features and qualitatively compare QuickGraph to existing annotation tools.A demonstration of our system is available at: https://youtu.be/ZlzH-AAoGXs.
How Do Seq2Seq Models Perform on End-to-End Data-to-Text Generation?

With the rapid development of deep learning, Seq2Seq paradigm has become prevalent for end-to-end data-to-text generation, and the BLEU scores have been increasing in recent years.However, it is widely recognized that there is still a gap between the quality of the texts generated by models and the texts written by human.In order to better understand the ability of Seq2Seq models, evaluate their performance and analyze the results, we choose to use Multidimensional Quality Metric(MQM) to evaluate several representative Seq2Seq models on end-to-end data-to-text generation.We annotate the outputs of five models on four datasets with eight error types and find that 1) copy mechanism is helpful for the improvement in Omission and Inaccuracy Extrinsic errors but it increases other types of errors such as Addition; 2) pre-training techniques are highly effective, and pre-training strategy and model size are very significant; 3) the structure of the dataset also influences the model's performance greatly; 4) some specific types of errors are generally challenging for seq2seq models.
ABLIMET @LT-EDI-ACL2022: A Roberta based Approach for Homophobia/Transphobia Detection in Social Media

This paper describes our system that participated in LT-EDI-ACL2022-Homophobia/Transphobia Detection in Social Media.Sexual minorities face a lot of unfair treatment and discrimination in our world.This creates enormous stress and many psychological problems for sexual minorities.There is a lot of hate speech on the internet, and homophobia/transphobia is one against sexual minorities.Identifying and processing homophobia/transphobia through natural language processing technology can improve the efficiency of processing it, and can quickly screen out it on the Internet.The organizer of the competition constructs a homophobia/ transphobia detection dataset based on YouTube comments for English and Tamil.We use a RoBERTa-based approach to conduct our experiments on the dataset of the competition, and get better results.
SSNCSE_NLP@LT-EDI-ACL2022: Homophobia/Transphobia Detection in Multiple Languages using SVM Classifiers and BERT-based Transformers

Over the years, there has been a slow but steady change in the attitude of society towards different kinds of sexuality.However, on social media platforms, where people have the license to be anonymous, toxic comments targeted at homosexuals, transgenders and the LGBTQ+ community are not uncommon.Detection of homophobic comments on social media can be useful in making the internet a safer place for everyone.For this task, we used a combination of word embeddings and SVM Classifiers as well as some BERT-based transformers.We achieved a weighted F1-score of 0.93 on the English dataset, 0.75 on the Tamil dataset and 0.87 on the Tamil-English Code-Mixed dataset.
Blockchain and Accounting Fraud Prevention: A Case Study on Luckin Coffee

Based on the case analysis of recent accounting scandal of Luckin Coffee, the paper studies whether and how the blockchain technology would help to prevent and detect accounting fraud, using the fraud triangle model.I find that the three characteristics of the blockchain technology would help to break the fraud triangle.First, the decentralization (rather than centralized authorization) will largely increase the fraud cost, discouraging fraudulent behavior in financial reporting.Second, the append-only linear form of transactional data (rather than relational database) enhances the tracking of tokenized assets, making the data much more difficult to be modified than the data in a traditional relational database with many data operation possibilities.Third, with smart-contracts serving as automatic controls, the blockchain removes the human factor, thus enhancing the control environment.My findings shed light on how blockchain, as one of the most disruptive and promising emerging technologies, will significantly cause landscape changes in the accounting and auditing fields.
iFace 1.1: A Proof-of-Concept of a Facial Authentication Based Digital ID for Smart Cities

Smart Cities" are a viable option to various issues caused by accelerated urban growth.To make smart cities a reality, smart citizens need to be connected to the "Smart City" through a digital ID.A digital ID allows citizens to make easy and effective use of amenities provided by the smart city such as healthcare, transport, finance, and energy.In this paper, we propose a proof-of-concept of facial authentication based end-to-end digital ID system for a smart city.Facial authentication systems are prone to various biometric template attacks and cyber security attacks.Our proposed system is designed to detect the first type of attack especially deepfake and presentation attacks.Users are authenticated each time before using facilities in the smart city.Authentication is performed on edge devices, making the process more secure as no data leaves the device during authentication.Facial data is stored at the cloud in a look up table format with an unidentifiable username.Our proposed solution achieved 97% accuracy in authentication with a False Rejection Ratio of 2% and False Acceptance Ratio of 3%.
Quantitative Analysis of Software Security through Fuzzy PROMETHEE-II Methodology: A Design Perspective

The objective of this research study is to develop secure and multi-functional software or web application with controlled complexity.The demand of software security in different IT sectors is the main focus of the present endeavor.The different design factors and their prioritization are the need and demand of the system.We have selected the case of banking software or application.Security assessment is an integral part of risk management practices which provides an analytical mechanism to control and integrate security features for valuable opinion during the design phase.The designing of secure software and the impact of security factor is adopted and evaluated by the Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE)-II method.The PROMETHEE-II methodology evaluates the impact of factors with respect to the design alternatives.The current priority is to work on the state-of-theart security attributes or alternatives of software design.Decision makers are generally responsible for evaluating various responses within their technical or scientific jurisdiction and rank them accordingly.Fuzzy set theories are the most appropriate tools to provide results for modeling qualitative information because of their ability to handle the impreciseness that is common in rating alternatives.The proposed work highlights the effectiveness of fuzzy PROMETHEE-II method in this context.We have enlisted this methodology for comparing software security factors in design perspective by using linguistic variables.The quantitative analysis attempted in our study was highly accurate for evaluating the security attributes and ranking them as per their priority, particularly in the context of banking software design.The study concludes with the advantages of employing the Fuzzy PROMETHEE-II vis-à -vis the other methodologies in analyzing the software security in the context of design.
PENGEMBANGAN MEDIA PEMBELAJARAN BERBASIS GOOGLE SITES PADA MATA PELAJASAN SOSIOLOGI KELAS X

This article describes the development of Google Sites-based learning media and the feasibility of Google Sites learning media from the media and material aspects. The method used is a research and development (RnD) method using the ADDIE development model developed by Dick and Carry. However, in its implementation, only three stages are carried out including Analysis (analysis), Design (design), and Development (development).The results of this study indicate that the development of Google Sites learning media is carried out through three stages, namely the analysis stage, the design stage, and the development stage. The learning media made consist of the Main Page, Competency Page, Material Page, Evaluation Page, Games Page, Announcement Page, and Credit Page. This media was validated by two media experts, two materials experts, and two sociology teachers. The feasibility of Google Sites media is based on the score obtained for each aspect by media experts, material experts, and sociology teachers to get an assessment result with an average score of 4.62 which is included in the appropriate category.