pwc-archive/papers-with-abstracts huggingface dataset:
Active Learning for Abstractive Text Summarization
Authors: Akim Tsvigun, Ivan Lysenko, Danila Sedashov, Ivan Lazichny, Eldar Damirov, Vladimir Karlov, Artemy Belousov, Leonid Sanochkin, Maxim Panov, Alexander Panchenko, Mikhail Burtsev, Artem Shelmanov
Date: 2023-01-09 00:00:00
Tasks: Abstractive Text Summarization, Active Learning, Diversity, Self-Learning, text-classification, Text Classification, Text Summarization

Construction of human-curated annotated datasets for abstractive text summarization (ATS) is very time-consuming and expensive because creating each instance requires a human annotator to read a long document and compose a shorter summary that would preserve the key information relayed by the original document. Active Learning (AL) is a technique developed to reduce the amount of annotation required to achieve a certain level of machine learning model performance. In information extraction and text classification, AL can reduce the amount of labor up to multiple times. Despite its potential for aiding expensive annotation, as far as we know, there were no effective AL query strategies for ATS. This stems from the fact that many AL strategies rely on uncertainty estimation, while as we show in our work, uncertain instances are usually noisy, and selecting them can degrade the model performance compared to passive annotation. We address this problem by proposing the first effective query strategy for AL in ATS based on diversity principles. We show that given a certain annotation budget, using our strategy in AL annotation helps to improve the model performance in terms of ROUGE and consistency scores. Additionally, we analyze the effect of self-learning and show that it can further increase the performance of the model.
Applying Automated Machine Translation to Educational Video Courses
Authors: Linden Wang
Date: 2023-01-09 00:00:00
Tasks: Machine Translation, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis, Translation, Video Synchronization

We studied the capability of automated machine translation in the online video education space by automatically translating Khan Academy videos with state-of-the-art translation models and applying text-to-speech synthesis and audio/video synchronization to build engaging videos in target languages. We also analyzed and established two reliable translation confidence estimators based on round-trip translations in order to efficiently manage translation quality and reduce human translation effort. Finally, we developed a deployable system to deliver translated videos to end users and collect user corrections for iterative improvement.
Automatic Generation of German Drama Texts Using Fine Tuned GPT-2 Models
Authors: Mariam Bangura, Kristina Barabashova, Anna Karnysheva, Sarah Semczuk, Yifan Wang
Date: 2023-01-08 00:00:00

This study is devoted to the automatic generation of German drama texts. We suggest an approach consisting of two key steps: fine-tuning a GPT-2 model (the outline model) to generate outlines of scenes based on keywords and fine-tuning a second model (the generation model) to generate scenes from the scene outline. The input for the neural model comprises two datasets: the German Drama Corpus (GerDraCor) and German Text Archive (Deutsches Textarchiv or DTA). In order to estimate the effectiveness of the proposed method, our models are compared with baseline GPT-2 models. Our models perform well according to automatic quantitative evaluation, but, conversely, manual qualitative analysis reveals a poor quality of generated texts. This may be due to the quality of the dataset or training inputs.
MEGAnno: Exploratory Labeling for NLP in Computational Notebooks
Authors: Dan Zhang, Hannah Kim, Rafael Li Chen, Eser Kandogan, Estevam Hruschka
Date: 2023-01-08 00:00:00
Tasks: Sentiment Analysis

We present MEGAnno, a novel exploratory annotation framework designed for NLP researchers and practitioners. Unlike existing labeling tools that focus on data labeling only, our framework aims to support a broader, iterative ML workflow including data exploration and model development. With MEGAnno's API, users can programmatically explore the data through sophisticated search and automated suggestion functions and incrementally update task schema as their project evolve. Combined with our widget, the users can interactively sort, filter, and assign labels to multiple items simultaneously in the same notebook where the rest of the NLP project resides. We demonstrate MEGAnno's flexible, exploratory, efficient, and seamless labeling experience through a sentiment analysis use case.
Topic Modelling of Swedish Newspaper Articles about Coronavirus: a Case Study using Latent Dirichlet Allocation Method
Authors: Bernadeta Griciūtė, Lifeng Han, Goran Nenadic
Date: 2023-01-08 00:00:00
Tasks: Articles, Natural Language Understanding

Topic Modelling (TM) is from the research branches of natural language understanding (NLU) and natural language processing (NLP) that is to facilitate insightful analysis from large documents and datasets, such as a summarisation of main topics and the topic changes. This kind of discovery is getting more popular in real-life applications due to its impact on big data analytics. In this study, from the social-media and healthcare domain, we apply popular Latent Dirichlet Allocation (LDA) methods to model the topic changes in Swedish newspaper articles about Coronavirus. We describe the corpus we created including 6515 articles, methods applied, and statistics on topic changes over approximately 1 year and two months period of time from 17th January 2020 to 13th March 2021. We hope this work can be an asset for grounding applications of topic modelling and can be inspiring for similar case studies in an era with pandemics, to support socio-economic impact research as well as clinical and healthcare analytics. Our data and source code are openly available at https://github. com/poethan/Swed_Covid_TM Keywords: Latent Dirichlet Allocation (LDA); Topic Modelling; Coronavirus; Pandemics; Natural Language Understanding; BERT-topic
Analyzing the Representational Geometry of Acoustic Word Embeddings
Authors: Badr M. Abdullah, Dietrich Klakow
Date: 2023-01-08 00:00:00
Tasks: Keyword Spotting, Word Embeddings

Acoustic word embeddings (AWEs) are vector representations such that different acoustic exemplars of the same word are projected nearby in the embedding space. In addition to their use in speech technology applications such as spoken term discovery and keyword spotting, AWE models have been adopted as models of spoken-word processing in several cognitively motivated studies and have been shown to exhibit human-like performance in some auditory processing tasks. Nevertheless, the representational geometry of AWEs remains an under-explored topic that has not been studied in the literature. In this paper, we take a closer analytical look at AWEs learned from English speech and study how the choice of the learning objective and the architecture shapes their representational profile. To this end, we employ a set of analytic techniques from machine learning and neuroscience in three different analyses: embedding space uniformity, word discriminability, and representational consistency. Our main findings highlight the prominent role of the learning objective on shaping the representation profile compared to the model architecture.
Building a Parallel Corpus and Training Translation Models Between Luganda and English
Authors: Richard Kimera, Daniela N. Rim, Heeyoul Choi
Date: 2023-01-07 00:00:00
Tasks: Machine Translation, NMT, Translation

Neural machine translation (NMT) has achieved great successes with large datasets, so NMT is more premised on high-resource languages. This continuously underpins the low resource languages such as Luganda due to the lack of high-quality parallel corpora, so even 'Google translate' does not serve Luganda at the time of this writing. In this paper, we build a parallel corpus with 41,070 pairwise sentences for Luganda and English which is based on three different open-sourced corpora. Then, we train NMT models with hyper-parameter search on the dataset. Experiments gave us a BLEU score of 21.28 from Luganda to English and 17.47 from English to Luganda. Some translation examples show high quality of the translation. We believe that our model is the first Luganda-English NMT model. The bilingual dataset we built will be available to the public.
Generative Antibody Design for Complementary Chain Pairing Sequences through Encoder-Decoder Language Model
Authors: Simon K. S. Chu, Kathy Y. Wei
Date: 2023-01-06 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling

Current protein language models (pLMs) predominantly focus on single-chain protein sequences and often have not accounted for constraints on generative design imposed by protein-protein interactions. To address this gap, we present paired Antibody T5 (pAbT5), an encoder-decoder model to generate complementary heavy or light chain from its pairing partner. We show that our model respects conservation in framework regions and variability in hypervariable domains, demonstrated by agreement with sequence alignment and variable-length CDR loops. We also show that our model captures chain pairing preferences through the recovery of ground-truth chain type and gene families. Our results showcase the potential of pAbT5 in generative antibody design, incorporating biological constraints from chain pairing preferences.
A comprehensive review of automatic text summarization techniques: method, data, evaluation and coding
Authors: Daniel O. Cajueiro, Arthur G. Nery, Igor Tavares, Maísa K. De Melo, Silvia A. dos Reis, Li Weigang, Victor R. R. Celestino
Date: 2023-01-04 00:00:00
Tasks: Text Summarization

We provide a literature review about Automatic Text Summarization (ATS) systems. We consider a citation-based approach. We start with some popular and well-known papers that we have in hand about each topic we want to cover and we have tracked the "backward citations" (papers that are cited by the set of papers we knew beforehand) and the "forward citations" (newer papers that cite the set of papers we knew beforehand). In order to organize the different methods, we present the diverse approaches to ATS guided by the mechanisms they use to generate a summary. Besides presenting the methods, we also present an extensive review of the datasets available for summarization tasks and the methods used to evaluate the quality of the summaries. Finally, we present an empirical exploration of these methods using the CNN Corpus dataset that provides golden summaries for extractive and abstractive methods.
Neural network models
Authors: Plamen Dimitrov
Date: 2023-01-08 00:00:00

This work presents the current collection of mathematical models related to neural networks and proposes a new family of such with extended structure and dynamics in order to attain a selection of cognitive capabilities. It starts by providing a basic background to the morphology and physiology of the biological and the foundations and advances of the artificial neural networks. The first part then continues with a survey of all current mathematical models and some of their derived properties. In the second part, a new family of models is formulated, compared with the rest, and developed analytically and numerically. Finally, important additional aspects and any limitations to deal with in the future are discussed.
EMAHA-DB1: A New Upper Limb sEMG Dataset for Classification of Activities of Daily Living
Authors: Naveen Kumar Karnam, Anish Chand Turlapaty, Shiv Ram Dubey, Balakrishna Gokaraju
Date: 2023-01-09 00:00:00
Tasks: Activity Recognition, Classification

In this paper, we present electromyography analysis of human activity - database 1 (EMAHA-DB1), a novel dataset of multi-channel surface electromyography (sEMG) signals to evaluate the activities of daily living (ADL). The dataset is acquired from 25 able-bodied subjects while performing 22 activities categorised according to functional arm activity behavioral system (FAABOS) (3 - full hand gestures, 6 - open/close office draw, 8 - grasping and holding of small office objects, 2 - flexion and extension of finger movements, 2 - writing and 1 - rest). The sEMG data is measured by a set of five Noraxon Ultium wireless sEMG sensors with Ag/Agcl electrodes placed on a human hand. The dataset is analyzed for hand activity recognition classification performance. The classification is performed using four state-ofthe-art machine learning classifiers, including Random Forest (RF), Fine K-Nearest Neighbour (KNN), Ensemble KNN (sKNN) and Support Vector Machine (SVM) with seven combinations of time domain and frequency domain feature sets. The state-of-theart classification accuracy on five FAABOS categories is 83:21% by using the SVM classifier with the third order polynomial kernel using energy feature and auto regressive feature set ensemble. The classification accuracy on 22 class hand activities is 75:39% by the same SVM classifier with the log moments in frequency domain (LMF) feature, modified LMF, time domain statistical (TDS) feature, spectral band powers (SBP), channel cross correlation and local binary patterns (LBP) set ensemble. The analysis depicts the technical challenges addressed by the dataset. The developed dataset can be used as a benchmark for various classification methods as well as for sEMG signal analysis corresponding to ADL and for the development of prosthetics and other wearable robotics.
Randomized Block-Coordinate Optimistic Gradient Algorithms for Root-Finding Problems
Authors: Quoc Tran-Dinh, Yang Luo
Date: 2023-01-08 00:00:00
Tasks: Federated Learning

In this paper, we develop two new randomized block-coordinate optimistic gradient algorithms to approximate a solution of nonlinear equations in large-scale settings, which are called root-finding problems. Our first algorithm is non-accelerated with constant stepsizes, and achieves $\mathcal{O}(1/k)$ best-iterate convergence rate on $\mathbb{E}[ \Vert Gx^k\Vert^2]$ when the underlying operator $G$ is Lipschitz continuous and satisfies a weak Minty solution condition, where $\mathbb{E}[\cdot]$ is the expectation and $k$ is the iteration counter. Our second method is a new accelerated randomized block-coordinate optimistic gradient algorithm. We establish both $\mathcal{O}(1/k^2)$ and $o(1/k^2)$ last-iterate convergence rates on both $\mathbb{E}[ \Vert Gx^k\Vert^2]$ and $\mathbb{E}[ \Vert x^{k+1} - x^{k}\Vert^2]$ for this algorithm under the co-coerciveness of $G$. In addition, we prove that the iterate sequence $\{x^k\}$ converges to a solution almost surely, and $k\Vert Gx^k\Vert$ attains a $o(1/k)$ almost sure convergence rate. Then, we apply our methods to a class of large-scale finite-sum inclusions, which covers prominent applications in machine learning, statistical learning, and network optimization, especially in federated learning. We obtain two new federated learning-type algorithms and their convergence rate guarantees for solving this problem class.
Solo: Data Discovery Using Natural Language Questions Via A Self-Supervised Approach
Authors: Qiming Wang, Raul Castro Fernandez
Date: 2023-01-09 00:00:00

Most deployed data discovery systems, such as Google Datasets, and open data portals only support keyword search. Keyword search is geared towards general audiences but limits the types of queries the systems can answer. We propose a new system that lets users write natural language questions directly. A major barrier to using this learned data discovery system is it needs expensive-to-collect training data, thus limiting its utility. In this paper, we introduce a self-supervised approach to assemble training datasets and train learned discovery systems without human intervention. It requires addressing several challenges, including the design of self-supervised strategies for data discovery, table representation strategies to feed to the models, and relevance models that work well with the synthetically generated questions. We combine all the above contributions into a system, Solo, that solves the problem end to end. The evaluation results demonstrate the new techniques outperform state-of-the-art approaches on well-known benchmarks. All in all, the technique is a stepping stone towards building learned discovery systems. The code is open-sourced at https://github.com/TheDataStation/solo
Doc2Query--: When Less is More
Authors: Mitko Gospodinov, Sean MacAvaney, Craig Macdonald
Date: 2023-01-09 00:00:00
Tasks: Hallucination, Retrieval

Doc2Query -- the process of expanding the content of a document before indexing using a sequence-to-sequence model -- has emerged as a prominent technique for improving the first-stage retrieval effectiveness of search engines. However, sequence-to-sequence models are known to be prone to "hallucinating" content that is not present in the source text. We argue that Doc2Query is indeed prone to hallucination, which ultimately harms retrieval effectiveness and inflates the index size. In this work, we explore techniques for filtering out these harmful queries prior to indexing. We find that using a relevance model to remove poor-quality queries can improve the retrieval effectiveness of Doc2Query by up to 16%, while simultaneously reducing mean query execution time by 23% and cutting the index size by 33%. We release the code, data, and a live demonstration to facilitate reproduction and further exploration at https://github.com/terrierteam/pyterrier_doc2query.
MangngalApp -- An integrated package of technology for COVID-19 response and rural development: Acceptability and usability using TAM
Authors: Billy S. Javier, Leo P. Paliuanan, James Karl A. Agpalza, Jesty S. Agoto
Date: 2023-01-07 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction, Descriptive

The COVID19 pandemic has challenged universities and organizations to devise mechanisms to uplift the well-being and welfare of people and communities. In response, the design and development of an integrated package of technologies, MangngalApp -- A web-based portal and mobile responsive application for rural development served as an opportunity. It showcases different packets of technologies that were outputs of R&D in the field of fisheries and aqua-culture, innovations that were IP-protected, and technologies that harness locally available resources for post-harvest development and aiding in sustaining growth and development in the communities. This paper focused on the usability and acceptability of the MangngalApp implementing a descriptive research design using the Technology Acceptance Model or TAM and ISO 25010 software quality standards. Constrained by government health restrictions due to COVID-19, a Google form-based questionnaire was forwarded to consented participants via an email with the attached consent and evaluation form. Results revealed that the MangngalApp was found to be very acceptable and usable, and compliant to ISO 25010 software quality characteristics to the higher extent. From the results, it is concluded that the developed MangngalApp will be a usable and responsive technology that aids to rural development especially among target users: fishers, gatherers, processors, traders, and farmers. Considering compatibility and usefulness, the MangngalApp is expected to provide greater social development in the community.
Cost-optimal Seeding Strategy During a Botanical Pandemic in Domesticated Fields
Authors: Teddy Lazebnik
Date: 2023-01-07 00:00:00

Botanical pandemics cause enormous economic damage and food shortages around the globe. However, since botanical pandemics are here to stay in the short-medium term, domesticated field owners can strategically seed their fields to optimize each session's economic profit. In this work, we propose a novel epidemiological-economic mathematical model that describes the economic profit from a field of plants during a botanical pandemic. We describe the epidemiological dynamics using a spatio-temporal extended Susceptible-Infected-Recovered epidemiological model with a non-linear output economic model. We provide an algorithm to obtain an optimal grid-formed seeding strategy to maximize economic profit, given field and pathogen properties. We show that the recovery and basic infection rates have a similar economic influence. Unintuitively, we show that a larger farm does not promise higher economic profit. Our results demonstrate a significant benefit of using the proposed seeding strategy and shed more light on the dynamics of the botanical pandemic.
SONNET: A Self-Guided Ordinal Regression Neural Network for Segmentation and Classification of Nuclei in Large-Scale Multi-Tissue Histology Images
Authors: Tan N. N. Doan, Boram Song, Trinh T. L. Vuong, Kyungeun Kim, Jin T. Kwak
Date: 2022-02-09 00:00:00
Proceeding: IEEE Journal of Biomedical and Health Informatics 2022 2
Tasks: Classification, Multi-tissue Nucleus Segmentation, Nuclear Segmentation, Nuclei Classification, regression, Segmentation

Automated nuclei segmentation and classification are the keys to analyze and understand the cellular characteristics and functionality, supporting computer-aided digital pathology in disease diagnosis. However, the task still remains challenging due to the intrinsic variations in size, intensity, and morphology of different types of nuclei. Herein, we propose a self-guided ordinal regression neural network for simultaneous nuclear segmentation and classification that can exploit the intrinsic characteristics of nuclei and focus on highly uncertain areas during training. The proposed network formulates nuclei segmentation as an ordinal regression learning by introducing a distance decreasing discretization strategy, which stratifies nuclei in a way that inner regions forming a regular shape of nuclei are separated from outer regions forming an irregular shape. It also adopts a self-guided training strategy to adaptively adjust the weights associated with nuclear pixels, depending on the difficulty of the pixels that is assessed by the network itself. To evaluate the performance of the proposed network, we employ large-scale multi-tissue datasets with 276349 exhaustively annotated nuclei. We show that the proposed network achieves the state-of-the-art performance in both nuclei segmentation and classification in comparison to several methods that are recently developed for segmentation and/or classification.
Dynamic Functional Connectivity
Authors: Christine Ahrends, Diego Vidaurre
Date: 2023-01-09 00:00:00
Tasks: Clustering, Functional Connectivity

Most generally, dynamic functional connectivity (FC) refers to the non-instantaneous couplings across timeseries from a set of brain areas, here as measured by fMRI. This is in contrast to static FC, which is defined as purely instantaneous relations. In this chapter, we provide a hands-on description of a non-exhaustive selection of different methods used to estimate dynamic FC (such as sliding windows, clustering approaches, Hidden Markov Models, and multivariate autoregressive models), and we explain, using practical examples, how data should be prepared for dynamic FC analyses and how models of dynamic FC can be evaluated. We also discuss current developments in the dynamic FC research field, including challenges of reliability and reproducibility, and perspectives of using dynamic FC for prediction.