pwc-archive/papers-with-abstracts huggingface dataset:
Plasticity Neural Network Based on Astrocytic effects at Critical Period, Synaptic Competition and Strength Rebalance by Current and Mnemonic Brain Plasticity and Synapse Formation
Authors: Jun-Bo Tao, Bai-Qing Sun, Wei-Dong Zhu, Shi-You Qu, Ling-Kun Chen, Jia-Qiang Li, Guo-Qi Li, Chong Wu, Yu Xiong, Jiaxuan Zhou
Date: 2022-03-19 00:00:00

In addition to the weights of synaptic shared connections, PNN includes weights of synaptic effective ranges [14-25]. PNN considers synaptic strength balance in dynamic of phagocytosing of synapses and static of constant sum of synapses length [14], and includes the lead behavior of the school of fish. Synapse formation will inhibit dendrites generation to a certain extent in experiments and PNN simulations [15]. The memory persistence gradient of retrograde circuit similar to the Enforcing Resilience in a Spring Boot. The relatively good and inferior gradient information stored in memory engram cells in synapse formation of retrograde circuit like the folds in brain [16]. The controversy was claimed if human hippocampal neurogenesis persists throughout aging, may have a new and longer circuit in late iteration [17,18]. Closing the critical period will cause neurological disorder in experiments and PNN simulations [19]. Considering both negative and positive memories persistence help activate synapse length changes better than only considering positive memory [20]. Astrocytic phagocytosis will avoid the local accumulation of synapses by simulation, Lack of astrocytic phagocytosis causes excitatory synapses and functionally impaired synapses accumulate in experiments and lead to destruction of cognition, but local longer synapses and worse results in PNN simulations [21]. It gives relationship of intelligence and cortical thickness, individual differences [22]. The PNN considered the memory engram cells that strengthened Synapses [23]. The effects of PNN's memory structure and tPBM may be the same for powerful penetrability of signals [24].And PNN considers the non-classical quantum computer in brain plasticity[25]. Memory persistence inhibit local synaptic accumulation. It may introduce the relatively good and inferior solution in PSO. The simple PNN only has the synaptic phagocytosis.
Distinguishing Non-natural from Natural Adversarial Samples for More Robust Pre-trained Language Model
Authors: Jiayi Wang, Rongzhou Bao, Zhuosheng Zhang, Hai Zhao
Date: 2022-03-19 00:00:00
Proceeding: Findings (ACL) 2022 5
Tasks: Data Augmentation, Language Modeling, Language Modelling

Recently, the problem of robustness of pre-trained language models (PrLMs) has received increasing research interest. Latest studies on adversarial attacks achieve high attack success rates against PrLMs, claiming that PrLMs are not robust. However, we find that the adversarial samples that PrLMs fail are mostly non-natural and do not appear in reality. We question the validity of current evaluation of robustness of PrLMs based on these non-natural adversarial samples and propose an anomaly detector to evaluate the robustness of PrLMs with more natural adversarial samples. We also investigate two applications of the anomaly detector: (1) In data augmentation, we employ the anomaly detector to force generating augmented data that are distinguished as non-natural, which brings larger gains to the accuracy of PrLMs. (2) We apply the anomaly detector to a defense framework to enhance the robustness of PrLMs. It can be used to defend all types of attacks and achieves higher accuracy on both adversarial samples and compliant samples than other defense frameworks.
Machine Learning based Laser Failure Mode Detection
Authors: Khouloud Abdelli, Danish Rafique, Stephan Pachnicke
Date: 2022-03-19 00:00:00
Tasks: BIG-bench Machine Learning, Fault Detection, regression

Laser degradation analysis is a crucial process for the enhancement of laser reliability. Here, we propose a data-driven fault detection approach based on Long Short-Term Memory (LSTM) recurrent neural networks to detect the different laser degradation modes based on synthetic historical failure data. In comparison to typical threshold-based systems, attaining 24.41% classification accuracy, the LSTM-based model achieves 95.52% accuracy, and also outperforms classical machine learning (ML) models namely Random Forest (RF), K-Nearest Neighbours (KNN) and Logistic Regression (LR).
Machine Learning based Data Driven Diagnostic and Prognostic Approach for Laser Reliability Enhancement
Authors: Khouloud Abdelli, Helmut Griesser, Stephan Pachnicke
Date: 2022-03-19 00:00:00
Tasks: BIG-bench Machine Learning, Diagnostic

In this paper, a data-driven diagnostic and prognostic approach based on machine learning is proposed to detect laser failure modes and to predict the remaining useful life (RUL) of a laser during its operation. We present an architecture of the proposed cognitive predictive maintenance framework and demonstrate its effectiveness using synthetic data.
Transformer based ensemble for emotion detection
Authors: Aditya Kane, Shantanu Patankar, Sahil Khose, Neeraja Kirtane
Date: 2022-03-22 00:00:00
Proceeding: WASSA (ACL) 2022 5

Detecting emotions in languages is important to accomplish a complete interaction between humans and machines. This paper describes our contribution to the WASSA 2022 shared task which handles this crucial task of emotion detection. We have to identify the following emotions: sadness, surprise, neutral, anger, fear, disgust, joy based on a given essay text. We are using an ensemble of ELECTRA and BERT models to tackle this problem achieving an F1 score of $62.76\%$. Our codebase (https://bit.ly/WASSA_shared_task) and our WandB project (https://wandb.ai/acl_wassa_pictxmanipal/acl_wassa) is publicly available.
SU-NLP at SemEval-2022 Task 11: Complex Named Entity Recognition with Entity Linking
Authors: Buse Çarık, Fatih Beyhan, Reyyan Yeniterzi
Date: 2022-03-22 00:00:00
Proceeding: SemEval (NAACL) 2022 7
Tasks: Entity Linking, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

This paper describes the system proposed by Sabanc{\i} University Natural Language Processing Group in the SemEval-2022 MultiCoNER task. We developed an unsupervised entity linking pipeline that detects potential entity mentions with the help of Wikipedia and also uses the corresponding Wikipedia context to help the classifier in finding the named entity type of that mention. Our results showed that our pipeline improved performance significantly, especially for complex entities in low-context settings.
Demo of the Linguistic Field Data Management and Analysis System -- LiFE
Authors: Siddharth Singh, Ritesh Kumar, Shyam Ratan, Sonal Sinha
Date: 2022-03-22 00:00:00
Tasks: Management

In the proposed demo, we will present a new software - Linguistic Field Data Management and Analysis System - LiFE (https://github.com/kmi-linguistics/life) - an open-source, web-based linguistic data management and analysis application that allows for systematic storage, management, sharing and usage of linguistic data collected from the field. The application allows users to store lexical items, sentences, paragraphs, audio-visual content with rich glossing / annotation; generate interactive and print dictionaries; and also train and use natural language processing tools and models for various purposes using this data. Since its a web-based application, it also allows for seamless collaboration among multiple persons and sharing the data, models, etc with each other. The system uses the Python-based Flask framework and MongoDB in the backend and HTML, CSS and Javascript at the frontend. The interface allows creation of multiple projects that could be shared with the other users. At the backend, the application stores the data in RDF format so as to allow its release as Linked Data over the web using semantic web technologies - as of now it makes use of the OntoLex-Lemon for storing the lexical data and Ligt for storing the interlinear glossed text and then internally linking it to the other linked lexicons and databases such as DBpedia and WordNet. Furthermore it provides support for training the NLP systems using scikit-learn and HuggingFace Transformers libraries as well as make use of any model trained using these libraries - while the user interface itself provides limited options for tuning the system, an externally-trained model could be easily incorporated within the application; similarly the dataset itself could be easily exported into a standard machine-readable format like JSON or CSV that could be consumed by other programs and pipelines.
Task-guided Disentangled Tuning for Pretrained Language Models
Authors: Jiali Zeng, Yufan Jiang, Shuangzhi Wu, Yongjing Yin, Mu Li
Date: 2022-03-22 00:00:00
Proceeding: Findings (ACL) 2022 5

Pretrained language models (PLMs) trained on large-scale unlabeled corpus are typically fine-tuned on task-specific downstream datasets, which have produced state-of-the-art results on various NLP tasks. However, the data discrepancy issue in domain and scale makes fine-tuning fail to efficiently capture task-specific patterns, especially in the low data regime. To address this issue, we propose Task-guided Disentangled Tuning (TDT) for PLMs, which enhances the generalization of representations by disentangling task-relevant signals from the entangled representations. For a given task, we introduce a learnable confidence model to detect indicative guidance from context, and further propose a disentangled regularization to mitigate the over-reliance problem. Experimental results on GLUE and CLUE benchmarks show that TDT gives consistently better results than fine-tuning with different PLMs, and extensive analysis demonstrates the effectiveness and robustness of our method. Code is available at https://github.com/lemon0830/TDT.
VLSP 2021 - ViMRC Challenge: Vietnamese Machine Reading Comprehension
Authors: Kiet Van Nguyen, Son Quoc Tran, Luan Thanh Nguyen, Tin Van Huynh, Son T. Luu, Ngan Luu-Thuy Nguyen
Date: 2022-03-22 00:00:00
Tasks: Language Modelling, Machine Reading Comprehension, Natural Language Inference, Natural Language Understanding, Question Answering, Question Generation, Question-Generation, Reading Comprehension, Vietnamese Datasets, Vietnamese Machine Reading Comprehension

One of the emerging research trends in natural language understanding is machine reading comprehension (MRC) which is the task to find answers to human questions based on textual data. Existing Vietnamese datasets for MRC research concentrate solely on answerable questions. However, in reality, questions can be unanswerable for which the correct answer is not stated in the given textual data. To address the weakness, we provide the research community with a benchmark dataset named UIT-ViQuAD 2.0 for evaluating the MRC task and question answering systems for the Vietnamese language. We use UIT-ViQuAD 2.0 as a benchmark dataset for the challenge on Vietnamese MRC at the Eighth Workshop on Vietnamese Language and Speech Processing (VLSP 2021). This task attracted 77 participant teams from 34 universities and other organizations. In this article, we present details of the organization of the challenge, an overview of the methods employed by shared-task participants, and the results. The highest performances are 77.24% in F1-score and 67.43% in Exact Match on the private test set. The Vietnamese MRC systems proposed by the top 3 teams use XLM-RoBERTa, a powerful pre-trained language model based on the transformer architecture. The UIT-ViQuAD 2.0 dataset motivates researchers to further explore the Vietnamese machine reading comprehension task and related tasks such as question answering, question generation, and natural language inference.
Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection
Authors: Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, Julian McAuley
Date: 2022-03-22 00:00:00
Proceeding: ACL 2022 5
Tasks: Informativeness, Specificity

A limitation of current neural dialog models is that they tend to suffer from a lack of specificity and informativeness in generated responses, primarily due to dependence on training data that covers a limited variety of scenarios and conveys limited knowledge. One way to alleviate this issue is to extract relevant knowledge from external sources at decoding time and incorporate it into the dialog response. In this paper, we propose a post-hoc knowledge-injection technique where we first retrieve a diverse set of relevant knowledge snippets conditioned on both the dialog history and an initial response from an existing dialog model. We construct multiple candidate responses, individually injecting each retrieved snippet into the initial response using a gradient-based decoding method, and then select the final response with an unsupervised ranking step. Our experiments in goal-oriented and knowledge-grounded dialog settings demonstrate that human annotators judge the outputs from the proposed method to be more engaging and informative compared to responses from prior dialog systems. We further show that knowledge-augmentation promotes success in achieving conversational goals in both experimental settings.
Efficient Classification of Long Documents Using Transformers
Authors: Hyunji Hayley Park, Yogarshi Vyas, Kashif Shah
Date: 2022-03-21 00:00:00
Proceeding: ACL 2022 5
Tasks: Classification, Document Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Several methods have been proposed for classifying long textual documents using Transformers. However, there is a lack of consensus on a benchmark to enable a fair comparison among different approaches. In this paper, we provide a comprehensive evaluation of the relative efficacy measured against various baselines and diverse datasets -- both in terms of accuracy as well as time and space overheads. Our datasets cover binary, multi-class, and multi-label classification tasks and represent various ways information is organized in a long text (e.g. information that is critical to making the classification decision is at the beginning or towards the end of the document). Our results show that more complex models often fail to outperform simple baselines and yield inconsistent performance across datasets. These findings emphasize the need for future studies to consider comprehensive baselines and datasets that better represent the task of long document classification to develop robust models.
DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization
Authors: Zheng Li, Zijian Wang, Ming Tan, Ramesh Nallapati, Parminder Bhatia, Andrew Arnold, Bing Xiang, Dan Roth
Date: 2022-03-21 00:00:00
Proceeding: ACL 2022 5
Tasks: Knowledge Distillation, Model Compression, Quantization, Text Generation

Large-scale pre-trained sequence-to-sequence models like BART and T5 achieve state-of-the-art performance on many generative NLP tasks. However, such models pose a great challenge in resource-constrained scenarios owing to their large memory requirements and high latency. To alleviate this issue, we propose to jointly distill and quantize the model, where knowledge is transferred from the full-precision teacher model to the quantized and distilled low-precision student model. Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16.5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets. We further pushed the limit of compression ratio to 27.7x and presented the performance-efficiency trade-off for generative tasks using pre-trained models. To the best of our knowledge, this is the first work aiming to effectively distill and quantize sequence-to-sequence pre-trained models for language generation tasks.
Adaptative clustering by minimization of the mixing entropy criterion
Authors: Thierry Dumont
Date: 2022-03-22 00:00:00
Tasks: Clustering

We present a clustering method and provide a theoretical analysis and an explanation to a phenomenon encountered in the applied statistical literature since the 1990's. This phenomenon is the natural adaptability of the order when using a clustering method derived from the famous EM algorithm. We define a new statistic, the relative entropic order, that represents the number of clumps in the target distribution. We prove in particular that the empirical version of this relative entropic order is consistent. Our approach is easy to implement and has a high potential of applications. Perspectives of this works are algorithmic and theoretical, with possible natural extensions to various cases such as dependent or multidimensional data.
Evolution is Driven by Natural Autoencoding: Reframing Species, Interaction Codes, Cooperation, and Sexual Reproduction
Authors: Irun R. Cohen, Assaf Marron
Date: 2022-03-22 00:00:00
Tasks: Dimensionality Reduction

The continuity of life and its evolution, we proposed, emerge from an interactive group process manifested in networks of interaction. We term this process \textit{survival-of-the-fitted}. Here, we reason that survival of the fitted results from a natural computational process we term \textit{natural autoencoding}. Natural autoencoding works by retaining repeating biological interactions while non-repeatable interactions disappear. (1) We define a species by its \textit{species interaction code}, which consists of a compact description of the repeating interactions of species organisms with their external and internal environments. Species interaction codes are descriptions recorded in the biological infrastructure that enables repeating interactions. Encoding and decoding are interwoven. (2) Evolution proceeds by natural autoencoding of sustained changes in species interaction codes. DNA is only one element in natural autoencoding. (3) Natural autoencoding accounts for the paradox of genome randomization in sexual reproduction -- recombined genomes are analogous to the diversified inputs required for artificial autoencoding. The increase in entropy generated by genome randomization compensates for the decrease in entropy generated by organized life. (4) Natural autoencoding and artificial autoencoding algorithms manifest defined similarities and differences. Recognition of the importance of fittedness could well serve the future of a humanly livable biosphere.
Running Time Analysis of the Non-dominated Sorting Genetic Algorithm II (NSGA-II) using Binary or Stochastic Tournament Selection
Authors: Chao Bian, Chao Qian
Date: 2022-03-22 00:00:00
Tasks: Evolutionary Algorithms

Evolutionary algorithms (EAs) have been widely used to solve multi-objective optimization problems, and have become the most popular tool. However, the theoretical foundation of multi-objective EAs (MOEAs), especially the essential theoretical aspect, i.e., running time analysis, has been still largely underdeveloped. The few existing theoretical works mainly considered simple MOEAs, while the non-dominated sorting genetic algorithm II (NSGA-II), probably the most influential MOEA, has not been analyzed except for a very recent work considering a simplified variant without crossover. In this paper, we present a running time analysis of the standard NSGA-II for solving LOTZ, OneMinMax and COCZ, the three commonly used bi-objective optimization problems. Specifically, we prove that the expected running time (i.e., number of fitness evaluations) is $O(n^3)$ for LOTZ, and $O(n^2\log n)$ for OneMinMax and COCZ, which is surprisingly as same as that of the previously analyzed simple MOEAs, GSEMO and SEMO. Next, we introduce a new parent selection strategy, stochastic tournament selection (i.e., $k$ tournament selection where $k$ is uniformly sampled at random), to replace the binary tournament selection strategy of NSGA-II, decreasing the required expected running time to $O(n^2)$ for all the three problems. Experiments are also conducted, suggesting that the derived running time upper bounds are tight for LOTZ, and almost tight for OneMinMax and COCZ.
Centered plug-in estimation of Wasserstein distances
Authors: Tamás P. Papp, Chris Sherlock
Date: 2022-03-22 00:00:00

The plug-in estimator of the squared Euclidean 2-Wasserstein distance is conservative, however due to its large positive bias it is often uninformative. We eliminate most of this bias using a simple centering procedure based on linear combinations. We construct a pair of centered plug-in estimators that decrease with the true Wasserstein distance, and are therefore guaranteed to be informative, for any finite sample size. Crucially, we demonstrate that these estimators can often be viewed as complementary upper and lower bounds on the squared Wasserstein distance. Finally, we apply the estimators to Bayesian computation, developing methods for estimating (i) the bias of approximate inference methods and (ii) the convergence of MCMC algorithms.
A Locally Adaptive Algorithm for Multiple Testing with Network Structure
Authors: Ziyi Liang, T. Tony Cai, Wenguang Sun, Yin Xia
Date: 2022-03-22 00:00:00
Tasks: Transfer Learning

Incorporating auxiliary information alongside primary data can significantly enhance the accuracy of simultaneous inference. However, existing multiple testing methods face challenges in efficiently incorporating complex side information, especially when it differs in dimension or structure from the primary data, such as network side information. This paper introduces a locally adaptive structure learning algorithm (LASLA), a flexible framework designed to integrate a broad range of auxiliary information into the inference process. Although LASLA is specifically motivated by the challenges posed by network-structured data, it also proves highly effective with other types of side information, such as spatial locations and multiple auxiliary sequences. LASLA employs a $p$-value weighting approach, leveraging structural insights to derive data-driven weights that prioritize the importance of different hypotheses. Our theoretical analysis demonstrates that LASLA asymptotically controls the false discovery rate (FDR) under independent or weakly dependent $p$-values, and achieves enhanced power in scenarios where the auxiliary data provides valuable side information. Simulation studies are conducted to evaluate LASLA's numerical performance, and its efficacy is further illustrated through two real-world applications.
Mathematical modeling of glioma invasion and therapy approaches via kinetic theory of active particles
Authors: Martina Conte, Yvonne Dzierma, Sven Knobe, Christina Surulescu
Date: 2022-03-22 00:00:00

A multiscale model for glioma spread in brain tissue under the influence of vascularization and vascular endothelial growth factors is proposed. It accounts for the interplay between the different components of the neoplasm and the healthy tissue and it investigates and compares various therapy approaches. Precisely, these involve radio- and chemotherapy in a concurrent or adjuvant manner together with anti-angiogenic therapy affecting the vascular component of the system. We assess tumor growth and spread on the basis of DTI data, which allows us to reconstruct a realistic brain geometry and tissue structure, and we apply our model to real glioma patient data. In this latter case, a space-dependent radiotherapy description is considered using data about the corresponding isodose curves.