Reviewerly/exHarmony huggingface dataset:
Obstructions to Compositionality

Compositionality is at the heart of computer science and several other areas of applied category theory such as computational linguistics, categorical quantum mechanics, interpretable AI, dynamical systems, compositional game theory, and Petri nets. However, the meaning of the term seems to vary across the many different applications. This work contributes to understanding, and in particular qualifying, different kinds of compositionality. Formally, we introduce invariants of categories that we call zeroth and first homotopy posets, generalising in a precise sense the pi0 and pi1 of a groupoid. These posets can be used to obtain a qualitative description of how far an object is from being terminal and a morphism is from being iso. In the context of applied category theory, this formal machinery gives us a way to qualitatively describe the "failures of compositionality", seen as failures of certain (op)lax functors to be strong, by classifying obstructions to the (op)laxators being isomorphisms. Failure of compositionality, for example for the interpretation of a categorical syntax in a semantic universe, can both be a bad thing and a good thing, which we illustrate by respective examples in graph theory and quantum theory.
Keyboarding Frege's concept writing

Gottlob Frege’s 1879 book Begriffsschrift (Concept Writing) poses a number of problems for the would-be digitizer, most notably its extensive use of a two-dimensional notation not easily reduced to a linear sequence of characters. With careful study, however, the notation can be seen to possess a clear internal logic, and an easily keyboardable serialized form for data capture can be designed, which makes it easy to transcribe even complex formulas. Invisible XML supports the key step of parsing that data capture format and turning it into usefully structured XML, from which routine stylesheets can generate SVG images of the two-dimensional formulae and other useful outputs.
Information Monitoring of Transmission Lines Based on Internet of Things Technology

To solve the problem of difficult real-time monitoring of current transmission lines, this article proposes an information-based monitoring system for transmission lines based on Internet of Things technology. The system utilizes the characteristics of strong scalability, good fault tolerance, low power consumption, and low cost of the Internet of Things. Taking the ultra-low power consumption MSP430 microcontroller and CC2430 radio frequency module as the core, a line monitoring system based on the Internet of Things is designed. The proposed design uses ZigBee wireless sensor network technology which is powered by solar energy. The collection, transmission, processing and judgment of various environmental parameters of the line are realized. The data information is transferred to the monitoring center of the upper computer through GPRS. When there is an abnormality, it can send a mobile phone short message to the person in charge to feedback the abnormal content in time. The distribution network's load symmetry allowed for the development of several locating procedures. For the three-phase symmetric scheme, the fault location approach based on line supply characteristics was employed, and for the three-phase asymmetric scheme, the fault location technique based on line impedance is proposed. One of the most vital uses for the Internet of Things is in the mitigation of power transmission line failures and disasters. Improved power transmission dependability, less financial loss, and fewer power outages are all possible thanks to the Internet of Things' cutting-edge sensing and communication technology. This research introduced the use of IoT in online monitoring system of electricity transmission line with a focus on the characteristics of the construction and development of smart grid. The results indicated that the system's highest temperature difference is 0.31 C, while the maximum humidity difference is 1.38%. The system increases the safety and manageability of electricity transmission while also fostering the widespread adoption and technical integration of the smart grid and the Internet of Things.
A global-frequency-domain network for medical image segmentation

The UNet series networks have been a leader in the field of medical image segmentation since their introduction. However, encoder and decoder structures of the traditional UNet series network are complex, with a large number of parameters and floating-point operations. This requires a large amount of data as support for model training, but most medical datasets only contain limited numbers of samples. To address this issue, we propose a global frequency domain UNet (GFUNet), a novel architecture for fast medical image segmentation. Inspired by recent modified Multi-Layer Perceptron(MLP)-like models, we combine Fourier Transform with UNet structure to achieve more efficient and effective encoding and decoding processes. Meanwhile, A dual-domain encoding module is designed to improve the performance of the encoder and decoder by fully used frequency domain feature. Furthermore, due to the excellent property of the Fourier Transform and its optimization, our network greatly reduces the number of parameters compared to other UNets. We evaluate GFUNet on several medical segmentation tasks, achieving improved segmentation performance compared to state-of-the-art network architectures for medical image segmentation. Compared to the original UNet, the results show that we reduce the number of parameters by 46 times, reduce computational complexity by 114 times, and improved the considerable dice score.
Perspective on Solid‐State Single‐Photon Sources in the Infrared for Quantum Technology

Abstract Solid‐state single‐photon sources in the infrared region are crucial for advancing quantum technologies, in particular long‐distance fiber or on‐chip quantum communication, quantum key distribution, quantum computing, quantum sensing and metrology, and fundamental research. The ability to generate and manipulate individual photons in the infrared spectrum is an area of expansion due to the availability of advanced material quality or new emerging materials. The latest advancements in the discovery and utilization of single‐photon sources in various material platforms and their foreseeable future development are here summarized.
Learning to Imagine: Visually-Augmented Natural Language Generation

People often imagine relevant scenes to aid in the writing process.In this work, we aim to utilize visual information for composition in the same manner as humans.We propose a method, LIVE, that makes pre-trained language models (PLMs) Learn to Imagine for Visuallyaugmented natural language gEneration.First, we imagine the scene based on the text: we use a diffusion model to synthesize high-quality images conditioned on the input texts.Second, we use CLIP to determine whether the text can evoke the imagination in a posterior way.Finally, our imagination is dynamic, and we conduct synthesis for each sentence rather than generate only one image for an entire paragraph.Technically, we propose a novel plug-and-play fusion layer to obtain visually-augmented representations for each text.Our vision-text fusion layer is compatible with Transformerbased architecture.We have conducted extensive experiments on four generation tasks using BART and T5, and the automatic results and human evaluation demonstrate the effectiveness of our proposed method.We will release the code, model, and data at the link: https://github.com/RUCAIBox/LIVE.
A Practical Toolkit for Multilingual Question and Answer Generation

Generating questions along with associated answers from a text has applications in several domains, such as creating reading comprehension tests for students, or improving document search by providing auxiliary questions and answers based on the query.Training models for question and answer generation (QAG) is not straightforward due to the expected structured output (i.e. a list of question and answer pairs), as it requires more than generating a single sentence.This results in a small number of publicly accessible QAG models.In this paper, we introduce AutoQG, an online service for multilingual QAG, along with lmqg, an allin-one Python package for model fine-tuning, generation, and evaluation.We also release QAG models in eight languages fine-tuned on a few variants of pre-trained encoder-decoder language models, which can be used online via AutoQG or locally via lmqg.With these resources, practitioners of any level can benefit from a toolkit that includes a web interface for end users, and easy-to-use code for developers who require custom models or fine-grained controls for generation.
MOBA-E2C: Generating MOBA Game Commentaries via Capturing Highlight Events from the Meta-Data

MOBA (Multiplayer Online Battle Arena) games such as Dota2 are currently one of the most popular e-sports gaming genres.Following professional commentaries is a great way to understand and enjoy a MOBA game.However, massive game competitions lack commentaries because of the shortage of professional human commentators.As an alternative, employing machine commentators that can work at any time and place is a feasible solution.Considering the challenges in modeling MOBA games, we propose a data-driven MOBA commentary generation framework, MOBA-E2C, allowing a model to generate commentaries based on the game meta-data.Subsequently, to alleviate the burden of collecting supervised data, we propose a MOBA-FuseGPT generator to generate MOBA game commentaries by fusing the power of a rule-based generator and a generative GPT generator.Finally, in the experiments, we take a popular MOBA game Dota2 as our case and construct a Chinese Dota2 commentary generation dataset Dota2-Commentary.Experimental results demonstrate the superior performance of our approach.
SemEval-2023 Task 1: Visual Word Sense Disambiguation

This paper presents the Visual Word Sense Disambiguation (Visual-WSD) task.The objective of Visual-WSD is to identify among a set of ten images the one that corresponds to the intended meaning of a given ambiguous word which is accompanied with minimal context.The task provides datasets for three different languages: English, Italian, and Farsi.We received a total of 96 different submissions.Out of these, 40 systems outperformed a strong zero-shot CLIPbased baseline (Radford et al., 2021).Participating systems proposed different zero-and few-shot approaches, often involving generative models and data augmentation.More information can be found on the task's website:
CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval

Multi-vector retrieval methods combine the merits of sparse (e.g.BM25) and dense (e.g.DPR) retrievers and achieve state-of-the-art performance on various retrieval tasks.These methods, however, are orders of magnitude slower and need more space to store their indexes compared to their single-vector counterparts.In this paper, we unify different multi-vector retrieval models from a token routing viewpoint and propose conditional token interaction via dynamic lexical routing, namely CITADEL, for efficient and effective multi-vector retrieval.CITADEL learns to route each token vector to the predicted lexical "keys" such that a query token vector only interacts with document token vectors routed to the same key.This design significantly reduces the computation cost while maintaining high accuracy.Notably, CITADEL achieves the same or slightly better performance than the previous state of the art, ColBERT-v2, on both in-domain (MS MARCO) and out-of-domain (BEIR) evaluations, while being nearly 40 times faster.Source
Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction

Continual few-shot relation extraction (RE) aims to continuously train a model for new relations with few labeled training data, of which the major challenges are the catastrophic forgetting of old relations and the overfitting caused by data sparsity.In this paper, we propose a new model, namely SCKD, to accomplish the continual few-shot RE task.Specifically, we design serial knowledge distillation to preserve the prior knowledge from previous models and conduct contrastive learning with pseudo samples to keep the representations of samples in different relations sufficiently distinguishable.Our experiments on two benchmark datasets validate the effectiveness of SCKD for continual few-shot RE and its superiority in knowledge transfer and memory utilization over stateof-the-art models.
An Empirical Comparison of LM-based Question and Answer Generation Methods

Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context (e.g. a paragraph).This task has a variety of applications, such as data augmentation for question answering (QA) models, information retrieval and education.In this paper, we establish baselines with three different QAG methodologies that leverage sequence-to-sequence language model (LM) fine-tuning.Experiments show that an endto-end QAG model, which is computationally light at both training and inference times, is generally robust and outperforms other more convoluted approaches.However, there are differences depending on the underlying generative LM.Finally, our analysis shows that QA models fine-tuned solely on generated questionanswer pairs can be competitive when compared to supervised QA models trained on human-labeled data.
Small Pre-trained Language Models Can be Fine-tuned as Large Models via Over-Parameterization

By scaling the model size, large pre-trained language models (PLMs) have shown remarkable performance in various natural language processing tasks, mostly outperforming small PLMs by a large margin.However, due to the high computational cost, the huge number of parameters also restricts the applicability of large PLMs in real-world systems.In this paper, we focus on scaling up the parameters of PLMs only during fine-tuning, to benefit from the over-parameterization, while without increasing the inference latency.Given a relatively small PLM, we over-parameterize it by employing a matrix product operator, an efficient and almost lossless decomposition method to factorize its contained parameter matrices into a set of higher-dimensional tensors.Considering the efficiency, we further propose both static and dynamic strategies to select the most important parameter matrices for over-parameterization.Extensive experiments have demonstrated that our approach can significantly boost the finetuning performance of small PLMs and even help small PLMs outperform 3× parameterized larger ones.Our code is publicly available at https://github.com/zfgao66/OPF.
Entity-based SpanCopy for Abstractive Summarization to Improve the Factual Consistency

Discourse-aware techniques, including entityaware approaches, play a crucial role in summarization.In this paper, we propose an entity-based SpanCopy mechanism to tackle the entity-level factual inconsistency problem in abstractive summarization, i.e. reducing the mismatched entities between the generated summaries and the source documents.Complemented by a Global Relevance component to identify summary-worthy entities, our approach demonstrates improved factual consistency while preserving saliency on four summarization datasets, contributing to the effective application of discourse-aware methods summarization tasks. 1
Investigating Glyph-Phonetic Information for Chinese Spell Checking: What Works and What’s Next?

While pre-trained Chinese language models have demonstrated impressive performance on a wide range of NLP tasks, the Chinese Spell Checking (CSC) task remains a challenge.Previous research has explored using information such as glyphs and pronunciations to improve the ability of CSC models to distinguish misspelled characters, with good results at the accuracy level on public datasets.However, the generalization ability of these CSC models has not been well understood: it is unclear whether they incorporate glyph-phonetic information and, if so, whether this information is fully utilized.In this paper, we aim to better understand the role of glyph-phonetic information in the CSC task and suggest directions for improvement.Additionally, we propose a new, more challenging, and practical setting for testing the generalizability of CSC models.Our code will be released at https://github.com/piglaker/ConfusionCluster.
ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation

We study the text generation task under the approach of pre-trained language models (PLMs).Typically, an auto-regressive (AR) method is adopted for generating texts in a token-bytoken manner.Despite many advantages of AR generation, it usually suffers from inefficient inference.Therefore, non-autoregressive (NAR) models are proposed to generate all target tokens simultaneously.However, NAR models usually generate texts of lower quality due to the absence of token dependency in the output text.In this paper, we propose ELMER: an Efficient and effective PLM for NAR tExt geneRation to explicitly model the token dependency during NAR generation.By leveraging the early exit technique, ELMER enables the token generations at different layers, according to their prediction confidence (a more confident token will exit at a lower layer).Besides, we propose a novel pre-training objective, Layer Permutation Language Modeling, to pre-train ELMER by permuting the exit layer for each token in sequences.Experiments on three text generation tasks show that ELMER significantly outperforms NAR models and further narrows the performance gap with AR PLMs (e.g., ELMER (29.92) vs BART (30.61)ROUGE-L in XSUM) while achieving over 10 times inference speedup.
Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation

The predictions of question answering (QA) systems are typically evaluated against manually annotated finite sets of one or more answers.This leads to a coverage limitation that results in underestimating the true performance of systems, and is typically addressed by extending over exact match (EM) with predefined rules or with the token-level F 1 measure.In this paper, we present the first systematic conceptual and data-driven analysis to examine the shortcomings of token-level equivalence measures.To this end, we define the asymmetric notion of answer equivalence (AE), accepting answers that are equivalent to or improve over the reference, and publish over 23k human judgments for candidates produced by multiple QA systems on SQuAD. 1 Through a careful analysis of this data, we reveal and quantify several concrete limitations of the F 1 measure, such as a false impression of graduality, or missing dependence on the question.Since collecting AE annotations for each evaluated model is expensive, we learn a BERT matching (BEM) measure to approximate this task.Being a simpler task than QA, we find BEM to provide significantly better AE approximations than F 1 , and to more accurately reflect the performance of systems.Finally, we demonstrate the practical utility of AE and BEM on the concrete application of minimal accurate prediction sets, reducing the number of required answers by up to ×2.6.
ToKen: Task Decomposition and Knowledge Infusion for Few-Shot Hate Speech Detection

Badr AlKhamissi, Faisal Ladhak, Srinivasan Iyer, Veselin Stoyanov, Zornitsa Kozareva, Xian Li, Pascale Fung, Lambert Mathias, Asli Celikyilmaz, Mona Diab. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.
Towards Intelligent Clinically-Informed Language Analyses of People with Bipolar Disorder and Schizophrenia

NLP offers a myriad of opportunities to support mental health research.However, prior work has almost exclusively focused on social media data, for which diagnoses are difficult or impossible to validate.We present a first-ofits-kind dataset of manually transcribed interactions with people clinically diagnosed with bipolar disorder and schizophrenia, as well as healthy controls.Data was collected through validated clinical tasks and paired with diagnostic measures.We extract 100+ temporal, sentiment, psycholinguistic, emotion, and lexical features from the data and establish classification validity using a variety of models to study language differences between diagnostic groups.Our models achieve strong classification performance (maximum F 1 =0.93-0.96),and lead to the discovery of interesting associations between linguistic features and diagnostic class.It is our hope that this dataset will offer high value to clinical and NLP researchers, with potential for widespread broader impacts.
A Multi-Modal Knowledge Graph for Classical Chinese Poetry

Classical Chinese poetry has a long history and is a precious cultural heritage of humankind.Displaying the classical Chinese poetry in a visual way, helps to cross cultural barriers in different countries, making it enjoyable for all the people.In this paper, we construct a multimodal knowledge graph for classical Chinese poetry (PKG), in which the visual information of words in the poetry are incorporated.Then a multi-modal pre-training language model, i.e., PKG-Bert, is proposed to obtain the poetry representation with visual information, which bridges the semantic gap between different modalities.PKG-Bert achieves the state-of-theart performance on the poetry-image retrieval task, showing the effectiveness of incorporating the multi-modal knowledge.The large-scale multi-modal knowledge graph of classical Chinese poetry will be released to promote the researches in classical Chinese culture area 1 .
Detecting Dementia from Long Neuropsychological Interviews

Neuropsychological exams are commonly used to diagnose various kinds of cognitive impairment.They typically involve a trained examiner who conducts a series of cognitive tests with a subject.In recent years, there has been growing interest in developing machine learning methods to extract speech and language biomarkers from exam recordings to provide automated input for cognitive assessment.Inspired by recent findings suggesting that the examiner's language can influence cognitive impairment classifications, in this paper, we study the influence of the examiner on automatic dementia identification decisions in real-world neuropsychological exams.To mitigate the influence of the examiner, we propose a systematic three-stage pipeline for detecting dementia from exam recordings.In the first stage, we perform audio-based speaker diarization (i.e., estimating who spoke when?) by incorporating speaker discriminative features.In the second stage, we employ text-based language models to identify the role of the speaker (i.e., examiner or subject).Finally, in the third stage, we employ text-and audio-based models to detect cognitive impairment from hypothesized subject segments.Our studies suggest that incorporating audio-based diarization followed by text-based role identification helps mitigate the influences from the examiner's segments.Further, we found that the text and audio modalities complement each other, and the performance improves when we use both modalities.We also perform several carefully designed experimental studies to assess the performance of each stage.Task