Miracle-dz/UniFAR-FTUs huggingface dataset:
Research on the Evaluation of Token Imbalance Degree of NMT Corpus

As a kind of classifier, neural machine trans-001 lation (NMT) is known to perform better with 002 balanced tokens during training. Studying the 003 token distribution in NMT corpus is of guiding 004 significance to improve its quality and the trans-005 lation effect. Due to the existing researches 006 on token imbalance degree have deficiencies 007 in algorithm performance and word segmenta-008 tion scope, we propose the Dispersion of Token 009 Distribution (DTD) algorithm, and use it to 010 evaluate corpus from three segmentation levels: 011 character, subword and word. Our experiments 012 show that this algorithm has an improvement in 013 accuracy, effectiveness and robustness. Mean-014 while, we find that the token imbalance degree 015 of NMT corpus varies greatly at different seg-016 mentation levels, among which character has 017 the highest, word has the lowest and subword is 018 in between. In addition, we also find the regu-019 larities of token imbalance degree in languages 020 German (DE), English (EN), French (FR) and 021 Russian (RU). 022
C AUSAL R: Causal Reasoning over Natural Language Rulebases

Transformers have been shown to be able to 001 perform deductive reasoning on a logical rule-002 base containing rules and statements written 003 in natural language. Recent works show that 004 such models can also produce the reasoning 005 steps (i.e., the proof graph ) that emulate the 006 model’s logical reasoning process. But these 007 models behave as a black-box unit that emu-008 lates the reasoning process without any causal 009 constraints in the reasoning steps, thus ques-010 tioning the faithfulness. In this work, we frame 011 the deductive logical reasoning task as a causal 012 process by defining three modular components: 013 rule selection, fact selection, and knowledge 014 composition. The rule and fact selection steps 015 select the candidate rule and facts to be used 016 and then the knowledge composition combines 017 them to generate new inferences. This ensures 018 model faithfulness by assured causal relation 019 from the proof step to the inference reasoning. 020 To test our causal reasoning framework, we 021 propose C AUSAL R where the above three com-022 ponents are independently modeled by trans-023 formers. We observe that C AUSAL R is robust 024 to novel language perturbations, and is com-025 petitive with previous works on existing rea-026 soning datasets. Furthermore, the errors made 027 by C AUSAL R are more interpretable due to 028 the multi-modular approach compared to black-029 box generative models. 1 030
A Benchmark for Text Quantification Learning Under Real-World Temporal Distribution Shift

Text quantification is a supervised learning task 001 estimating the relative frequency of each class 002 for a collection of uncategorized text docu-003 ments. Quantification learning has an increas-004 ing number of applications in practice and 005 presents unique challenges that are often over-006 looked in classification problems, such as deal-007 ing with distribution shift. Many studies on 008 quantification use artificially re-sampled test 009 sets to evaluate models under varying target 010 label distributions. Despite being a convenient 011 solution, label-based biased sampling changes 012 the underlying test data distribution and makes 013 it hard to rely on the results to deploy models 014 in practice. This paper introduces a text quan-015 tification benchmark consisting of 8 datasets 016 across sentiment analysis, document categoriza-017 tion, and toxicity classification. We compare 018 popular quantification baselines on the bench-019 mark and show that there is no model consis-020 tently outperforming others. Therefore, we be-021 lieve the benchmark should enable new com-022 munity research to tackle text quantification 023 under temporal distribution shift and develop 024 reliable models in real-world applications. 025
Kinetic Study of the Removal of Methyl Orange Dye by Coupling WO 3 / H 2 O 2 N ’ goran Sévérin Eroi

In the present work, the heterogeneous Fenton-like process was employed to investigate the kinetic models of the degradation of methyl orange (MO) using tungsten oxide/hydrogen peroxide couple. Tungsten oxide particles were successfully synthesized by reux without surfactant and characterized by using XRD, SEM, TEM, and FT-IR techniques. e inuence of parameters such as temperature and concentration of MO was studied and pseudo rst-order and second-order models were applied. WO3/H2O2 showed high eciency in the removal of methyl orange and attained more than 92.8% in 180min. e rst-order kinetic model was described by the removal process with the correlation coecient of R 0.99.
Extended-Spectrum Beta-Lactamases Producing Enterobacteriaceae in the USA Dairy Cattle Farms and Implications for Public Health

Antimicrobial resistance (AMR) is one of the top global health threats of the 21th century. Recent studies are increasingly reporting the rise in extended-spectrum beta-lactamases producing Enterobacteriaceae (ESBLs-Ent) in dairy cattle and humans in the USA. The causes of the increased prevalence of ESBLs-Ent infections in humans and commensal ESBLs-Ent in dairy cattle farms are mostly unknown. However, the extensive use of beta-lactam antibiotics, especially third-generation cephalosporins (3GCs) in dairy farms and human health, can be implicated as a major driver for the rise in ESBLs-Ent. The rise in ESBLs-Ent, particularly ESBLs-Escherichia coli and ESBLs-Klebsiella species in the USA dairy cattle is not only an animal health issue but also a serious public health concern. The ESBLs-E. coli and -Klebsiella spp. can be transmitted to humans through direct contact with carrier animals or indirectly through the food chain or via the environment. The USA Centers for Disease Control and Prevention reports also showed continuous increase in community-associated human infections caused by ESBLs-Ent. Some studies attributed the elevated prevalence of ESBLs-Ent infections in humans to the frequent use of 3GCs in dairy farms. However, the status of ESBLs-Ent in dairy cattle and their contribution to human infections caused by ESBLs-producing enteric bacteria in the USA is the subject of further study. The aims of this review are to give in-depth insights into the status of ESBL-Ent in the USA dairy farms and its implication for public health and to highlight some critical research gaps that need to be addressed.
Heterogeneous Language Model Optimization in Automatic Speech Recognition

The rising data privacy risks make it difﬁ-001 cult for automatic speech recognition (ASR) 002 systems to acquire complete training data in 003 practical application. Recently, the merge 004 paradigm for acoustic model has been pro-005 posed to solve the issue. However, ASR still 006 suffers from another salient issue on language 007 model. Current efforts mainly focus on iso-008 morphic neural network models, while lan-009 guage model optimization is characterized by 010 merging and matching heterogeneous models 011 including n -gram and neural network models. 012 In this paper, we propose a novel Match-and-013 Merge paradigm to ﬁll up the vacuum for the 014 language model optimization. Based on dif-015 ferent training datasets, we train multiple lan-016 guage model pairs. In order to merge them into 017 a target pair with the best performance, we ﬁrst 018 propose a Genetic Match-and-Merge (GMM) 019 method that can be speciﬁcally adopted to op-020 timize heterogeneous models. To improve the 021 algorithm efﬁciency, we further propose a Re-022 inforced Match-and-Merge (RMM) method, 023 which maintains superior recognition accu-024 racy while reducing convergence time. Exten-025 sive experiments demonstrate the effectiveness 026 and generalization of our proposed methods, 027 which signiﬁcantly establishes the new state-028 of-the-art. 029
HOT-DUST-POOR TYPE 1 ACTIVE GALACTIC NUCLEI IN THE COSMOS SURVEY

We report a sizable class of type 1 active galactic nuclei (AGNs) with unusually weak near-infrared (1–3 μm) emission in the XMM-COSMOS type 1 AGN sample. The fraction of these “hot-dust-poor” AGNs increases with redshift from 6% at low redshift (z < 2) to 20% at moderate high redshift (2 < z < 3.5). There is no clear trend of the fraction with other parameters: bolometric luminosity, Eddington ratio, black hole mass, and X-ray luminosity. The 3 μm emission relative to the 1 μm emission is a factor of 2–4 smaller than the typical Elvis et al. AGN spectral energy distribution (SED), which indicates a “torus” covering factor of 2%–29%, a factor of 3–40 smaller than required by unified models. The weak hot dust emission seems to expose an extension of the accretion disk continuum in some of the source SEDs. We estimate the outer edge of their accretion disks to lie at (0.3–2.0) × 104 Schwarzschild radii, ∼10–23 times the gravitational stability radii. Formation scenarios for these sources are discussed.
Knowledge Extraction for Art History: the Case of Vasari's The Lives of The Artists

Knowledge Extraction (KE) techniques are used to convert unstructured information present in texts to Knowledge Graphs (KGs) which can be queried and explored. Despite their potential for cultural heritage domains, such as Art History, these techniques often encounter limitations if applied to domain-specific data. In this paper we present the main challenges that KE has to face on art-historical texts, by using as case study Giorgio Vasari’s The Lives of The Artists . This paper discusses the following NLP tasks for art-historical texts, namely entity recognition and linking, coreference resolution, time extraction, motif extraction and artwork extraction. Several strategies to annotate art-historical data for these tasks and evaluate NLP models are also proposed.
Hardness Masking via Auto-Regressive Language Model

Pre-trained masked language models have 001 achieved tremendous success in natural lan-002 guage processing. Most of these methods rely 003 on recovering randomly masked tokens, which 004 is in general not as good as when tokens are 005 masked based on how well the model can pre-006 dict. However, it is costly for a large-scale 007 model to self-identify tokens that it still strug-008 gles to predict. On the other hand, we observe 009 that a smaller language model can often effec-010 tively find what a large model fails to learn. In-011 spired by this observation, we propose to lever-012 age a compact bi-directional auto-regressive 013 language model to dynamically discover tokens 014 that a large language model has not learned well 015 and guide its training via hardness masking . 016 Comprehensive experiments demonstrate that 017 our masking method can effectively boost the 018 performance of pre-trained language models on 019 general language understanding benchmarks. 020
Reducing Short Circuits in Multiple-Choice Natural Language Reasoning Models with Data Augmentation

Statistical biases in the training data may 001 lead to fragility in neural models that makes 002 choices in multiple-choice natural language 003 reasoning problems without referring to the 004 context or premises. To encourage the models 005 to pay more attention to the relations between 006 the premise and the choices, we propose two 007 biologically inspired operations that can gen-008 erate new training data that “forces” the model 009 to look at the premises and reducing short cir-010 cuits. They can augment any type of multiple 011 choice reasoning dataset, and can be applied to 012 any supervised learning models. Results show 013 that models trained with the augmented data 014 become more robust against both stress test 015 and original test. 016
SumHiS: Extractive Summarization Exploiting Hidden Sctructure

Extractive summarization is a task of high-001 lighting the most important parts of the text. 002 We introduce a new approach to extractive 003 summarization task using hidden clustering 004 structure of the text. Experimental results 005 on CNN/DailyMail demonstrate that our ap-006 proach generates more accurate summaries 007 than both extractive and abstractive methods, 008 achieving state-of-the-art results in terms of 009 ROUGE-2 metric exceeding the previous ap-010 proaches by 10%. Additionally, we show that 011 hidden structure of the text could be inter-012 preted as aspects. 013
Efﬁcient Weighted Deduction Systems for Earley’s Algorithm Anonymous ACL submission

The parsing algorithm of Earley (1970), 001 as presented, has a runtime complexity of 002 O (cid:0) N 3 |G||R| (cid:1) where N is the length of the 003 sentence, |G| is the size of the grammar, and 004 |R| is the number of productions in the gram-005 mar. This is unworkable for the large gram-006 mars that arise in natural language process-007 ing. Fortunately, the dynamic programming 008 algorithm can be improved to run in time 009 O (cid:0) N 3 |G| (cid:1) , matching the complexity of run-010 ning CKY on a binarized version of G . Some 011 of the necessary speed-ups have been pre-012 sented in part or in full in various parts of the 013 literature. However, there has been no uni-014 ﬁed, formal treatment that is written as a de-015 duction system 1 or covers the weighted case. 016 We present such a treatment in terms of ﬁve 017 proof rules that can be used in weighted deduc-018 tion, which reﬁne Earley’s P REDICT , S CAN 019 and C OMPLETE actions. We also provide a 020 generalization of Earley’s algorithm that uses 021 a ﬁnite-state automaton to represent the gram-022 mar, and whose runtime is proportional to the 023 size of the automaton (and the usual O (cid:0) N 3 (cid:1) 024 term), or more precisely the size of the portion 025 of the automaton that is reached while parsing 026 the input sentence. Further speed-ups can then 027 be achieved by minimizing the automaton so 028 that similar productions share transitions. 029
Bidirectional Modeling for Simultaneous Neural Machine Translation

Simultaneous Neural Machine Translation (SimulNMT) generates the output before the entire input sentence is available and only uses the unidirectional attention from left-to-right so that its decoding highly relies on future forecast according to word ordering rules. However, it is utopian that the word order strictly obeys the grammar rules in a language, especially in oral. To address the mismatch be-tween SimulNMT expecting strict word order and free word order in real scenario, we pro-pose a bidirectional modeling. In detail, we train another backward model where the input sentence is from right-to-left and keep the target sentence from left-to-right. Then we join this backward model into the standard forward SimulNMT model during decoding. This strategy enhances the robustness of SimulNMT and empowers the model to be more adaptable for the inconstant word ordering phenomenon. Experiments show that our method brings improvement over the strong baselines.
Emphasis on Easy Samples for Distantly Supervised Relation Extraction

There are many wrongly-labeled samples and 001 low-quality samples in automatically gener-002 ated Distantly Supervised Relation Extraction 003 datasets. Overfitting these samples leads to de-004 cline of generalization. To address this issue, 005 the learning of high-quality samples should be 006 prioritized. In this paper, we propose the Em-007 phasis on Easy Samples (EES) mechanism to 008 emphasize high-quality samples using weight 009 distribution regularization at sentence level and 010 priority weighting at bag level. Experiments 011 on a widely used benchmark show that our ap-012 proach achieves significant improvements. 013
Compositional Data Augmentation for Abstractive Conversation Summarization

Recent abstractive conversation summarization systems generally rely on large-scale datasets with annotated summaries. However, collecting and annotating these conversations can be a time-consuming and labor-intensive task. To address this issue, in this work, we present a sub-structure level compositional data augmentation method, Compo, for generating diverse and high-quality pairs of conversations and summaries. Specifically, Compo first extracts conversation structures like topic splits and action triples as basic units. Then we organize these semantically meaningful conversation snippets compositionally to create new training instances.Additionally, we explore noise-tolerant settings in both self-training and joint-training paradigms to make the most of these augmented samples. Our experiments on benchmark datasets, SAMSum and DialogSum, show that Compo substantially outperforms prior baseline methods by achieving a nearly 10% increase of ROUGE scores with limited data. Code is available at https://github.com/ozyyshr/Compo.
COPD_A_313439 2023..2037

1School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine, Shanghai, People’s Republic of China; 2Department of Rehabilitation, Huashan Hospital, Fudan University, Shanghai, People’s Republic of China; 3Engineering Research Center of Intelligent Rehabilitation of Traditional Chinese Medicine, Ministry of Education, Shanghai University of Traditional Chinese Medicine, Shanghai, People’s Republic of China Introduction: Diaphragm dysfunction is a significant extrapulmonary effect in chronic obstructive pulmonary disease (COPD), which is manifested by changes in diaphragm structure and reduced diaphragm strength. Acupuncture is a traditional rehabilitation technique in China, which has been used in rehabilitation for COPD. But whether acupuncture can improve the diaphragm function of COPD patients remains to be verified. Objective: The objective of this study was to evaluate the rehabilitation effects of acupuncture on diaphragm dysfunction in patients with COPD. Methods: The authors retrieved in CNKI, VIP, SinoMed, PubMed, Ebsco, Web of Science, from inception to November 2020, for relevant randomized control trials. Two researchers independently screened the articles and extracted the data. The quality of the included studies was evaluated by Physiotherapy Evidence Database scale. The primary outcome measures were maximal inspiratory pressure and the scale for accessory respiratory muscle mobilization, the secondary outcome measures were pulmonary function-related indicators and arterial blood gas indicators. Results: Nine articles were finally obtained. Seven studies added acupuncture to standard treatment for patients with diaphragm dysfunction in COPD and found statistically significant changes in the maximum inspiratory pressure and the scale for accessory respiratory muscle mobilization. Two studies have proved that use acupuncture combined with other Traditional Chinese Medicine methods in the rehabilitation for COPD can effectively improve the diaphragm strength and diaphragmatic motor performance. Seven studies showed that acupuncture has obvious improvement in pulmonary ventilation function. Seven studies reported significant differences in arterial blood gas preto post-intervention. Conclusion: This systematic review found that acupuncture can effectively enhance the diaphragm strength, relieve respiratory muscle fatigue, it can also play a promoting role in improving lung function, hypoxia, and carbon dioxide retention, as well as preventing and alleviating respiratory failure. The generalizability of these results is limited by the design of the included studies.
Isospin dependence of the nuclear equation of state near the critical point

We discuss experimental evidence for a nuclear phase transition driven by the different concentrations of neutrons to protons. Different ratios of the neutron to proton concentrations lead to different critical points for the phase transition. This is analogous to the phase transitions occurring in He-4-He-3 liquid mixtures. We present experimental results that reveal the N/A (or Z/A) dependence of the phase transition and discuss possible implications of these observations in terms of the Landau free energy description of critical phenomena.
C60 AS A PROBE FOR ASTROPHYSICAL ENVIRONMENTS

The C60 molecule has been recently detected in a wide range of astrophysical environments through its four active intramolecular vibrational modes (T1u) near 18.9, 17.4, 8.5, and 7.0 μm. The strengths of the mid-infrared emission bands have been used to infer astrophysical conditions in the fullerene-rich regions. Widely varying values of the relative intrinsic strengths (RIS) of these four bands are reported in laboratory and theoretical papers, which impedes the derivation of the excitation mechanism of C60 in the astrophysical sources. The spectroscopic analysis of the C60 samples produced with our method delivers highly reproducible RIS values of 100, 25 ± 1, 26 ± 1 and 40 ± 4. A comparison of the inferred C60 emission band strengths with the astrophysical data shows that the observed strengths cannot be explained in terms of fluorescent or thermal emission alone. The large range in the observed 17.4 μm/18.9 μm emission ratios indicates that either the emission bands contain significant contributions from emitters other than C60, or that the population distribution among the C60 vibrational modes is affected by physical processes other than thermal or UV excitation, such as chemo-luminescence from nascent C60 or possibly Poincaré fluorescence resulting from an inverse internal energy conversion. We have carefully analyzed the effect of the weakly active fundamental modes and second order modes in the mid-infrared spectrum of C60, and propose that neutral C60 is the carrier of the unidentified emission band at 6.49 μm which has been observed in fullerene-rich environments.
Gravitational wave emission from binary supermassive black holes

Massive black hole binaries (MBHBs) are unavoidable outcomes of the hierarchical structure formation process, and, according to the theory of general relativity, are expected to be the loudest gravitational wave (GW) sources in the Universe. In this paper I provide a broad overview of MBHBs as GW sources. After reviewing the basics of GW emission from binary systems and of MBHB formation, evolution and dynamics, I describe in some details the connection between binary properties and the emitted gravitational waveform. Direct GW observations will provide an unprecedented wealth of information about the physical nature and the astrophysical properties of these extreme objects, allowing to reconstruct their cosmic history, dynamics and coupling with their dense stellar and gaseous environment. In this context I describe ongoing and future efforts to make a direct detection with space based interferometry and pulsar timing arrays, highlighting the invaluable scientific payouts of such enterprises.