Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning to Reason for Text Generation from Scientific Tables

In this paper, we introduce SciGen, a new challenge dataset for the task of reasoning-aware data-to-text generation consisting of tables from scientific articles and their corresponding descriptions. Describing scientific tables goes beyond the surface realization of the table content and requires reasoning over table values. The unique properties of SciGen are that (1) tables mostly contain numerical values, and (2) the corresponding descriptions require arithmetic reasoning. SciGen is therefore the first dataset that assesses the arithmetic reasoning capabilities of generation models on complex input structures, i.e., tables from scientific articles. We study the effectiveness of state-of-the-art data-to-text generation models on SciGen and evaluate the results using common metrics as well as human evaluation. Our results and analyses show that (a) while humans like to reason for describing scientific tables, the ability of state-of-the-art models is severely limited on this task, (b) while adding more training data improves the results, it is not the solution for reasoning-aware text generation, and (c) one of the main bottlenecks for this task is the lack of proper automatic evaluation metrics. The data, code, and annotations for human evaluation will be available at https://github.com/UKPLab/SciGen. SciGen opens new avenues for future research in reasoning-aware text generation and evaluation.
Emergence of Lie Symmetries in Functional Architectures Learned by CNNs

In this paper we study the spontaneous development of symmetries in the early layers of a Convolutional Neural Network (CNN) during learning on natural images. Our architecture is built in such a way to mimic some properties of the early stages of biological visual systems. In particular, it contains a pre-filtering step ℓ0 defined in analogy with the Lateral Geniculate Nucleus (LGN). Moreover, the first convolutional layer is equipped with lateral connections defined as a propagation driven by a learned connectivity kernel, in analogy with the horizontal connectivity of the primary visual cortex (V1). We first show that the ℓ0 filter evolves during the training to reach a radially symmetric pattern well approximated by a Laplacian of Gaussian (LoG), which is a well-known model of the receptive profiles of LGN cells. In line with previous works on CNNs, the learned convolutional filters in the first layer can be approximated by Gabor functions, in agreement with well-established models for the receptive profiles of V1 simple cells. Here, we focus on the geometric properties of the learned lateral connectivity kernel of this layer, showing the emergence of orientation selectivity w.r. t. the tuning of the learned filters. We also examine the short-range connectivity and association fields induced by this connectivity kernel, and show qualitative and quantitative comparisons with known group-based models of V1 horizontal connections. These geometric properties arise spontaneously during the training of the CNN architecture, analogously to the emergence of symmetries in visual systems thanks to brain plasticity driven by external stimuli.
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation

Large pretrained generative models like GPT-3 often suffer from hallucinating non-existent or incorrect content, which undermines their potential merits in real applications. Existing work usually attempts to detect these hallucinations based on a corresponding oracle reference at a sentence or document level. However ground-truth references may not be readily available for many free-form text generation applications, and sentence- or document-level detection may fail to provide the fine-grained signals that would prevent fallacious content in real time. As a first step to addressing these issues, we propose a novel token-level, reference-free hallucination detection task and an associated annotated dataset named HaDeS (HAllucination DEtection dataSet). To create this dataset, we first perturb a large number of text segments extracted from English language Wikipedia, and then verify these with crowd-sourced annotations. To mitigate label imbalance during annotation, we utilize an iterative model-in-loop strategy. We conduct comprehensive data analyses and create multiple baseline models.
Case-based Reasoning for Natural Language Queries over Knowledge Bases

It is often challenging to solve a complex problem from scratch, but much easier if we can access other similar problems with their solutions — a paradigm known as case-based reasoning (CBR). We propose a neuro-symbolic CBR approach (CBR-KBQA) for question answering over large knowledge bases. CBR-KBQA consists of a nonparametric memory that stores cases (question and logical forms) and a parametric model that can generate a logical form for a new question by retrieving cases that are relevant to it. On several KBQA datasets that contain complex questions, CBR-KBQA achieves competitive performance. For example, on the CWQ dataset, CBR-KBQA outperforms the current state of the art by 11% on accuracy. Furthermore, we show that CBR-KBQA is capable of using new cases without any further training: by incorporating a few human-labeled examples in the case memory, CBR-KBQA is able to successfully generate logical forms containing unseen KB entities as well as relations.
Fusing Information Streams in End-to-End Audio-Visual Speech Recognition

End-to-end acoustic speech recognition has quickly gained widespread popularity and shows promising results in many studies. Specifically the joint transformer/CTC model pro-vides very good performance in many tasks. However, under noisy and distorted conditions, the performance still degrades notably. While audio-visual speech recognition can significantly improve the recognition rate of end-to-end models in such poor conditions, it is not obvious how to best utilize any available information on acoustic and visual signal quality and reliability in these models. We thus consider the question of how to optimally inform the transformer/CTC model of any time-variant reliability of the acoustic and visual in-formation streams. We propose a new fusion strategy, incorporating reliability information in a decision fusion net that considers the temporal effects of the attention mechanism. This approach yields significant improvements compared to a state-of-the-art baseline model on the Lip Reading Sentences 2 and 3 (LRS2 and LRS3) corpus. On average, the new sys-tem achieves a relative word error rate reduction of 43% com-pared to the audio-only setup and 31% compared to the audio-visual end-to-end baseline.
Emotion Classification in a Resource Constrained Language Using Transformer-based Approach

Although research on emotion classification has significantly progressed in high-resource languages, it is still infancy for resource-constrained languages like Bengali. However, unavailability of necessary language processing tools and deficiency of benchmark corpora makes the emotion classification task in Bengali more challenging and complicated. This work proposes a transformer-based technique to classify the Bengali text into one of the six basic emotions: anger, fear, disgust, sadness, joy, and surprise. A Bengali emotion corpus consists of 6243 texts is developed for the classification task. Experimentation carried out using various machine learning (LR, RF, MNB, SVM), deep neural networks (CNN, BiLSTM, CNN+BiLSTM) and transformer (Bangla-BERT, m-BERT, XLM-R) based approaches. Experimental outcomes indicate that XLM-R outdoes all other techniques by achieving the highest weighted f_1-score of 69.73% on the test data.
The Topic Confusion Task: A Novel Scenario for Authorship Attribution

Authorship attribution is the problem of identifying the most plausible author of an anonymous text from a set of candidate authors. Researchers have investigated same-topic and cross-topic scenarios of authorship attribution, which differ according to whether unseen topics are used in the testing phase. However, neither scenario allows us to explain whether errors are caused by failure to capture authorship style, by the topic shift or by other factors. Motivated by this, we propose the topic confusion task, where we switch the author-topic conﬁg-uration between training and testing set. This setup allows us to probe errors in the attribution process. We investigate the accuracy and two error measures: one caused by the models’ confusion by the switch because the features capture the topics, and one caused by the features’ inability to capture the writing styles, leading to weaker models. By evaluating different features, we show that stylometric features with part-of-speech tags are less susceptible to topic variations and can increase the accuracy of the attribution process. We further show that combining them with word-level n - grams can outperform the state-of-the-art technique in the cross-topic scenario. Finally, we show that pretrained language models such as BERT and RoBERTa perform poorly on this task, and are outperformed by simple n -gram features.
Carbonate Complexation of Neptunium(IV) and Analogous Complexation of Ground-Water Uranium

The solubility of Np(IV) in a carbonate solution in the pH range of 8 —12.5 was studied. The total carbonate concentration was varied between 5 x l 0 ~ 4 — 5 x l 0 ~ 3 M. The result was interpreted by considering two kinds of reactions: a. below pH 10 where bicarbonate ion was the dominant species, Np(OH)4(am) + 2HCO3 = Np(0H) 2 (C0 3 ) i" + 2H 20, ^_JQ-0.35±0.33 or, Np(OH)4(am) + 2HCO3= N p ( 0 H ) 4 ( H C 0 3 ) r , jq-0.35±033 b. above pH 10 where carbonate ion was considered predominant, Np(OH)4(am) + 2CO3 = N p ( 0 H ) 4 ( C 0 3 ) r K = X Q 1 . 4 3 + 0.33 The solubility of uranium in Swedish ground-water was analyzed by assuming the presence of a typical complex of U(0H)2(C03)2~, and a good agreement was achieved with the data observed. The recommended stability constant of U(0H)2(C03)2~ was estimated to be >10 4 1 · 7 0 . This suggests the importance of such complex species in the natural environment.
Contrastive Out-of-Distribution Detection for Pretrained Transformers

Pretrained Transformers achieve remarkable performance when training and test data are from the same distribution. However, in real-world scenarios, the model often faces out-of-distribution (OOD) instances that can cause severe semantic shift problems at inference time. Therefore, in practice, a reliable model should identify such instances, and then either reject them during inference or pass them over to models that handle another distribution. In this paper, we develop an unsupervised OOD detection method, in which only the in-distribution (ID) data are used in training. We propose to fine-tune the Transformers with a contrastive loss, which improves the compactness of representations, such that OOD instances can be better differentiated from ID ones. These OOD instances can then be accurately detected using the Mahalanobis distance in the model’s penultimate layer. We experiment with comprehensive settings and achieve near-perfect OOD detection performance, outperforming baselines drastically. We further investigate the rationales behind the improvement, finding that more compact representations through margin-based contrastive learning bring the improvement. We release our code to the community for future research.
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP

Humans can learn a new language task efficiently with only few examples, by leveraging their knowledge obtained when learning prior tasks. In this paper, we explore whether and how such cross-task generalization ability can be acquired, and further applied to build better few-shot learners across diverse NLP tasks. We introduce CrossFit, a problem setup for studying cross-task generalization ability, which standardizes seen/unseen task partitions, data access during different learning stages, and the evaluation protocols. To instantiate different seen/unseen task partitions in CrossFit and facilitate in-depth analysis, we present the NLP Few-shot Gym, a repository of 160 diverse few-shot NLP tasks created from open-access NLP datasets and converted to a unified text-to-text format. Our analysis reveals that the few-shot learning ability on unseen tasks can be improved via an upstream learning stage using a set of seen tasks. We also observe that the selection of upstream learning tasks can significantly influence few-shot performance on unseen tasks, asking further analysis on task similarity and transferability.
Locate Who You Are: Matching Geo-location to Text for Anchor Link Prediction

Nowadays, users are encouraged to activate across multiple on-line social networks simultaneously. Anchor link prediction, which aims to reveal the correspondence among different accounts of the same user across networks, has been regarded as a fundamental problem for user profiling, marketing, cybersecurity, and recommendation. Existing methods mainly address the prediction problem by utilizing profile, content, or structural features of users in symmetric ways. However, encouraged by online services, users would also post asymmetric information across networks, such as geo-locations and texts. It leads to an emerged challenge in aligning users with asymmetric information across networks. Instead of similarity evaluation applied in previous works, we formalize correlation between geo-locations and texts and propose a novel anchor link prediction framework for matching users across networks. Moreover, our model can alleviate the label scarcity problem by introducing external data. Experimental results on real-world datasets show that our approach outperforms existing methods and achieves state-of-the-art results.
SUPERDENSITY OF NON-INTEGRABLE BILLIARDS

Consider any rectangular polygon region tiled with unit squares, with possibly some extra walls inside the polygon. We prove that a half-infinite billiard orbit in such a region is superdense, a best possible form of time-quantitative density, if and only if the initial slope of the orbit is a badly approximable number.
Fifty Three Matrix Factorizations: A systematic approach

The success of matrix factorizations such as the singular value decomposition (SVD) has motivated the search for even more factorizations. We catalog 53 matrix factorizations, most of which we believe to be new. Our systematic approach, inspired by the generalized Cartan decomposition of Lie theory, also encompasses known factorizations such as the SVD, the symmetric eigendecomposition, the CS decomposition, the hyperbolic SVD, structured SVDs, the Takagi factorization, and others thereby covering familiar matrix factorizations as well as ones that were waiting to be discovered. We suggest that Lie theory has one way or another been lurking hidden in the foundations of the very successful field of matrix computations with applications routinely used in so many areas of computation. In this paper, we investigate consequences of the Cartan decomposition and the little known generalized Cartan decomposition for matrix factorizations. We believe that these factorizations once properly identified can lead to further work on algorithmic computations and applications.
Optimal Per-Antenna ADC Bit Allocation in Correlated and Cell-Free Massive MIMO

In Massive MIMO base stations (BSs), the hardware design needs to balance high spectral efficiency (SE) with low complexity. The level of hardware impairments (HWIs) indicates how strong the signal distortion introduced by hardware imperfections is. In particular, the analog-to-digital converters (ADCs) have an important impact on signal distortion and power consumption. This article addresses the fundamental problem of selecting the optimal hardware quality in the Massive MIMO space. In particular, we examine the optimal HWI and ADC bit allocation per BS antenna to maximize the SE. The results show that in co-located arrays with low channel gain variations across antennas, equal ADC bit allocation is optimal. In contrast, cell-free Massive MIMO systems benefit the most from optimizing the ADC bit allocation achieving improvements in the order of 2 [bit-per-channel-use] per user equipment when using regularized zero-forcing (RZF). In addition, when including the impact of power consumption in cell-free Massive MIMO with RZF, allocating low values of mixed ADC bit resolutions across the BS antennas can increase the energy efficiency up to 30% compared to equal ADC bit allocation.
Knowledge Neurons in Pretrained Transformers

Large-scale pretrained language models are surprisingly good at recalling factual knowledge presented in the training corpus. In this paper, we present preliminary studies on how factual knowledge is stored in pretrained Transformers by introducing the concept of knowledge neurons. Specifically, we examine the fill-in-the-blank cloze task for BERT. Given a relational fact, we propose a knowledge attribution method to identify the neurons that express the fact. We find that the activation of such knowledge neurons is positively correlated to the expression of their corresponding facts. In our case studies, we attempt to leverage knowledge neurons to edit (such as update, and erase) specific factual knowledge without fine-tuning. Our results shed light on understanding the storage of knowledge within pretrained Transformers.
Temperature dependence of the viscosity of In-Sb alloy melts

The temperature dependence of the dynamic viscosity of In-Sb alloy melts has been investigated using a torsional oscillation viscometer. The results show that no anomalous change in viscosity occurs in In-1 wt. %Sb melt and an anomalous change occurs at about 800°C both in In-55 wt. %Sb and In-69.5 wt. %Sb alloy melts. A low temperature area (LTA) and a high temperature area (HTA) can be used to describe the viscosity behaviour of In-55 wt. %Sb and In-69.5 wt. %Sb alloy melts. The activation energy of In-Sb alloy melts has been obtained. Variation of activation energy with temperature of In-Sb melts has been observed. The structural variation of In-Sb alloy melts has also been discussed on the basis of the change in viscosity and DSC analysis.
Linguistic Dependencies and Statistical Dependence

Are pairs of words that tend to occur together also likely to stand in a linguistic dependency? This empirical question is motivated by a long history of literature in cognitive science, psycholinguistics, and NLP. In this work we contribute an extensive analysis of the relationship between linguistic dependencies and statistical dependence between words. Improving on previous work, we introduce the use of large pretrained language models to compute contextualized estimates of the pointwise mutual information between words (CPMI). For multiple models and languages, we extract dependency trees which maximize CPMI, and compare to gold standard linguistic dependencies. Overall, we find that CPMI dependencies achieve an unlabelled undirected attachment score of at most \approx 0.5. While far above chance, and consistently above a non-contextualized PMI baseline, this score is generally comparable to a simple baseline formed by connecting adjacent words. We analyze which kinds of linguistic dependencies are best captured in CPMI dependencies, and also find marked differences between the estimates of the large pretrained language models, illustrating how their different training schemes affect the type of dependencies they capture.
Structural Change of Wheat Starch Granule by Ball-mill Treatment

Wheat starch granules were perfectly damaged but not cracked to pieces with rolling type ball-mill treatment for a long period (320 h), however, the granule surface changed from smooth to rough. The properties (amylase susceptivity and water absorption activity) of treated starch changed distinctly after 10 h of treatment. DSC results and X-ray diffraction patterns of treated samples also showed the same tendency. It was concluded that ball-mill treatment gelatizes starch granules over 10 h without water and heat. GPCs of a starch sample and a debranched sample showed that amylopectin was degraded and the cluster size fragment possibly liberated. Long-term treatment makes the molecular size of both components of starch small. A HPAEC analysis showed the liberation of very short chains with long-term treatment. A '3C solid-state-NMR spectra of starch samples suggests that disruption occurs at the glycosidic linkage during treatment.
Stein’s method of normal approximation: Some recollections and reflections

This paper is a short exposition of Stein's method of normal approximation from my personal perspective. It focuses mainly on the characterization of the normal distribution and the construction of Stein identities. Through examples, it provides glimpses into the many approaches to constructing Stein identities and the diverse applications of Stein's method to mathematical problems. It also includes anecdotes of historical interest, including how Stein discovered his method and how I found an unpublished proof of his of the Berry-Esseen theorem.
Detecting cold pools from soundings during EUREC4A

<p>We develop a novel method to detect cold pools from atmospheric soundings over tropical oceans and apply it to sounding data from EUREC<sup>4</sup>A. The proposed method exploits the fact that the air in a cold pool is denser than the air above it. It leads us to define cold pool soundings as those for which the mixed-layer height is smaller than 400 m. We first test this criterion by verifying its consistency with surface temperature and precipitation in a realistic high-resolution simulation over the western tropical Atlantic. Applying to EUREC<sup>4</sup>A data, we then identify 7 % of EUREC<sup>4</sup>A dropsondes and radiosondes as cold pool soundings. In two selected case studies, we find that cold pool soundings coincide with mesoscale cloud arcs and temperature drops in the surface time series. Statistics for the entire campaign further characterize the signature of cold pools in temperature, humidity and wind profiles. In the presence of wind shear, we show in particular that the spreading of cold pools is favored downshear, suggesting downward momentum transport by unsaturated downdrafts. These results support the robustness of our simple method in different environmental conditions and illustrate the new insights it offers for the characterization of cold pools and their environment. &#160;</p>
Pressures for multi-potentials in semigroup dynamics

On compact metric spaces X we introduce and study the amalgamated pressure, condensed pressure, trajectory pressure, exhaustive pressure, and their capacities on C(X,Rm), for non-compact sets Y ⊂ X , with respect to finitely generated semigroups G acting on X . Properties and comparisons between these pressures, and relations with inverse limits are given. Next we introduce a measuretheoretic amalgamated entropy, and prove a Partial Variational Principle for amalgamated pressure. Local amalgamated and local exhaustive entropies are studied for measures on X , and they provide estimates for amalgamated and exhaustive topological entropies. For marginal ergodic measures ν on X , we find an upper bound for the local exhaustive entropy. These notions can be used for classification of semigroup actions. We apply the amalgamated pressure of unstable multipotentials, also to estimate the dimensions of slices in G-invariant saddle-type sets. MSC 2020: 37A35, 37A05, 37D35, 37H12, 46E27, 46G10, 46G12.