Miracle-dz/UniFAR-FTUs huggingface dataset:
Improving Weak PINNs for Hyperbolic Conservation Laws: Dual Norm Computation, Boundary Conditions and Systems

We consider the approximation of entropy solutions of nonlinear hyperbolic conservation laws using neural networks. We provide explicit computations that highlight why classical PINNs will not work for discontinuous solutions to nonlinear hyperbolic conservation laws and show that weak (dual) norms of the PDE residual should be used in the loss functional. This approach has been termed"weak PINNs"recently. We suggest some modifications to weak PINNs that make their training easier, which leads to smaller errors with less training, as shown by numerical experiments. Additionally, we extend wPINNs to scalar conservation laws with weak boundary data and to systems of hyperbolic conservation laws. We perform numerical experiments in order to assess the accuracy and efficiency of the extended method.
Modelling the Dynamic Dependence Structure in Multivariate Financial Time Series

Abstract. The dependence structure in multivariate financial time series is of great importance in portfolio management. By studying daily return histories of 17 exchange‐traded index funds, we identify important features of the data, and we propose two new models to capture these features. The first is an extension of the multivariate BEKK (Baba, Engle, Kraft, Kroner) model, which includes a multivariate t‐type error distribution with different degrees of freedom. We demonstrate that this error distribution is able to accommodate different levels of heavy‐tailed behaviour and thus provides a better fit than models based on a multivariate t‐with a common degree of freedom. The second model is copula based, and can be regarded as an extension of the standard and the generalized dynamic conditional correlation model [Engle, Journal of Business and Economics Statistics (2002) Vol. 17, 425–446; Cappiello et al. (2003) Working paper, UCSD] to a Student copula. Model comparison is carried out using criteria including the Akaike information criteria and Bayesian information criteria. We also evaluate the two models from an asset‐allocation perspective using a three‐asset portfolio as an example, constructing optimal portfolios based on the Markowitz theory. Our results indicate that, for our data, the proposed models both outperform the standard BEKK model, with the copula model performing better than the extension of the BEKK model.
Universal Dynamics of Damped-Driven Systems: The Logistic Map as a Normal Form for Energy Balance

Damped-driven systems are ubiquitous in engineering and science. Despite the diversity of physical processes observed in a broad range of applications, the underlying instabilities observed in practice have a universal characterization which is determined by the overall gain and loss curves of a given system. The universal behavior of damped-driven systems can be understood from a geometrical description of the energy balance with a minimal number of assumptions. The assumptions on the energy dynamics are as follows: the energy increases monotonically as a function of increasing gain, and the losses become increasingly larger with increasing energy, i.e. there are many routes for dissipation in the system for large input energy. The intersection of the gain and loss curves deﬁne an energy balanced solution. By constructing an iterative map between the loss and gain curves, the dynamics can be shown to be homeomorphic to the logistic map, which exhibits a period doubling cascade to chaos. Indeed, the loss and gain curves allow for a geometrical description of the dynamics through a simple Verhulst diagram (cobweb plot). Thus irrespective of the physics and its complexities, this simple geometrical description dictates the universal set of logistic map instabilities that arise in complex damped-driven systems. More broadly, damped-driven systems are a class of non-equilibrium pattern forming systems which have a canonical set of instabilities that are manifest in practice.
Robust High-dimensional Tuning Free Multiple Testing

A stylized feature of high-dimensional data is that many variables have heavy tails, and robust statistical inference is critical for valid large-scale statistical inference. Yet, the existing developments such as Winsorization, Huberization and median of means require the bounded second moments and involve variable-dependent tuning parameters, which hamper their fidelity in applications to large-scale problems. To liberate these constraints, this paper revisits the celebrated Hodges-Lehmann (HL) estimator for estimating location parameters in both the one- and two-sample problems, from a non-asymptotic perspective. Our study develops Berry-Esseen inequality and Cram\'{e}r type moderate deviation for the HL estimator based on newly developed non-asymptotic Bahadur representation, and builds data-driven confidence intervals via a weighted bootstrap approach. These results allow us to extend the HL estimator to large-scale studies and propose \emph{tuning-free} and \emph{moment-free} high-dimensional inference procedures for testing global null and for large-scale multiple testing with false discovery proportion control. It is convincingly shown that the resulting tuning-free and moment-free methods control false discovery proportion at a prescribed level. The simulation studies lend further support to our developed theory.
T EMPORAL D ISENTANGLEMENT OF R EPRESENTATIONS FOR I MPROVED G ENERALISATION IN R EINFORCEMENT L EARNING

Reinforcement Learning (RL) agents are often unable to generalise well to environment variations in the state space that were not observed during training. This issue is especially problematic for image-based RL, where a change in just one variable, such as the background colour, can change many pixels in the image, which can lead to drastic changes in the agent’s latent representation of the image, causing the learned policy to fail. To learn more robust representations, we introduce TEmporal Disentanglement (TED), a self-supervised auxiliary task that leads to disentangled image representations exploiting the sequential nature of RL observations. We find empirically that RL algorithms utilising TED as an auxiliary task adapt more quickly to changes in environment variables with continued training compared to state-of-the-art representation learning methods. Since TED enforces a disentangled structure of the representation, we also find that policies trained with TED generalise better to unseen values of variables irrelevant to the task (e.g. background colour) as well as unseen values of variables that affect the optimal policy (e.g. goal positions).
High-resolution urban land-cover classification using a competitive multi-scale object-based approach

In this study, a two-step classification procedure was used for classifying urban land cover. First, a hierarchy of seven image segmentations of different scales was created for an urban scene, and preliminary classifications were performed for each of the segmentations using a classification algorithm that provides the probability that a segment belongs to a land-cover class in addition to the class assignment. A higher probability for the assigned class indicates that a segment is more likely to have been classified correctly. The class assignment and probability for image segments in each of the six coarser segmentations were added to the segments that they contained in the finest scale segmentation to allow for a comparison across all of the scales. For the final classification, the finest scale segments were assigned to the class that gave the highest probability in any of the segmentations. The incorporation of preliminary class assignments and probabilities across multiple scales led to an increase in overall accuracy from 78.1% to 82.1%.
Symmetries in transmission electron microscopy imaging of crystals with strain

Transmission electron microscopy (TEM) images of strained crystals often exhibit symmetries, the source of which is not always clear. To understand these symmetries, we distinguish between symmetries that occur from the imaging process itself and symmetries of the inclusion that might affect the image. For the imaging process, we prove mathematically that the intensities are invariant under specific transformations. A combination of these invariances with specific properties of the strain profile can then explain symmetries observed in TEM images. We demonstrate our approach to the study of symmetries in TEM images using selected examples in the field of semiconductor nanostructures such as quantum wells and quantum dots.
“Good morning, COVID!” the inertia of journalistic imaginaries in morning shows’ online comments

This textual analysis examines meanings of user comments to Facebook posts of three UK breakfast programs as COVID-19 entered England in 2020. This analysis suggests that during this time of crisis and uncertainty, users’ – even if trolling, interacting through incidental media use, or commenting as regular contributors to the pages – relied on traditional and lasting interpretations of conventional journalistic standards in their discourse surrounding “soft news” content. We argue that such comments represent an “inertia” of longstanding journalistic imaginaries that have survived an increased hybridity of news.
Mental disorders and their impact on school performance and high school completion by gender in Australia: A matched population-based cohort study

Background: Young people with a mental disorder often perform poorly at school and can fail to complete high school. This study aims to compare scholastic performance and high school completion of young people hospitalised with a mental disorder compared to young people not hospitalised for a mental disorder health condition by gender. Method: A population-based matched case-comparison cohort study of young people aged ⩽18 years hospitalised for a mental disorder during 2005–2018 in New South Wales, Australia using linked birth, health, education and mortality records. The comparison cohort was matched on age, gender and residential postcode. Generalised linear mixed modelling examined risk of school performance below the national minimum standard and generalised linear regression examined risk of not completing high school for young people with a mental disorder compared to matched peers. Results: Young males with a mental disorder had over a 1.7 times higher risk of not achieving the national minimum standard for numeracy (adjusted relative risk: 1.71; 95% confidence interval: [1.35, 2.15]) and reading (adjusted relative risk: 1.99; 95% confidence interval: [1.80, 2.20]) compared to matched peers. Young females with a mental disorder had around 1.5 times higher risk of not achieving the national minimum standard for numeracy (adjusted relative risk: 1.50; 95% confidence interval: [1.14, 1.96]) compared to matched peers. Both young males and females with a disorder had around a three times higher risk of not completing high school compared to peers. Young males with multiple disorders had up to a sixfold increased risk and young females with multiple disorders had up to an eightfold increased risk of not completing high school compared to peers. Conclusion: Early recognition and support could improve school performance and educational outcomes for young people who were hospitalised with a mental disorder. This support should be provided in conjunction with access to mental health services and school involvement and assistance.
Fractality of sentiment arcs for literary quality assessment: The case of Nobel laureates

In the few works that have used NLP to study literary quality, sentiment and emotion analysis have often been considered valuable sources of information. At the same time, the idea that the nature and polarity of the sentiments expressed by a novel might have something to do with its perceived quality seems limited at best. In this paper, we argue that the fractality of narratives, specifically the long-term memory of their sentiment arcs, rather than their simple shape or average valence, might play an important role in the perception of literary quality by a human audience. In particular, we argue that such measure can help distinguish Nobel-winning writers from control groups in a recent corpus of English language novels. To test this hypothesis, we present the results from two studies: (i) a probability distribution test, where we compute the probability of seeing a title from a Nobel laureate at different levels of arc fractality; (ii) a classification test, where we use several machine learning algorithms to measure the predictive power of both sentiment arcs and their fractality measure. Our findings seem to indicate that despite the competitive and complex nature of the task, the populations of Nobel and non-Nobel laureates seem to behave differently and can to some extent be told apart by a classifier.
ParsSimpleQA: The Persian Simple Question Answering Dataset and System over Knowledge Graph

The simple question answering over the knowledge graph concerns answering single-relation questions by querying the facts in the knowledge graph. This task has drawn significant attention in recent years. However, there is a demand for a simple question dataset in the Persian language to study open-domain simple question answering. In this paper, we present the first Persian single-relation question answering dataset and a model that uses a knowledge graph as a source of knowledge to answer questions. We create the ParsSimpleQA dataset semi-automatically in two steps. First, we build single-relation question templates. Next, we automatically create simple questions and answers using templates, entities, and relations from Farsbase. To present the reliability of the presented dataset, we proposed a simple question-answering system that receives questions and uses deep learning and information retrieval techniques for answering questions. The experimental results presented in this paper show that the ParsSimpleQA dataset is very promising for the Persian simple question-answering task.
Dual-Encoder Transformers with Cross-modal Alignment for Multimodal Aspect-based Sentiment Analysis

Multimodal aspect-based sentiment analysis (MABSA) aims to extract the aspect terms from text and image pairs, and then analyze their corresponding sentiment. Recent studies typically use either a pipeline method or a unified transformer based on a cross-attention mechanism. However, these methods fail to explicitly and effectively incorporate the alignment between text and image. Supervised finetuning of the universal transformers for MABSA still requires a certain number of aligned image-text pairs. This study proposes a dual-encoder transformer with cross-modal alignment (DTCA). Two auxiliary tasks, including text-only extraction and text-patch alignment are introduced to enhance cross-attention performance. To align text and image, we propose an unsupervised approach which minimizes the Wasserstein distance between both modalities, forcing both encoders to produce more appropriate representations for the final extraction. Experimental results on two benchmarks demonstrate that DTCA consistently outperforms existing methods.
Neural Readability Pairwise Ranking for Sentences in Italian Administrative Language

Automatic Readability Assessment aims at assigning a complexity level to a given text, which could help improve the accessibility to information in specific domains, such as the administrative one. In this paper, we investigate the behavior of a Neural Pairwise Ranking Model (NPRM) for sentence-level readability assessment of Italian administrative texts. To deal with data scarcity, we experiment with cross-lingual, cross- and in-domain approaches, and test our models on Admin-It, a new parallel corpus in the Italian administrative language, containing sentences simplified using three different rewriting strategies. We show that NPRMs are effective in zero-shot scenarios (~0.78 ranking accuracy), especially with ranking pairs containing simplifications produced by overall rewriting at the sentence-level, and that the best results are obtained by adding in-domain data (achieving perfect performance for such sentence pairs). Finally, we investigate where NPRMs failed, showing that the characteristics of the training data, rather than its size, have a bigger effect on a model’s performance.
TaxFree: a Visualization Tool for Candidate-free Taxonomy Enrichment

Taxonomies are widely used in a various number of downstream NLP tasks and, therefore, should be kept up-to-date. In this paper, we present TaxFree, an open source system for taxonomy visualisation and automatic Taxonomy Enrichment without pre-defined candidates on the example of WordNet-3.0. As oppose to the traditional task formulation (where the list of new words is provided beforehand), we provide an approach for automatic extension of a taxonomy using a large pre-trained language model. As an advantage to the existing visualisation tools of WordNet, TaxFree also integrates graphic representations of synsets from ImageNet. Such visualisation tool can be used for both updating taxonomies and inspecting them for the required modifications.
Dynamic Context Extraction for Citation Classification

We investigate the effect of varying citation context window sizes on model performance in citation intent classification. Prior studies have been limited to the application of fixed-size contiguous citation contexts or the use of manually curated citation contexts. We introduce a new automated unsupervised approach for the selection of a dynamic-size and potentially non-contiguous citation context, which utilises the transformer-based document representations and embedding similarities. Our experiments show that the addition of non-contiguous citing sentences improves performance beyond previous results. Evalu- ating on the (1) domain-specific (ACL-ARC) and (2) the multi-disciplinary (SDP-ACT) dataset demonstrates that the inclusion of additional context beyond the citing sentence significantly improves the citation classifi- cation model’s performance, irrespective of the dataset’s domain. We release the datasets and the source code used for the experiments at: https://github.com/oacore/dynamic_citation_context
VISCOSE - a Kanji Dictionary Enriched with VISual, COmpositional, and SEmantic Information

In this paper, we present a novel approach for building kanji dictionaries by enriching the lexical data of 3,500 kanji with images, structural decompositions, and semantically based cross-media mappings from the textual to the visual dimension. Our kanji dictionary is part of a Web-based contextual language learning environment based on augmented browsing technology. We display our multimodal kanji information as kanji cards in the Web browser, offering a versatile representation that can be integrated into other advanced creative language learning applications, such as memorization puzzles, creative storytelling assignments, or educational games.
Cross-Lingual Open-Domain Question Answering with Answer Sentence Generation

Open-Domain Generative Question Answering has achieved impressive performance in English by combining document-level retrieval with answer generation. These approaches, which we refer to as GenQA, can generate complete sentences, effectively answering both factoid and non-factoid questions. In this paper, we extend to the multilingual and cross-lingual settings. For this purpose, we first introduce GenTyDiQA, an extension of the TyDiQA dataset with well-formed and complete answers for Arabic, Bengali, English, Japanese, and Russian. Based on GenTyDiQA, we design a cross-lingual generative model that produces full-sentence answers by exploiting passages written in multiple languages, including languages different from the question. Our cross-lingual generative system outperforms answer sentence selection baselines for all 5 languages and monolingual generative pipelines for three out of five languages studied.
Towards Unified Representations of Knowledge Graph and Expert Rules for Machine Learning and Reasoning

With a knowledge graph and a set of if-then rules, can we reason about the conclusions given a set of observations? In this work, we formalize this question as the cognitive inference problem, and introduce the Cognitive Knowledge Graph (CogKG) that unifies two representations of heterogeneous symbolic knowledge: expert rules and relational facts. We propose a general framework in which the unified knowledge representations can perform both learning and reasoning. Specifically, we implement the above framework in two settings, depending on the availability of labeled data. When no labeled data are available for training, the framework can directly utilize symbolic knowledge as the decision basis and perform reasoning. When labeled data become available, the framework casts symbolic knowledge as a trainable neural architecture and optimizes the connection weights among neurons through gradient descent. Empirical study on two clinical diagnosis benchmarks demonstrates the superiority of the proposed method over time-tested knowledge-driven and data-driven methods, showing the great potential of the proposed method in unifying heterogeneous symbolic knowledge, i.e., expert rules and relational facts, as the substrate of machine learning and reasoning models.
An Empirical Study on Topic Preservation in Multi-Document Summarization

Multi-document summarization (MDS) is a process of generating an informative and concise summary from multiple topic-related documents. Many studies have analyzed the quality of MDS dataset or models, however no work has been done from the perspective of topic preservation. In this work, we fill the gap by performing an empirical analysis on two MDS datasets and study topic preservation on generated summaries from 8 MDS models.Our key findings include i) Multi-News dataset has better gold summaries compared to Multi-XScience in terms of its topic distribution consistency and ii) Extractive approaches perform better than abstractive approaches in preserving topic information from source documents. We hope our findings could help develop a summarization model that can generate topic-focused summary and also give inspiration to researchers in creating dataset for such challenging task.
Classification Systems: Combining taxonomical and perceptual lexical meaning

Lexical meaning includes both perceptual and logical aspects. We present a method for combining a taxonomy with perceptual classiﬁers, and show that in the few-shot setting, it out-performs other methods of injecting taxonomical information in image classiﬁcation. We use this method to deﬁne witness conditions for types in a rich type system with probabilistic type judgments and suggest how such a type system can be used as the basis for a new type of hybrid NLU architecture.
Machines in the media: semantic change in the lexicon of mechanization in 19th-century British newspapers

The industrialization process associated with the so-called Industrial Revolution in 19th-century Great Britain was a time of profound changes, including in the English lexicon. An important yet understudied phenomenon is the semantic shift in the lexicon of mechanisation. In this paper we present the first large-scale analysis of terms related to mechanization over the course of the 19th-century in English. We draw on a corpus of historical British newspapers comprising 4.6 billion tokens and train historical word embedding models. We test existing semantic change detection techniques and analyse the results in light of previous historical linguistic scholarship.