Miracle-dz/UniFAR-FTUs huggingface dataset:
Birds Have Four Legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models

Recent works show that pre-trained language models (PTLMs), such as BERT, possess certain commonsense and factual knowledge. They suggest that it is promising to use PTLMs as "neural knowledge bases" via predicting masked words. Surprisingly, we find that this may not work for numerical commonsense knowledge (e.g., a bird usually has two legs). In this paper, we investigate whether and to what extent we can induce numerical commonsense knowledge from PTLMs as well as the robustness of this process. To study this, we introduce a novel probing task with a diagnostic dataset, NumerSense, containing 13.6k masked-word-prediction probes (10.5k for fine-tuning and 3.1k for testing). Our analysis reveals that: (1) BERT and its stronger variant RoBERTa perform poorly on the diagnostic dataset prior to any fine-tuning; (2) fine-tuning with distant supervision brings some improvement; (3) the best supervised model still performs poorly as compared to human performance (54.06% vs 96.3% in accuracy).
Neural Networks and Value at Risk

Utilizing a generative regime switching framework, we perform Monte-Carlo simulations of asset returns for Value at Risk threshold estimation. Using equity markets and long term bonds as test assets in the global, US, Euro area and UK setting over an up to 1,250 weeks sample horizon ending in August 2018, we investigate neural networks along three design steps relating (i) to the initialization of the neural network, (ii) its incentive function according to which it has been trained and (iii) the amount of data we feed. First, we compare neural networks with random seeding with networks that are initialized via estimations from the best-established model (i.e. the Hidden Markov). We find latter to outperform in terms of the frequency of VaR breaches (i.e. the realized return falling short of the estimated VaR threshold). Second, we balance the incentive structure of the loss function of our networks by adding a second objective to the training instructions so that the neural networks optimize for accuracy while also aiming to stay in empirically realistic regime distributions (i.e. bull vs. bear market frequencies). In particular this design feature enables the balanced incentive recurrent neural network (RNN) to outperform the single incentive RNN as well as any other neural network or established approach by statistically and economically significant levels. Third, we half our training data set of 2,000 days. We find our networks when fed with substantially less data (i.e. 1,000 days) to perform significantly worse which highlights a crucial weakness of neural networks in their dependence on very large data sets .
Identification and Validation of QTL for Grain Quality Traits in a Cross of Soft Wheat Cultivars Pioneer Brand 25R26 and Foster

End-use quality is vital to the wheat (Triticum aestivum L.) industry. Understanding the genetics of components of quality is needed for effi cient genetic improvement. Our objectives were to determine the genetics of soft wheat quality in a population derived from soft wheat cultivars and to validate the results. We developed 171 families from the cross ‘Foster’ × Pioneer ‘25R26’ and obtained quality data from four environments. We assessed fl our yield (FY) and fl our protein (FP), softness equivalent (SE), and water (WA), sodium carbonate (SO), sucrose (SU), and lactic acid (LA) solvent retention capacities (SRCs). Families were genotyped with markers and quantitative trait loci (QTL) analyses were performed. All traits were heritable (0.71– 0.93) and transgressive segregants were noted. The traits FY, WA, SO, and SU were correlated to one another as were SE and FP. A total of 29 QTL were detected from 11 genomic regions. Quantitative trait loci for correlated traits were generally coincident. Two regions of chromosome 1B had the greatest effect on WA, SO, FY, and LA (r 2 = 0.23 to 0.30); these QTL appeared to be associated with the 1BL.1RS rye (Secale cereale L.) translocation and the Glu-B1 locus. A region of chromosome 2B had the greatest effect on FP, SE, and SU (r 2 = 0.13 to 0.23). The 1B and 2B QTL were validated in crosses between each of the original mapping parents and ‘Hopewell’. We documented the relative importance for soft wheat quality of two known regions of 1B and report on a novel QTL on 2B. Marker-assisted selection for these three regions could have a signifi cant impact on soft wheat quality.
Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering

Existing work on augmenting question answering (QA) models with external knowledge (e.g., knowledge graphs) either struggle to model multi-hop relations efficiently, or lack transparency into the model's prediction rationale. In this paper, we propose a novel knowledge-aware approach that equips pre-trained language models (PTLMs) with a multi-hop relational reasoning module, named multi-hop graph relation network (MHGRN). It performs multi-hop, multi-relational reasoning over subgraphs extracted from external knowledge graphs. The proposed reasoning module unifies path-based reasoning methods and graph neural networks to achieve better interpretability and scalability. We also empirically show its effectiveness and scalability on CommonsenseQA and OpenbookQA datasets, and interpret its behaviors with case studies.
A review of issues related to the quantification of bacteria from the phyllosphere

Quantification of the size of epiphytic bacterial populations and characterization of their composition involves definition of a sampling strategy in time and space, the choice of methods for liberating the bacteria from the leaf surface and for recovering them for subsequent determination of the number of viable or culturable cells. This literature review focuses on some of the issues related to these choices. After briefly reviewing the different types of epiphytic colonizers we consider the biological, methodological, and statistical consequences of the choice of the sampling unit and of the spatial and temporal variability of population size and composition for epiphytic bacteria. The different methods available for the detection and enumeration of naturally occurring microorganisms in the phyllosphere are discussed. Advantages and drawbacks of each are described in this review designed as a ‘hands-on’ guide.
Expression of the Nicotinic Receptor α7 Gene in Tendon and Periosteum During Early Development

Abstract: One of the most abundant nicotinic acetylcholine receptors expressed in the central and peripheral nervous systems is a species that contains the α7 gene product, binds α‐bungarotoxin with high affinity, and has a high relative permeability to calcium. The α7 gene is also expressed at low levels in embryonic muscle tissue. We show here that the α7 gene is expressed in tendon fibroblasts and periosteal cells during development. In situ hybridizations identify α7 transcripts in tissue sections containing embryonic tendon and periosteum. RNase protection experiments demonstrate α7 mRNA in primary tendon cells grown in culture. Immunofluorescence with subunit‐specific monoclonal antibodies reveals α7 protein in embryonic tendon. Immunoprecipitation assays with the antibodies indicate that the α7‐containing species in tendon is capable of binding α‐bungarotoxin and that a similar species can be identified at low levels on the surface of fibroblasts in culture. The results show that the α7 gene product is expressed in a range of tissues, including cells thought to be nonexcitable. The distribution of α7 expression early in development and the ability of α7‐containing receptors to elevate intracellular calcium suggest that the gene may influence a variety of calcium‐dependent events during embryogenesis.
Discussion of digital gaming's impact on players' well-being during the COVID-19 lockdown

This research discusses how to utilise digital gaming to support the well-being of its users and sustain their physical and mental health during the COVID-19 lockdown in which people's activities are limited. The published academic literature that is written in English and available for access on online databases was reviewed to develop key take-aways and a framework for discussing how to enhance people's well-being in the COVID-19 lockdown. Interaction with other players in virtual communities has been found to have a positive influence on the mental health of those suffering from a lack of societal connection. A framework for further research has also been developed that focuses on the critical situation of the COVID-19 lockdown,as this is an urgent topic with a huge impact on our health.Some gaming service providers have been proactive in redesigning game programming to be suitable for the lockdown situation, and this enables players to enjoy physical activities even at home.
Code replicability in computer graphics

Being able to duplicate published research results is an important process of conducting research whether to build upon these findings or to compare with them. This process is called "replicability" when using the original authors' artifacts (e.g., code), or "reproducibility" otherwise (e.g., re-implementing algorithms). Reproducibility and replicability of research results have gained a lot of interest recently with assessment studies being led in various fields, and they are often seen as a trigger for better result diffusion and transparency. In this work, we assess replicability in Computer Graphics, by evaluating whether the code is available and whether it works properly. As a proxy for this field we compiled, ran and analyzed 151 codes out of 374 papers from 2014, 2016 and 2018 SIGGRAPH conferences. This analysis shows a clear increase in the number of papers with available and operational research codes with a dependency on the subfields, and indicates a correlation between code replicability and citation count. We further provide an interactive tool to explore our results and evaluation data.
Implementation of the ADMM to Parabolic Optimal Control Problems with Control Constraints and Beyond

Parabolic optimal control problems with control constraints are generally challenging, from either theoretical analysis or algorithmic design perspectives. Conceptually, the well-known alternating direction method of multipliers (ADMM) can be directly applied to such a problem. An attractive advantage of this direct ADMM application is that the control constraint can be untied from the parabolic PDE constraint; these two inherently different constraints thus can be treated individually in iterations. At each iteration of the ADMM, the main computation is for solving an unconstrained parabolic optimal control problem. Because of its high dimensionality after discretization, the unconstrained parabolic optimal control problem at each iteration can be solved only inexactly by implementing certain numerical scheme internally and thus a two-layer nested iterative scheme is required. It then becomes important to find an easily implementable and efficient inexactness criterion to execute the internal iterations, and to prove the overall convergence rigorously for the resulting two-layer nested iterative scheme. To implement the ADMM efficiently, we propose an inexactness criterion that is independent of the mesh size of the involved discretization, and it can be executed automatically with no need to set empirically perceived constant accuracy a prior. The inexactness criterion turns out to allow us to solve the resulting unconstrained optimal control problems to medium or even low accuracy and thus saves computation significantly, yet convergence of the overall two-layer nested iterative scheme can be still guaranteed rigorously. Efficiency of this ADMM implementation is promisingly validated by preliminary numerical results. Our methodology can also be extended to a range of optimal control problems constrained by other linear PDEs such as elliptic equations and hyperbolic equations.
Fluctuation of linear eigenvalue statistics of reverse circulant matrices with independent entries

In this article, we study the fluctuations of linear eigenvalue statistics of reverse circulant $(RC_n)$ matrices with independent entries which satisfy some moment conditions. We show that $\frac{1}{\sqrt{n}} \text{Tr} \phi(RC_n)$ obey the central limit theorem (CLT) type result, where $\phi$ is a nice test function.
Laboratory study of cannibalism and interspecific predation in ladybirds

Abstract. 1 In the absence of aphids, adult females of Adalia bipunctata (L.) showed a greater reluctance to eat eggs than males. 2 Eggs and young larvae were more vulnerable to cannibalism than older larvae and starved larvae were more vulnerable than well‐fed larvae. 3 Both egg and larval cannibalism is inversely related to the abundance of aphids. 4 Eggs are a better food, in terms of larval growth and survival, than aphids. 5 In the absence of aphids interspecific predation occurred, but not equally, between the coccinellids A.bipunctata, A.decempunctata (L.), Coccinella septempunctata L. and C.undecempunctata L. 6 Larvae and adults of A. bipunctata and C.septempunctata were reluctant to eat conspecific eggs painted with a water extract of the other species' eggs and larvae of C. septempunctata were more likely to die after eating a few eggs of A.bipunctata than vice versa. 7 These results indicate that cannibalism occurs mainly when aphid prey is scarce and is adaptive in that it improves the chances of survival, and coccinellids, to varying degrees, are defended against interspecific predation.
Improving Adversarial Text Generation by Modeling the Distant Future

Auto-regressive text generation models usually focus on local fluency, and may cause inconsistent semantic meaning in long text generation. Further, automatically generating words with similar semantics is challenging, and hand-crafted linguistic rules are difficult to apply. We consider a text planning scheme and present a model-based imitation-learning approach to alleviate the aforementioned issues. Specifically, we propose a novel guider network to focus on the generative process over a longer horizon, which can assist next-word prediction and provide intermediate rewards for generator optimization. Extensive experiments demonstrate that the proposed method leads to improved performance.
GoEmotions: A Dataset of Fine-Grained Emotions

Understanding emotion expressed in language has a wide range of applications, from building empathetic chatbots to detecting harmful online behavior. Advancement in this area can be improved using large-scale datasets with a fine-grained typology, adaptable to multiple downstream tasks. We introduce GoEmotions, the largest manually annotated dataset of 58k English Reddit comments, labeled for 27 emotion categories or Neutral. We demonstrate the high quality of the annotations via Principal Preserved Component Analysis. We conduct transfer learning experiments with existing emotion benchmarks to show that our dataset generalizes well to other domains and different emotion taxonomies. Our BERT-based model achieves an average F1-score of . 46 across our proposed taxonomy, leaving much room for improvement.
Deep Convolutional Neural Networks to Diagnose COVID-19 and other Pneumonia Diseases from Posteroanterior Chest X-Rays

The article explores different deep convolutional neural network architectures trained and tested on posteroanterior chest X-rays of 327 patients who are healthy (152 patients), diagnosed with COVID-19 (125), and other types of pneumonia (48). In particular, this paper looks at the deep convolutional neural networks VGG16 and VGG19, InceptionResNetV2 and InceptionV3, as well as Xception, all followed by a flat multi-layer perceptron and a final 30% drop-out. The paper has found that the best performing network is VGG16 with a final $30$% drop-out trained over 3 classes (COVID-19, No Finding, Other Pneumonia). It has an internal cross-validated accuracy of $93.9(\pm3.4)$%, a COVID-19 sensitivity of $87.7(-1.9,+2)$%, and a No Finding sensitivity of $96.8(\pm0.8)$%. The respective external cross-validated values are $84.1(\pm13.5)$%, $87.7(-1.9,2)$%, and $96.8(\pm0.8)$%. The model optimizer was Adam with a 1e-4 learning rate, and categorical cross-entropy loss. It is hoped that, once this research will be put to practice in hospitals, healthcare professionals will be able in the medium to long-term to diagnosing through machine learning tools possible pneumonia, and if detected, whether it is linked to a COVID-19 infection, allowing the detection of new possible COVID-19 foyers after the end of possible "stop-and-go" lockdowns as expected by until a vaccine is found and widespread. Furthermore, in the short-term, it is hoped practitioners can compare the diagnosis from the deep convolutional neural networks with possible RT-PCR testing results, and if clashing, a Computed Tomography could be performed as they are more accurate in showing COVID-19 pneumonia.
A Decoupled Learning Strategy for Massive Access Optimization in Cellular IoT Networks

Cellular-based networks are expected to offer connectivity for massive Internet of Things (mIoT) systems. However, their Random Access CHannel (RACH) procedure suffers from unreliability, due to the collision from the simultaneous massive access. Despite that this collision problem has been treated in existing RACH schemes, these schemes usually organize IoT devices’ transmission and re-transmission along with fixed parameters, thus can hardly adapt to time-varying traffic patterns. Without adaptation, the RACH procedure easily suffers from high access delay, high energy consumption, or even access unavailability. With the goal of improving the RACH procedure, this paper targets to optimize the RACH procedure in real-time by maximizing a long-term hybrid multi-objective function, which consists of the number of access success devices, the average energy consumption, and the average access delay. To do so, we first optimize the long-term objective in the number of access success devices by using Deep Reinforcement Learning (DRL) algorithms for different RACH schemes, including Access Class Barring (ACB), Back-Off (BO), and Distributed Queuing (DQ). The converging capability and efficiency of different DRL algorithms including Policy Gradient (PG), Actor-Critic (AC), Deep Q-Network (DQN), and Deep Deterministic Policy Gradient (DDPG) are compared. Inspired by the results from this comparison, a decoupled learning strategy is developed to jointly and dynamically adapt the access control factors of those three access schemes. This decoupled strategy integrates predicted traffic into the learning process to improve training efficiency, where a Recurrent Neural Network (RNN) model is first employed to predict the real-time traffic values of the network environment, and then multiple DRL agents are employed to cooperatively configure parameters of each RACH scheme. Our results demonstrate that the decoupled strategy remarkably accelerate the training speedy.
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems

In this tutorial article, we aim to provide the reader with the conceptual tools needed to get started on research on offline reinforcement learning algorithms: reinforcement learning algorithms that utilize previously collected data, without additional online data collection. Offline reinforcement learning algorithms hold tremendous promise for making it possible to turn large datasets into powerful decision making engines. Effective offline reinforcement learning methods would be able to extract policies with the maximum possible utility out of the available data, thereby allowing automation of a wide range of decision-making domains, from healthcare and education to robotics. However, the limitations of current algorithms make this difficult. We will aim to provide the reader with an understanding of these challenges, particularly in the context of modern deep reinforcement learning methods, and describe some potential solutions that have been explored in recent work to mitigate these challenges, along with recent applications, and a discussion of perspectives on open problems in the field.
A Transformer-based Approach for Source Code Summarization

Generating a readable summary that describes the functionality of a program is known as source code summarization. In this task, learning code representation by modeling the pairwise relationship between code tokens to capture their long-range dependencies is crucial. To learn code representation for summarization, we explore the Transformer model that uses a self-attention mechanism and has shown to be effective in capturing long-range dependencies. In this work, we show that despite the approach is simple, it outperforms the state-of-the-art techniques by a significant margin. We perform extensive analysis and ablation studies that reveal several important findings, e.g., the absolute encoding of source code tokens’ position hinders, while relative encoding significantly improves the summarization performance. We have made our code publicly available to facilitate future research.
The Dark Photon

The dark photon is a new gauge boson whose existence has been conjectured. It is dark because it arises from a symmetry of a hypothetical dark sector comprising particles completely neutral under the Standard Model interactions. Dark though it is, this new gauge boson can be detected because of its kinetic mixing with the ordinary, visible photon. We review its physics from the theoretical and the experimental point of view. We discuss the difference between the massive and the massless case. We explain how the dark photon enters laboratory, astrophysical and cosmological observations as well as dark matter physics. We survey the current and future experimental limits on the parameters of the massless and massive dark photons together with the related bounds on milli-charged fermions.
A 3D view of the Taurus star-forming region by Gaia and Herschel

Context. Taurus represents an ideal region to study the three-dimensional distribution of the young stellar population and relate it to the associated molecular cloud. Aims. The second Gaia data release (DR2) enables us to investigate the Taurus complex in three dimensions, starting from a previously defined robust membership. The molecular cloud structured in filaments can be traced in emission using the public far-infrared maps from Herschel. Methods. From a compiled catalog of spectroscopically confirmed members, we analyze the 283 sources with reliable parallax and proper motions in the Gaia DR2 archive. We fit the distribution of parallaxes and proper motions with multiple populations described by multivariate Gaussians. We compute the cartesian Galactic coordinates (X,Y,Z) and, for the populations associated with the main cloud, also the galactic space velocity (U,V,W). We discuss the spatial distribution of the populations in relation to the structure of the filamentary molecular cloud traced by Herschel. Results. We discover the presence of six populations which are all well defined in parallax and proper motions, with the only exception being Taurus D. The derived distances range between ~130 and ~160 pc. We do not find a unique relation between stellar population and the associated molecular cloud: while the stellar population seems to be on the cloud surface, both lying at similar distances, this is not the case when the molecular cloud is structured in filaments. Taurus B is probably moving in the direction of Taurus A, while Taurus E appears to be moving towards them. Conclusions. The Taurus region is the result of a complex star formation history which most probably occurred in clumpy and filamentary structures that are evolving independently.