Miracle-dz/UniFAR-FTUs huggingface dataset:
Group-based Bi-Directional Recurrent Wavelet Neural Networks for Video Super-Resolution

Video super-resolution (VSR) aims to estimate a high-resolution (HR) frame from a low-resolution (LR) frames. The key challenge for VSR lies in the effective exploitation of spatial correlation in an intra-frame and temporal dependency between consecutive frames. However, most of the previous methods treat different types of the spatial features identically and extract spatial and temporal features from the separated modules. It leads to lack of obtaining meaningful information and enhancing the fine details. In VSR, there are three types of temporal modeling frameworks: 2D convolutional neural networks (CNN), 3D CNN, and recurrent neural networks (RNN). Among them, the RNN-based approach is suitable for sequential data. Thus the SR performance can be greatly improved by using the hidden states of adjacent frames. However, at each of time step in a recurrent structure, the RNN-based previous works utilize the neighboring features restrictively. Since the range of accessible motion per time step is narrow, there are still limitations to restore the missing details for dynamic or large motion. In this paper, we propose a group-based bi-directional recurrent wavelet neural networks (GBR-WNN) to exploit the sequential data and spatio-temporal information effectively for VSR. The proposed group-based bi-directional RNN (GBR) temporal modeling framework is built on the well-structured process with the group of pictures (GOP). We propose a temporal wavelet attention (TWA) module, in which attention is adopted for both spatial and temporal features. Experimental results demonstrate that the proposed method achieves superior performance compared with state-of-the-art methods in both of quantitative and qualitative evaluations.
A Multi Polarization Square Patch Antenna with a Reconfigurable Feeding Network

A multi-polarization square patch antenna with a reconfigurable feeding network is presented in this paper. The reconfigurable feeding network of this antenna is implemented on an FR-4 substrate by a Wilkinson power divider and a branch line coupler which perform amplitude distribution in the feeding network. Besides, two switching circuits which consist of one PIN diode (BAR63-02w) and its DC biasing circuit manage the RF signal flow on this feeding network. These switching circuits control the phase of the RF signal applied to the square patch, so it can provide linear polarization, left-hand and right-hand circular polarization at 2.45 GHz which has many applications in wireless networks. The simulated and measured results are presented which illuminate acceptable axial ratio bandwidth (ARBW) for both right-hand and left-hand circular polarization in (2.38-2.48 GHz) and minimum -10 dB return loss at 2.45 GHz.
Nonlinear signal transduction network with multistate

Signal transduction is an important and basic mechanism to cell life activities. The stochastic state transition of receptor induces the release of signaling molecular, which triggers the state transition of other receptors. It constructs a nonlinear sigaling network, and leads to robust switchlike properties which are critical to biological function. Network architectures and state transitions of receptor affect the performance of this biological network. In this work, we perform a study of nonlinear signaling on biological polymorphic network by analyzing network dynamics of the Ca induced Ca release mechanism, where fast and slow processes are involved and the receptor has four conformational states. Three types of networks, Erdös-Rényi network, Watts-Strogatz network and BaraBási-Albert network, are considered with different parameters. The dynamics of the biological networks exhibit different patterns at different time scales. At short time scale, the second open state is essential to reproduce the quasi-bistable regime, which emerges at a critical strength of connection for all three states involved in the fast processes and disappears at another critical point. The pattern at short time scale is not sensitive to the network architecture. At long time scale, only monostable regime is observed, and difference of network architectures affects the results more seriously. Our finding identifies features of nonlinear signaling networks with multistate that may underlie their biological function.
Marangoni instability triggered by selective evaporation of a binary liquid inside a Hele-Shaw cell

Abstract Interfacial stability is important for many processes involving heat and mass transfer across two immiscible phases. For the evaporation of a binary solution with one component more volatile than the other, gradients in surface tension can arise. These gradients can ultimately destabilize the liquid–gas interface. We study the evaporation of an ethanol–water solution, for which ethanol is more volatile. The solution is contained in a horizontal Hele-Shaw cell which is open from one end to allow for evaporation into air. A Marangoni instability is then triggered at the liquid–air interface. We study the temporal evolution of the instability through its effects on the bulk of the liquid. More specifically, the growth of convective cells is measured with confocal microscopy and the velocity field with microparticle image velocimetry. The results of numerical simulations based on quasi-two-dimensional equations satisfactorily compare with the experimental observations, even without consideration of evaporative cooling, although this cooling can play an extra role in experiments. Furthermore, a linear stability analysis applied to a simplified version of the quasi-two-dimensional equations showed reasonably good agreement with the results from simulations at early times, when the instability has just been triggered and before coarsening. In particular, we find a critical Marangoni number below which a regime of stability is predicted.
Neural Combinatory Constituency Parsing

We propose two fast neural combinatory models for constituency parsing: binary and multi-branching. Our models decompose the bottom-up parsing process into 1) classification of tags, labels, and binary orientations or chunks and 2) vector composition based on the computed orientations or chunks. These models have theoretical sub-quadratic complexity and empirical linear complexity. The binary model achieves an F1 score of 92.54 on Penn Treebank, speeding at 1327.2 sents/sec. Both the models with XLNet provide near state-of-the-art accuracies for English. Syntactic branching tendency and headedness of a language are observed during the training and inference processes for Penn Treebank, Chinese Treebank, and Keyaki Treebank (Japanese).
Last Layer Marginal Likelihood for Invariance Learning

Data augmentation is often used to incorporate inductive biases into models. Traditionally, these are hand-crafted and tuned with cross validation. The Bayesian paradigm for model selection provides a path towards end-to-end learning of invariances using only the training data, by optimising the marginal likelihood. Computing the marginal likelihood is hard for neural networks, but success with tractable approaches that compute the marginal likelihood for the last layer only raises the question of whether this convenient approach might be employed for learning invariances. We show partial success on standard benchmarks, in the low-data regime and on a medical imaging dataset by designing a custom optimisation routine. Introducing a new lower bound to the marginal likelihood allows us to perform inference for a larger class of likelihood functions than before. On the other hand, we demonstrate failure modes on the CIFAR10 dataset, where the last layer approximation is not sufficient due to the increased complexity of our neural network. Our results indicate that once more sophisticated approximations become available the marginal likelihood is a promising approach for invariance learning in neural networks.
Understanding Information Spreading Mechanisms During COVID-19 Pandemic by Analyzing the Impact of Tweet Text and User Features for Retweet Prediction

COVID-19 has affected the world economy and the daily life routine of almost everyone. It has been a hot topic on social media platforms such as Twitter, Facebook, etc. These social media platforms enable users to share information with other users who can reshare this information, thus causing this information to spread. Twitter's retweet functionality allows users to share the existing content with other users without altering the original content. Analysis of social media platforms can help in detecting emergencies during pandemics that lead to taking preventive measures. One such type of analysis is predicting the number of retweets for a given COVID-19 related tweet. Recently, CIKM organized a retweet prediction challenge for COVID-19 tweets focusing on using numeric features only. However, our hypothesis is, tweet text may play a vital role in an accurate retweet prediction. In this paper, we combine numeric and text features for COVID-19 related retweet predictions. For this purpose, we propose two CNN and RNN based models and evaluate the performance of these models on a publicly available TweetsCOV19 dataset using seven different evaluation metrics. Our evaluation results show that combining tweet text with numeric features improves the performance of retweet prediction significantly.
Origin of stellar prolate rotation in a cosmologically simulated faint dwarf galaxy

Stellar prolate rotation in dwarf galaxies is rather uncommon, with only two known galaxies in the Local Group showing such feature (Phoenix and And II). Cosmological simulations show that in massive early-type galaxies prolate rotation likely arises from major mergers. However, the origin of such kinematics in the dwarf galaxies regime has only been explored using idealized simulations. Here, we made use of hydrodynamical cosmological simulations of dwarfs galaxies with stellar mass between 3 × 105 and 5 × 108 M⊙ to explore the formation of prolate rotators. Out of 27 dwarfs, only one system showed clear rotation around the major axis, whose culprit is a major merger at $z$ = 1.64, which caused the transition from an oblate to a prolate configuration. Interestingly, this galaxy displays a steep metallicity gradient, reminiscent of the one measured in Phoenix and And II: this is the outcome of the merger event that dynamically heats old, metal-poor stars, and of the centrally concentrated residual star formation. Major mergers in dwarf galaxies offer a viable explanation for the formation of such peculiar systems, characterized by steep metallicity gradients and prolate rotation.
An Empirical Survey of Data Augmentation for Limited Data Learning in NLP

NLP has achieved great progress in the past decade through the use of neural models and large labeled datasets. The dependence on abundant data prevents NLP models from being applied to low-resource settings or novel tasks where significant time, money, or expertise is required to label massive amounts of textual data. Recently, data augmentation methods have been explored as a means of improving data efficiency in NLP. To date, there has been no systematic empirical overview of data augmentation for NLP in the limited labeled data setting, making it difficult to understand which methods work in which settings. In this paper, we provide an empirical survey of recent progress on data augmentation for NLP in the limited labeled data setting, summarizing the landscape of methods (including token-level augmentations, sentence-level augmentations, adversarial augmentations, and hidden-space augmentations) and carrying out experiments on 11 datasets covering topics/news classification, inference tasks, paraphrasing tasks, and single-sentence tasks. Based on the results, we draw several conclusions to help practitioners choose appropriate augmentations in different settings and discuss the current challenges and future directions for limited data learning in NLP.
Single inertial particle statistics in turbulent flows from Lagrangian velocity models

Jan Friedrich, 2 Bianca Viggiano, 3 Mickael Bourgoin, Raúl Bayoán Cal, 2 and Laurent Chevillard Institute of Physics and For Wind, University of Oldenburg, 26129 Oldenburg, Germany Univ. Lyon, ENS de Lyon, Univ. Claude Bernard, CNRS, Laboratoire de Physique, F-69342, Lyon, France Department of Mechanical and Materials Engineering, Portland State University, Portland, Oregon, USA (Dated: June 15, 2021)
An Online Riemannian PCA for Stochastic Canonical Correlation Analysis

We present an efficient stochastic algorithm (RSG+) for canonical correlation analysis (CCA) using a reparametrization of the projection matrices. We show how this reparametrization (into structured matrices), simple in hindsight, directly presents an opportunity to repurpose/adjust mature techniques for numerical optimization on Riemannian manifolds. Our developments nicely complement existing methods for this problem which either require O(d 3) time complexity per iteration with O ( 1 t ) convergence rate (where d is the dimensionality) or only extract the top 1 component with O ( 1 t ) convergence rate. In contrast, our algorithm offers an improvement: it achieves O(d 2 k) runtime complexity per iteration for extracting the top k canonical components with O ( 1 t ) convergence rate. While our paper focuses more on the formulation and the algorithm, our experiments show that the empirical behavior on common datasets is quite promising. We also explore a potential application in training fair models with missing sensitive attributes.
Synthesis of Lactide from Alkyl Lactate via a Prepolymer Route

Poly(lactic acid) is a biodegradable polymer that has enormous potential for use as a replacement for some petroleum-based materials. However, the properties of this polymer depend strongly on the quality of the lactide monomer from which it is often synthesized. In this work, lactide was synthesized from alkyl lactate via a prepolymer route, and the reaction kinetics was compared with lactide synthesis from lactic acid. A number of different parameters were investigated in order to obtain the highest possible yield of lactide, as well as to achieve l-isomer selectivity. Among the various acid catalysts tested, Sn(Oct)2 was found to be the most effective for L-lactide selectivity as well as for producing a high oligomer yield from the alkyl lactate. The effect of the alkyl group length of the starting materials was investigated with the highest lactide yield being obtained from ethyl lactate, over methyl and butyl lactate. The deoligomerization reaction was also studied in detail. It was found that an oli.
Learning on Abstract Domains: A New Approach for Verifiable Guarantee in Reinforcement Learning

Formally verifying Deep Reinforcement Learning (DRL) systems is a challenging task due to the dynamic continuity of system behaviors and the black-box feature of embedded neural networks. In this paper, we propose a novel abstraction-based approach to train DRL systems on finite abstract domains instead of concrete system states. It yields neural networks whose input states are finite, making hosting DRL systems directly verifiable using model checking techniques. Our approach is orthogonal to existing DRL algorithms and off-the-shelf model checkers. We implement a resulting prototype training and verification framework and conduct extensive experiments on the state-of-the-art benchmark. The results show that the systems trained in our approach can be verified more efficiently while they retain comparable performance against those that are trained without abstraction.
Low-memory stochastic backpropagation with multi-channel randomized trace estimation

Thanks to the combination of state-of-the-art accelerators and highly optimized open software frameworks, there has been tremendous progress in the performance of deep neural networks. While these developments have been responsible for many breakthroughs, progress towards solving large-scale problems, such as video encoding and semantic segmentation in 3D, is hampered because access to on-premise memory is often limited. Instead of relying on (optimal) checkpointing or invertibility of the network layers -- to recover the activations during backpropagation -- we propose to approximate the gradient of convolutional layers in neural networks with a multi-channel randomized trace estimation technique. Compared to other methods, this approach is simple, amenable to analyses, and leads to a greatly reduced memory footprint. Even though the randomized trace estimation introduces stochasticity during training, we argue that this is of little consequence as long as the induced errors are of the same order as errors in the gradient due to the use of stochastic gradient descent. We discuss the performance of networks trained with stochastic backpropagation and how the error can be controlled while maximizing memory usage and minimizing computational overhead.
EDEN: A High-Performance, General-Purpose, NeuroML-Based Neural Simulator

Modern neuroscience employs in silico experimentation on ever-increasing and more detailed neural networks. The high modeling detail goes hand in hand with the need for high model reproducibility, reusability and transparency. Besides, the size of the models and the long timescales under study mandate the use of a simulation system with high computational performance, so as to provide an acceptable time to result. In this work, we present EDEN (Extensible Dynamics Engine for Networks), a new general-purpose, NeuroML-based neural simulator that achieves both high model flexibility and high computational performance, through an innovative model-analysis and code-generation technique. The simulator runs NeuroML-v2 models directly, eliminating the need for users to learn yet another simulator-specific, model-specification language. EDEN's functional correctness and computational performance were assessed through NeuroML models available on the NeuroML-DB and Open Source Brain model repositories. In qualitative experiments, the results produced by EDEN were verified against the established NEURON simulator, for a wide range of models. At the same time, computational-performance benchmarks reveal that EDEN runs from one to nearly two orders-of-magnitude faster than NEURON on a typical desktop computer, and does so without additional effort from the user. Finally, and without added user effort, EDEN has been built from scratch to scale seamlessly over multiple CPUs and across computer clusters, when available.
Named Entity Normalization Model Using Edge Weight Updating Neural Network: Assimilation Between Knowledge-Driven Graph and Data-Driven Graph

Discriminating the matched named entity pairs or identifying the entities' canonical forms are critical in text mining tasks. More precise named entity normalization in text mining will benefit other subsequent text analytic applications. We built the named entity normalization model with a novel Edge Weight Updating Neural Network. Our proposed model when tested on four different datasets achieved state-of-the-art results. We, next, verify our model's performance on NCBI Disease, BC5CDR Disease, and BC5CDR Chemical databases, which are widely used named entity normalization datasets in the bioinformatics field. We also tested our model with our own financial named entity normalization dataset to validate the efficacy for more general applications. Using the constructed dataset, we differentiate named entity pairs. Our model achieved the highest named entity normalization performances in terms of various evaluation metrics.
Making ab initio QED functional(s): Nonperturbative and photon-free effective frameworks for strong light–matter coupling

Significance Strong interaction between light and matter recently attracted enormous interest after seminal work highlighted how materials and chemical reactions can be effectively controlled with cavities, suggesting new quantum technological applications and alternative paths to catalysis. Combining the quantized nature of light and matter is, however, a nontrivial task due to the overwhelming complexity of the combined system. We first motivate a nonperturbative photon-free reformulation of quantum electrodynamics (QED) that recovers, even in its simplest form, the correct physical limits of the original QED problem, providing us with an efficient framework for further developments. By combination with density-functional theory, we obtain the first nonperturbative local density–type functional that describes the quantized interaction between light and matter while remaining computationally feasible. Strong light–matter coupling provides a promising path for the control of quantum matter where the latter is routinely described from first principles. However, combining the quantized nature of light with this ab initio tool set is challenging and merely developing as the coupled light–matter Hilbert space is conceptually different and computational cost quickly becomes overwhelming. In this work, we provide a nonperturbative photon-free formulation of quantum electrodynamics (QED) in the long-wavelength limit, which is formulated solely on the matter Hilbert space and can serve as an accurate starting point for such ab initio methods. The present formulation is an extension of quantum mechanics that recovers the exact results of QED for the zero- and infinite-coupling limit and the infinite-frequency as well as the homogeneous limit, and we can constructively increase its accuracy. We show how this formulation can be used to devise approximations for quantum-electrodynamical density-functional theory (QEDFT), which in turn also allows us to extend the ansatz to the full minimal-coupling problem and to nonadiabatic situations. Finally, we provide a simple local density–type functional that takes the strong coupling to the transverse photon degrees of freedom into account and includes the correct frequency and polarization dependence. This QEDFT functional accounts for the quantized nature of light while remaining computationally simple enough to allow its application to a large range of systems. All approximations allow the seamless application to periodic systems.
Ab initio
 study of lattice dynamics of group IV semiconductors using pseudohybrid functionals for extended Hubbard interactions

We study the lattice dynamics of group IV semiconductors using fully ab-initio extended Hubbard functional. The onsite and intersite Hubbard interactions are determined self-consistently with recently developed pseudohybrid functionals and included in force calculations. We analyze the Pulay forces by the choice of atomic orbital projectors and the force contribution of the onsite and intersite Hubbard terms. The phonon dispersions, Grüneisen parameters, and lattice thermal conductivities of diamond, silicon, and germanium, which are most-representative covalent-bonding semiconductors, are calculated and compared with the results using local, semilocal, and hybrid functionals. The extended Hubbard functional produces increased phonon velocities and lifetimes, and thus lattice thermal conductivities compared to local and semilocal functionals, agreeing with experiments very well. Considering that our computational demand is comparable to simple local functionals, this work thus suggests a way to perform high-throughput electronic and structural calculations with a higher accuracy.