Miracle-dz/UniFAR-FTUs huggingface dataset:
A Formal Hierarchy of RNN Architectures

We develop a formal hierarchy of the expressive capacity of RNN architectures. The hierarchy is based on two formal properties: space complexity, which measures the RNN’s memory, and rational recurrence, defined as whether the recurrent update can be described by a weighted finite-state machine. We place several RNN variants within this hierarchy. For example, we prove the LSTM is not rational, which formally separates it from the related QRNN (Bradbury et al., 2016). We also show how these models’ expressive capacity is expanded by stacking multiple layers or composing them with different pooling functions. Our results build on the theory of “saturated” RNNs (Merrill, 2019). While formally extending these findings to unsaturated RNNs is left to future work, we hypothesize that the practical learnable capacity of unsaturated RNNs obeys a similar hierarchy. We provide empirical results to support this conjecture. Experimental findings from training unsaturated networks on formal languages support this conjecture.
Commutators of random matrices from the unitary and orthogonal groups

We investigate the statistical properties of C = uvu−1 v−1, when u and v are independent random matrices, uniformly distributed with respect to the Haar measure of the groups U( N) and O( N). An exact formula is derived for the average value of power sum symmetric functions of C, and also for products of the matrix elements of C, similar to Weingarten functions. The density of eigenvalues of C is shown to become constant in the large- N limit, and the first N−1 correction is found.
Predicting 21 cm-line map from Lyman-α emitter distribution with generative adversarial networks

The radio observation of 21cm-line signal from the Epoch of Reionization (EoR) enables us to explore the evolution of galaxies and intergalactic medium in the early universe. However, the detection and imaging of 21cm-line signal are tough due to the foreground and instrumental systematics. In order to overcome these obstacles, as a new approach, we propose to take a cross correlation between observed 21cm-line data and 21cm-line images generated from the distribution of the Lyman-$\alpha$ emitters (LAEs) through machine learning. In order to create 21cm-line maps from LAE distribution, we apply conditional Generative Adversarial Network (cGAN) trained with the results of our numerical simulations. We find that 21cm-line brightness temperature maps and the neutral fraction maps can be well reproduced at large scales. Furthermore, we show that the cross correlation is detectable at $k < 0.2~{\rm Mpc}^{-1}$ by combing 400 hours of MWA Phase II observation and LAE deep survey of the Subaru Hyper Suprime Camera. Our new approach of cross correlation with image construction using the cGAN can not only boost the detectability of EoR 21cm-line signal but also allow us to estimate the 21cm-line auto-power spectrum.
Ultra Thin Films of Yttrium Iron Garnet with Very Low Damping: A Review

Thin film Yttrium Iron Garnet (YIG) is a promising material for integrated magnonics. To introduce YIG into nanofabrication processes it is necessary to fabricate very thin YIG films with a thickness well below 100 nm while retaining the extraordinary magnetic properties of the material, especially its long magnon lifetime and spin wave propagation length. Herein, a brief introduction into the topic is given and the various results published over the last decade in this area are reviewed and discussed. Especially for ultrathin films it turns out that pulsed layer deposition and sputtering are the most promising candidates. In addition, the application of room‐temperature deposition and annealing for lift‐off based nanopatterning is discussed, as well as the properties of nanostructures obtained by this method over the past years.
Extractive Summarization as Text Matching

This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems. Instead of following the commonly used framework of extracting sentences individually and modeling the relationship between sentences, we formulate the extractive summarization task as a semantic text matching problem, in which a source document and candidate summaries will be (extracted from the original text) matched in a semantic space. Notably, this paradigm shift to semantic matching framework is well-grounded in our comprehensive analysis of the inherent gap between sentence-level and summary-level extractors based on the property of the dataset. Besides, even instantiating the framework with a simple form of a matching model, we have driven the state-of-the-art extractive result on CNN/DailyMail to a new level (44.41 in ROUGE-1). Experiments on the other five datasets also show the effectiveness of the matching framework. We believe the power of this matching-based summarization framework has not been fully exploited. To encourage more instantiations in the future, we have released our codes, processed dataset, as well as generated summaries in https://github.com/maszhongming/MatchSum.
Pixton’s formula and Abel–Jacobi theory on the Picard stack

Let $A=(a_1,\ldots,a_n)$ be a vector of integers with $d=\sum_{i=1}^n a_i$. By partial resolution of the classical Abel-Jacobi map, we construct a universal twisted double ramification cycle $\mathsf{DR}^{\mathsf{op}}_{g,A}$ as an operational Chow class on the Picard stack $\mathfrak{Pic}_{g,n,d}$ of $n$-pointed genus $g$ curves carrying a degree $d$ line bundle. The method of construction follows the log (and b-Chow) approach to the standard double ramification cycle with canonical twists on the moduli space of curves [arXiv:1707.02261, arXiv:1711.10341, arXiv:1708.04471]. Our main result is a calculation of $\mathsf{DR}^{\mathsf{op}}_{g,A}$ on the Picard stack $\mathfrak{Pic}_{g,n,d}$ via an appropriate interpretation of Pixton's formula in the tautological ring. The basic new tool used in the proof is the theory of double ramification cycles for target varieties [arXiv:1812.10136]. The formula on the Picard stack is obtained from [arXiv:1812.10136] for target varieties $\mathbb{CP}^n$ in the limit $n \rightarrow \infty$. The result may be viewed as a universal calculation in Abel-Jacobi theory. As a consequence of the calculation of $\mathsf{DR}^{\mathsf{op}}_{g,A}$ on the Picard stack $\mathfrak{Pic}_{g,n,d}$, we prove that the fundamental classes of the moduli spaces of twisted meromorphic differentials in $\overline{\mathcal{M}}_{g,n}$ are exactly given by Pixton's formula (as conjectured in the appendix to [arXiv:1508.07940] and in [arXiv:1607.08429]). The comparison result of fundamental classes proven in [arXiv:1909.11981] plays a crucial role in our argument. We also prove the set of relations in the tautological ring of the Picard stack $\mathfrak{Pic}_{g,n,d}$ associated to Pixton's formula.
Dynamic Matching Algorithms in Practice

In recent years, significant advances have been made in the design and analysis of fully dynamic maximal matching algorithms. However, these theoretical results have received very little attention from the practical perspective. Few of the algorithms are implemented and tested on real datasets, and their practical potential is far from understood. In this paper, we attempt to bridge the gap between theory and practice that is currently observed for the fully dynamic maximal matching problem. We engineer several algorithms and empirically study those algorithms on an extensive set of dynamic instances.
Hot-Starting the Ac Power Flow with Convolutional Neural Networks

Obtaining good initial conditions to solve the Newton-Raphson (NR) based ac power flow (ACPF) problem can be a very difficult task. In this paper, we propose a framework to obtain the initial bus voltage magnitude and phase values that decrease the solution iterations and time for the NR based ACPF model, using the dc power flow (DCPF) results and one dimensional convolutional neural networks (1D CNNs). We generate the dataset used to train the 1D CNNs by sampling from a distribution of load demands, and by computing the DCPF and ACPF results for each sample. Experiments on the IEEE 118-bus and \textsc{Pegase} 2869-bus study systems show that we can achieve 33.56\% and 30.06\% reduction in solution time, and 66.47% and 49.52% reduction in solution iterations per case, respectively. We include the 1D CNN architectures and the hyperparameters used, which can be expanded on by the future studies on this topic.
High-resolution transmission spectroscopy of MASCARA-2 b with EXPRES

We report detections of atomic species in the atmosphere of MASCARA-2 b, using the first transit observations obtained with the newly commissioned EXPRES spectrograph. EXPRES is a highly stabilized optical echelle spectrograph, designed to detect stellar reflex motions with amplitudes down to 30 cm s−1, and has recently been deployed at the Lowell Discovery Telescope. By analyzing the transmission spectrum of the ultra-hot Jupiter MASCARA-2 b using the cross-correlation method, we confirm previous detections of Fe I, Fe II, and Na I, which likely originate in the upper regions of the inflated atmosphere. In addition, we report significant detections of Mg I and Cr II. The absorption strengths change slightly with time, possibly indicating different temperatures and chemistry in the day- and nightside terminators. Using the effective stellar line-shape variation induced by the transiting planet, we constrain the projected spin-orbit misalignment of the system to 1.6 ± 3.1 degrees, consistent with an aligned orbit. We demonstrate that EXPRES joins a suite of instruments capable of phase-resolved spectroscopy of exoplanet atmospheres.
Exclusive Hierarchical Decoding for Deep Keyphrase Generation

Keyphrase generation (KG) aims to summarize the main ideas of a document into a set of keyphrases. A new setting is recently introduced into this problem, in which, given a document, the model needs to predict a set of keyphrases and simultaneously determine the appropriate number of keyphrases to produce. Previous work in this setting employs a sequential decoding process to generate keyphrases. However, such a decoding method ignores the intrinsic hierarchical compositionality existing in the keyphrase set of a document. Moreover, previous work tends to generate duplicated keyphrases, which wastes time and computing resources. To overcome these limitations, we propose an exclusive hierarchical decoding framework that includes a hierarchical decoding process and either a soft or a hard exclusion mechanism. The hierarchical decoding process is to explicitly model the hierarchical compositionality of a keyphrase set. Both the soft and the hard exclusion mechanisms keep track of previously-predicted keyphrases within a window size to enhance the diversity of the generated keyphrases. Extensive experiments on multiple KG benchmark datasets demonstrate the effectiveness of our method to generate less duplicated and more accurate keyphrases.
CWY Parametrization for Scalable Learning of Orthogonal and Stiefel Matrices

In this paper we propose a new approach for optimization over orthogonal groups. We parametrize an orthogonal matrix as a product of Householder reflections. To overcome low parallelization capabilities of computing Householder reflections sequentially, we employ an accumulation scheme called the compact WY (or CWY) transform---a compact matrix representation for the series of Householder reflections which can be computed efficiently on highly parallelizable computation units such as GPU and TPU. We further introduce the Truncated CWY (or T-CWY)---a novel approach for Stiefel manifold parametrization which has a competitive complexity estimate compared to other methods and, again, has an advantage when computed on GPU and TPU. We apply these proposed parametrizations to train recurrent neural network architectures in the tasks of neural machine translation and video prediction and demonstrate superiority in both computational and learning aspects compared to other methods from the literature.
Buoyancy and capillary effects on floating liquid lenses

We study the equilibrium shape of a liquid drop resting on top of a liquid surface, i.e., a floating lens. We consider the surface tension forces in non--wetting situations (negative spreading factor), as well as the effects of gravity. We obtain analytical expressions for the drop shape when gravity can be neglected. Perhaps surprisingly, when including gravity in the analysis, we find two different families of equilibrium solutions for the same set of physical parameters. These solutions correspond to drops whose center of mass is above or below the level of the external liquid surface. By means of energetic considerations we determine the family that has the smallest energy, and therefore is the most probable to be found in nature. A detailed explanation of the geometrical differences between the both types of solutions is provided.
Unsupervised vehicle counting via multiple camera domain adaptation

Monitoring vehicle flows in cities is crucial to improve the urban environment and quality of life of citizens. Images are the best sensing modality to perceive and assess the flow of vehicles in large areas. Current technologies for vehicle counting in images hinge on large quantities of annotated data, preventing their scalability to city-scale as new cameras are added to the system. This is a recurrent problem when dealing with physical systems and a key research area in Machine Learning and AI. We propose and discuss a new methodology to design image-based vehicle density estimators with few labeled data via multiple camera domain adaptations.
The Bright Reverse Shock Emission in the Optical Afterglows of Gamma-Ray Bursts in a Stratified Medium

The reverse shock (RS) model is generally introduced to interpret the optical afterglows with the rapid rising and decaying, such as the early optical afterglow of GRB 990123 (which is also called the optical flash). In this paper, we collected 11 gamma-ray burst (GRB) early optical afterglows, which have signatures of dominant RS emission. Since the temporal slopes of the optical flashes are determined by both the medium density distribution index k and the electron spectral index p, we apply the RS model of the thin shell case to the optical flashes and determine the ambient medium of the progenitors. We find that the k value is in the range of 0–1.5. The k value in this paper is consistent with the result in Yi et al., where the forward shock model was applied to some onset bumps. However, the method adopted in this paper is only applicable to GRB afterglows with significant sharp rising and decaying RS emission. Our results indicate that the RS model can also be applied to confirm the circumburst medium, further implying that GRBs may have diverse circumburst media.
On the Unusual Effectiveness of Type-aware Mutations for Testing SMT Solvers

We propose type-aware operator mutation, a simple, but unusually effective approach for testing SMT solvers. The key idea is to mutate operators of conforming types within the seed formulas to generate well-typed mutant formulas. These mutant formulas are then used as the test cases for SMT solvers. We realized type-aware operator mutation within the OpFuzz tool and used it to stress-test Z3 and CVC4, two state-of-the-art SMT solvers. Type-aware operator mutations are unusually effective: During nine months of extensive testing with OpFuzz, we reported 909 bugs in Z3 and CVC4, out of which 632 bugs were confirmed and 531 of the confirmed bugs were fixed by the developers. The detected bugs are highly diverse -- we found bugs of many different types (soundness bugs, invalid model bugs, crashes, etc.), logics and solver configurations. We have further conducted an in-depth study on the bugs found by OpFuzz. The study results show that the bugs found by OpFuzz are of high quality. Many of them affect core components of the SMT solvers' codebases, and some required major changes for the developers to fix. Among the 909 bugs found by OpFuzz, 130 were soundness bugs, the most critical bugs in SMT solvers, and 501 were in the default modes of the solvers. Notably, OpFuzz found 16 critical soundness bugs in CVC4, which has proved to be a very stable SMT solver.
ferify: A Virtual Machine File Protection System against Zero-Day Attacks

Most existing solutions for protecting VMs assume known attack patterns or signatures and focus on detecting malicious manipulations of system files and kernel level memory structures. In this research we develop a system called ferify, which leverages VM introspection (VMI) to protect user files hosted on a VM against unauthorized access even after an attacker has managed to obtain root privileges on the VM. ferify maintains in the hypervisor domain a shadow file access control list (SACL) that is totally transparent to the VM. It uses the SACL to perform independent access control on all system calls that may operate on the target files. Further, ferify prevents kernel modification, ensures the integrity of process ownership, and supports hypervisor based user authentication. We have developed a ferify prototype for Linux and through a set of controlled experiments we show that the system is able to mitigate a range of zero-day attacks that otherwise may evade signature-based solutions. In addition, we analyze the root cause of the observed high processing overhead from trapping of system calls, and propose a general solution that can potentially cut that overhead by half.
Smart Radio Environments Empowered by Reconfigurable Intelligent Surfaces: How it Works, State of Research, and Road Ahead

What is a reconfigurable intelligent surface? What is a smart radio environment? What is a metasurface? How do metasurfaces work and how to model them? How to reconcile the mathematical theories of communication and electromagnetism? What are the most suitable uses and applications of reconfigurable intelligent surfaces in wireless networks? What are the most promising smart radio environments for wireless applications? What is the current state of research? What are the most important and challenging research issues to tackle? These are a few of the many questions that we investigate in this short opus, which has the threefold objective of introducing the emerging research field of smart radio environments empowered by reconfigurable intelligent surfaces, putting forth the need of reconciling and reuniting C. E. Shannon's mathematical theory of communication with G. Green's and J. C. Maxwell's mathematical theories of electromagnetism, and reporting pragmatic guidelines and recipes for employing appropriate physics-based models of metasurfaces in wireless communications.
How Do Neural Networks Estimate Optical Flow? A Neuropsychology-Inspired Study

End-to-end trained convolutional neural networks have led to a breakthrough in optical flow estimation. The most recent advances focus on improving the optical flow estimation by improving the architecture and setting a new benchmark on the publicly available MPI-Sintel dataset. Instead, in this article, we investigate how deep neural networks estimate optical flow. A better understanding of how these networks function is important for (i) assessing their generalization capabilities to unseen inputs, and (ii) suggesting changes to improve their performance. For our investigation, we focus on FlowNetS, as it is the prototype of an encoder-decoder neural network for optical flow estimation. Furthermore, we use a filter identification method that has played a major role in uncovering the motion filters present in animal brains in neuropsychological research. The method shows that the filters in the deepest layer of FlowNetS are sensitive to a variety of motion patterns. Not only do we find translation filters, as demonstrated in animal brains, but thanks to the easier measurements in artificial neural networks, we even unveil dilation, rotation, and occlusion filters. Furthermore, we find similarities in the refinement part of the network and the perceptual filling-in process which occurs in the mammal primary visual cortex.
DGL-KE: Training Knowledge Graph Embeddings at Scale

Knowledge graphs have emerged as a key abstraction for organizing information in diverse domains and their embeddings are increasingly used to harness their information in various information retrieval and machine learning tasks. However, the ever growing size of knowledge graphs requires computationally efficient algorithms capable of scaling to graphs with millions of nodes and billions of edges. This paper presents DGL-KE, an open-source package to efficiently compute knowledge graph embeddings. DGL-KE introduces various novel optimizations that accelerate training on knowledge graphs with millions of nodes and billions of edges using multi-processing, multi-GPU, and distributed parallelism. These optimizations are designed to increase data locality, reduce communication overhead, overlap computations with memory accesses, and achieve high operation efficiency. Experiments on knowledge graphs consisting of over 86M nodes and 338M edges show that DGL-KE can compute embeddings in 100 minutes on an EC2 instance with 8 GPUs and 30 minutes on an EC2 cluster with 4 machines with 48 cores/machine. These results represent a 2× ~ 5× speedup over the best competing approaches. DGL-KE is available on https://github.com/awslabs/dgl-ke.
On the (Un)Reliability of Privacy Policies in Android Apps

The access to privacy-sensitive information on Android is a growing concern in the mobile community. Albeit Google Play recently introduced some privacy guidelines, it is still an open problem to soundly verify whether apps actually comply with such rules. To this aim, in this paper, we discuss a novel methodology based on a fruitful combination of static analysis, dynamic analysis, and machine learning techniques, which allows assessing such compliance. More in detail, our methodology checks whether each app i) contains a privacy policy that complies with the Google Play privacy guidelines, and ii) accesses privacy-sensitive information only upon the acceptance of the policy by the user. Furthermore, the methodology also allows checking the compliance of third-party libraries embedded in the apps w.r. t. the same privacy guidelines.We implemented our methodology in a tool, 3PDroid, and we carried out an assessment on a set of recent and most-downloaded Android apps in the Google Play Store. Experimental results suggest that more than 95% of apps access user’s privacy-sensitive information, but just a negligible subset of them (≈ 1%) fully complies with the Google Play privacy guidelines.