Miracle-dz/UniFAR-FTUs huggingface dataset:
A Practical Survey on Faster and Lighter Transformers

Recurrent neural networks are effective models to process sequences. However, they are unable to learn long-term dependencies because of their inherent sequential nature. As a solution, Vaswani et al. introduced the Transformer, a model solely based on the attention mechanism that is able to relate any two positions of the input sequence, hence modelling arbitrary long dependencies. The Transformer has improved the state-of-the-art across numerous sequence modelling tasks. However, its effectiveness comes at the expense of a quadratic computational and memory complexity with respect to the sequence length, hindering its adoption. Fortunately, the deep learning community has always been interested in improving the models’ efficiency, leading to a plethora of solutions such as parameter sharing, pruning, mixed-precision, and knowledge distillation. Recently, researchers have directly addressed the Transformer’s limitation by designing lower-complexity alternatives such as the Longformer, Reformer, Linformer, and Performer. However, due to the wide range of solutions, it has become challenging for researchers and practitioners to determine which methods to apply in practice to meet the desired tradeoff between capacity, computation, and memory. This survey addresses this issue by investigating popular approaches to make Transformers faster and lighter and by providing a comprehensive explanation of the methods’ strengths, limitations, and underlying assumptions.
A Graph-based Conflict-free Cooperation Method for Intelligent Electric Vehicles at Unsignalized Intersections

Electric, intelligent, and network are the most important future development directions of automobiles. Intelligent electric vehicles have shown great potentials to improve traffic mobility and reduce emissions, especially at unsignalized intersections. Previous research has shown that vehicle passing order is the key factor in traffic mobility improvement. In this paper, we propose a graph-based cooperation method to formalize the conflict-free scheduling problem at unsignalized intersections. Firstly, conflict directed graphs and coexisting undirected graphs are built to describe the conflict relationship of the vehicles. Then, two graph-based methods are introduced to solve the vehicle passing order. One method is an optimized depth - first spanning tree method which aims to find the local optimal passing order for each vehicle. The other method is a maximum matching algorithm that solves the global optimal problem. The computational complexity of both methods is also derived. Numerical simulation results demonstrate the effectiveness of the proposed algorithms.
3D Point Cloud Registration with Multi-Scale Architecture and Self-supervised Fine-tuning

We present MS-SVConv, a fast multi-scale deep neural network that outputs features from point clouds for 3D registration between two scenes. We compute features using a 3D sparse voxel convolutional network on a point cloud at different scales and then fuse the features through fully-connected layers. With supervised learning, we show signiﬁcant improvements compared to state-of-the-art meth-ods on the competitive and well-known 3DMatch benchmark. We also achieve a better generalization through different source and target datasets, with very fast computation. Finally, we present a strategy to ﬁne-tune MS-SVConv on unknown datasets in a self-supervised way, which leads to state-of-the-art results on ETH and TUM datasets. The code is publicly available at: https://github.com/ humanpose1/MS-SVConv .
Cavity-induced exciton localization and polariton blockade in two-dimensional semiconductors coupled to an electromagnetic resonator

Emil V. Denning, 2, 3, ∗ Martijn Wubs, 2, 4 Nicolas Stenger, 2, 4 Jesper Mørk, 2 and Philip Trøst Kristensen 2 Department of Photonics Engineering, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark NanoPhoton Center for Nanophotonics, Technical University of Denmark, Ørsteds Plads 345A, DK-2800 Kgs. Lyngby, Denmark Nichtlineare Optik und Quantenelektronik, Institut für Theoretische Physik, Technische Universität Berlin, 10623 Berlin, Germany Center for Nanostructured Graphene, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark (Dated: February 23, 2022)
Dodrio: Exploring Transformer Models with Interactive Visualization

Why do large pre-trained transformer-based models perform so well across a wide variety of NLP tasks? Recent research suggests the key may lie in multi-headed attention mechanism’s ability to learn and represent linguistic information. Understanding how these models represent both syntactic and semantic knowledge is vital to investigate why they succeed and fail, what they have learned, and how they can improve. We present Dodrio, an open-source interactive visualization tool to help NLP researchers and practitioners analyze attention mechanisms in transformer-based models with linguistic knowledge. Dodrio tightly integrates an overview that summarizes the roles of different attention heads, and detailed views that help users compare attention weights with the syntactic structure and semantic information in the input text. To facilitate the visual comparison of attention weights and linguistic knowledge, Dodrio applies different graph visualization techniques to represent attention weights scalable to longer input text. Case studies highlight how Dodrio provides insights into understanding the attention mechanism in transformer-based models. Dodrio is available at https://poloclub.github. io/dodrio/.
MASTER OT J172758.09+380021.5: a peculiar ER UMa-type dwarf nova, probably a missed nova in the recent past

A CCD photometry of the dwarf nova MASTER OT J172758.09 +380021.5 was carried out in 2019 during 134 nights. Observations covered three superoutbursts, five normal outbursts and quiescence between them. The available ASASSN and ZTF data for 2014 – 2020 were also examined. Spectral observations were done in 2020 when the object was in quiescence. Spectra and photometry revealed that the star is an H-rich active ER UMa-type dwarf nova with a highly variable supercycle (time interval between two successive superoutbursts) of ∼ 50 – 100 d that implies a high and variable mass-transfer rate. MASTER OT J172758.09+380021.5 demonstrated peculiar behaviour: short-lasted superoutbursts (a week); a slow superoutburst decline and cases MASTER OT J172758.09+380021.5 139 of rebrightenings; low frequency (from none to a few) of the normal outbursts during the supercycle. In 2019 a mean period of positive superhumps was found to be 0.05829 d during the superoutbursts. Late superhumps with a mean period of 0.057915 d which lasted about ∼ 20 d after the end of superoutburst and were replaced by an orbital period of 0.057026 d or its orbital-negative superhump beat period were detected. An absence of eclipse in the orbital light curve and its moderate amplitude are consistent with the orbital inclination of about 40◦ found from spectroscopy. The blue peaks of the V-Ic and B-Rc colour indices of superhumps during the superoutburst coincided with minima of the light curves, while B-Rc of the late superhumps coincided with a rising branch of the light curves. We found that a low mass ratio q = 0.08 could explain most of the peculiarities of MASTER OT J172758.09+380021.5. The mass-transfer rate should be accordingly higher than what is expected from gravitational radiation only, this assumes the object is in a post-nova state and underwent a nova eruption relatively recently – hundreds of years ago. This object would provide probably the first observational evidence that a nova eruption can occur even in CVs near the period minimum.
A Dual Characterization of the Stability of the Wonham Filter

This paper revisits the classical question of the stability of the nonlinear Wonham filter. The novel contributions of this paper are two-fold: (i) definition of the stabilizability for the (control-theoretic) dual to the nonlinear filter; and (ii) the use of this definition to obtain conclusions on the stability of the Wonham filter. Specifically, it is shown that the stabilizability of the dual system is necessary for filter stability and conversely stabilizability implies that the filter asymptotically detects the correct ergodic class. The formulation and the proofs are based upon a recently discovered duality result whereby the nonlinear filtering problem is cast as a stochastic optimal control problem for a backward stochastic differential equation (BSDE). The control-theoretic proof techniques and results may be viewed as a generalization of the classical work on the stability of the Kalman filter.
Model-Free Learning of Safe yet Effective Controllers

We study the problem of learning safe control policies that are also effective; i.e., maximizing the probability of satisfying a linear temporal logic (LTL) specification of a task, and the discounted reward capturing the (classic) control performance. We consider unknown environments modeled as Markov decision processes. We propose a model-free reinforcement learning algorithm that learns a policy that first maximizes the probability of ensuring safety, then the probability of satisfying the given LTL specification and lastly, the sum of discounted Quality of Control rewards. Finally, we illustrate applicability of our RL-based approach.
Nearly Horizon-Free Offline Reinforcement Learning

We revisit offline reinforcement learning on episodic time-homogeneous Markov Decision Processes (MDP). For tabular MDP with $S$ states and $A$ actions, or linear MDP with anchor points and feature dimension $d$, given the collected $K$ episodes data with minimum visiting probability of (anchor) state-action pairs $d_m$, we obtain nearly horizon $H$-free sample complexity bounds for offline reinforcement learning when the total reward is upper bounded by $1$. Specifically: 1. For offline policy evaluation, we obtain an $\tilde{O}\left(\sqrt{\frac{1}{Kd_m}} \right)$ error bound for the plug-in estimator, which matches the lower bound up to logarithmic factors and does not have additional dependency on $\mathrm{poly}\left(H, S, A, d\right)$ in higher-order term. 2.For offline policy optimization, we obtain an $\tilde{O}\left(\sqrt{\frac{1}{Kd_m}} + \frac{\min(S, d)}{Kd_m}\right)$ sub-optimality gap for the empirical optimal policy, which approaches the lower bound up to logarithmic factors and a high-order term, improving upon the best known result by \cite{cui2020plug} that has additional $\mathrm{poly}\left(H, S, d\right)$ factors in the main term. To the best of our knowledge, these are the \emph{first} set of nearly horizon-free bounds for episodic time-homogeneous offline tabular MDP and linear MDP with anchor points. Central to our analysis is a simple yet effective recursion based method to bound a"total variance"term in the offline scenarios, which could be of individual interest.
Variation of the 
Δ
 baryon mass and hybrid star properties in static and rotating conditions

The possible conditions for hadron-quark phase transition in hybrid star cores are investigated in the present work. For the hadronic matter part the effective chiral model is adopted. Exotic baryonic degrees like hyperons and the delta baryons are also taken into account. As ∆s posses Breit-Wigner mass distribution (1232 ± 120 MeV), the hadronic equation of state is obtained by varying the mass of the delta baryons in this range. For the quark phase the MIT bag model is chosen with repulsive effects of the unpaired quarks. Phase transition is achieved using Gibbs construction and the gross properties of the resultant hybrid star are calculated in both static and rotating conditions and compared with the various constraints on them from different observational and empirical perspectives. The work presents a thorough study of the phase transition properties like the critical density of appearance of quarks, the density range for the persistence of the mixed phase and the population of different hadrons and quarks in hybrid star matter. The hybrid star properties, calculated in both static and rotating conditions, are found to be consistent with the bounds on them from different perspectives.
DAGN: Discourse-Aware Graph Network for Logical Reasoning

Recent QA with logical reasoning questions requires passage-level relations among the sentences. However, current approaches still focus on sentence-level relations interacting among tokens. In this work, we explore aggregating passage-level clues for solving logical reasoning QA by using discourse-based information. We propose a discourse-aware graph network (DAGN) that reasons relying on the discourse structure of the texts. The model encodes discourse information as a graph with elementary discourse units (EDUs) and discourse relations, and learns the discourse-aware features via a graph network for downstream QA tasks. Experiments are conducted on two logical reasoning QA datasets, ReClor and LogiQA, and our proposed DAGN achieves competitive results. The source code is available at https://github.com/Eleanor-H/DAGN.
Comparative determination of the electron temperature in Ar- and N2-plasmas with electrostatic probes, optical emission spectroscopy OES and energy dispersive mass spectrometry EDMS

The electron temperature T e as the often most essential parameter for applications of low pressures plasma has been derived from comparative in situ measurements with a single and a double electrostatic probe, and with optical emission spectroscopy OES and energy dispersive mass spectrometry EDMS as remote techniques. Electrodeless rf discharges maintained by electron cyclotron wave resonance ECWR in pure Ar and N 2 in the pressure regime from 10 −2 to 1 Pa have been used as sample plasmas. The evaluation of the OES- and EDMS-signals is described in detail. The pressure dependence of the T e -results derived therefrom is found to compare well with the data from the probe measurements, and with calculations from a charge carrier balance equation. By matching the OES data to the absolute T e -values from the probe measurements, numerical expressions have been obtained by which T e can be quantitatively calculated from the intensity ratios between selected emission lines from the Ar- and the N 2 -plasma. Furthermore, the EDMS-results are also shown to deliver quantitative information about T e .
Superiority of Mild Interventions against COVID-19 on Public Health and Economic Measures

(1) Background: During the global spread of COVID-19, Japan has been among the top countries to maintain a relatively low number of infections, despite implementing limited institutional interventions and its high population density. This study investigated how limited intervention policies have affected public health and economic conditions in the COVID-19 context and aimed to gain insight into the effective and sustainable measures against new infectious diseases in densely inhabited areas. (2) Methods: A system dynamics approach was employed. Qualitative causal loop analysis and stock and quantitative flow model analysis were performed, using a Tokyo Metropolitan area dataset. (3) Results: A causal loop analysis suggested that there were risks in prematurely terminating such interventions. Based on this result and the subsequent quantitative modeling, we found that the short-term effectiveness of a short-term pre-emptive stay-at-home request caused a resurgence in the number of positive cases, whereas an additional request provided a limited negative add-on effect for economic measures (e.g., number of electronic word-of-mouth communications and restaurant visits). (4) Conclusions: These findings suggest the superiority of a mild and continuous intervention as a long-term countermeasure under epidemic pressures when compared with strong intermittent interventions.
A Machine Learning Pipeline for Automatic Extraction of Statistic Reports and Experimental Conditions from Scientific Papers

—We address the problem of extracting reports of statistics along with information about the experimental conditions and topics from scientiﬁc publications. A common writing style for statistical results are the recommendations of the American Psychology Association, known as APA-style. However, in practice, writing styles vary as reports are not 100% following APA-style or parameters are not reported despite being mandatory. In addition, the statistics are not reported in isolation but in context of experimental conditions investigated and the general topic. We address these challenges by proposing a ﬂexible pipeline STEREO based on active wrapper induction and unsupervised aspect extraction. Hierarchical active wrapper learning is applied to learn rules to extract the reported statistics. Challenge here is to apply active wrapper on an information extraction task without having formatting landmarks as they can be exploited in HTML pages. Result of step 1 is a set of extracted statistic reports together with sentences in which the reports were found. This is used as input to step 2 of STEREO, which has two two parts. We extract experimental conditions using a grammar-based active wrapper. Furthermore, we identify the experimental topic using an unsupervised attention-based aspect extraction approach adapted to our problem domain. We applied our pipeline to the over 100 , 000 documents in the CORD-19 dataset. It required only 0 . 25% of the corpus (about 500 documents) to learn statistics extraction rules that cover 95% of the sentences in CORD-19. The statistic extraction has nearly 100% precision on APA-conform and 95% precision on non-APA writing styles. In total, we were able to extract 113 k reported statistics, of which only < 1% is APA conform. We could extract in 46% the correct conditions from APA-conform reports ( 30% for non-APA). The best model for topic extraction achieves a precision of 75% on statistics reported in APA style ( 73% for non-APA conform). We conclude that STEREO is a good foundation for automatic statistic extraction and future developments for scientiﬁc paper analysis. Particularly the extraction of non-APA conform reports is important and allows applications such as giving feedback to authors about what is missing and could be changed. Finally, STEREO complements the portfolio of existing metadata extraction tools and can be integrated in a general scientiﬁc paper analysis pipeline.
Divide-and-Conquer: A Distributed Hierarchical Factor Approach to Modeling Large-Scale Time Series Data

This paper proposes a hierarchical approximate-factor approach to analyzing highdimensional, large-scale heterogeneous time series data using distributed computing. The new method employs a multiple-fold dimension reduction procedure using Principal Component Analysis (PCA) and shows great promises for modeling large-scale data that cannot be stored nor analyzed by a single machine. Each computer at the basic level performs a PCA to extract common factors among the time series assigned to it and transfers those factors to one and only one node of the second level. Each 2nd-level computer collects the common factors from its subordinates and performs another PCA to select the 2ndlevel common factors. This process is repeated until the central server is reached, which collects common factors from its direct subordinates and performs a final PCA to select the global common factors. The noise terms of the 2nd-level approximate factor model are the unique common factors of the 1st-level clusters. We focus on the case of 2 levels in our theoretical derivations, but the idea can easily be generalized to any finite number of hierarchies. We discuss some clustering methods when the group memberships are unknown and introduce a new diffusion index approach to forecasting. We further extend the analysis to unit-root nonstationary time series. Asymptotic properties of the proposed method are derived for the diverging dimension of the data in each computing unit and the sample size T . We use both simulated data and real examples to assess the performance of the proposed method in finite samples, and compare our method with the commonly used ones in the literature concerning the forecastability of extracted factors.
Band‐gap narrowing in highly doped n‐ and p‐type GaAs studied by photoluminescence spectroscopy

Band‐gap narrowing of GaAs as a function of doping concentration has been measured using photoluminescence spectroscopy on samples grown by molecular beam epitaxy. Both n‐ (Si) and p‐ (Be) doped samples with concentrations varying from 3×1017 to 3×1018 cm−3 have been measured. The experimental results obtained from a line‐shape analysis of the spectra taking tailing effects into account are in good agreement with recent theoretical calculations. A simple expression for the band‐gap narrowing as a function of concentration for both n‐and p‐doped GaAs is given.
Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes

Grasping unseen objects in unconstrained, cluttered environments is an essential skill for autonomous robotic manipulation. Despite recent progress in full 6-DoF grasp learning, existing approaches often consist of complex sequential pipelines that possess several potential failure points and run-times unsuitable for closed-loop grasping. Therefore, we propose an end-to-end network that efficiently generates a distribution of 6-DoF parallel-jaw grasps directly from a depth recording of a scene. Our novel grasp representation treats 3D points of the recorded point cloud as potential grasp contacts. By rooting the full 6-DoF grasp pose and width in the observed point cloud, we can reduce the dimensionality of our grasp representation to 4-DoF which greatly facilitates the learning process. Our class-agnostic approach is trained on 17 million simulated grasps and generalizes well to real world sensor data. In a robotic grasping study of unseen objects in structured clutter we achieve over 90% success rate, cutting the failure rate in half compared to a recent state-of-the-art method. Video of the real world experiments and code are available at https://research.nvidia. com/publication/2021-03_Contact-GraspNet%3A--Efficient.
Super-Resolving Compressed Video in Coding Chain

Scaling and lossy coding are widely used in video transmission and storage. Previous methods for enhancing the resolution of such videos often ignore the inherent interference between resolution loss and compression artifacts, which compromises perceptual video quality. To address this problem, we present a mixed-resolution coding framework, which cooperates with a reference-based DCNN. In this novel coding chain, the reference-based DCNN learns the direct mapping from low-resolution (LR) compressed video to their high-resolution (HR) clean version at the decoder side. We further improve reconstruction quality by devising an efficient deformable alignment module with receptive field block to handle various motion distances and introducing a disentangled loss that helps networks distinguish the artifact patterns from texture. Extensive experiments demonstrate the effectiveness of proposed innovations by comparing with state-of-the-art single image, video and reference-based restoration methods.
A local in time existence and uniqueness result of an inverse problem for the Kelvin-Voigt fluids

In this paper, we consider an inverse problem for three dimensional viscoelastic fluid flow equations, which arises from the motion of Kelvin–Voigt fluids in bounded domains. This inverse problem aims to reconstruct the velocity and kernel of the memory term simultaneously, from the measurement described as an integral overdetermination condition. By using the contraction mapping principle in an appropriate space, a local in time existence and uniqueness result for the inverse problem of Kelvin–Voigt fluids are obtained. Furthermore, using similar arguments, a global in time existence and uniqueness result for an inverse problem for Oseen type equations are also achieved.