Miracle-dz/UniFAR-FTUs huggingface dataset:
Unsupervised Skill Discovery with Bottleneck Option Learning

Having the ability to acquire inherent skills from environments without any external rewards or supervision like humans is an important problem. We propose a novel unsupervised skill discovery method named Information Bottleneck Option Learning (IBOL). On top of the linearization of environments that promotes more various and distant state transitions, IBOL enables the discovery of diverse skills. It provides the abstraction of the skills learned with the information bottleneck framework for the options with improved stability and encouraged disentanglement. We empirically demonstrate that IBOL outperforms multiple state-of-the-art unsupervised skill discovery methods on the information-theoretic evaluations and downstream tasks in MuJoCo environments, including Ant, HalfCheetah, Hopper and D'Kitty.
Feature Combination Meets Attention: Baidu Soccer Embeddings and Transformer based Temporal Detection

With rapidly evolving internet technologies and emerging tools, sports related videos generated online are increasing at an unprecedentedly fast pace. To automate sports video editing/highlight generation process, a key task is to precisely recognize and locate the events in the long untrimmed videos. In this tech report, we present a two-stage paradigm to detect what and when events happen in soccer broadcast videos. Specifically, we fine-tune multiple action recognition models on soccer data to extract high-level semantic features, and design a transformer based temporal detection module to locate the target events. This approach achieved the state-of-the-art performance in both two tasks, i.e., action spotting and replay grounding, in the SoccerNet-v2 Challenge, under CVPR 2021 ActivityNet workshop. Our soccer embedding features are released at https://github.com/baidu-research/vidpress-sports. By sharing these features with the broader community, we hope to accelerate the research into soccer video understanding.
XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages

Contemporary works on abstractive text summarization have focused primarily on high-resource languages like English, mostly due to the limited availability of datasets for low/mid-resource ones. In this work, we present XL-Sum, a comprehensive and diverse dataset comprising 1 million professionally annotated article-summary pairs from BBC, extracted using a set of carefully designed heuristics. The dataset covers 44 languages ranging from low to high-resource, for many of which no public dataset is currently available. XL-Sum is highly abstractive, concise, and of high quality, as indicated by human and intrinsic evaluation. We fine-tune mT5, a state-of-the-art pretrained multilingual model, with XL-Sum and experiment on multilingual and low-resource summarization tasks. XL-Sum induces competitive results compared to the ones obtained using similar monolingual datasets: we show higher than 11 ROUGE-2 scores on 10 languages we benchmark on, with some of them exceeding 15, as obtained by multilingual training. Additionally, training on low-resource languages individually also provides competitive performance. To the best of our knowledge, XL-Sum is the largest abstractive summarization dataset in terms of the number of samples collected from a single source and the number of languages covered. We are releasing our dataset and models to encourage future research on multilingual abstractive summarization. The resources can be found at \url{https://github.com/csebuetnlp/xl-sum}.
Midpoint Regularization: from High Uncertainty Training to Conservative Classification

Label Smoothing (LS) improves model generalization through penalizing models from generating overconfident output distributions. For each training sample the LS strategy smooths the one-hot encoded training signal by distributing its distribution mass over the non-ground truth classes. We extend this technique by considering example pairs, coined PLS. PLS first creates midpoint samples by averaging random sample pairs and then learns a smoothing distribution during training for each of these midpoint samples, resulting in midpoints with high uncertainty labels for training. We empirically show that PLS significantly outperforms LS, achieving up to 30% of relative classification error reduction. We also visualize that PLS produces very low winning softmax scores for both in and out of distribution samples.
The eROSITA Final Equatorial Depth Survey (eFEDS): The X-ray catalog

Context. The eROSITA X-ray telescope on board the Spectrum-Roentgen-Gamma (SRG) observatory combines a large field of view and collecting area in the energy range $\sim$0.2 to $\sim$8.0 keV with the capability to perform uniform scanning observations of large sky areas. Aims. SRG/eROSITA performed scanning observations of the $\sim$140 square degrees eROSITA Final Equatorial Depth Survey (eFEDS) field as part of its performance verification phase. The observing time was chosen to slightly exceed the depth of equatorial fields after the completion of the eROSITA all-sky survey. We present a catalog of detected X-ray sources in the eFEDS field providing source positions and extent information, as well as fluxes in multiple energy bands and document a suite of tools and procedures developed for eROSITA data processing and analysis, validated and optimized by the eFEDS work. Methods. A multi-stage source detection procedure was optimized and calibrated by performing realistic simulations of the eROSITA eFEDS observations. We cross-matched the eROSITA eFEDS source catalog with previous XMM-ATLAS observations, confirming excellent agreement of the eROSITA and XMM-ATLAS source fluxes. Result. We present a primary catalog of 27910 X-ray sources, including 542 with significant spatial extent, detected in the 0.2-2.3 keV energy range with detection likelihoods $\ge 6$, corresponding to a (point source) flux limit of $\approx 6.5 \times 10^{-15}$ erg/cm$^2$/s in the 0.5-2.0 keV energy band (80% completeness). A supplementary catalog contains 4774 low-significance source candidates with detection likelihoods between 5 and 6. In addition, a hard band sample of 246 sources detected in the energy range 2.3-5.0 keV above a detection likelihood of 10 is provided. The dedicated data analysis software package, calibration database, and calibrated data products are described in an appendix.
GSmart: An Efficient SPARQL Query Engine Using Sparse Matrix Algebra - Full Version

Ecient execution of SPARQL queries over large RDF datasets is a topic of considerable interest due to increased use of RDF to encode data. Most of this work has followed either relational or graph-based approaches. In this paper, we propose an alternative query engine, called gSmart, based on matrix algebra. This approach can potentially better exploit the computing power of high-performance heterogeneous architectures that we target. gSmart incorporates: (1) grouped incident edge-based SPARQL query evaluation, in which all unevaluated edges of a vertex are evaluated together using a series of matrix operations to fully utilize query constraints and narrow down the solution space; (2) a graph query planner that determines the order in which vertices in query graphs should be evaluated; (3) memoryand computation-ecient data structures including the light-weight sparse matrix (LSpM) storage for RDF data and the tree-based representation for evaluation results; (4) a multi-stage data partitioner to map the incident edge-based query evaluation into heterogeneous HPC architectures and develop multi-level parallelism; and (5) a parallel executor that uses the ne-grained processing scheme, pre-pruning technique, and tree-pruning technique to lower inter-node communication and enable high throughput. Evaluations of gSmart on a CPU+GPU HPC architecture show execution time speedups of up to 46920.00⇥ compared to the existing SPARQL query engines on a single node machine. Additionally, gSmart on the Tianhe-1A supercomputer achieves a maximum speedup of 6.90⇥ scaling from 2 to 16 CPU+GPU nodes. PVLDB Reference Format: Yuedan Chen, M. Tamer Özsu, Guoqing Xiao, Zhuo Tang, and Kenli Li. gSmart: An Ecient SPARQL Query Engine Using Sparse Matrix Algebra – Full Version. PVLDB, 15(1): XXX-XXX, 2021. doi:XX.XX/XXX.XX PVLDB Artifact Availability: The source code, data, and/or other artifacts have been made available at https://github.com/Danniedan/gSmart.
SDEs with critical time dependent drifts: Weak solutions

We prove the unique weak solvability of time-inhomogeneous stochastic differential equations with additive noises and drifts in critical Lebsgue space $L^q([0,T]; L^{p}(\mathbb{R}^d))$ with $d/p+2/q=1$. The weak uniqueness is obtained by solving corresponding Kolmogorov's backward equations in some second order Sobolev spaces, which is analytically interesting in itself.
Channel Pruning in a White Box for Efﬁcient Image Classiﬁcation

—Channel Pruning has been long studied to compress CNNs for efﬁcient image classiﬁcation. Prior works implement channel pruning in an unexplainable manner, which tends to reduce the ﬁnal classiﬁcation errors while failing to consider the internal inﬂuence of each channel. In this paper, we conduct channel pruning in a white box. Through deep visualization of feature maps activated by different channels, we observe that different channels have a varying contribution to different categories in image classiﬁcation. Inspired by this, we choose to preserve channels contributing to most categories. Speciﬁcally, to model the contribution of each channel to differentiating categories, we develop a class-wise mask for each channel, implemented in a dynamic training manner w.r. t. the input image’s category. On the basis of the learned class-wise mask, we perform a global voting mechanism to remove channels with less category discrimination. Lastly, a ﬁne-tuning process is conducted to recover the performance of the pruned model. To our best knowledge, it is the ﬁrst time that CNN interpretability theory is considered to guide channel pruning. Extensive experiments on representative image classiﬁcation tasks demonstrate the superiority of our White-Box over many state-of-the-arts. For instance, on CIFAR-10, it reduces 65.23% FLOPs with even 0.62% accuracy improvement for ResNet-110. On ILSVRC-2012, White-Box achieves a 45.6% FLOPs reduction with only a small loss of 0.83% in the top-1 accuracy for ResNet-50.
Quadratic First Integrals of Time-Dependent Dynamical Systems of the Form q¨a=-Γbcaq˙bq˙c-ω(t)Qa(q)

We consider the time-dependent dynamical system q¨a=−Γbcaq˙bq˙c−ω(t)Qa(q) where ω(t) is a non-zero arbitrary function and the connection coefficients Γbca are computed from the kinetic metric (kinetic energy) of the system. In order to determine the quadratic first integrals (QFIs) I we assume that I=Kabq˙aq˙b+Kaq˙a+K where the unknown coefficients Kab,Ka,K are tensors depending on t,qa and impose the condition dIdt=0. This condition leads to a system of partial differential equations (PDEs) involving the quantities Kab,Ka,K,ω(t) and Qa(q). From these PDEs, it follows that Kab is a Killing tensor (KT) of the kinetic metric. We use the KT Kab in two ways: a. We assume a general polynomial form in t both for Kab and Ka; b. We express Kab in a basis of the KTs of order 2 of the kinetic metric assuming the coefficients to be functions of t. In both cases, this leads to a new system of PDEs whose solution requires that we specify either ω(t) or Qa(q). We consider first that ω(t) is a general polynomial in t and find that in this case the dynamical system admits two independent QFIs which we collect in a Theorem. Next, we specify the quantities Qa(q) to be the generalized time-dependent Kepler potential V=−ω(t)rν and determine the functions ω(t) for which QFIs are admitted. We extend the discussion to the non-linear differential equation x¨=−ω(t)xμ+ϕ(t)x˙(μ≠−1) and compute the relation between the coefficients ω(t),ϕ(t) so that QFIs are admitted. We apply the results to determine the QFIs of the generalized Lane–Emden equation.
On holographic braneworld cosmology

Recently, S. Antonini and B. Swingle builded a holographic model for braneworld cosmology by introducing an ``end-of-the-world"(ETW) brane moving in a charged black hole spactime. In this paper, we will show that a holographic description of braneworld cosmology is possible for a general black hole spacetime with a pure tension brane if one implements a mixed boundary condition on the ETW brane. As a simple example, we study AdS-Schwarzschild black holes and show that a sensible Euclidean braneworld solution is compatible with localization of gravity on the ETW brane.
Slope detection and toroidal 3-manifolds

The $L$-space conjecture asserts the equivalence, for prime 3-manifolds, of three properties: not being an $L$-space, having a left-orderable fundamental group, and admitting a co-oriented taut foliation. We investigate these properties for toroidal $3$-manifolds using various notions of slope detection. Our main technical result gives sufficient conditions for certain slopes on the boundaries of rational homology solid tori to be detected by left-orders, foliations, and Heegaard Floer homology. This leads to a proof that toroidal integer homology spheres have left-orderable fundamental groups, as predicted by the $L$-space conjecture. It also allows us to show that the cyclic branched covers of prime satellite knots are not $L$-spaces and have left-orderable fundamental groups, as conjectured by Gordon and Lidman. Similarly we show that a cyclic branched cover of a satellite knot admits a co-oriented taut foliation when it has a fibred companion. A partial extension of these results to toroidal links leads to a proof that prime quasi-alternating links are either hyperbolic or $(2, m)$-torus links, which generalises Menasco's classical theorem that non-split alternating links are either hyperbolic or $(2, m)$-torus links.
Rethinking Token-Mixing MLP for MLP-based Vision Backbone

In the past decade, we have witnessed rapid progress in the machine vision backbone. By introducing the inductive bias from the image processing, convolution neural network (CNN) has achieved excellent performance in numerous computer vision tasks and has been established as \emph{de facto} backbone. In recent years, inspired by the great success achieved by Transformer in NLP tasks, vision Transformer models emerge. Using much less inductive bias, they have achieved promising performance in computer vision tasks compared with their CNN counterparts. More recently, researchers investigate using the pure-MLP architecture to build the vision backbone to further reduce the inductive bias, achieving good performance. The pure-MLP backbone is built upon channel-mixing MLPs to fuse the channels and token-mixing MLPs for communications between patches. In this paper, we re-think the design of the token-mixing MLP. We discover that token-mixing MLPs in existing MLP-based backbones are spatial-specific, and thus it is sensitive to spatial translation. Meanwhile, the channel-agnostic property of the existing token-mixing MLPs limits their capability in mixing tokens. To overcome those limitations, we propose an improved structure termed as Circulant Channel-Specific (CCS) token-mixing MLP, which is spatial-invariant and channel-specific. It takes fewer parameters but achieves higher classification accuracy on ImageNet1K benchmark.
Revelio: ML-Generated Debugging Queries for Distributed Systems

A major difficulty in debugging distributed systems lies in manually determining which of the many available debugging tools to use and how to query its logs. Our own study of a production debugging workflow confirms the magnitude of this burden. This paper explores whether a machine-learning model can assist developers in distributed systems debugging. We present Revelio, a debugging assistant which takes user reports and system logs as input, and outputs debugging queries that developers can use to find a bug's root cause. The key challenges lie in (1) combining inputs of different types (e.g., natural language reports and quantitative logs) and (2) generalizing to unseen faults. Revelio addresses these by employing deep neural networks to uniformly embed diverse input sources and potential queries into a high-dimensional vector space. In addition, it exploits observations from production systems to factorize query generation into two computationally and statistically simpler learning tasks. To evaluate Revelio, we built a testbed with multiple distributed applications and debugging tools. By injecting faults and training on logs and reports from 800 Mechanical Turkers, we show that Revelio includes the most helpful query in its predicted list of top-3 relevant queries 96% of the time. Our developer study confirms the utility of Revelio.
AMU-EURANOVA at CASE 2021 Task 1: Assessing the stability of multilingual BERT

This paper explains our participation in task 1 of the CASE 2021 shared task. This task is about multilingual event extraction from news. We focused on sub-task 4, event information extraction. This sub-task has a small training dataset and we fine-tuned a multilingual BERT to solve this sub-task. We studied the instability problem on the dataset and tried to mitigate it.
Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization

We study the asymmetric low-rank factorization problem: \[\min_{\mathbf{U} \in \mathbb{R}^{m \times d}, \mathbf{V} \in \mathbb{R}^{n \times d}} \frac{1}{2}\|\mathbf{U}\mathbf{V}^\top -\mathbf{\Sigma}\|_F^2\] where $\mathbf{\Sigma}$ is a given matrix of size $m \times n$ and rank $d$. This is a canonical problem that admits two difficulties in optimization: 1) non-convexity and 2) non-smoothness (due to unbalancedness of $\mathbf{U}$ and $\mathbf{V}$). This is also a prototype for more complex problems such as asymmetric matrix sensing and matrix completion. Despite being non-convex and non-smooth, it has been observed empirically that the randomly initialized gradient descent algorithm can solve this problem in polynomial time. Existing theories to explain this phenomenon all require artificial modifications of the algorithm, such as adding noise in each iteration and adding a balancing regularizer to balance the $\mathbf{U}$ and $\mathbf{V}$. This paper presents the first proof that shows randomly initialized gradient descent converges to a global minimum of the asymmetric low-rank factorization problem with a polynomial rate. For the proof, we develop 1) a new symmetrization technique to capture the magnitudes of the symmetry and asymmetry, and 2) a quantitative perturbation analysis to approximate matrix derivatives. We believe both are useful for other related non-convex problems.
Standardized Long Gamma-Ray Bursts as a Cosmic Distance Indicator

Gamma-ray bursts (GRBs) are the most luminous explosions in and can be detectable out to the edge of the universe. They have long been thought to be able to extend the Hubble diagram to very high redshifts. Several correlations between temporal or spectral properties and GRB luminosities have been proposed to make GRBs cosmological tools. However, those correlations cannot be properly standardized. In this paper, we select a long-GRB sample with X-ray plateau phases produced by electromagnetic dipole emissions from central newborn magnetars. A tight correlation is found between the plateau luminosity and the end time of the plateau in the X-ray afterglows out to the redshift z = 5.91. We standardize these long-GRB X-ray light curves to a universal behavior through this correlation, with a luminosity dispersion of 0.5 dex. The derived distance–redshift relation of GRBs is in agreement with the standard ΛCDM model both at low and high redshifts. The evidence for an accelerating universe from this GRB sample is 3σ, which is the highest statistical significance from GRBs to date.
A Knowledge-Grounded Dialog System Based on Pre-Trained Language Models

We present a knowledge-grounded dialog system developed for the ninth Dialog System Technology Challenge (DSTC9) Track 1 - Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access. We leverage transfer learning with existing language models to accomplish the tasks in this challenge track. Specifically, we divided the task into four sub-tasks and fine-tuned several Transformer models on each of the sub-tasks. We made additional changes that yielded gains in both performance and efficiency, including the combination of the model with traditional entity-matching techniques, and the addition of a pointer network to the output layer of the language model.
A Comprehensive Radiative Magnetohydrodynamics Simulation of Active Region Scale Flux Emergence from the Convection Zone to the Corona

We present a comprehensive radiative magnetohydrodynamic simulation of the quiet Sun and large solar active regions. The 197 Mm wide simulation domain spans from 18(10) Mm beneath the photosphere to 113 Mm in the solar corona. Radiative transfer assuming local thermal equilibrium, optically thin radiative losses, and anisotropic conduction transport provide the necessary realism for synthesizing observables to compare with remote-sensing observations of the photosphere and corona. This model self-consistently reproduces observed features of the quiet Sun, emerging and developed active regions, and solar flares up to M class. Here, we report an overview of the first results. The surface magneto-convection yields an upward Poynting flux that is dissipated in the corona and heats the plasma to over 1 MK. The quiescent corona also presents ubiquitous propagating waves, jets, and bright points with sizes down to 2 Mm. Magnetic flux bundles emerge into the photosphere and give rise to strong and complex active regions with over 1023 Mx magnetic flux. The coronal free magnetic energy, which is approximately 18% of the total magnetic energy, accumulates to approximately 1033 erg. The coronal magnetic field is clearly non-force-free, as the Lorentz force needs to balance the pressure force and viscous stress as well as drive magnetic field evolution. The emission measure from log10T=4.5 to log10T>7 provides a comprehensive view of the active region corona, such as coronal loops of various lengths and temperatures, mass circulation by evaporation and condensation, and eruptions from jets to large-scale mass ejections.
HALF: Holistic Auto Machine Learning for FPGAs

Deep Neural Networks (DNNs) are capable of solving complex problems in domains related to embedded systems, such as image and natural language processing. To efficiently implement DNNs on a specific FPGA platform for a given cost criterion, e.g., energy efficiency, an enormous amount of design parameters must be considered from the topology down to the final hardware implementation. Interdependencies between the different design layers must be taken into account and explored efficiently, making it hardly possible to find optimized solutions manually. An automatic, holistic design approach can improve the quality of DNN implementations on FPGA significantly. To this end, we present a cross-layer design space exploration methodology. It comprises optimizations starting from a hardware-aware topology search for DNNs down to the final optimized implementation for a given FPGA platform. The methodology is implemented in our Holistic Auto machine Learning for FPGAs (HALF) framework, which combines an evolutionary search algorithm, various optimization steps, and a library of parametrizable hardware DNN modules. HALF automates both the exploration process and the implementation of optimized solutions on a target FPGA platform for various applications. We demonstrate the performance of HALF on a medical use case for arrhythmia detection for three different design goals, i.e., low-energy, low-power, and high-throughput. Our FPGA implementation outperforms a TensorRT optimized model on an Nvidia Jetson platform in both throughput and energy consumption.