Miracle-dz/UniFAR-FTUs huggingface dataset:
PhoBERT: Pre-trained language models for Vietnamese

We present PhoBERT with two versions, PhoBERT-base and PhoBERT-large, the first public large-scale monolingual language models pre-trained for Vietnamese. Experimental results show that PhoBERT consistently outperforms the recent best pre-trained multilingual model XLM-R (Conneau et al., 2020) and improves the state-of-the-art in multiple Vietnamese-specific NLP tasks including Part-of-speech tagging, Dependency parsing, Named-entity recognition and Natural language inference. We release PhoBERT to facilitate future research and downstream applications for Vietnamese NLP. Our PhoBERT models are available at https://github.com/VinAIResearch/PhoBERT
Ab initio path integral monte carlo simulation of the uniform electron gas in the high energy density regime

The response of the uniform electron gas (UEG) to an external perturbation is of paramount importance for many applications. Recently, highly accurate results for the static density response function and the corresponding local field correction have been provided both for warm dense matter [2019 J. Chem. Phys. 151 194104] and strongly coupled electron liquid [2020 Phys. Rev. B 101 045129] conditions based on exact ab initio path integral Monte Carlo (PIMC) simulations. In the present work, we further complete our current description of the UEG by exploring the high energy density regime, which is relevant for, e.g. astrophysical applications and inertial confinement fusion experiments. To this end, we present extensive new PIMC results for the static density response in the range of 0.05 ≤ rs≤ 0.5 and 0.85 ≤ θ ≤ 8. These data are subsequently used to benchmark the accuracy of the widely used random phase approximation and the dielectric theory by Singwi, Tosi, Land, and Sjölander (STLS). Moreover, we compare our results to configuration PIMC data where they are available and find perfect agreement with a relative accuracy of 0.001 − 0.01%. All PIMC data are available online.
Least-squares for linear elasticity eigenvalue problem

We study the approximation of the spectrum of least-squares operators arising from linear elasticity. We consider a two-field (stress/displacement) and a three-field (stress/displacement/vorticity) formulation; other formulations might be analyzed with similar techniques. We prove a priori estimates and we confirm the theoretical results with simple two-dimensional numerical experiments.
On the asymptotic properties of an interesting diffraction integral.

We introduce and study a new special integral, denoted $I_{+-}^{\varepsilon}$, depending on two complex parameters $\alpha_1$ and $\alpha_2$. It arises from the canonical problem of wave diffraction by a quarter-plane, and is heuristically constructed to capture the complex field near the tip and edges. We establish some region of analyticity of this integral in $\mathbb{C}^2$, and derive its rich asymptotic behaviour as $|\alpha_1|$ and $|\alpha_2 |$ tend to infinity. We also study the decay properties of the function obtained from applying a specific double Cauchy integral operator to this integral. These results allow us to show that this integral shares all of the asymptotic properties expected from the key unknown function $G_{+-}$ arising when the quarter-plane diffraction problem is studied via a two-complex-variables Wiener--Hopf technique (see Assier \& Abrahams, arXiv:1905.03863, 2019). As a result, the integral $I_{+ -}^{\varepsilon}$ can be used to mimic the unknown function $G_{+ -}$ and to build an efficient `educated' approximation to the quarter-plane problem.
Multi-Frequency Electromagnetic Tomography for Acute Stroke Detection Using Frequency-Constrained Sparse Bayesian Learning

Imaging the bio-impedance distribution of the brain can provide initial diagnosis of acute stroke. This paper presents a compact and non-radiative tomographic modality, i.e. multi-frequency Electromagnetic Tomography (mfEMT), for the initial diagnosis of acute stroke. The mfEMT system consists of 12 channels of gradiometer coils with adjustable sensitivity and excitation frequency. To solve the image reconstruction problem of mfEMT, we propose an enhanced Frequency-Constrained Sparse Bayesian Learning (FC-SBL) to simultaneously reconstruct the conductivity distribution at all frequencies. Based on the Multiple Measurement Vector (MMV) model in the Sparse Bayesian Learning (SBL) framework, FC-SBL can recover the underlying distribution pattern of conductivity among multiple images by exploiting the frequency constraint information. A realistic 3D head model was established to simulate stroke detection scenarios, showing the capability of mfEMT to penetrate the highly resistive skull and improved image quality with FC-SBL. Both simulations and experiments showed that the proposed FC-SBL method is robust to noisy data for image reconstruction problems of mfEMT compared to the single measurement vector model, which is promising to detect acute strokes in the brain region with enhanced spatial resolution and in a baseline-free manner.
Augmenting Visual Place Recognition With Structural Cues

In this letter, we propose to augment image-based place recognition with structural cues. Specifically, these structural cues are obtained using structure-from-motion, such that no additional sensors are needed for place recognition. This is achieved by augmenting the 2D convolutional neural network (CNN) typically used for image-based place recognition with a 3D CNN that takes as input a voxel grid derived from the structure-from-motion point cloud. We evaluate different methods for fusing the 2D and 3D features and obtain best performance with global average pooling and simple concatenation. On the Oxford RobotCar dataset, the resulting descriptor exhibits superior recognition performance compared to descriptors extracted from only one of the input modalities, including state-of-the-art image-based descriptors. Especially at low descriptor dimensionalities, we outperform state-of-the-art descriptors by up to 90%.
Hypernetwork approach to generating point clouds

In this work, we propose a novel method for generating 3D point clouds that leverage properties of hyper networks. Contrary to the existing methods that learn only the representation of a 3D object, our approach simultaneously finds a representation of the object and its 3D surface. The main idea of our HyperCloud method is to build a hyper network that returns weights of a particular neural network (target network) trained to map points from a uniform unit ball distribution into a 3D shape. As a consequence, a particular 3D shape can be generated using point-by-point sampling from the assumed prior distribution and transforming sampled points with the target network. Since the hyper network is based on an auto-encoder architecture trained to reconstruct realistic 3D shapes, the target network weights can be considered a parametrization of the surface of a 3D shape, and not a standard representation of point cloud usually returned by competitive approaches. The proposed architecture allows finding mesh-based representation of 3D objects in a generative manner while providing point clouds en pair in quality with the state-of-the-art methods.
Deep Variational Luenberger-type Observer for Stochastic Video Prediction

Considering the inherent stochasticity and uncertainty, predicting future video frames is exceptionally challenging. In this work, we study the problem of video prediction by combining interpretability of stochastic state space models and representation learning of deep neural networks. Our model builds upon an variational encoder which transforms the input video into a latent feature space and a Luenberger-type observer which captures the dynamic evolution of the latent features. This enables the decomposition of videos into static features and dynamics in an unsupervised manner. By deriving the stability theory of the nonlinear Luenberger-type observer, the hidden states in the feature space become insensitive with respect to the initial values, which improves the robustness of the overall model. Furthermore, the variational lower bound on the data log-likelihood can be derived to obtain the tractable posterior prediction distribution based on the variational principle. Finally, the experiments such as the Bouncing Balls dataset and the Pendulum dataset are provided to demonstrate the proposed model outperforms concurrent works.
Collective All‐Carbon Magnetism in Triangulene Dimers†

Abstract Triangular zigzag nanographenes, such as triangulene and its π‐extended homologues, have received widespread attention as organic nanomagnets for molecular spintronics, and may serve as building blocks for high‐spin networks with long‐range magnetic order, which are of immense fundamental and technological relevance. As a first step towards these lines, we present the on‐surface synthesis and a proof‐of‐principle experimental study of magnetism in covalently bonded triangulene dimers. On‐surface reactions of rationally designed precursor molecules on Au(111) lead to the selective formation of triangulene dimers in which the triangulene units are either directly connected through their minority sublattice atoms, or are separated via a 1,4‐phenylene spacer. The chemical structures of the dimers have been characterized by bond‐resolved scanning tunneling microscopy. Scanning tunneling spectroscopy and inelastic electron tunneling spectroscopy measurements reveal collective singlet–triplet spin excitations in the dimers, demonstrating efficient intertriangulene magnetic coupling.
ADF-GA: Data Flow Criterion Based Test Case Generation for Ethereum Smart Contracts

Testing is an important technique to improve the quality of Ethereum smart contract programs. However, current work on testing smart contract only focus on static problems of smart contract programs. A data flow oriented test case generation approach for dynamic testing of smart contract programs is still missing. To address this problem, this paper proposes a novel test case generation approach, called ADF-GA (All-uses Data Flow criterion based test case generation using Genetic Algorithm), for Solidity based Ethereum smart contract programs. ADF-GA aims to efficiently generate a valid set of test cases via three stages. First, the corresponding program control flow graph is constructed from the source codes. Second, the generated control flow graph is analyzed to obtain the variable information in the Solidity programs, locate the require statements, and also get the definition-use pairs to be tested. Finally, a genetic algorithm is used to generate test cases, in which an improved fitness function is proposed to calculate the definition-use pairs coverage of each test case with program instrumentation. Experimental studies are performed on several representative Solidity programs. The results show that ADF-GA can effectively generate test cases, achieve better coverage, and reduce the number of iterations in genetic algorithm.
3D Object Detection From LiDAR Data Using Distance Dependent Feature Extraction

This paper presents a new approach to 3D object detection that leverages the properties of the data obtained by a LiDAR sensor. State-of-the-art detectors use neural network architectures based on assumptions valid for camera images. However, point clouds obtained from LiDAR are fundamentally different. Most detectors use shared filter kernels to extract features which do not take into account the range dependent nature of the point cloud features. To show this, different detectors are trained on two splits of the KITTI dataset: close range (objects up to 25 meters from LiDAR) and long-range. Top view images are generated from point clouds as input for the networks. Combined results outperform the baseline network trained on the full dataset with a single backbone. Additional research compares the effect of using different input features when converting the point cloud to image. The results indicate that the network focuses on the shape and structure of the objects, rather than exact values of the input. This work proposes an improvement for 3D object detectors by taking into account the properties of LiDAR point clouds over distance. Results show that training separate networks for close-range and long-range objects boosts performance for all KITTI benchmark difficulties.
Identification of Primary and Collateral Tracks in Stuttered Speech

Disfluent speech has been previously addressed from two main perspectives: the clinical perspective focusing on diagnostic, and the Natural Language Processing (NLP) perspective aiming at modeling these events and detect them for downstream tasks. In addition, previous works often used different metrics depending on whether the input features are text or speech, making it difficult to compare the different contributions. Here, we introduce a new evaluation framework for disfluency detection inspired by the clinical and NLP perspective together with the theory of performance from (Clark, 1996) which distinguishes between primary and collateral tracks. We introduce a novel forced-aligned disfluency dataset from a corpus of semi-directed interviews, and present baseline results directly comparing the performance of text-based features (word and span information) and speech-based (acoustic-prosodic information). Finally, we introduce new audio features inspired by the word-based span features. We show experimentally that using these features outperformed the baselines for speech-based predictions on the present dataset.
A Hybrid Stochastic Policy Gradient Algorithm for Reinforcement Learning

We propose a novel hybrid stochastic policy gradient estimator by combining an unbiased policy gradient estimator, the REINFORCE estimator, with another biased one, an adapted SARAH estimator for policy optimization. The hybrid policy gradient estimator is shown to be biased, but has variance reduced property. Using this estimator, we develop a new Proximal Hybrid Stochastic Policy Gradient Algorithm (ProxHSPGA) to solve a composite policy optimization problem that allows us to handle constraints or regularizers on the policy parameters. We first propose a single-looped algorithm then introduce a more practical restarting variant. We prove that both algorithms can achieve the best-known trajectory complexity $\mathcal{O}\left(\varepsilon^{-3}\right)$ to attain a first-order stationary point for the composite problem which is better than existing REINFORCE/GPOMDP $\mathcal{O}\left(\varepsilon^{-4}\right)$ and SVRPG $\mathcal{O}\left(\varepsilon^{-10/3}\right)$ in the non-composite setting. We evaluate the performance of our algorithm on several well-known examples in reinforcement learning. Numerical results show that our algorithm outperforms two existing methods on these examples. Moreover, the composite settings indeed have some advantages compared to the non-composite ones on certain problems.
Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation

Since annotating pixel-level labels for semantic segmentation is laborious, leveraging synthetic data is an attractive solution. However, due to the domain gap between synthetic domain and real domain, it is challenging for a model trained with synthetic data to generalize to real data. In this paper, considering the fundamental difference between the two domains as the texture, we propose a method to adapt to the target domain's texture. First, we diversity the texture of synthetic images using a style transfer algorithm. The various textures of generated images prevent a segmentation model from overfitting to one specific (synthetic) texture. Then, we fine-tune the model with self-training to get direct supervision of the target texture. Our results achieve state-of-the-art performance and we analyze the properties of the model trained on the stylized dataset with extensive experiments.
Multi-View Learning for Vision-and-Language Navigation

Learning to navigate in a visual environment following natural language instructions is a challenging task because natural language instructions are highly variable, ambiguous, and under-specified. In this paper, we present a novel training paradigm, Learn from EveryOne (LEO), which leverages multiple instructions (as different views) for the same trajectory to resolve language ambiguity and improve generalization. By sharing parameters across instructions, our approach learns more effectively from limited training data and generalizes better in unseen environments. On the recent Room-to-Room (R2R) benchmark dataset, LEO achieves 16% improvement (absolute) over a greedy agent as the base agent (25.3% $\rightarrow$ 41.4%) in Success Rate weighted by Path Length (SPL). Further, LEO is complementary to most existing models for vision-and-language navigation, allowing for easy integration with the existing techniques, leading to LEO+, which creates the new state of the art, pushing the R2R benchmark to 62% (9% absolute improvement).
Hypocoercivity with Schur complements

We propose an approach to directly obtaining estimates on the resolvent of hypocoercive operators, using Schur complements, rather than obtaining bounds on the resolvent from an exponential decay of the evolution semigroup. We present applications to Langevin-like dynamics and Fokker--Planck equations, as well as the linear Boltzmann equation (which is also the generator of randomized Hybrid Monte Carlo in molecular dynamics). In particular, we make precise the dependence of the resolvent bounds on the parameters of the dynamics and on the dimension. We also highlight relationships with other hypocoercive approaches.
The STEM-ECR Dataset: Grounding Scientific Entity References in STEM Scholarly Content to Authoritative Encyclopedic and Lexicographic Sources

We introduce the STEM (Science, Technology, Engineering, and Medicine) Dataset for Scientific Entity Extraction, Classification, and Resolution, version 1.0 (STEM-ECR v1.0). The STEM-ECR v1.0 dataset has been developed to provide a benchmark for the evaluation of scientific entity extraction, classification, and resolution tasks in a domain-independent fashion. It comprises abstracts in 10 STEM disciplines that were found to be the most prolific ones on a major publishing platform. We describe the creation of such a multidisciplinary corpus and highlight the obtained findings in terms of the following features: 1) a generic conceptual formalism for scientific entities in a multidisciplinary scientific context; 2) the feasibility of the domain-independent human annotation of scientific entities under such a generic formalism; 3) a performance benchmark obtainable for automatic extraction of multidisciplinary scientific entities using BERT-based neural models; 4) a delineated 3-step entity resolution procedure for human annotation of the scientific entities via encyclopedic entity linking and lexicographic word sense disambiguation; and 5) human evaluations of Babelfy returned encyclopedic links and lexicographic senses for our entities. Our findings cumulatively indicate that human annotation and automatic learning of multidisciplinary scientific concepts as well as their semantic disambiguation in a wide-ranging setting as STEM is reasonable.
Benchmarking Graph Neural Networks

Graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. As the field grows, it becomes critical to identify key architectures and validate new ideas that generalize to larger, more complex datasets. Unfortunately, it has been increasingly difficult to gauge the effectiveness of new models in the absence of a standardized benchmark with consistent experimental settings. In this paper, we introduce a reproducible GNN benchmarking framework, with the facility for researchers to add new models conveniently for arbitrary datasets. We demonstrate the usefulness of our framework by presenting a principled investigation into the recent Weisfeiler-Lehman GNNs (WL-GNNs) compared to message passing-based graph convolutional networks (GCNs) for a variety of graph tasks, i.e. graph regression/classification and node/link prediction, with medium-scale datasets.
Cluster-Based Social Reinforcement Learning

Social Reinforcement Learning methods, which model agents in large networks, are useful for fake news mitigation, personalized teaching/healthcare, and viral marketing, but it is challenging to incorporate inter-agent dependencies into the models effectively due to network size and sparse interaction data. Previous social RL approaches either ignore agents dependencies or model them in a computationally intensive manner. In this work, we incorporate agent dependencies efficiently in a compact model by clustering users (based on their payoff and contribution to the goal) and combine this with a method to easily derive personalized agent-level policies from cluster-level policies. We also propose a dynamic clustering approach that captures changing user behavior. Experiments on real-world datasets illustrate that our proposed approach learns more accurate policy estimates and converges more quickly, compared to several baselines that do not use agent correlations or only use static clusters.
Relational Deep Feature Learning for Heterogeneous Face Recognition

Heterogeneous Face Recognition (HFR) is a task that matches faces across two different domains such as visible light (VIS), near-infrared (NIR), or the sketch domain. Due to the lack of databases, HFR methods usually exploit the pre-trained features on a large-scale visual database that contain general facial information. However, these pre-trained features cause performance degradation due to the texture discrepancy with the visual domain. With this motivation, we propose a graph-structured module called Relational Graph Module (RGM) that extracts global relational information in addition to general facial features. Because each identity’s relational information between intra-facial parts is similar in any modality, the modeling relationship between features can help cross-domain matching. Through the RGM, relation propagation diminishes texture dependency without losing its advantages from the pre-trained features. Furthermore, the RGM captures global facial geometrics from locally correlated convolutional features to identify long-range relationships. In addition, we propose a Node Attention Unit (NAU) that performs node-wise recalibration to concentrate on the more informative nodes arising from relation-based propagation. Furthermore, we suggest a novel conditional-margin loss function ( $C$ -softmax) for the efficient projection learning of the embedding vector in HFR. The proposed method outperforms other state-of-the-art methods on five HFR databases. Furthermore, we demonstrate performance improvement on three backbones because our module can be plugged into any pre-trained face recognition backbone to overcome the limitations of a small HFR database.