Miracle-dz/UniFAR-FTUs huggingface dataset:
Clustering Semantic Predicates in the Open Research Knowledge Graph

When semantically describing knowledge graphs (KGs), users have to make a critical choice of a vocabulary (i.e. predicates and re-sources). The success of KG building is determined by the convergence of shared vocabularies so that meaning can be established. The typical lifecycle for a new KG construction can be deﬁned as follows: nascent phases of graph construction experience terminology divergence, while later phases of graph construction experience terminology convergence and reuse. In this paper, we describe our approach tailoring two AI-based clustering algorithms for recommending predicates (in RDF statements) about resources in the Open Research Knowledge Graph (ORKG) https://orkg.org/. Such a service to recommend existing predicates to semantify new incoming data of scholarly publications is of paramount importance for fostering terminology convergence in the ORKG. Our experiments show very promising results: a high precision with relatively high recall in linear runtime performance. Furthermore, this work oﬀers novel insights into the predicate groups that automatically accrue loosely as generic semantiﬁcation patterns for semantiﬁcation of scholarly knowledge spanning 44 research ﬁelds.
Hypergraph patterns and collaboration structure

Humans collaborate in different contexts such as in creative or scientific projects, in workplaces and in sports. Depending on the project and external circumstances, a newly formed collaboration may include people that have collaborated before in the past, and people with no collaboration history. Such existing relationships between team members have been reported to influence the performance of teams. However, it is not clear how existing relationships between team members should be quantified, and whether some relationships are more likely to occur in new collaborations than others. Here we introduce a new family of structural patterns, m-patterns, which formalize relationships between collaborators and we study the prevalence of such structures in data and a simple random-hypergraph null model. We analyze the frequency with which different collaboration structures appear in our null model and show how such frequencies depend on size and hyperedge density in the hypergraphs. Comparing the null model to data of human and non-human collaborations, we find that some collaboration structures are vastly under- and overrepresented in empirical datasets. Finally, we find that structures of scientific collaborations on COVID-19 papers in some cases are statistically significantly different from those of non-COVID-19 papers. Examining citation counts for 4 different scientific fields, we also find indications that repeat collaborations are more successful for 2-author scientific publications and less successful for 3-author scientific publications as compared to other collaboration structures.
Flexible Instrumental Variable Models With Bayesian Additive Regression Trees

Methods utilizing instrumental variables have been a fundamental statistical approach to estimation in the presence of unmeasured confounding, usually occurring in non-randomized observational data common to fields such as economics and public health. However, such methods usually make constricting linearity and additivity assumptions that are inapplicable to the complex modeling challenges of today. The growing body of observational data being collected will necessitate flexible regression modeling while also being able to control for confounding using instrumental variables. Therefore, this article presents a nonlinear instrumental variable regression model based on Bayesian regression tree ensembles to estimate such relationships, including interactions, in the presence of confounding. One exciting application of this method is to use genetic variants as instruments, known as Mendelian randomization. Body mass index is one factor that is hypothesized to have a nonlinear relationship with cardiovascular risk factors such as blood pressure while interacting with age. Heterogeneity in patient characteristics such as age could be clinically interesting from a precision medicine perspective where individualized treatment is emphasized. We present our flexible Bayesian instrumental variable regression tree method with an example from the UK Biobank where body mass index is related to blood pressure using genetic variants as the instruments.
What and When to Look?: Temporal Span Proposal Network for Video Relation Detection

Identifying relations between objects is central to understanding the scene. While several works have been proposed for relation modeling in the image domain, there have been many constraints in the video domain due to challenging dynamics of spatio-temporal interactions (e.g., between which objects are there an interaction? when do relations start and end? ). To date, two representative methods have been proposed to tackle Video Visual Relation Detection (VidVRD): segment-based and window-based. We first point out limitations of these methods and propose a novel approach named Temporal Span Proposal Network (TSPN). TSPN tells what to look: it sparsifies relation search space by scoring relationness of object pair, i.e., measuring how probable a relation exist. TSPN tells when to look: it simultaneously predicts start-end timestamps (i.e., temporal spans) and categories of the all possible relations by utilizing full video context. These two designs enable a win-win scenario: it accelerates training by 2X or more than existing methods and achieves competitive performance on two VidVRD benchmarks (ImageNet-VidVDR and VidOR). Moreover, comprehensive ablative experiments demonstrate the effectiveness of our approach. Codes are available at https://github.com/sangminwoo/Temporal-Span-Proposal-Network-VidVRD.
A Counterexample to a Directed KKL Inequality

We show that the natural directed analogues of the KKL theorem [KKL88] and the Eldan--Gross inequality [EG20] from the analysis of Boolean functions fail to hold. This is in contrast to several other isoperimetric inequalities on the Boolean hypercube (such as the Poincare inequality, Margulis's inequality [Mar74] and Talagrand's inequality [Tal93]) for which directed strengthenings have recently been established.
H OW E RD ¨ OS AND R ´ ENYI W IN THE L OTTERY

Random masks define surprisingly effective sparse neural network models, as has been shown empirically. The resulting Erdös-Rényi (ER) random graphs can often compete with dense architectures and state-of-the-art lottery ticket pruning algorithms struggle to outperform them, even though the random baselines do not rely on computationally expensive pruning-training iterations but can be drawn initially without significant computational overhead. We offer a theoretical explanation of how such ER masks can approximate arbitrary target networks if they are wider by a logarithmic factor in the inverse sparsity 1/ log(1/sparsity). While we are the first to show theoretically and experimentally that random ER source networks contain strong lottery tickets, we also prove the existence of weak lottery tickets that require a lower degree of overparametrization than strong lottery tickets. These unusual results are based on the observation that ER masks are well trainable in practice, which we verify in experiments with varied choices of random masks. Some of these data-free choices outperform previously proposed random approaches on standard image classification benchmark datasets.
Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based Reinforcement Learning

We consider how to most efficiently leverage teleoperator time to collect data for learning robust image-based value functions and policies for sparse reward robotic tasks. To accomplish this goal, we modify the process of data collection to include more than just successful demonstrations of the desired task. Instead we develop a novel protocol that we call Visual Backtracking Teleoperation (VBT), which deliberately collects a dataset of visually similar failures, recoveries, and successes. VBT data collection is particularly useful for efficiently learning accurate value functions from small datasets of image-based observations. We demonstrate VBT on a real robot to perform continuous control from image observations for the deformable manipulation task of T-shirt grasping. We find that by adjusting the data collection process we improve the quality of both the learned value functions and policies over a variety of baseline methods for data collection. Specifically, we find that offline reinforcement learning on VBT data outperforms standard behavior cloning on successful demonstration data by 13 % when both methods are given equal-sized datasets of 60 minutes of data from the real robot.
MOA-2020-BLG-208Lb: Cool Sub-Saturn-mass Planet within Predicted Desert

We analyze the MOA-2020-BLG-208 gravitational microlensing event and present the discovery and characterization of a new planet, MOA-2020-BLG-208Lb, with an estimated sub-Saturn mass. With a mass ratio q=3.17−0.26+0.28×10−4 , the planet lies near the peak of the mass-ratio function derived by the MOA collaboration and near the edge of expected sample sensitivity. For these estimates we provide results using two mass-law priors: one assuming that all stars have an equal planet-hosting probability, and the other assuming that planets are more likely to orbit around more massive stars. In the first scenario, we estimate that the lens system is likely to be a planet of mass mplanet=46−24+42M⊕ and a host star of mass Mhost=0.43−0.23+0.39M⊙ , located at a distance DL=7.49−1.13+0.99kpc . For the second scenario, we estimate mplanet=69−34+37M⊕ , Mhost=0.66−0.32+0.35M⊙ , and DL=7.81−0.93+0.93kpc . The planet has a projected separation as a fraction of the Einstein ring radius s=1.3807−0.0018+0.0018 . As a cool sub-Saturn-mass planet, this planet adds to a growing collection of evidence for revised planetary formation models.
Improving Sentiment Analysis By Emotion Lexicon Approach on Vietnamese Texts

The sentiment analysis task has various applications in practice. In the sentiment analysis task, words and phrases that represent positive and negative emotions are important. Finding out the words that represent the emotion from the text can improve the performance of the classification models for the sentiment analysis task. In this paper, we propose a methodology that combines the emotion lexicon with the classification model to enhance the accuracy of the models. Our experimental results show that the emotion lexicon combined with the classification model improves the performance of models.
A quickest detection problem with false negatives

We formulate and solve a quickest detection problem with false negatives. A standard Brownian motion acquires a drift at an independent exponential random time which is not directly observable. Based on the observation in continuous time of the sample path of the process, an optimiser must detect the drift as quickly as possible after it has appeared. The optimiser can inspect the system multiple times upon payment of a fixed cost per inspection. If a test is performed on the system before the drift has appeared then, naturally, the test will return a negative outcome. However, if a test is performed after the drift has appeared, then the test may fail to detect it and return a false negative with probability $\epsilon\in(0,1)$. The optimisation ends when the drift is eventually detected. The problem is formulated mathematically as an optimal multiple stopping problem and it is shown to be equivalent to a recursive optimal stopping problem. Exploiting such connection and free boundary methods we find explicit formulae for the expected cost and the optimal strategy. We also show that when $\epsilon = 0$ our expected cost coincides with the one in Shiryaev's classical optimal detection problem.
Direct evidence of a charge depletion region at the interface of van der Waals monolayers and dielectric oxides: The case of superconducting FeSe/STO

The discovery of two dimensional Van der Waals materials has opened up several possibilities for designing novel devices. Yet a more promising way of designing exotic heterostutures with improved physical properties is to grow a monolayer of these materials on a substrate. For example, in the field of superconductivity it has been demonstrated that the superconducting transition temperature of a monolayer of FeSe grown on some oxide substrates e.g., strontium titanate (STO) is by far higher than its bulk counterpart. Although the system has been considered as a model system for understanding the phenomenon of high-temperature superconductivity, the physical mechanism responsible for this high transition temperature is still highly under debate. Here using momentum and energy resolved high-resolution electron energy-loss spectroscopy we probe the dynamic charge response of the FeSe/STO(001) system and demonstrate that the frequency- and momentum-dependent dynamic charge response is not compatible with a simple film/substrate model. Our analysis reveals the existence of a depletion region at the interface between this Van der Waals monolayer and the substrate. The presence of the depletion layer, accompanied with a considerably large charge transfer from STO into the FeSe monolayer, leads to a strong renormalization of the STO energy bands and a substantial band bending at the interface. Our results shed light on the electronic complexities of the FeSe/oxide interfaces and pave the way of designing novel low-dimensional high-temperature superconductors through interface engineering. We anticipate that the observed phenomenon is rather general and can take place in many two dimensional Van der Waals monolayers brought in contact with dielectric oxides or semiconducting substrates.
GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks

Time series synthesis is an important research topic in the field of deep learning, which can be used for data augmentation. Time series data types can be broadly classified into regular or irregular. However, there are no existing generative models that show good performance for both types without any model changes. Therefore, we present a general purpose model capable of synthesizing regular and irregular time series data. To our knowledge, we are the first designing a general purpose time series synthesis model, which is one of the most challenging settings for time series synthesis. To this end, we design a generative adversarial network-based method, where many related techniques are carefully integrated into a single framework, ranging from neural ordinary/controlled differential equations to continuous time-flow processes. Our method outperforms all existing methods.
Modelling tree survival for investigating climate change effects

Using German forest health monitoring data we investigate the main drivers leading to tree mortality and the association between defoliation and mortality; in particular (a) whether defoliation is a proxy for other covariates (climate, soil, water budget); (b) whether defoliation is a tree response that mitigates the effects of climate change and (c) whether there is a threshold of defoliation which could be used as an early warning sign for irreversible damage. Results show that environmental drivers leading to tree mortality differ by species, but some are always required in the model. The defoliation effect on mortality differs by species but it is always strong and monotonic. There is some evidence that a defoliation threshold exists for spruce, ﬁr and beech. We model tree survival with a smooth additive Cox model allowing for random effects taking care of dependence between neighbouring trees and non-linear functions of spatial time varying and functional predictors on defoliation, climate, soil and hydrology characteristics. Due to the large sample size and large number of parameters, we use parallel computing combined with marginal discretization of covariates. We propose a ’boost forward penalise backward’ model selection scheme based on combining component-wise gradient boosting with integrated backward selection.
Alternating Differentiation for Optimization Layers

The idea of embedding optimization problems into deep neural networks as optimization layers to encode constraints and inductive priors has taken hold in recent years. Most existing methods focus on implicitly differentiating Karush-Kuhn-Tucker (KKT) conditions in a way that requires expensive computations on the Jacobian matrix, which can be slow and memory-intensive. In this paper, we developed a new framework, named Alternating Differentiation (Alt-Diff), that differentiates optimization problems (here, specifically in the form of convex optimization problems with polyhedral constraints) in a fast and recursive way. Alt-Diff decouples the differentiation procedure into a primal update and a dual update in an alternating way. Accordingly, Alt-Diff substantially decreases the dimensions of the Jacobian matrix especially for optimization with large-scale constraints and thus increases the computational speed of implicit differentiation. We show that the gradients obtained by Alt-Diff are consistent with those obtained by differentiating KKT conditions. In addition, we propose to truncate Alt-Diff to further accelerate the computational speed. Under some standard assumptions, we show that the truncation error of gradients is upper bounded by the same order of variables' estimation error. Therefore, Alt-Diff can be truncated to further increase computational speed without sacrificing much accuracy. A series of comprehensive experiments validate the superiority of Alt-Diff.
Multiclass Learnability Beyond the PAC Framework: Universal Rates and Partial Concept Classes

In this paper we study the problem of multiclass classification with a bounded number of different labels $k$, in the realizable setting. We extend the traditional PAC model to a) distribution-dependent learning rates, and b) learning rates under data-dependent assumptions. First, we consider the universal learning setting (Bousquet, Hanneke, Moran, van Handel and Yehudayoff, STOC '21), for which we provide a complete characterization of the achievable learning rates that holds for every fixed distribution. In particular, we show the following trichotomy: for any concept class, the optimal learning rate is either exponential, linear or arbitrarily slow. Additionally, we provide complexity measures of the underlying hypothesis class that characterize when these rates occur. Second, we consider the problem of multiclass classification with structured data (such as data lying on a low dimensional manifold or satisfying margin conditions), a setting which is captured by partial concept classes (Alon, Hanneke, Holzman and Moran, FOCS '21). Partial concepts are functions that can be undefined in certain parts of the input space. We extend the traditional PAC learnability of total concept classes to partial concept classes in the multiclass setting and investigate differences between partial and total concepts.
Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning

Dataset discovery from data lakes is essential in many real application scenarios. In this paper, we propose Starmie, an end-to-end framework for dataset discovery from data lakes (with table union search as the main use case). Our proposed framework features a contrastive learning method to train column encoders from pre-trained language models in a fully unsupervised manner. The column encoder of Starmie captures the rich contextual semantic information within tables by leveraging a contrastive multi-column pre-training strategy. We utilize the cosine similarity between column embedding vectors as the column unionability score and propose a filter-and-verification framework that allows exploring a variety of design choices to compute the unionability score between two tables accordingly. Empirical results on real table benchmarks show that Starmie outperforms the best-known solutions in the effectiveness of table union search by 6.8 in MAP and recall. Moreover, Starmie is the first to employ the HNSW (Hierarchical Navigable Small World) index to accelerate query processing of table union search which provides a 3,000X performance gain over the linear scan baseline and a 400X performance gain over an LSH index (the state-of-the-art solution for data lake indexing).
ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild

Benchmarking initiatives support the meaningful comparison of competing solutions to prominent problems in speech and language processing. Successive benchmarking evaluations typically reflect a progressive evolution from ideal lab conditions towards to those encountered in the wild. ASVspoof, the spoofing and deepfake detection initiative and challenge series, has followed the same trend. This article provides a summary of the ASVspoof 2021 challenge and the results of 54 participating teams that submitted to the evaluation phase. For the logical access (LA) task, results indicate that countermeasures are robust to newly introduced encoding and transmission effects. Results for the physical access (PA) task indicate the potential to detect replay attacks in real, as opposed to simulated physical spaces, but a lack of robustness to variations between simulated and real acoustic environments. The Deepfake (DF) task, new to the 2021 edition, targets solutions to the detection of manipulated, compressed speech data posted online. While detection solutions offer some resilience to compression effects, they lack generalization across different source datasets. In addition to a summary of the top-performing systems for each task, new analyses of influential data factors and results for hidden data subsets, the article includes a review of post-challenge results, an outline of the principal challenge limitations and a road-map for the future of ASVspoof.
Fast prescramblers

We consider the process of diffusion or"pre-scrambling"of information in a quantum system. We define a measure for this spreading or"pre-scrambling"of the wavefunction in terms of a minimum probability threshold for the states in the system's Hilbert space. We illustrate our findings on the example of a prototype model with enhanced memory capacity. We conjecture: (1) The fastest pre-scramblers require a time logarithmic in the number of degrees of freedom. (2) The investigated enhanced memory capacity model is a fast pre-scrambler. (3) (Fast) pre-scrambling occurs not later than (fast) scrambling. (4) Fast scramblers are fast pre-scramblers. (5) Black holes are fast pre-scramblers.