Miracle-dz/UniFAR-FTUs huggingface dataset:
Entropy Regularised Deterministic Optimal Control: From Path Integral Solution to Sample-Based Trajectory Optimisation

Sample-based trajectory optimisers are a promising tool for the control of robotics with non-differentiable dynamics and cost functions. Contemporary approaches derive from a restricted subclass of stochastic optimal control where the optimal policy can be expressed in terms of an expectation over stochastic paths. By estimating the expectation with Monte Carlo sampling and reinterpreting the process as exploration noise, a stochastic search algorithm is obtained tailored to (deterministic) trajectory optimisation. For the purpose of future algorithmic development, it is essential to properly understand the underlying theoretical foundations that allow for a principled derivation of such methods. In this paper we make a connection between entropy regularisation in optimisation and deterministic optimal control. We then show that the optimal policy is given by a belief function rather than a deterministic function. The policy belief is governed by a Bayesian-type update where the likelihood can be expressed in terms of a conditional expectation over paths induced by a prior policy. Our theoretical investigation firmly roots sample-based trajectory optimisation in the larger family of control as inference. It allows us to justify a number of heuristics that are common in the literature and motivate a number of new improvements that benefit convergence.
Genomic data analysis using a two stage expectation propagation algorithm for analysis of sparse Bayesian high-dimensional instrumental variables regression

Simultaneous analysis of gene expression data and genetic variants is highly of interest, especially when the number of gene expressions and genetic variants are both greater than the sample size. Association of both causal genes and effective SNPs makes the use of sparse modeling of such genetic data sets, highly important. The high-dimensional sparse instrumental variables models are one of such useful association models, which models the simultaneous relation of the gene expressions and genetic variants with complex traits. From a Bayesian viewpoint, the sparsity can be favored using sparsity-enforcing priors such as spike-and-slab priors. A two-stage modification of the expectation propagation (EP) algorithm is proposed and examined for approximate inference in high-dimensional sparse instrumental variables models with spike-and-slab priors. This method is an adoption of the classical two-stage least squares method, to be used with the Bayes context. A simulation study is performed to examine the performance of the methods. The proposed method is applied to analysis of the mouse obesity data.
Variance function estimation in regression model via aggregation procedures

In the regression problem, we consider the problem of estimating the variance function by means of aggregation methods. We focus on two particular aggregation setting: Model Selection aggregation (MS) and Convex aggregation (C), where the goal is to select the best candidate and to build the best convex combination of candidates, respectively, among a collection of candidates. In both cases, the construction of the estimator relies on a two-step procedure and requires two independent samples. The first step exploits the first sample to build the candidate estimators for the variance function by the residual-based method and then the second dataset is used to perform the aggregation step. We show the consistency of the proposed method with respect to the -error both for MS and C aggregations. We evaluate the performance of these two methods in the heteroscedastic model and illustrate their interest in both the regression problem with reject option and the quantile regression.
Weakly-supervised Text Classification Based on Keyword Graph

Weakly-supervised text classification has received much attention in recent years for it can alleviate the heavy burden of annotating massive data. Among them, keyword-driven methods are the mainstream where user-provided keywords are exploited to generate pseudo-labels for unlabeled texts. However, existing methods treat keywords independently, thus ignore the correlation among them, which should be useful if properly exploited. In this paper, we propose a novel framework called ClassKG to explore keyword-keyword correlation on keyword graph by GNN. Our framework is an iterative process. In each iteration, we first construct a keyword graph, so the task of assigning pseudo labels is transformed to annotating keyword subgraphs. To improve the annotation quality, we introduce a self-supervised task to pretrain a subgraph annotator, and then finetune it. With the pseudo labels generated by the subgraph annotator, we then train a text classifier to classify the unlabeled texts. Finally, we re-extract keywords from the classified texts. Extensive experiments on both long-text and short-text datasets show that our method substantially outperforms the existing ones.
Multicolor Tur\'an numbers

We consider a natural generalisation of Turán’s forbidden subgraph problem and the RuzsaSzemerédi problem by studying the maximum number exF (n,G) of edge-disjoint copies of a fixed graph F can be placed on an n-vertex ground set without forming a subgraph G whose edges are from different F -copies. We determine the pairs {F,G} for which the order of magnitude of exF (n,G) is quadratic and prove several asymptotic results using various tools from the regularity lemma and supersaturation to graph packing results.
Modeling charge transport in gold nanogranular films

Cluster-assembled metallic films show interesting electrical properties, both in the near-topercolation regime, when deposited clusters do not form a complete layer yet, and when the film thickness is well above the electrical percolation threshold. Correctly estimating their electrical conductivity is crucial, but, particularly for the latter regime, standard theoretical tools are not quite adequate. We therefore developed a procedure based on an atomically informed mesoscopic model in which ab-initio estimates of electronic transport at the nanoscale are used to reconstruct the conductivity of nanogranular gold films generated by molecular dynamics. An equivalent resistor network is developed, appropriately accounting for ballistic transport. The method is shown to correctly capture the non-monotonic behavior of the conductivity as a function of the film thickness, namely a signature feature of nanogranular films.
Primordial black holes as dark matter candidates

We review the formation and evaporation of primordial black holes (PBHs) and their possible contribution to dark matter. Various constraints suggest they could only provide most of it in the mass windows 10^{17}\;1017 – 10^{23}\;1023g or 1010 – 10^2\;M_{\odot}102M⊙, with the last possibility perhaps being suggested by the LIGO/Virgo observations. However, PBHs could have important consequences even if they have a low cosmological density. Sufficiently large ones might generate cosmic structures and provide seeds for the supermassive black holes in galactic nuclei. Planck-mass relics of PBH evaporations or stupendously large black holes bigger than 10^{12}\;M_{\odot}1012M⊙ could also be an interesting dark component.
Pretrained Transformers for Offensive Language Identification in Tanglish

This paper describes the system submitted to Dravidian-Codemix-HASOC2021: Hate Speech and Offensive Language Identification in Dravidian Languages (Tamil-English and Malayalam-English). This task aims to identify offensive content in code-mixed comments/posts in Dravidian Languages collected from social media. Our approach utilizes pooling the last layers of pretrained transformer multilingual BERT for this task which helped us achieve rank nine on the leaderboard with a weighted average score of 0.61 for the Tamil-English dataset in subtask B. After the task deadline, we sampled the dataset uniformly and used the MuRIL pretrained model, which helped us achieve a weighted average score of 0.67, the top score in the leaderboard. Furthermore, our approach to utilizing the pretrained models helps reuse our models for the same task with a different dataset. Our code and models are available in https://github.com/seanbenhur/tanglish-offensive-language-identification
NON-PARAMETRIC INTERPRETABLE SCORE BASED ESTIMATION OF HETEROGENEOUS TREATMENT EFFECTS

In the study of causal inference, statisticians show growing interest in estimating and analyzing heterogeneity in causal effects in observational studies. However, there usually exists a trade-off between accuracy and interpretability when developing a desirable estimator for treatment effects. To make efforts to address the issue, we propose a non-parametric framework for estimating the Conditional Average Treatment Effect (CATE) function in this paper. The framework integrates two components: (i) leverage the joint use of propensity and prognostic scores in a matching algorithm to obtain a proxy of the heterogeneous treatment effects for each observation, (ii) utilize non-parametric regression trees to construct an estimator for the CATE function conditioning on the two scores. The method naturally stratifies treatment effects into subgroups over a 2d grid whose axis are the propensity and prognostic scores. We conduct benchmark experiments on multiple simulated data and demonstrate clear advantages of the proposed estimator over state of the art methods. We also evaluate empirical performance in real-life settings, using two observational social studies in the United States, and interpret policy implications following the numerical results. The R code for implementing the method introduced in the paper is publicly available on one of the author’s Github page (https://github.com/stevenysw/causal_pp).
An Unconstrained Layer-Peeled Perspective on Neural Collapse

Neural collapse is a highly symmetric geometric pattern of neural networks that emerges during the terminal phase of training, with profound implications on the generalization performance and robustness of the trained networks. To understand how the last-layer features and classifiers exhibit this recently discovered implicit bias, in this paper, we introduce a surrogate model called the unconstrained layer-peeled model (ULPM). We prove that gradient flow on this model converges to critical points of a minimum-norm separation problem exhibiting neural collapse in its global minimizer. Moreover, we show that the ULPM with the cross-entropy loss has a benign global landscape for its loss function, which allows us to prove that all the critical points are strict saddle points except the global minimizers that exhibit the neural collapse phenomenon. Empirically, we show that our results also hold during the training of neural networks in real-world tasks when explicit regularization or weight decay is not used.
Optical resonances in silica-nanosphere dimers and trimers probed with high-energy electrons at the nanoscale

Optical dielectric resonances have the ability to modulate optical energy, and by coupling multiple resonators into photonic molecules, new opportunities emerge in spectral engineering. Using electron-beam spectroscopy, in a 300-keV scanning transmission electron microscope, we probe optical modes of photonic molecules at the nanometer scale. Monochromated electron energy-loss spectra of 400nm-silica-sphere monomers, dimers and trimers display series of sharp peaks in the ultraviolet region. For monomers, the peaks are assigned to low angular number optical whispering gallery modes. For dimers and trimers, experimental observations, supported by simulations, reveal energy shifts and peak broadening resulting from coupling between spherical modes. Variations in spectra for different electron probe positions demonstrate the possibility to extract information on local photonic density of states at the nanometer scale.
Quantum Semi-Supervised Learning with Quantum Supremacy

Quantum machine learning promises to eﬃciently solve important problems. There are two persistent challenges in classical machine learning: the lack of labeled data, and the limit of computational power. We propose a novel framework that resolves both issues: quantum semi-supervised learning. Moreover, we provide a protocol in systematically designing quantum machine learning algorithms with quantum supremacy, which can be extended beyond quantum semi-supervised learning. In the meantime, we show that naive quantum matrix product estimation algorithm outperforms the best known classical matrix multiplication algorithm. We showcase two concrete quantum semi-supervised learning algorithms: a quantum self-training algorithm named the propagating nearest-neighbor classiﬁer, and the quantum semi-supervised K-means clustering algorithm. By doing time complexity analysis, we conclude that they indeed possess quantum supremacy.
Determining Joint Periodicities in Multi-Time Data with Sampling Uncertainties

In this work, we introduce a novel approach for determining a joint sparse spectrum from several non-uniformly sampled data sets, where each data set is assumed to have its own, and only partially known, sampling times. The problem originates in paleoclimatology, where each data point derives from a separate ice core measurement, resulting in that even though all measurements reflect the same periodicities, the sampling times and phases differ among the data sets, with the sampling times being only approximately known. The proposed estimator exploits all available data using a sparse reconstruction framework allowing for a reliable and robust estimation of the underlying periodicities. The performance of the method is illustrated using both simulated and measured ice core data sets.
Blind Super-Resolution via Projected Gradient Descent

Blind super-resolution can be cast as a low rank matrix recovery problem by exploiting the inherent simplicity of the signal. In this paper, we develop a simple yet efficient non-convex method for this problem based on the low rank structure of the vectorized Hankel matrix associated with the target matrix. Theoretical guarantees have been established under the similar conditions as convex approaches. Numerical experiments are also conducted to demonstrate its performance.
Improving Real-time Score Following in Opera by Combining Music with Lyrics Tracking

Fully automatic opera tracking is challenging because of the acoustic complexity of the genre, combining musical and linguistic information (singing, speech) in complex ways. In this paper, we propose a new pipeline for complete opera tracking. The pipeline is based on two trackers. A music tracker that has proven to be effective at tracking orchestral parts, will lead the tracking process. In addition, a lyrics tracker, that has recently been shown to reliably track the lyrics of opera songs, will correct the music tracker when tracking parts that have a text dominance over the music. We will demonstrate the efficiency of this method on the opera Don Giovanni, showing that this technique helps improving accuracy and robustness of a complete opera tracker.
Equivalent complex conductivities representing the effects of T-tubules and folded surface membranes on the electrical admittance and impedance of skeletal muscles measured by external-electrode method

In order to represent the effects of T-tubules and folded surface membranes on the electrical admittance and impedance of skeletal muscles measured by the external-electrode method, analytical relations for the equivalent complex conductivities of hypothetical smooth surface membranes were derived. In the relations, the effects of each tubule were represented by the admittance of a straight cable. The effects of the folding of a surface membrane were represented by the increased area of surface membranes. The equivalent complex conductivities were represented as summation of these effects, and the effects of the T-tubules were different between the transversal and longitudinal directions. The validity of the equivalent complex conductivities was supported by the results of finite-difference method (FDM) calculations made using three-dimensional models in which T-tubules and folded surface membranes were represented explicitly. FDM calculations using the equivalent complex conductivities suggested that the electrically inhomogeneous structure due to the existence of muscle cells with T-tubules was sufficient for explaining the experimental results previously obtained using the external-electrode method. Results of FDM calculations in which the structural changes caused by muscle contractions were taken into account were consistent with the reported experimental results.
Geometric Algebra Attention Networks for Small Point Clouds

Much of the success of deep learning is drawn from building architectures that properly respect underlying symmetry and structure in the data on which they operate - a set of considerations that have been united under the banner of geometric deep learning. Often problems in the physical sciences deal with relatively small sets of points in two- or three-dimensional space wherein translation, rotation, and permutation equivariance are important or even vital for models to be useful in practice. In this work, we present rotation- and permutation-equivariant architectures for deep learning on these small point clouds, composed of a set of products of terms from the geometric algebra and reductions over those products using an attention mechanism. The geometric algebra provides valuable mathematical structure by which to combine vector, scalar, and other types of geometric inputs in a systematic way to account for rotation invariance or covariance, while attention yields a powerful way to impose permutation equivariance. We demonstrate the usefulness of these architectures by training models to solve sample problems relevant to physics, chemistry, and biology.
Spectral Bias in Practice: The Role of Function Frequency in Generalization

Despite their ability to represent highly expressive functions, deep learning models seem to find simple solutions that generalize surprisingly well. Spectral bias -- the tendency of neural networks to prioritize learning low frequency functions -- is one possible explanation for this phenomenon, but so far spectral bias has primarily been observed in theoretical models and simplified experiments. In this work, we propose methodologies for measuring spectral bias in modern image classification networks on CIFAR-10 and ImageNet. We find that these networks indeed exhibit spectral bias, and that interventions that improve test accuracy on CIFAR-10 tend to produce learned functions that have higher frequencies overall but lower frequencies in the vicinity of examples from each class. This trend holds across variation in training time, model architecture, number of training examples, data augmentation, and self-distillation. We also explore the connections between function frequency and image frequency and find that spectral bias is sensitive to the low frequencies prevalent in natural images. On ImageNet, we find that learned function frequency also varies with internal class diversity, with higher frequencies on more diverse classes. Our work enables measuring and ultimately influencing the spectral behavior of neural networks used for image classification, and is a step towards understanding why deep models generalize well.
Cookie Banners, What's the Purpose?: Analyzing Cookie Banner Text Through a Legal Lens

A cookie banner pops up when a user visits a website for the first time, requesting consent to the use of cookies and other trackers for a variety of purposes. Unlike prior work that has focused on evaluating the user interface (UI) design of cookie banners, this paper presents an in-depth analysis of what cookie banners say to users to get their consent. We took an interdisciplinary approach to determiningwhat cookie banners should say. Following the legal requirements of the ePrivacy Directive (ePD) and the General Data Protection Regulation (GDPR), we manually annotated around 400 cookie banners presented on the most popular English-speaking websites visited by users residing in the EU. We focused on analyzing the purposes of cookie banners and how these purposes were expressed (e.g., any misleading or vague language, any use of jargon). We found that 89% of cookie banners violated applicable laws. In particular, 61% of banners violated the purpose specificity requirement by mentioning vague purposes, including "\emphuser experience enhancement ''. Further, 30% of banners used positive framing, breaching the freely given and informed consent requirements. Based on these findings, we provide recommendations that regulators can find useful. We also describe future research directions.
A Topological View of Rule Learning in Knowledge Graphs

Inductive relation prediction is an important learning task for knowledge graph completion. One can use the existence of rules, namely a sequence of relations, to predict the relation between two entities. Previous works view rules as paths and primarily focus on the searching of paths between entities. The space of paths is huge, and one has to sacrifice either efficiency or accuracy. In this paper, we consider rules in knowledge graphs as cycles and show that the space of cycles has a unique structure based on the theory of algebraic topology. By exploring the linear structure of the cycle space, we can improve the searching efficiency of rules. We propose to collect cycle bases that span the space of cycles. We build a novel GNN framework on the collected cycles to learn the representations of cycles, and to predict the existence/non-existence of a relation. Our method achieves state-of-the-art performance on benchmarks.
On the Impact of Stable Ranks in Deep Nets

A recent line of work has established intriguing connections between the generalization/compression properties of a deep neural network (DNN) model and the so-called layer weights' stable ranks. Intuitively, the latter are indicators of the effective number of parameters in the net. In this work, we address some natural questions regarding the space of DNNs conditioned on the layers' stable rank, where we study feed-forward dynamics, initialization, training and expressivity. To this end, we first propose a random DNN model with a new sampling scheme based on stable rank. Then, we show how feed-forward maps are affected by the constraint and how training evolves in the overparametrized regime (via Neural Tangent Kernels). Our results imply that stable ranks appear layerwise essentially as linear factors whose effect accumulates exponentially depthwise. Moreover, we provide empirical analysis suggesting that stable rank initialization alone can lead to convergence speed ups.
Towards Robotic Knee Arthroscopy: Multi-Scale Network for Tissue-Tool Segmentation

Tissue awareness has a great demand to improve surgical accuracy in minimally invasive procedures. In arthroscopy, it is one of the challenging tasks due to surgical sites exhibit limited features and textures. Moreover, arthroscopic surgical video shows high intra-class variations. Arthroscopic videos are recorded with endoscope known as arthroscope which records tissue structures at proximity, therefore, frames contain minimal joint structure. As consequences, fully conventional network-based segmentation model suffers from long- and short- term dependency problems. In this study, we present a densely connected shape aware multi-scale segmentation model which captures multi-scale features and integrates shape features to achieve tissue-tool segmentations. The model has been evaluated with three distinct datasets. Moreover, with the publicly available polyp dataset our proposed model achieved 5.09 % accuracy improvement.