Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning Context-Aware Representations of Subtrees

This thesis tackles the problem of learning efficient representations of complex, structured data with a natural application to web page and element classification. We hypothesise that the context around the element inside the web page is of high value to the problem and is currently under exploited. This thesis aims to solve the problem of classifying web elements as subtrees of a DOM tree by also considering their context. To achieve this, first we discuss current expert knowledge systems that work on structures, such as Tree-LSTM. Then, we propose context-aware extensions to this model. We show that the new model achieves an average F1-score of 0.7973 on a multi-class web classification task. This model generates better representations for various subtrees and may be used for applications such element classification, state estimators in reinforcement learning over the Web and more.
Off-policy Imitation Learning from Visual Inputs

Recently, various successful applications utilizing expert states in imitation learning (IL) have been witnessed. However, IL from visual inputs (ILfVI), which has a greater promise to be widely applied by using online visual resources, suffers from low data-efficiency and poor performance resulted from on-policy learning and high-dimensional visual inputs. We propose OPIfVI (Off-Policy Imitation from Visual Inputs), which is composed of an off-policy learning manner, data augmentation, and encoder techniques, to tackle the mentioned challenges, respectively. More specifically, to improve data-efficiency, OPIfVI conducts IL in an off-policy manner, with which sampled data used multiple times. In addition, we enhance the stability of OPIfVI with spectral normalization to mitigate the side effect of off-policy training. The core factor, contributing to the poor performance of ILfVI, that we think is agents could not extract meaningful features from visual inputs. Hence, OPIfVI employs data augmentation from computer vision to help train encoders to better extract features from visual inputs. Besides, a specific structure of gradient backpropagation for the encoder is designed to stabilize the encoder training. At last, we demonstrate that OPIfVI can achieve expert-level performance and outperform existing baselines via extensive experiments using DeepMind Control Suite.
Rate-optimal cluster-randomized designs for spatial interference

We consider a potential outcomes model in which interference may be present between any two units but the extent of interference diminishes with spatial distance. The causal estimand is the global average treatment effect, which compares counterfactual outcomes when all units are treated to those when none are. We study a class of designs in which space is partitioned into clusters that are randomized into treatment and control. For each design, we estimate the treatment effect using a Horvitz-Thompson estimator that compares the average outcomes of units with all neighbors treated to units with no treated neighbors, where the neighborhood radius is of the same order as the cluster size dictated by the design. We derive the estimator’s rate of convergence as a function of the design and degree of interference and use this to obtain estimator-design pairs that achieve near-optimal rates of convergence under relatively minimal assumptions on interference. We prove that the estimators are asymptotically normal and provide a variance estimator. For practical implementation of the designs, we suggest partitioning space using clustering algorithms. only be directly observed in the data under an extreme design that assigns all units to the same treatment arm, which would necessarily preclude observation of the other counterfactual. Common designs used in the literature, including those studied here, assign different units to different treatment arms, so neither average is directly observed in the data. Nonetheless, we show that asymptotic inference on θ n is possible for a class of cluster-randomized designs under spatial interference where the degree of interference diminishes with distance.
CORE: a COmplex event Recognition Engine

Complex Event Recognition (CER) systems are a prominent technology for finding user-defined query patterns over large data streams in real time. CER query evaluation is known to be computationally challenging, since it requires maintaining a set of partial matches, and this set quickly grows super-linearly in the number of processed events. We present CORE, a novel COmplex event Recognition Engine that focuses on the efficient evaluation of a large class of complex event queries, including time windows as well as the partition-by event correlation operator. This engine uses a novel automaton-based evaluation algorithm that circumvents the super-linear partial match problem: under data complexity, it takes constant time per input event to maintain a data structure that compactly represents the set of partial matches and, once a match is found, the query results may be enumerated from the data structure with output-linear delay. We experimentally compare CORE against state-of-the-art CER systems on real-world data. We show that (1) CORE's performance is stable with respect to both query and time window size, and (2) CORE outperforms the other systems by up to five orders of magnitude on different workloads.
Holographic Superconductors in a Non-minimally Coupled Einstein-Maxwell-scalar Model

In this paper, we investigate holographic superconductors dual to asymptotically anti-de Sitter black holes in an Einstein-Maxwell-scalar model with a non-minimal coupling between the scalar and Maxwell fields. In the probe limit, it shows that the scalar condensate occurs below the critical temperature $T_{c}$, and decreases with the increase of the coupling constant $\alpha$. On the other hand, the the critical temperature $T_{c}$ increases as the coupling constant $\alpha$ grows. We also calculate the optical conductivity of the holographic superconductor, and observe that a gap forms below $T_{c}$. Interestingly, the non-minimal coupling can lead to a spike occurring in the gap at a low temperature.
Content-aware media retargeting based on deep importance map

We present a neural network to estimate the visual information of important pixels in image and video, which is used in content-aware media retargeting applications. Existing techniques are successful in proposing retargeting methods. Yet, the serious distortion and the shrunk problem in the retargeted results still need to be investigated due to the limitations in the methods used to analyze visual attention. To accomplish this, we propose a network to define the importance map, which is sufficient to describe the energy of the significant regions in image/video. With this strategy, more ideal results are obtained from our system. Besides, the objective evaluation presented in this paper shows that our media retargeting system can achieve better and more plausible results than those of other works. In addition, our proposed importance map performs well in the enlarge operator and on the"difficult-to-resize"images.
Universal fine grained asymptotics of free and weakly coupled Quantum Field Theory

We give a rigorous proof that in any free quantum field theory with a finite group global symmetry $\mathrm{G}$, on a compact spatial manifold, at sufficiently high energy, the density of states $\rho_\alpha(E)$ for each irreducible representation $\alpha$ of $\mathrm{G}$ obeys a universal formula as conjectured by Harlow and Ooguri. We further prove that this continues to hold in a weakly coupled quantum field theory, given an appropriate scaling of the coupling with temperature. This generalizes similar results that were previously obtained in $(1+1)$-D to higher spacetime dimension. We discuss the role of averaging in the density of states, and we compare and contrast with the case of continuous group $\mathrm{G}$, where we prove a universal, albeit different, behavior.
Robust and Information-theoretically Safe Bias Classifier against Adversarial Attacks

In this paper, the bias classifier is introduced, that is, the bias part of a DNN with Relu as the activation function is used as a classifier. The work is motivated by the fact that the bias part is a piecewise constant function with zero gradient and hence cannot be directly attacked by gradient-based methods to generate adversaries, such as FGSM. The existence of the bias classifier is proved and an effective training method for the bias classifier is given. It is proved that by adding a proper random first-degree part to the bias classifier, an information-theoretically safe classifier against the original-model gradient attack is obtained in the sense that the attack will generate a totally random attacking direction. This seems to be the first time that the concept of information-theoretically safe classifier is proposed. Several attack methods for the bias classifier are proposed and numerical experiments are used to show that the bias classifier is more robust than DNNs with similar size against these attacks in most cases.
A space of goals: the cognitive geometry of informationally bounded agents

Traditionally, Euclidean geometry is treated by scientists as a priori and objective. However, when we take the position of an agent, the problem of selecting a best route should also factor in the abilities of the agent, its embodiment and particularly its cognitive effort. In this paper, we consider geometry in terms of travel between states within a world by incorporating information processing costs with the appropriate spatial distances. This induces a geometry that increasingly differs from the original geometry of the given world as information costs become increasingly important. We visualize this ‘cognitive geometry’ by projecting it onto two- and three-dimensional spaces showing distinct distortions reflecting the emergence of epistemic and information-saving strategies as well as pivot states. The analogies between traditional cost-based geometries and those induced by additional informational costs invite a generalization of the notion of geodesics as cheapest routes towards the notion of infodesics. In this perspective, the concept of infodesics is inspired by the property of geodesics that, travelling from a given start location to a given goal location along a geodesic, not only the goal, but all points along the way are visited at optimal cost from the start.
Linguistic Cues of Deception in a Multilingual April Fools' Day Context

In this work we consider the collection of deceptive April Fools' Day(AFD) news articles as a useful addition in existing datasets for deception detection tasks. Such collections have an established ground truth and are relatively easy to construct across languages. As a result, we introduce a corpus that includes diachronic AFD and normal articles from Greek newspapers and news websites. On top of that, we build a rich linguistic feature set, and analyze and compare its deception cues with the only AFD collection currently available, which is in English. Following a current research thread, we also discuss the individualism/collectivism dimension in deception with respect to these two datasets. Lastly, we build classifiers by testing various monolingual and crosslingual settings. The results showcase that AFD datasets can be helpful in deception detection studies, and are in alignment with the observations of other deception detection works.
Higher-dimensional black holes with multiple equal rotations

We study a limit of the Kerr–(A)dS spacetime in a general dimension where an arbitrary number of its rotational parameters is set equal. The resulting metric after the limit formally splits into two parts — the first part has the form of the Kerr–NUT–(A)dS metric analogous to the metric of the entire spacetime, but only for the directions not subject to the limit, and the second part can be interpreted as the Kähler metrics. However, this separation is not integrable, thus it does not lead to a product of independent manifolds. We also reconstruct the original number of explicit and hidden symmetries associated with Killing vectors and Killing tensors. Therefore, the resulting spacetime represents a special subcase of the generalized Kerr–NUT–(A)dS metric that retains the full Killing tower of symmetries. In D = 6, we present evidence of an enhanced symmetry structure after the limit. Namely, we find additional Killing vectors and show that one of the Killing tensors becomes reducible as it can be decomposed into Killing vectors.
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective

Federated Averaging (FedAvg), also known as Local SGD, is one of the most popular algorithms in Federated Learning (FL). Despite its simplicity and popularity, the convergence rate of FedAvg has thus far been undetermined. Even under the simplest assumptions (convex, smooth, homogeneous, and bounded covariance), the best-known upper and lower bounds do not match, and it is not clear whether the existing analysis captures the capacity of the algorithm. In this work, we first resolve this question by providing a lower bound for FedAvg that matches the existing upper bound, which shows the existing FedAvg upper bound analysis is not improvable. Additionally, we establish a lower bound in a heterogeneous setting that nearly matches the existing upper bound. While our lower bounds show the limitations of FedAvg, under an additional assumption of third-order smoothness, we prove more optimistic state-of-the-art convergence results in both convex and non-convex settings. Our analysis stems from a notion we call iterate bias, which is defined by the deviation of the expectation of the SGD trajectory from the noiseless gradient descent trajectory with the same initialization. We prove novel sharp bounds on this quantity, and show intuitively how to analyze this quantity from a Stochastic Differential Equation (SDE) perspective.
Bounds-Preserving Lax-Wendroff Discontinuous Galerkin Schemes for Quadrature-Based Moment-Closure Approximations of Kinetic Models

The quadrature-based method of moments (QMOM) offers a promising class of approximation techniques for reducing kinetic equations to fluid equations that are valid beyond thermodynamic equilibrium. A major challenge with these and other closures is that whenever the flux function must be evaluated (e.g., in a numerical update), a moment-inversion problem must be solved that computes the flux from the known input moments. In this work we study a particular five-moment variant of QMOM known as HyQMOM and establish that this system is momentinvertible over a convex region in solution space. We then develop a high-order Lax-Wendroff discontinuous Galerkin scheme for solving the resulting fluid system. The scheme is based on a predictor-corrector approach, where the prediction step is a localized space-time discontinuous Galerkin scheme. The nonlinear algebraic system that arises in this prediction step is solved using a Picard iteration. The correction step is a straightforward explicit update using the predicted solution in order to evaluate space-time flux integrals. In the absence of additional limiters, the proposed highorder scheme does not in general guarantee that the numerical solution remains in the convex set over which HyQMOM is moment-invertible. To overcome this challenge, ★ Corresponding author This research was partially funded by NSF Grants DMS–1620128 and DMS–2012699. Erica R. Johnson Southwest Research Institute, Space Science Department, 6220 Culebra Road, San Antonio, Texas 78238, USA E-mail: erica.johnson@swri.edu James A. Rossmanith Iowa State University, Department of Mathematics, 411 Morrill Road, Ames, Iowa 50011, USA E-mail: rossmani@iastate.edu Christine Vaughan Iowa State University, Department of Mathematics, 411 Morrill Road, Ames, Iowa 50011, USA E-mail: cwiersma@iastate.edu ar X iv :2 11 1. 03 70 9v 1 [ m at h. N A ] 5 N ov 2 02 1 2 E.R. Johnson, J.A. Rossmanith, and C. Vaughan we introduce novel limiters that rigorously guarantee that the computed solution does not leave the convex set over which moment-invertible and hyperbolicity of the fluid system is guaranteed. We develop positivity-preserving limiters in both the prediction and correction steps, as well as an oscillation-limiter that damps unphysical oscillations near shocks and rarefactions. Finally, we perform convergence tests to verify the order of accuracy of the scheme, as well as test the scheme on Riemann data to demonstrate the shock-capturing and robustness of the method.
Efficiently Learning Any One Hidden Layer ReLU Network From Queries

Model extraction attacks have renewed interest in the classic problem of learning neural networks from queries. In this work we give the first polynomial-time algorithm for learning arbitrary one hidden layer neural networks activations provided black-box access to the network. Formally, we show that if $F$ is an arbitrary one hidden layer neural network with ReLU activations, there is an algorithm with query complexity and running time that is polynomial in all parameters that outputs a network $F'$ achieving low square loss relative to $F$ with respect to the Gaussian measure. While a number of works in the security literature have proposed and empirically demonstrated the effectiveness of certain algorithms for this problem, ours is the first with fully polynomial-time guarantees of efficiency even for worst-case networks (in particular our algorithm succeeds in the overparameterized setting).
Minimum-lap-time Control Strategies for All-wheel Drive Electric Race Cars via Convex Optimization

This paper presents a convex optimization frame-work to compute the minimum-lap-time control strategies of all-wheel drive (AWD) battery electric race cars, accounting for the grip limitations of the individual tyres. Specifically, we first derive the equations of motion (EOM) of the race car and simplify them to a convex form. Second, we leverage convex models of the electric motors (EMs) and battery, and frame the time-optimal final-drives design and EMs control problem in space domain. The resulting optimization problem is fully convex and can be efficiently solved with global optimality guarantees using second-order conic programming algorithms. Finally, we validate our modeling assumptions via the original non-convex EOM, and simulate our framework on the Formula Student Netherlands endurance race track. Thereby, we compare a torque vectoring with a fixed power split configuration, showing that via torque vectoring we can make a better use of the individual tyre grip, and significantly improve the achievable lap time by more than 4%. Finally, we present a design study investigating the respective impact of the front and rear EM size on lap time, revealing that the rear motor sizing is predominant due to the higher vertical rear tyre load caused by the center of pressure position and rearwards load transfer under acceleration.
The Role of Pre-Training in High-Resolution Remote Sensing Scene Classification

Due to the scarcity of labeled data, using models pre-trained on ImageNet is a de facto standard in remote sensing scene classiﬁcation. Although, recently, several larger high resolution remote sensing (HRRS) datasets have appeared with a goal of establishing new benchmarks, attempts at training models from scratch on these datasets are sporadic. In this paper, we show that training models from scratch on several newer datasets yields comparable results to ﬁne-tuning the models pre-trained on ImageNet. Furthermore, the representations learned on HRRS datasets transfer to other HRRS scene classiﬁcation tasks better or at least similarly as those learned on ImageNet. Finally, we show that in many cases the best representations are obtained by using a second round of pre-training using in-domain data, i.e. domain-adaptive pre-training. The source code and pre-trained models are available at https: //github.com/risojevicv/RSSC-transfer.
A General Formulation for Evaluating the Performance of Linear Power Flow Models

 Abstract—Linear power flow (LPF) models are essential in power system analysis. Various LPF models are proposed, but some crucial questions are still remained: what is the performance bound (e.g., the error bound) of LPF models, how to know a branch is applicable for LPF models or not, and what is the best LPF model. In this paper, these crucial questions are answered and a general formulation (GF) for evaluating the performance of LPF models is proposed. The GF actually figure out two core difficulties, the one is how to define the definition range of the LPF models, and the second is how to analytically obtain the best LPF model and evaluate the performance of a given LPF model. Besides, the key factors that affect the performance of LPF models are also analyzed through the proposed framework. The case studies compare the proposed LPF model with the DC power flow model, the physical-model-driven LPF model, and the data-driven LPF model, and the results show the effectiveness as well as the superiority of the proposed method.
Near-Optimal Statistical Query Hardness of Learning Halfspaces with Massart Noise

We study the problem of PAC learning halfspaces with Massart noise. Given labeled samples $(x, y)$ from a distribution $D$ on $\mathbb{R}^{d} \times \{ \pm 1\}$ such that the marginal $D_x$ on the examples is arbitrary and the label $y$ of example $x$ is generated from the target halfspace corrupted by a Massart adversary with flipping probability $\eta(x) \leq \eta \leq 1/2$, the goal is to compute a hypothesis with small misclassification error. The best known $\mathrm{poly}(d, 1/\epsilon)$-time algorithms for this problem achieve error of $\eta+\epsilon$, which can be far from the optimal bound of $\mathrm{OPT}+\epsilon$, where $\mathrm{OPT} = \mathbf{E}_{x \sim D_x} [\eta(x)]$. While it is known that achieving $\mathrm{OPT}+o(1)$ error requires super-polynomial time in the Statistical Query model, a large gap remains between known upper and lower bounds. In this work, we essentially characterize the efficient learnability of Massart halfspaces in the Statistical Query (SQ) model. Specifically, we show that no efficient SQ algorithm for learning Massart halfspaces on $\mathbb{R}^d$ can achieve error better than $\Omega(\eta)$, even if $\mathrm{OPT} = 2^{-\log^{c} (d)}$, for any universal constant $c \in (0, 1)$. Furthermore, when the noise upper bound $\eta$ is close to $1/2$, our error lower bound becomes $\eta - o_{\eta}(1)$, where the $o_{\eta}(1)$ term goes to $0$ when $\eta$ approaches $1/2$. Our results provide strong evidence that known learning algorithms for Massart halfspaces are nearly best possible, thereby resolving a longstanding open problem in learning theory.
Masked Training of Neural Networks with Partial Gradients

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extragradient), limiting SGD updates to a subset of parameters for increased efficiency (such as meProp) or a combination of both (such as Dropout). However, the convergence of these methods is often not studied in theory. We propose a unified theoretical framework to study such SGD variants -- encompassing the aforementioned algorithms and additionally a broad variety of methods used for communication efficient training or model compression. Our insights can be used as a guide to improve the efficiency of such methods and facilitate generalization to new applications. As an example, we tackle the task of jointly training networks, a version of which (limited to sub-networks) is used to create Slimmable Networks. By training a low-rank Transformer jointly with a standard one we obtain superior performance than when it is trained separately.