Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning-to-Learn Personalised Human Activity Recognition Models

Human Activity Recognition~(HAR) is the classification of human movement, captured using one or more sensors either as wearables or embedded in the environment~(e.g. depth cameras, pressure mats). State-of-the-art methods of HAR rely on having access to a considerable amount of labelled data to train deep architectures with many train-able parameters. This becomes prohibitive when tasked with creating models that are sensitive to personal nuances in human movement, explicitly present when performing exercises. In addition, it is not possible to collect training data to cover all possible subjects in the target population. Accordingly, learning personalised models with few data remains an interesting challenge for HAR research. We present a meta-learning methodology for learning to learn personalised HAR models for HAR; with the expectation that the end-user need only provides a few labelled data but can benefit from the rapid adaptation of a generic meta-model. We introduce two algorithms, Personalised MAML and Personalised Relation Networks inspired by existing Meta-Learning algorithms but optimised for learning HAR models that are adaptable to any person in health and well-being applications. A comparative study shows significant performance improvements against the state-of-the-art Deep Learning algorithms and the Few-shot Meta-Learning algorithms in multiple HAR domains.
Optimal Task Allocation for Mobile Edge Learning with Global Training Time Constraints

This paper proposes to maximize the accuracy of a distributed machine learning (ML) model trained on learners connected via the resource-constrained wireless edge. We jointly optimize the number of local/global updates and the task size allocation to minimize the loss while taking into account heterogeneous communication and computation capabilities of each learner. By leveraging existing bounds on the difference between the optimal and actual training loss, we derive an expression for the objective function in terms of the local updates. The resulting convex program is solved to obtain the optimal number of local updates which is used to obtain the total updates and batch sizes for each learner. The merits of the proposed solution, which is heterogeneity aware (HA), are exhibited by comparing its performance to the heterogeneity unaware (HU) approach.
Is the four-dimensional novel EGB theory equivalent to its regularized counterpart in a cylindrically symmetric spacetime?

Recently, a novel four-dimensional Einstein-Gauss-Bonnet (EGB) theory was presented to bypass the Lovelock's theorem and to give nontrivial effects on the four-dimensional local gravity. The main mechanism is to introduce a redefinition $\alpha\rightarrow\alpha/(D-4)$ and to take the limit $D\rightarrow4$. However, this theory does not have standard four-dimensional field equations. Some regularization produces are then proposed to address this problem [arXiv:2003.11552, arXiv:2003.12771, arXiv:2004.08362, arXiv:2004.09472, arXiv:2004.10716]. The resultant regularized four-dimensional EGB theory has the same on-shell action as the original theory. Thus it is expected that the novel four-dimensional EGB theory is equivalent to its regularized version. However, the equivalence of these two theories is symmetry-dependent. In this paper, we test the equivalence in a cylindrically symmetric spacetime. The well-defined field equations of the two theories are obtained, with which our follow-up analysis shows that they are equivalent in such spacetime. Cylindrical cosmic strings are then considered as specific examples of the metric. Three sets of solutions are obtained and the corresponding string mass densities are evaluated. The results reveal how the Gauss-Bonnet term in four dimensions contributes to the string geometry in the new theory.
Affective Conditioning on Hierarchical Networks applied to Depression Detection from Transcribed Clinical Interviews

In this work we propose a machine learning model for depression detection from transcribed clinical interviews. Depression is a mental disorder that impacts not only the subject's mood but also the use of language. To this end we use a Hierarchical Attention Network to classify interviews of depressed subjects. We augment the attention layer of our model with a conditioning mechanism on linguistic features, extracted from affective lexica. Our analysis shows that individuals diagnosed with depression use affective language to a greater extent than not-depressed. Our experiments show that external affective information improves the performance of the proposed architecture in the General Psychotherapy Corpus and the DAIC-WoZ 2017 depression datasets, achieving state-of-the-art 71.6 and 68.6 F1 scores respectively.
Adversarial Sparsity Attacks on Deep Neural Networks

Adversarial attacks have exposed serious vulnerabilities in Deep Neural Networks (DNNs) through their ability to force misclassifications through human-imperceptible perturbations to DNN inputs. We explore a new direction in the field of adversarial attacks by suggesting attacks that aim to degrade the computational efficiency of DNNs rather than their classification accuracy. Specifically, we propose and demonstrate sparsity attacks, which adversarial modify a DNN's inputs so as to reduce sparsity (or the presence of zero values) in its internal activation values. In resource-constrained systems, a wide range of hardware and software techniques have been proposed that exploit sparsity to improve DNN efficiency. The proposed attack increases the execution time and energy consumption of sparsity-optimized DNN implementations, raising concern over their deployment in latency and energy-critical applications. We propose a systematic methodology to generate adversarial inputs for sparsity attacks by formulating an objective function that quantifies the network's activation sparsity, and minimizing this function using iterative gradient-descent techniques. We launch both white-box and black-box versions of adversarial sparsity attacks on image recognition DNNs and demonstrate that they decrease activation sparsity by up to 1.82x. We also evaluate the impact of the attack on a sparsity-optimized DNN accelerator and demonstrate degradations up to 1.59x in latency, and also study the performance of the attack on a sparsity-optimized general-purpose processor. Finally, we evaluate defense techniques such as activation thresholding and input quantization and demonstrate that the proposed attack is able to withstand them, highlighting the need for further efforts in this new direction within the field of adversarial machine learning.
Computing Shortest Paths and Diameter in the Hybrid Network Model

The HYBRID model, introduced in [Augustine et al., SODA '20], provides a theoretical foundation for networks that allow multiple communication modes. The model follows the principles of synchronous message passing, whereas nodes are allowed to use two fundamentally different communication modes. First, a local mode where nodes may exchange arbitrary information per round over edges of a local communication graph G (akin to the LOCAL model). Second, a global mode where every node may exchange O(log n) messages of size O(log n) bits per round with arbitrary nodes in the network. The HYBRID model intends to reflect the conditions of many real hybrid networks, where high-bandwidth but inherently local communication is combined with highly flexible global communication with restricted bandwidth. We continue to explore the power and limitations of the HYBRID model by investigating the complexity of computing shortest paths and diameter of the local communication graph G. We show that the all pair shortest paths problem can be solved exactly in [EQUATION] rounds, which improves on the previous Õ(n2/3) round algorithm and closes the gap to the known [EQUATION] lower bound (up to polylog n factors). Furthermore, we give constant approximations for the k-source shortest paths problem (k-SSP) with runtime [EQUATION], provided that k is sufficiently large. As k-SSP has a lower bound of [EQUATION] even for large approximation ratios, our k-SSP algorithms are almost tight for large enough k. In the case of a single source we give an exact Õ(n2/5)-round algorithm, improving on the known [EQUATION]-round algorithm for graphs with large diameter D. For the diameter problem we provide algorithms with complexities Õ(n1/3/ε) and Õ(n0.397/ε) and approximation factors (3/2 + ε) and (1 + ε), respectively. On the negative side, we demonstrate that the classical 2-party set-disjointness framework can be adapted for the HYBRID model to prove a lower bound of [EQUATION] rounds for computing the diameter exactly. For the weighted diameter problem the same holds for computing (2 − ε)-approximations for any ε > 0.
Parametric Sequential Method for MRI-Based Wall Shear Stress Quantification

Wall shear stress (WSS) has been suggested as a potential biomarker in various cardiovascular diseases and it can be estimated from phase-contrast Magnetic Resonance Imaging (PC-MRI) velocity measurements. We present a parametric sequential method for MRI-based WSS quantification consisting of a geometry identification and a subsequent approximation of the velocity field. This work focuses on its validation, investigating well controlled high-resolution in vitro measurements of turbulent stationary flows and physiological pulsatile flows in phantoms. Initial tests for in vivo 2D PC-MRI data of the ascending aorta of three volunteers demonstrate basic applicability of the method to in vivo.
The intransitive dice kernel: $\frac{\mathbf{1}_{x\ge y}-\mathbf{1}_{x\le y}}{4} - \frac{3(x-y)(1+xy)}{8}$

Answering a pair of questions of Conrey, Gabbard, Grant, Liu, and Morrison, we prove that a triplet of dice drawn from the multiset model are intransitive with probability $1/4+o(1)$ and the probability a random pair of dice tie tends toward $\alpha n^{-1}$ for an explicitly defined constant $\alpha$. This extends and sharpens the recent results of Polymath regarding the balanced sequence model. We further show the distribution of larger tournaments converges to a universal tournamenton in both models. This limit naturally arises from the discrete spectrum of a certain skew-symmetric operator (given by the kernel in the title acting on $L^2([-1,1])$). The limit exhibits a degree of symmetry and can be used to prove that, for instance, the limiting probability that $A_i$ beats $A_{i+1}$ for $1\le i\le 4$ and that $A_5$ beats $A_1$ is $1/32+o(1)$. Furthermore, the limiting tournamenton has range contained in the discrete set $\{0,1\}$. This proves that the associated tournamenton is non-quasirandom in a dramatic fashion, vastly extending work of Cornacchia and H\k{a}z{\l}a regarding the continuous analogue of the balanced sequence model. The proof is based on a reduction to conditional central limit theorems (related to work of Polymath), the use of a"Poissonization"style method to reduce to computations with independent random variables, and the systematic use of switching-based arguments to extract cancellation in Fourier estimates when establishing local limit-type estimates.
Tight Nonparametric Convergence Rates for Stochastic Gradient Descent under the Noiseless Linear Model

In the context of statistical supervised learning, the noiseless linear model assumes that there exists a deterministic linear relation $Y = \langle \theta_*, X \rangle$ between the random output $Y$ and the random feature vector $\Phi(U)$, a potentially non-linear transformation of the inputs $U$. We analyze the convergence of single-pass, fixed step-size stochastic gradient descent on the least-square risk under this model. The convergence of the iterates to the optimum $\theta_*$ and the decay of the generalization error follow polynomial convergence rates with exponents that both depend on the regularities of the optimum $\theta_*$ and of the feature vectors $\Phi(u)$. We interpret our result in the reproducing kernel Hilbert space framework; as a special case, we analyze an online algorithm for estimating a real function on the unit interval from the noiseless observation of its value at randomly sampled points. The convergence depends on the Sobolev smoothness of the function and of a chosen kernel. Finally, we apply our analysis beyond the supervised learning setting to obtain convergence rates for the averaging process (a.k.a. gossip algorithm) on a graph depending on its spectral dimension.
Beyond Random Matrix Theory for Deep Networks

We investigate whether the Wigner semi-circle and Marcenko-Pastur distributions, often used for deep neural network theoretical analysis, match empirically observed spectral densities. We find that even allowing for outliers, the observed spectral shapes strongly deviate from such theoretical predictions. This raises major questions about the usefulness of these models in deep learning. We further show that theoretical results, such as the layered nature of critical points, are strongly dependent on the use of the exact form of these limiting spectral densities. We consider two new classes of matrix ensembles; random Wigner/Wishart ensemble products and percolated Wigner/Wishart ensembles, both of which better match observed spectra. They also give large discrete spectral peaks at the origin, providing a theoretical explanation for the observation that various optima can be connected by one dimensional of low loss values. We further show that, in the case of a random matrix product, the weight of the discrete spectral component at $0$ depends on the ratio of the dimensions of the weight matrices.
Learning Sparse Graphons and the Generalized Kesten-Stigum Threshold

The problem of learning graphons has attracted considerable attention across several scientific communities, with significant progress over the recent years in sparser regimes. Yet, the current techniques still require diverging degrees in order to succeed with efficient algorithms in the challenging cases where the local structure of the graph is homogeneous. This paper provides an efficient algorithm to learn graphons in the constant expected degree regime. The algorithm is shown to succeed in estimating the rank-$k$ projection of a graphon in the $L_2$ metric if the top $k$ eigenvalues of the graphon satisfy a generalized Kesten-Stigum condition.
Influence of powder-bed temperature on the microstructure and mechanical properties of Ti-6Al-4V produced by selective laser melting

Advanced characterisation techniques were used on LPBF Ti-6Al-4V samples produced on a heated base plate. When the substrate temperature is 100°C the elongation is 6\%, which increases and peaks at 10\% at 570°C, then sharply decreases to zero ductility at 770°C. At 100°C, a heavily strained and twinned microstructure, primarily composed of {\alpha}+{\alpha}', was observed and it was comparable to asbuilt microstructures obtained by conventional LPBF methods. At higher temperatures, twins are no longer present and instead nano-scale {\beta} precipitates are observed within {\alpha}' and {\alpha}, as well as dislocation networks (570°C) and tangles (770°C). Solute segregation at crystal defects was observed in all pre-heating conditions. Al and V segregation at microtwins was observed in the 100°C sample, reporting for the first time `selective' and mutually exclusive Al- and V-rich regions forming in adjacent twins. V segregation at dislocations was observed in the 570°C and 770°C samples, consistent with the higher preheating temperatures. High O contents were measured in all samples but with apparent opposing effects. At 100°C and 570°C was estimated to be below the critical threshold for O embrittlement and locally aids in maintaining a strength high by solid solution strengthening, whereas at 770°C it was above the threshold, therefore failing in a brittle fashion. Based on these observations, the initial increase in ductility from 100°C to 570°C is attributed to a reduction in microtwins and the dislocation networks acting as `soft barriers' for slip within a coarser microstructure. The lack of ductility at 770°C was attributed to local solute redistribution causing dislocation pinning and an increase of O content in this sample.
Quiver Representations and Dimension Reduction in Dynamical Systems

Dynamical systems often admit geometric properties that must be taken into account when studying their behaviour. We show that many such properties can be encoded by means of quiver representations. These properties include classical symmetry, hidden symmetry and feedforward structure, as well as subnetwork and quotient relations in network dynamical systems. A quiver equivariant dynamical system consists of a collection of dynamical systems with maps between them that send solutions to solutions. We prove that such quiver structures are preserved under Lyapunov-Schmidt reduction, center manifold reduction, and normal form reduction.
Markov-Lipschitz Deep Learning

We propose a novel framework, called Markov-Lipschitz deep learning (MLDL), to tackle geometric deterioration caused by collapse, twisting, or crossing in vector-based neural network transformations for manifold-based representation learning and manifold data generation. A prior constraint, called locally isometric smoothness (LIS), is imposed across-layers and encoded into a Markov random field (MRF)-Gibbs distribution. This leads to the best possible solutions for local geometry preservation and robustness as measured by locally geometric distortion and locally bi-Lipschitz continuity. Consequently, the layer-wise vector transformations are enhanced into well-behaved, LIS-constrained metric homeomorphisms. Extensive experiments, comparisons, and ablation study demonstrate significant advantages of MLDL for manifold learning and manifold data generation. MLDL is general enough to enhance any vector transformation-based networks. The code is available at this https URL.
ClusTR: Clustering Training for Robustness

This paper studies how encouraging semantically-aligned features during deep neural network training can increase network robustness. Recent works observed that Adversarial Training leads to robust models, whose learnt features appear to correlate with human perception. Inspired by this connection from robustness to semantics, we study the complementary connection: from semantics to robustness. To do so, we provide a tight robustness certificate for distance-based classification models (clustering-based classifiers), which we leverage to propose ClusTR (Clustering Training for Robustness), a clustering-based and adversary-free training framework to learn robust models. Interestingly, ClusTR outperforms adversarially-trained networks by up to 4\% under strong PGD attacks. Moreover, it can be equipped with simple and fast adversarial training to improve the current state-of-the-art in robustness by 16\%-29\% on CIFAR10, SVHN, and CIFAR100.
Self-Supervised Learning: Generative or Contrastive

Deep supervised learning has achieved great success in the last decade. However, its defects of heavy dependence on manual labels and vulnerability to attacks have driven people to find other paradigms. As an alternative, self-supervised learning (SSL) attracts many researchers for its soaring performance on representation learning in the last several years. Self-supervised representation learning leverages input data itself as supervision and benefits almost all types of downstream tasks. In this survey, we take a look into new self-supervised learning methods for representation in computer vision, natural language processing, and graph learning. We comprehensively review the existing empirical methods and summarize them into three main categories according to their objectives: generative, contrastive, and generative-contrastive (adversarial). We further collect related theoretical analysis on self-supervised learning to provide deeper thoughts on why self-supervised learning works. Finally, we briefly discuss open problems and future directions for self-supervised learning. An outline slide for the survey is provided<inline-formula><tex-math notation="LaTeX">$^1$</tex-math><alternatives><mml:math><mml:msup><mml:mrow/><mml:mn>1</mml:mn></mml:msup></mml:math><inline-graphic xlink:href="liu-ieq1-3090866.gif"/></alternatives></inline-formula>.
Estimation of a Function of Low Local Dimensionality by Deep Neural Networks

Deep neural networks (DNNs) achieve impressive results for complicated tasks like object detection on images and speech recognition. Motivated by this practical success, there is now a strong interest in showing good theoretical properties of DNNs. To describe for which tasks DNNs perform well and when they fail, it is a key challenge to understand their performance. The aim of this paper is to contribute to the current statistical theory of DNNs. We apply DNNs on high dimensional data and we show that the least squares regression estimates using DNNs are able to achieve dimensionality reduction in case that the regression function has locally low dimensionality. Consequently, the rate of convergence of the estimate does not depend on its input dimension <inline-formula> <tex-math notation="LaTeX">$d$ </tex-math></inline-formula>, but on its local dimension <inline-formula> <tex-math notation="LaTeX">$d^{*}$ </tex-math></inline-formula> and the DNNs are able to circumvent the curse of dimensionality in case that <inline-formula> <tex-math notation="LaTeX">$d^{*}$ </tex-math></inline-formula> is much smaller than <inline-formula> <tex-math notation="LaTeX">$d$ </tex-math></inline-formula>. In our simulation study we provide numerical experiments to support our theoretical result and we compare our estimate with other conventional nonparametric regression estimates. The performance of our estimates is also validated in experiments with real data.
The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry

The Fisher information matrix (FIM) is fundamental for understanding the trainability of deep neural networks (DNN) since it describes the local metric of the parameter space. We investigate the spectral distribution of the FIM given a single input by focusing on fully-connected networks achieving dynamical isometry. Then, while dynamical isometry is known to keep specific backpropagated signals independent of the depth, we find that the parameter space's local metric depends on the depth. In particular, we obtain an exact expression of the spectrum of the FIM given a single input and reveal that it concentrates around the depth point. Here, considering random initialization and the wide limit, we construct an algebraic methodology to examine the spectrum based on free probability theory, which is the algebraic wrapper of random matrix theory. As a byproduct, we provide the solvable spectral distribution in the two-hidden-layer case. Lastly, we empirically confirm that the spectrum of FIM with small batch-size has the same property as the single-input version. An experimental result shows that FIM's dependence on the depth determines the appropriate size of the learning rate for convergence at the initial phase of the online training of DNNs.