Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning Shared Filter Bases for Efficient ConvNets

Modern convolutional neural networks (ConvNets) achieve state-of-the-art performance for many computer vision tasks. However, such high performance requires millions of parameters and high computational costs. Recently, inspired by the iterative structure of modern ConvNets, such as ResNets, parameter sharing among repetitive convolution layers has been proposed to reduce the size of parameters. However, naive sharing of convolution filters poses many challenges such as overfitting and vanishing/exploding gradients. Furthermore, parameter sharing often increases computational complexity due to additional operations. In this paper, we propose to exploit the linear structure of convolution filters for effective and efficient sharing of parameters among iterative convolution layers. Instead of sharing convolution filters themselves, we hypothesize that a filter basis of linearly-decomposed convolution layers is a more effective unit for sharing parameters since a filter basis is an intrinsic and reusable building block constituting diverse high dimensional convolution filters. The representation power and peculiarity of individual convolution layers are further increased by adding a small number of layer-specific non-shared components to the filter basis. We show empirically that enforcing orthogonality to shared filter bases can mitigate the difficulty in training shared parameters. Experimental results show that our approach achieves significant reductions both in model parameters and computational costs while maintaining competitive, and often better, performance than non-shared baseline networks.
Weak c-ideals of Lie algebras

A subalgebra B of a Lie algebra L is called a weak c-ideal of L if there is a subideal C of L such that L = B+C and B\cap C \subseteq B_L where B_L is the largest ideal of L contained in B. This is analogous to the concept of weakly c- normal subgroups, which has been studied by a number of authors. We obtain some properties of weak c-ideals and use them to give some characterisations of solvable and supersolvable Lie algebras. We also note that one-dimensional weak c-ideals are c-ideals.
Unique Faces Recognition in Videos

This paper tackles face recognition in videos employing metric learning methods and similarity ranking models. The paper compares the use of the Siamese network with contrastive loss and Triplet Network with triplet loss implementing the following architectures: Google/Inception architecture, 3D Convolutional Network (C3D), and a 2-D Long short-term memory (LSTM) Recurrent Neural Network. We make use of still images and sequences from videos for training the networks and compare the performances implementing the above architectures. The dataset used was the YouTube Face Database designed for investigating the problem of face recognition in videos. The contribution of this paper is two-fold: to begin, the experiments have established 3-D Convolutional networks and 2-D LSTMs with the contrastive loss on image sequences do not outperform Google/Inception architecture with contrastive loss in top $n$ rank face retrievals with still images. However, the 3-D Convolution networks and 2-D LSTM with triplet Loss outperform the Google/Inception with triplet loss in top $n$ rank face retrievals on the dataset; second, a Support Vector Machine (SVM) was used in conjunction with the CNNs' learned feature representations for facial identification. The results show that feature representation learned with triplet loss is significantly better for n-shot facial identification compared to contrastive loss. The most useful feature representations for facial identification are from the 2-D LSTM with triplet loss. The experiments show that learning spatio-temporal features from video sequences is beneficial for facial recognition in videos.
On the Bottleneck of Graph Neural Networks and its Practical Implications

Graph neural networks (GNNs) were shown to effectively learn from highly structured data containing elements (nodes) with relationships (edges) between them. GNN variants differ in how each node in the graph absorbs the information flowing from its neighbor nodes. In this paper, we highlight an inherent problem in GNNs: the mechanism of propagating information between neighbors creates a bottleneck when every node aggregates messages from its neighbors. This bottleneck causes the over-squashing of exponentially-growing information into fixed-size vectors. As a result, the graph fails to propagate messages flowing from distant nodes and performs poorly when the prediction task depends on long-range information. We demonstrate that the bottleneck hinders popular GNNs from fitting the training data. We show that GNNs that absorb incoming edges equally, like GCN and GIN, are more susceptible to over-squashing than other GNN types. We further show that existing, extensively-tuned, GNN-based models suffer from over-squashing and that breaking the bottleneck improves state-of-the-art results without any hyperparameter tuning or additional weights.
Stochastic matrix games with bandit feedback

We study a version of the classical zero-sum matrix game with unknown payoff matrix and bandit feedback, where the players only observe each others actions and a noisy payoff. This generalizes the usual matrix game, where the payoff matrix is known to the players. Despite numerous applications, this problem has received relatively little attention. Although adversarial bandit algorithms achieve low regret, they do not exploit the matrix structure and perform poorly relative to the new algorithms. The main contributions are regret analyses of variants of UCB and K-learning that hold for any opponent, e.g., even when the opponent adversarially plays the best response to the learner's mixed strategy. Along the way, we show that Thompson fails catastrophically in this setting and provide empirical comparison to existing algorithms.
Graph Learning Under Partial Observability

Many optimization, inference, and learning tasks can be accomplished efficiently by means of decentralized processing algorithms where the network topology (i.e., the graph) plays a critical role in enabling the interactions among neighboring nodes. There is a large body of literature examining the effect of the graph structure on the performance of decentralized processing strategies. In this article, we examine the inverse problem and consider the reverse question: How much information does observing the behavior at the nodes of a graph convey about the underlying topology? For large-scale networks, the difficulty in addressing such inverse problems is compounded by the fact that usually only a limited fraction of the nodes can be probed, giving rise to a second important question: Despite the presence of unobserved nodes, can partial observations still be sufficient to discover the graph linking the probed nodes? The article surveys recent advances on this challenging learning problem and related questions.
Random Van der Waerden Theorem

In this paper, we prove a sparse random analogue of the Van der Waerden Theorem. We show that, for all $r > 2$ and all $q_1 \geq q_2 \geq \dotsb \geq q_r \geq 3 \in \mathbb{N}$, $n^{-\frac{q_2}{q_1(q_2-1)}}$ is a threshold for the following property: For every $r$-coloring of the $p$-random subset of $\{1,\dotsc,n\}$, there exists a monochromatic $q_i$-term arithmetic progression colored $i$, for some $i$. This extends the results of Rödl and Ruciński for the symmetric case $q_1 = q_2 = \dotsb = q_r$. The proof of the 1-statement is based on the Hypergraph Container Method by Balogh, Morris and Samotij and Saxton and Thomason. The proof of the 0-statement is an extension of Rödl and Ruciński's argument for the symmetric case.
A Survey on Generative Adversarial Networks: Variants, Applications, and Training

The Generative Models have gained considerable attention in unsupervised learning via a new and practical framework called Generative Adversarial Networks (GAN) due to their outstanding data generation capability. Many GAN models have been proposed, and several practical applications have emerged in various domains of computer vision and machine learning. Despite GANs excellent success, there are still obstacles to stable training. The problems are Nash equilibrium, internal covariate shift, mode collapse, vanishing gradient, and lack of proper evaluation metrics. Therefore, stable training is a crucial issue in different applications for the success of GANs. Herein, we survey several training solutions proposed by different researchers to stabilize GAN training. We discuss (I) the original GAN model and its modified versions, (II) a detailed analysis of various GAN applications in different domains, and (III) a detailed study about the various GAN training obstacles as well as training solutions. Finally, we reveal several issues as well as research outlines to the topic.
Assessing spectra and thermal inversions due to TiO in hot Jupiter atmospheres

Recent detections of thermal inversions in the dayside atmospheres of some hot Jupiters are motivating new avenues to understand the interplay between their temperature structures and other atmospheric conditions. In particular, TiO has long been proposed to cause thermal inversions in hot Jupiters, depending on other factors such as stellar irradiation, C/O, and vertical mixing. TiO also has spectral features in the optical and near-infrared that have been detected. However, interpretations of TiO signatures rely on the accuracy of TiO opacity used in the models. The recently reported toto TiO line list provides a new opportunity to investigate these dependences, which is the goal of this work. First, we investigate how the toto line list affects observable transmission and emission spectra of hot Jupiters at low and high resolutions. The improvement in the toto line list compared to a previous line list results in observable differences in the model spectra, particularly in the optical at high resolution. Secondly, we explore the interplay between temperature structure, irradiation, and composition with TiO as the primary source of optical opacity, using 1D self-consistent atmospheric models. Among other trends, we find that the propensity for thermal inversions due to TiO peaks at C/O ∼ 0.9, consistent with recent studies. Using these models, we further assess metrics to quantify thermal inversions due to TiO, compared to frequently used Spitzer photometry, over a range in C/O, irradiation, metallicity, gravity, and stellar type.
On Uniform Convergence and Low-Norm Interpolation Learning

We consider an underdetermined noisy linear regression model where the minimum-norm interpolating predictor is known to be consistent, and ask: can uniform convergence in a norm ball, or at least (following Nagarajan and Kolter) the subset of a norm ball that the algorithm selects on a typical input set, explain this success? We show that uniformly bounding the difference between empirical and population errors cannot show any learning in the norm ball, and cannot show consistency for any set, even one depending on the exact algorithm and distribution. But we argue we can explain the consistency of the minimal-norm interpolator with a slightly weaker, yet standard, notion, uniform convergence of zero-error predictors. We use this to bound the generalization error of low- (but not minimal-) norm interpolating predictors.
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks. Despite the strong empirical performance of fine-tuned models, fine-tuning is an unstable process: training the same model with multiple random seeds can result in a large variance of the task performance. Previous literature (Devlin et al., 2019; Lee et al., 2020; Dodge et al., 2020) identified two potential reasons for the observed instability: catastrophic forgetting and a small size of the fine-tuning datasets. In this paper, we show that both hypotheses fail to explain the fine-tuning instability. We analyze BERT, RoBERTa, and ALBERT, fine-tuned on three commonly used datasets from the GLUE benchmark and show that the observed instability is caused by optimization difficulties that lead to vanishing gradients. Additionally, we show that the remaining variance of the downstream task performance can be attributed to differences in generalization where fine-tuned models with the same training loss exhibit noticeably different test performance. Based on our analysis, we present a simple but strong baseline that makes fine-tuning BERT-based models significantly more stable than previously proposed approaches. Code to reproduce our results is available online: this https URL .
Hypoglossum caloglossoides sp. nov. (Delesseriaceae, Rhodophyta) from Lord Howe Island, South Pacific

An alga superficially resembling the genus Caloglossa in its habit is a previously undescribed species of Hypoglossum (Delesseriaceae, Rhodophyta). Hypoglossum caloglossoides sp. nov. occurs at Lord Howe Island in the South Pacific at a depth of 11 m, where it grows as a creeping system over coral. Additional material of this alga was found in collections from Samoa (misidentified as Calaglossa vieillardii) and from the Southern Marshall Islands (misidentified as C. leprieurii var. pygmaea). Observations are made on tetrasporangial, male and female specimens, and the relationships of this new species with other members of the genus and with Caloglossa are discussed.
Off-the-shelf sensor vs. experimental radar - How much resolution is necessary in automotive radar classification?

Radar-based road user detection is an important topic in the context of autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to refine during subsequent signal processing. On the other hand, a new sensor generation is waiting in the wings for its application in this challenging field. In this article, two sensors of different radar generations are evaluated against each other. The evaluation criterion is the performance on moving road user object detection and classification tasks. To this end, two data sets originating from an off-the-shelf radar and a high resolution next generation radar are compared. Special attention is given on how the two data sets are assembled in order to make them comparable. The utilized object detector consists of a clustering algorithm, a feature extraction module, and a recurrent neural network ensemble for classification. For the assessment, all components are evaluated both individually and, for the first time, as a whole. This allows for indicating where overall performance improvements have their origin in the pipeline. Furthermore, the generalization capabilities of both data sets are evaluated and important comparison metrics for automotive radar object detection are discussed. Results show clear benefits of the next generation radar. Interestingly, those benefits do not actually occur due to better performance at the classification stage, but rather because of the vast improvements at the clustering stage.
Using GAMA to probe the impact of small-scale galaxy physics on nonlinear redshift-space distortions

We present improved modelling of the redshift-space distortions (RSDs) of galaxy clustering that arise from peculiar velocities. We create mock galaxy catalogues in the framework of the halo model, using data from the Bolshoi project. These mock galaxy populations are inserted into the haloes with additional degrees of freedom that govern spatial and kinematical biases of the galaxy populations relative to the dark matter. We explore this generalized halo model with an Markov Chain Monte Carlo (MCMC) algorithm, comparing the predictions to data from the Galaxy And Mass Assembly survey, and thus derive one of the first constraints on the detailed kinematic degrees of freedom for satellite galaxies within haloes. With this approach, the distortions of the redshift-space galaxy autocorrelations can be accounted for down to spatial separations close to 10 kpc, opening the prospect of improved RSD measurements of the perturbation growth rate by the inclusion of data from nonlinear scales.
Identifying measures on non-abelian groups and modules by their moments via reduction to a local problem

Work on generalizations of the Cohen-Lenstra and Cohen-Martinet heuristics has drawn attention to probability measures on the space of isomorphism classes of profinite groups. As is common in probability theory, it would be desirable to know that these measures are determined by their moments, which in this context are the expected number of surjections to a fixed finite group. We show a wide class of measures, including those appearing in a recent paper of Liu, Wood, and Zurieck-Brown, have this property. The method is to work "locally" with groups that are extensions of a fixed group by a product of finite simple groups. This eventually reduces the problem to the case of powers of a fixed finite simple group, which can be handled by a simple explicit calculation. We can also prove a similar theorem for random modules over an algebra.
Population Dynamics of Lake Ontario Lake Trout during 1985–2007

Abstract Lake trout Salvelinus namaycush were extirpated from Lake Ontario circa 1950 owing to commercial and recreational fishing, predation by sea lampreys Petromyzon marinus, and habitat degradation. Since the 1970s, substantial efforts have been devoted to reestablishing a self-sustaining population through stocking, sea lamprey control, and harvest reduction. Although a stocking-supported population has been established, only limited natural reproduction has been detected. Since the 1990s, surveys have indicated a continuing decline in overall abundance despite fairly static stocking levels. We constructed a statistical catch-at-age model to describe the dynamics of Lake Ontario lake trout from 1985 to 2007 and explore what factor(s) could be causing the declines in abundance. Model estimates indicated that abundance had declined by approximately 76% since 1985. The factor that appeared most responsible for this was an increase in age-1 natural mortality rates from approximately 0.9 to 2.5 between 19.
Marine algae of the Solomon Islands

An account is given of the benthic marine algae (and sea grasses) collected on the 1965 Royal Society Expedition to the Solomon Islands. The known algal flora is fairly typical of such a tropical area, comprising some 71 species of Chlorophyta, 27 of Phaeophyta, 121 of Rhodophyta and 14 of Cyanophyta. Pseudobryopsis solomonensis , P. gracilis, Caulerpa spathulata and Cryptonemia ? subdichotoma are newly described. Seven species of sea grasses are also recorded.
Neural Networks, Ridge Splines, and TV Regularization in the Radon Domain

We develop a variational framework to understand the properties of the functions learned by neural networks fit to data. We propose and study of a family of continuous-domain linear inverse problems with total variation-like regularization in the Radon domain subject to data fitting constraints. We derive a representer theorem showing that finite-width, single-hidden layer neural networks are solutions to these inverse problems. We draw on many techniques from variational spline theory and so we propose the notion of a ridge spline, which corresponds to fitting data with a single-hidden layer neural network. The representer theorem is reminiscent of the classical Reproducing Kernel Hilbert space representer theorem, but the neural network problem is set in a non-Hilbertian Banach space. Although the learning problems are posed in the continuous-domain, similar to kernel methods, the problems can be recast as finite-dimensional neural network training problems. These neural network training problems have regularizers which are related to the well-known weight decay and path-norm regularizers. Thus, our result gives insight into functional characteristics of trained neural networks and also into the design neural network regularizers. We also show that these regularizers promote neural network solutions with desirable generalization properties.
Predicting the time variation of radio emission from MHD simulations of a flaring T-Tauri star

We model the time-dependent radio emission from a disc accretion event in a T-Tauri star using 3D, ideal magnetohydrodynamic simulations combined with a gyrosynchrotron emission and radiative transfer model. We predict for the first time, the multifrequency (1–1000 GHz) intensity and circular polarization from a flaring T-Tauri star. A flux tube, connecting the star with its circumstellar disc, is populated with a distribution of non-thermal electrons that is allowed to decay exponentially after a heating event in the disc and the system is allowed to evolve. The energy distribution of the electrons, as well as the non-thermal power-law index and loss rate, are varied to see their effect on the overall flux. Spectra are generated from different lines of sight, giving different views of the flux tube and disc. The peak flux typically occurs around 20–30 GHz and the radio luminosity is consistent with that observed from T-Tauri stars. For all simulations, the peak flux is found to decrease and move to lower frequencies with elapsing time. The frequency-dependent circular polarization can reach 10$-30{{\ \rm per\ cent}}$ but has a complex structure that evolves as the flare evolves. Our models show that observations of the evolution of the spectrum and its polarization can provide important constraints on physical properties of the flaring environment and associated accretion event.
Secure Groupcast: Extra-Entropic Structure and Linear Feasibility

In the secure groupcast problem, a transmitter wants to securely groupcast a message with the maximum rate to the first $N$ of $K$ receivers by broadcasting with the minimum bandwidth, where the $K$ receivers are each equipped with a key variable from a known joint distribution. Examples are provided to prove that different instances of secure groupcast that have the same entropic structure, i.e., the same entropy for all subsets of the key variables, can have different maximum groupcast rates and different minimum broadcast bandwidth. Thus, extra-entropic structure matters for secure groupcast. Next, the maximum groupcast rate is explored when the key variables are generic linear combinations of a basis set of independent key symbols, i.e., the keys lie in generic subspaces. The maximum groupcast rate is characterized when the dimension of each key subspace is either small or large, i.e., the extreme regimes. For the intermediate regime, various interference alignment schemes originated from wireless interference networks, such as eigenvector based and asymptotic schemes, are shown to be useful.