Miracle-dz/UniFAR-FTUs huggingface dataset:
Structure-preserving deep learning

Over the past few years, deep learning has risen to the foreground as a topic of massive interest, mainly as a result of successes obtained in solving large-scale image processing tasks. There are multiple challenging mathematical problems involved in applying deep learning: most deep learning methods require the solution of hard optimisation problems, and a good understanding of the trade-off between computational effort, amount of data and model complexity is required to successfully design a deep learning approach for a given problem. A large amount of progress made in deep learning has been based on heuristic explorations, but there is a growing effort to mathematically understand the structure in existing deep learning methods and to systematically design new deep learning methods to preserve certain types of structure in deep learning. In this article, we review a number of these directions: some deep neural networks can be understood as discretisations of dynamical systems, neural networks can be designed to have desirable properties such as invertibility or group equivariance and new algorithmic frameworks based on conformal Hamiltonian systems and Riemannian manifolds to solve the optimisation problems have been proposed. We conclude our review of each of these topics by discussing some open problems that we consider to be interesting directions for future research.
The timing of adult sockeye salmon migration into fresh water: adaptations by populations to prevailing thermal regimes

Anadromous fishes migrate to sea, apparently to take advantage of growing conditions, and return to fresh water to spawn. Despite favorable growing conditions at sea in summer, some populations leave the ocean in spring, many months prior to spawning. We hypothesized that this premature migration is a consequence of the fish having to avoid stressful summer temperatures in order to access certain suitable areas for spawning in the fall. We tested this idea in sockeye salmon, Oncorhynchus nerka, by compiling data on the timing of migration and spawning and the freshwater temperature regime along the migration route in populations throughout the species' North American range. The timing of migration varied among populations and was primarily related to temperature regime during migration and the timing of spawning. When temperatures were moderate (<19°C), sockeye salmon tended to migrate to the vicinity of the spawning grounds about 1 month prior to spawning, regardless of the length of the freshwater migra.
Aspect-based Sentiment Analysis of Scientific Reviews

Scientific papers are complex and understanding the usefulness of these papers requires prior knowledge. Peer reviews are comments on a paper provided by designated experts on that field and hold a substantial amount of information, not only for the editors and chairs to make the final decision, but also to judge the potential impact of the paper. In this paper, we propose to use aspect-based sentiment analysis of scientific reviews to be able to extract useful information, which correlates well with the accept/reject decision. While working on a dataset of close to 8k reviews from ICLR, one of the top conferences in the field of machine learning, we use an active learning framework to build a training dataset for aspect prediction, which is further used to obtain the aspects and sentiments for the entire dataset. We show that the distribution of aspect-based sentiments obtained from a review is significantly different for accepted and rejected papers. We use the aspect sentiments from these reviews to make an intriguing observation, certain aspects present in a paper and discussed in the review strongly determine the final recommendation. As a second objective, we quantify the extent of disagreement among the reviewers refereeing a paper. We also investigate the extent of disagreement between the reviewers and the chair and find that the inter-reviewer disagreement may have a link to the disagreement with the chair. One of the most interesting observations from this study is that reviews, where the reviewer score and the aspect sentiments extracted from the review text written by the reviewer are consistent, are also more likely to be concurrent with the chair's decision.
Triple descent and the two kinds of overfitting: where and why do they appear?

A recent line of research has highlighted the existence of a ‘double descent’ phenomenon in deep learning, whereby increasing the number of training examples N causes the generalization error of neural networks (NNs) to peak when N is of the same order as the number of parameters P. In earlier works, a similar phenomenon was shown to exist in simpler models such as linear regression, where the peak instead occurs when N is equal to the input dimension D. Since both peaks coincide with the interpolation threshold, they are often conflated in the literature. In this paper, we show that despite their apparent similarity, these two scenarios are inherently different. In fact, both peaks can co-exist when NNs are applied to noisy regression tasks. The relative size of the peaks is then governed by the degree of nonlinearity of the activation function. Building on recent developments in the analysis of random feature models, we provide a theoretical ground for this sample-wise triple descent. As shown previously, the nonlinear peak at N = P is a true divergence caused by the extreme sensitivity of the output function to both the noise corrupting the labels and the initialization of the random features (or the weights in NNs). This peak survives in the absence of noise, but can be suppressed by regularization. In contrast, the linear peak at N = D is solely due to overfitting the noise in the labels, and forms earlier during training. We show that this peak is implicitly regularized by the nonlinearity, which is why it only becomes salient at high noise and is weakly affected by explicit regularization. Throughout the paper, we compare analytical results obtained in the random feature model with the outcomes of numerical experiments involving deep NNs.
SAveRUNNER: A network-based algorithm for drug repurposing and its application to COVID-19

The novelty of new human coronavirus COVID-19/SARS-CoV-2 and the lack of effective drugs and vaccines gave rise to a wide variety of strategies employed to fight this worldwide pandemic. Many of these strategies rely on the repositioning of existing drugs that could shorten the time and reduce the cost compared to de novo drug discovery. In this study, we presented a new network-based algorithm for drug repositioning, called SAveRUNNER (Searching off-lAbel dRUg aNd NEtwoRk), which predicts drug–disease associations by quantifying the interplay between the drug targets and the disease-specific proteins in the human interactome via a novel network-based similarity measure that prioritizes associations between drugs and diseases locating in the same network neighborhoods. Specifically, we applied SAveRUNNER on a panel of 14 selected diseases with a consolidated knowledge about their disease-causing genes and that have been found to be related to COVID-19 for genetic similarity (i.e., SARS), comorbidity (e.g., cardiovascular diseases), or for their association to drugs tentatively repurposed to treat COVID-19 (e.g., malaria, HIV, rheumatoid arthritis). Focusing specifically on SARS subnetwork, we identified 282 repurposable drugs, including some the most rumored off-label drugs for COVID-19 treatments (e.g., chloroquine, hydroxychloroquine, tocilizumab, heparin), as well as a new combination therapy of 5 drugs (hydroxychloroquine, chloroquine, lopinavir, ritonavir, remdesivir), actually used in clinical practice. Furthermore, to maximize the efficiency of putative downstream validation experiments, we prioritized 24 potential anti-SARS-CoV repurposable drugs based on their network-based similarity values. These top-ranked drugs include ACE-inhibitors, monoclonal antibodies (e.g., anti-IFNγ, anti-TNFα, anti-IL12, anti-IL1β, anti-IL6), and thrombin inhibitors. Finally, our findings were in-silico validated by performing a gene set enrichment analysis, which confirmed that most of the network-predicted repurposable drugs may have a potential treatment effect against human coronavirus infections.
Low Complexity Adaptation for Reconfigurable Intelligent Surface-Based MIMO Systems

Reconfigurable intelligent surface (RIS)-based transmission technology offers a promising solution to enhance wireless communication performance cost-effectively through properly adjusting the parameters of a large number of passive reflecting elements. This letter proposes a cosine similarity theorem-based low-complexity algorithm for adapting the phase shifts of an RIS that assists a multiple-input multiple-output (MIMO) transmission system. A semi-analytical probabilistic approach is developed to derive the theoretical average bit error probability (ABEP) of the system. Furthermore, the validity of the theoretical analysis is supported through extensive computer simulations.
Spoken Dialect Identification in Twitter using a Multi-filter Architecture

This paper presents our approach for SwissText & KONVENS 2020 shared task 2, which is a multi-stage neural model for Swiss German (GSW) identification on Twitter. Our model outputs either GSW or non-GSW and is not meant to be used as a generic language identifier. Our architecture consists of two independent filters where the first one favors recall, and the second one filter favors precision (both towards GSW). Moreover, we do not use binary models (GSW vs. not-GSW) in our filters but rather a multi-class classifier with GSW being one of the possible labels. Our model reaches F1-score of 0.982 on the test set of the shared task.
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers

Transformer models have achieved state-of-the-art results across a diverse range of domains. However, concern over the cost of training the attention mechanism to learn complex dependencies between distant inputs continues to grow. In response, solutions that exploit the structure and sparsity of the learned attention matrix have blossomed. However, real-world applications that involve long sequences, such as biological sequence analysis, may fall short of meeting these assumptions, precluding exploration of these models. To address this challenge, we present a new Transformer architecture, Performer, based on Fast Attention Via Orthogonal Random features (FAVOR). Our mechanism scales linearly rather than quadratically in the number of tokens in the sequence, is characterized by sub-quadratic space complexity and does not incorporate any sparsity pattern priors. Furthermore, it provides strong theoretical guarantees: unbiased estimation of the attention matrix and uniform convergence. It is also backwards-compatible with pre-trained regular Transformers. We demonstrate its effectiveness on the challenging task of protein sequence modeling and provide detailed theoretical analysis.
GMAT: Global Memory Augmentation for Transformers

Transformer-based models have become ubiquitous in natural language processing thanks to their large capacity, innate parallelism and high performance. The contextualizing component of a Transformer block is the $\textit{pairwise dot-product}$ attention that has a large $\Omega(L^2)$ memory requirement for length $L$ sequences, limiting its ability to process long documents. This has been the subject of substantial interest recently, where multiple approximations were proposed to reduce the quadratic memory requirement using sparse attention matrices. In this work, we propose to augment sparse Transformer blocks with a dense attention-based $\textit{global memory}$ of length $M$ ($\ll L$) which provides an aggregate global view of the entire input sequence to each position. Our augmentation has a manageable $O(M\cdot(L+M))$ memory overhead, and can be seamlessly integrated with prior sparse solutions. Moreover, global memory can also be used for sequence compression, by representing a long input sequence with the memory representations only. We empirically show that our method leads to substantial improvement on a range of tasks, including (a) synthetic tasks that require global reasoning, (b) masked language modeling, and (c) reading comprehension.
Optimal filters for the moving lens effect

We assess the prospects for detecting the moving lens effect using cosmological surveys. The bulk motion of cosmological structure induces a small-scale dipolar temperature anisotropy of the cosmic microwave radiation (CMB), centered around halos and oriented along the transverse velocity field. We introduce a set of optimal filters for this signal, and forecast that a high significance detection can be made with upcoming experiments. We discuss the prospects for reconstructing the bulk transverse velocity field on large scales using matched filters, finding good agreement with previous work using quadratic estimators.
Augmenting Automated Kinship Verification with Targeted Human Input

Kinship verification is the problem whereby a third party determines whether two people are related. Despite previous research in Psychology and Machine Vision, the factors affecting a person's verification ability are poorly understood. Through an online crowdsourcing study, we investigate the impact of gender, race and medium type (image vs video) on kinship verification - taking into account the demographics of both raters and ratees. A total of 325 workers completed over 50,000 kinship verification tasks consisting of pairs of faces shown in images and videos from three widely used datasets. Our results identify an own-race bias and a higher verification accuracy for same-gender image pairs than opposite-gender image pairs. Our results demonstrate that humans can still outperform current state-of-the-art automated unsupervised approaches. Furthermore, we show that humans perform better when presented with videos instead of still images. Our findings contribute to the design of future human-in-the-loop kinship verification tasks, including time-critical use cases such as identifying missing persons.
Structure–property relationships and the mechanisms of multistep transitions in spin crossover materials and frameworks

Spin crossover frameworks and molecular crystals display fascinating collective behaviours. This includes multiple step transitions with hysteresis and a wide variety of long-range ordered patterns of high-spin and low-spin metal centres. From both practical and fundamental perspectives it is important to understand the mechanisms behind these collective behaviours. We study a simple model of elastic interactions and identify thirty six different spin-state ordered phases. We observe spin-state transitions with between one and eight steps. These include both sharp transitions and crossovers, leading to both complete and incomplete spin crossover. We demonstrate structure-property relationships that explain these differences. These arise because through-bond interactions favour metal centres with different spin-states; whereas, through-space interactions typically favour the same spin-states. In general, rigid materials with longer range elastic interactions lead to transitions with more steps and more diverse spin-state ordering, which may explain why both are prominent in framework materials.
Multiple Solutions to Cylindrically Symmetric Curl-Curl Problems and Related Schrödinger Equations with Singular Potentials

We look for multiple solutions $\mathbf{U}:\mathbb{R}^3\to\mathbb{R}^3$ to the curl-curl problem $$\nabla\times\nabla\times\mathbf{U}=h(x,\mathbf{U}),\qquad x\in\mathbb{R}^3,$$ with a nonlinear function $h:\mathbb{R}^3\times\mathbb{R}^3\to\mathbb{R}^3$ which has subcritical growth at infinity or is critical in $\mathbb{R}^3$ , i.e. $h(x, \mathbf{U}) = |\mathbf{U}|^4 \mathbf{U}$. If $h$ is radial in $\mathbf{U}$, $N=3$, $K=2$ and $a=1$ below, then we show that the solutions to the problem above are in one to one correspondence with the solutions to the following Schrodinger equation $$-\Delta u+\frac{a}{r^2}u=f(x,u),\qquad u:\mathbb{R}^N\to\mathbb{R},$$ where $x=(y, z)\in\mathbb{R}^K\times\mathbb{R}^{N-K}$, $N>K\ge2$, $r=|y|$ and $a>a_0\in(−∞, 0]$. In the subcritical case, applying a critical point theory to the Schrodinger equation above, we find infinitely many bound states for both problems. In the critical case, however, the multiplicity problem for the latter equation has been studied only in the autonomous case $a=0$ and the available methods seem to be insufficient for the problem involving the singular potential, i.e. $a\ne0$, due to the lack of conformal invariance. Therefore we develop methods for the critical curl-curl problem and show the multiplicity of bound states for both equations in the case $N=3$, $K=2$ and $a=1$.
Correlation between grain yield and its components in some Syrian barley

Five local barley genotypes grown in Damascus, Syria during the winter season of 2005/2006 were evaluated for variability parameters and correlations for five metric traits i.e., plant height, leaf area, number of productive tillers/plant, number of seeds and grain yield/plant. Significant genotypic differences were observed for all the traits studied indicating considerable amount of variation among genotypes for each character. All the traits showed significant reduction under LMS. Grain yield/plant showed highly significant positive correlation with number of tillers and number of seeds/plant. Study on the genetic behaviour of some local varieties under contrasting moisture regimes will provide a scientific basis for the utilization of these materials for efficient crop improvement.
Structure and Mechanical Properties on Tail Flukes of Dolphin

The objectives are to make the supporting mechanism of the tail flukes of dolphins clear and to obtain the useful knowledge for developing an artificial fin. The inner structure of the flukes was investigated by dissection of the flukes. The mechanical properties of a ligamentous layer and a dense connective tissue composing the flukes are investigated by the tensile test and compression test. It was found that the ligamentous layer had a high tensile modulus in the spanwise direction; the dense connective tissue had a high compressive modulus in the spanwise direction.
Effects of selection for yolk cholesterol on growth and meat quality in Japanese quail (Coturnix coturnix japonica)

Abstract Descendants () of S18 generation of Japanese quails (Coturnix coturnix japonica), selected for low (line 11, n=15) or for high (line 12, n=6) yolk cholesterol content were used for this experiment. The high (12) and low (11) lines were developed from a control egg type (line13) wild color plumage stock 07, from Poultry Breeding Station in Ivanka at Danube (Slovakia) that was maintained without conscious selection. The results of selected lines were compared to this one from unselected lines, egg type (line 13, n=15) and meat type (line 8, n=15), to study productive performance (growth and carcass weight) as well as meat quality traits (intramuscular collagen properties, fatty and cholesterol content). The birds were grown up to 35 days of age. Quails were fed ad libitum with commercial diets according to their ages. Animals had free access to water during the experiment. The body weight of the quails was controlled every week (from week 1 to 5). At the slaughter (at avg. weight of 104.0, 107.7, 110.6 g vs 209.3 g for lines 11, 12, 13 vs 8, respectively; P<0.05) the hot carcass weight was recorded and dressing percentage was calculated. Pectoralis superficialis muscles were removed from the carcasses to assess intramuscular collagen (IMC) properties (collagen and crosslink concentrations), intramuscular lipid and cholesterol contents. Growth and slaughter performance, intramuscular fat percentage and IMC properties of Pectoralis superficialis were not different between the birds selected for low (line 11) and high (line 12) yolk cholesterol content and were also found to be similar between the quails of egg type and those selected for yolk cholesterol content (lines 11 and 12). Compared to the birds of egg type line and those selected for yolk cholesterol content, the quails of meat type had higher growth and carcass traits (P<0.05), while meat quality traits were found to be similar. The meat of the quails selected for low cholesterol in the egg yolk (line 11) had lower (P<0.05) cholesterol (-35.9%) than that of the birds characterized by high cholesterol in the egg yolk (line 12). In conclusion, divergent selection for yolk cholesterol content did not significantly influence growth and slaughter traits, intramuscular fat and IMC properties of Japanese quail, while it affected the amount of cholesterol in meat. The quails of meat type showed the highest growth and slaughter traits.
A Primal-Dual SGD Algorithm for Distributed Nonconvex Optimization

The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of $n$ local cost functions by using local information exchange is considered. This problem is an important component of many machine learning techniques with data parallelism, such as deep learning and federated learning. We propose a distributed primal-dual stochastic gradient descent (SGD) algorithm, suitable for arbitrarily connected communication networks and any smooth (possibly nonconvex) cost functions. We show that the proposed algorithm achieves the linear speedup convergence rate $\mathcal{O}(1/\sqrt{nT})$ for general nonconvex cost functions and the linear speedup convergence rate $\mathcal{O}(1/(nT))$ when the global cost function satisfies the Polyak-Łojasiewicz (P-Ł) condition, where $T$ is the total number of iterations. We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum. We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.
Circumnuclear Molecular Gas in Low-redshift Quasars and Matched Star-forming Galaxies

A series of gravitational instabilities in a circumnuclear gas disk (CND) are required to trigger gas transport to a central supermassive black hole and ignite active galactic nuclei (AGNs). A test of this scenario is to investigate whether an enhanced molecular gas mass surface density (Σmol) is found in the CND-scale of quasars relative to a comparison sample of inactive galaxies. Here we performed subkiloparsec resolution CO(2–1) observations with the Atacama Large Millimeter/submillimeter Array (ALMA) of four low-redshift (z ∼ 0.06), luminous (∼1045 erg s−1) quasars with each matched to a different star-forming galaxy, having similar redshift, stellar mass, and star formation rate. We detected CO(2–1) emission from all quasars, which show diverse morphologies. Contrary to expectations, Σmol of the quasar sample, computed from the CO(2–1) luminosity, tends to be smaller than the comparison sample at r < 500 pc; there is no systematic enhancement of Σmol in our quasars. We discuss four possible scenarios that would explain the lower molecular gas content (or CO(2–1) luminosity as an actual observable) at the CND-scale of quasars, i.e., AGN-driven outflows, gas-rich minor mergers, time-delay between the onsets of a starburst-phase and a quasar-phase, and X-ray-dominated region (XDR) effects on the gas chemical abundance and excitation. While not extensively discussed in the literature, XDR effects can have an impact on molecular mass measurements particularly in the vicinity of luminous quasar nuclei; therefore higher resolution molecular gas observations, which are now viable using ALMA, need to be considered.