Miracle-dz/UniFAR-FTUs huggingface dataset:
Variational Auto-Regressive Gaussian Processes for Continual Learning

This paper proposes Variational Auto-Regressive Gaussian Process (VAR-GP), a principled Bayesian updating mechanism to incorporate new data for sequential tasks in the context of continual learning. It relies on a novel auto-regressive characterization of the variational distribution and inference is made scalable using sparse inducing point approximations. Experiments on standard continual learning benchmarks demonstrate the ability of VAR-GPs to perform well at new tasks without compromising performance on old ones, yielding competitive results to state-of-the-art methods. In addition, we qualitatively show how VAR-GP improves the predictive entropy estimates as we train on new tasks. Further, we conduct a thorough ablation study to verify the effectiveness of inferential choices.
A Note on Deepfake Detection with Low-Resources

Deepfakes are videos that include changes, quite often substituting face of a portrayed individual with a different face using neural networks. Even though the technology gained its popularity as a carrier of jokes and parodies it raises a serious threat to ones security - via biometric impersonation or besmearing. In this paper we present two methods that allow detecting Deepfakes for a user without significant computational power. In particular, we enhance MesoNet by replacing the original activation functions allowing a nearly 1% improvement as well as increasing the consistency of the results. Moreover, we introduced and verified a new activation function - Pish that at the cost of slight time overhead allows even higher consistency. Additionally, we present a preliminary results of Deepfake detection method based on Local Feature Descriptors (LFD), that allows setting up the system even faster and without resorting to GPU computation. Our method achieved Equal Error Rate of 0.28, with both accuracy and recall exceeding 0.7.
Better Together: Resnet-50 accuracy with $13x$ fewer parameters and at $3x$ speed

Recent research on compressing deep neural networks has focused on reducing the number of parameters. Smaller networks are easier to export and deploy on edge-devices. We introduce Adjoined networks as a training approach that can compress and regularize any CNN-based neural architecture. Our one-shot learning paradigm trains both the original and the smaller networks together. The parameters of the smaller network are shared across both the architectures. For resnet-50 trained on Imagenet, we are able to achieve a $13.7x$ reduction in the number of parameters and a $3x$ improvement in inference time without any significant drop in accuracy. For the same architecture on CIFAR-100, we are able to achieve a $99.7x$ reduction in the number of parameters and a $5x$ improvement in inference time. On both these datasets, the original network trained in the adjoint fashion gains about $3\%$ in top-1 accuracy as compared to the same network trained in the standard fashion.
Effects of habitat fragmentation and road density on the distribution pattern of the moor frog Rana arvalis

1. The effects of habitat fragmentation on the distribution pattern of the moor frog Rana arvalis were investigated. Also, the possible isolation effects of the road network were taken into account. 2. Indications were found that habitat fragmentation partly explains the distribution pattern of the moor frog. The statistical models showed a positive effect of pond size (or marsh area) and a negative effect of road density on the probability of occupation of a moorland pond. 3. Because of the strong correlation between habitat quality variables and isolation variables, no unambiguous effects of isolation, described as the amount of suitable terrestrial habitat (moorland) in the surroundings of a moorland pond in a radius of 100-2000 m, could be demonstrated. 4. Spatial differences in road density can play a role in the selection of optimal locations for nature protection areas. The regression model used in this study predicts a reduced occupation probability in 55% of the study area. In the part of the study area adjacent to a motorway, occupation probability is lowered to less than 30%. 5. European studies of habitat fragmentation on amphibian species revealed a mean distance between occupied ponds of <1 km in all studies. This could be a general rule of thumb for persistent amphibian populations. Effects of pond size on the probability of occupation were more variable. 6. When discussing the effects of habitat fragmentation on amphibians and other ground dwelling species, the negative effects of roads are often underestimated.
Interpretable Signal Analysis with Knockoffs Enhances Classification of Bacterial Raman Spectra

Sophisticated machine learning models are widely applied to signal data because they can detect complex patterns and leverage them effectively to make predictions. However, such models tend to be difficult to interpret, which is particularly concerning for critical biomedical applications, such as the identification of bacterial infections from spectroscopic data. Feature extraction and selection can identify structures in the data that are both informative and non-redundant, leading to simpler and more easily understandable models, without necessarily sacrificing predictive accuracy. In this paper, we present a signal classification method that combines wavelet-based feature extraction with a knockoff filter to control the false discovery rate. We apply the method to Raman spectroscopy data in order to classify bacterial samples. We show that the features thus obtained allow an intuitive logistic regression model to achieve predictive accuracy comparable to that of less understandable alternative approaches.
Speaker Diarization: Using Recurrent Neural Networks

Speaker Diarization is the problem of separating speakers in an audio. There could be any number of speakers and final result should state when speaker starts and ends. In this project, we analyze given audio file with 2 channels and 2 speakers (on separate channel). We train Neural Network for learning when a person is speaking. We use different type of Neural Networks specifically, Single Layer Perceptron (SLP), Multi Layer Perceptron (MLP), Recurrent Neural Network (RNN) and Convolution Neural Network (CNN) we achieve $\sim$92\% of accuracy with RNN. The code for this project is available at this https URL
Spatial-Temporal Dynamic Graph Attention Networks for Ride-hailing Demand Prediction

Ride-hailing demand prediction is an important prediction task in traffic prediction. An accurate prediction model can help the platform pre-allocate resources in advance to improve vehicle utilization and reduce the wait-time. This task is challenging due to the complicated spatial-temporal relationships among regions. Most existing methods mainly focus on Euclidean correlations among regions. Though there are some methods that use Graph Convolutional Networks (GCN) to capture the non-Euclidean pair-wise correlations, they only rely on the static topological structure among regions. Besides, they only consider fixed graph structures at different time intervals. In this paper, we propose a novel deep learning method called Spatial-Temporal Dynamic Graph Attention Network (STDGAT) to predict the taxi demand of multiple connected regions in the near future. The method uses Graph Attention Network (GAT), which achieves the adaptive allocation of weights for other regions, to capture the spatial information. Furthermore, we implement a Dynamic Graph Attention mode to capture the different spatial relationships at different time intervals based on the actual commuting relationships. Extensive experiments are conducted on a real-world large scale ride-hailing demand dataset, the results demonstrate the superiority of our method over existing methods.
$l_1$-based sparsification of energy interactions in two-dimensional turbulent flows

In this paper, sparsity-promoting regression techniques are employed to automatically identify from data relevant triadic interactions between modal structures in large Galerkin-based models of two-dimensional unsteady flows. The approach produces interpretable, sparsely-connected models that reproduce the original dynamical behaviour at a much lower computational cost, as fewer triadic interactions need to be evaluated. The key feature of the approach is that dominant interactions are selected systematically from the solution of a convex optimisation problem, with a unique solution, and no a priori assumptions on the structure of scale interactions are required. We demonstrate this approach on models of two-dimensional lid-driven cavity flow at Reynolds number $Re = 2 \times 10^4$, where fluid motion is chaotic. To understand the role of the subspace utilised for the Galerkin projection on sparsity characteristics, we consider two families of models obtained from two different modal decomposition techniques. The first uses energy-optimal Proper Orthogonal Decomposition modes, while the second uses modes oscillating at a single frequency obtained from Discrete Fourier Transform of the flow snapshots. We show that, in both cases, and despite no \textit{a-priori} physical knowledge is incorporated into the approach, relevant interactions across the hierarchy of modes are identified in agreement with the expected picture of scale interactions in two-dimensional turbulence. Yet, substantial structural changes in the interaction pattern and a quantitatively different sparsity are observed. Finally, although not directly enforced in the procedure, the sparsified models have excellent long-term stability properties and correctly reproduce the spatio-temporal evolution of dominant flow structures in the cavity.
Geometry-Aware Segmentation of Remote Sensing Images via implicit height estimation

Recent studies have shown the benefits of using additional elevation data (e.g., DSM) for enhancing the performance of the semantic segmentation of aerial images. However, previous methods mostly adopt 3D elevation information as additional inputs. While in many real-world applications, one does not have the corresponding DSM information at hand and the spatial resolution of acquired DSM images usually do not match the aerial images. To alleviate this data constraint and also take advantage of 3D elevation information, in this paper, we introduce a geometry-aware segmentation model that achieves accurate semantic labeling of aerial images via joint height estimation. Instead of using a single-stream encoder-decoder network for semantic labeling, we design a separate decoder branch to predict the height map and use the DSM images as side supervision to train this newly designed decoder branch. In this way, our model does not require DSM as model input and still benefits from the helpful 3D geometric information during training. Moreover, we develop a new geometry-aware convolution module that fuses the 3D geometric features from the height decoder branch and the 2D contextual features from the semantic segmentation branch. The fused feature embeddings can produce geometry-aware segmentation maps with enhanced performance. Our model is trained with DSM images as side supervision, while in the inference stage, it does not require DSM data and directly predicts the semantic labels in an end-to-end fashion. Experiments on ISPRS Vaihingen and Potsdam datasets demonstrate the effectiveness of the proposed method for the semantic segmentation of aerial images. The proposed model achieves remarkable performance on both datasets without using any hand-crafted features or post-processing.
Variational Optimization for the Submodular Maximum Coverage Problem

We examine the \emph{submodular maximum coverage problem} (SMCP), which is related to a wide range of applications. We provide the first variational approximation for this problem based on the Nemhauser divergence, and show that it can be solved efficiently using variational optimization. The algorithm alternates between two steps: (1) an E step that estimates a variational parameter to maximize a parameterized \emph{modular} lower bound; and (2) an M step that updates the solution by solving the local approximate problem. We provide theoretical analysis on the performance of the proposed approach and its curvature-dependent approximate factor, and empirically evaluate it on a number of public data sets and several application tasks.
Testing a theoretical prediction for bar formation in galaxies with bulges

Earlier studies have shown that massive bulges impede bar formation in disk galaxies. Recent N-body simulations have derived a bar formation criterion that depends on the radial bulge force in a galaxy disk. We use those simulations to show that bars can form only when the force constant FB < 0.13, where FB depends on the ratio of the bulge force to the total force of the galaxy at twice the disk scale length 2Rd. In this article, we test this theoretical prediction using observational data obtained from the literature. Our sample consists of 63 barred galaxies with a wide range of Hubble classes taken from the S4G catalog; for the galaxies that we include, bulge, disk, and bar decomposition has been carried out. We find that 92% of our sample galaxies satisfy the condition FB < 0.13 for bar formation in galaxies and hence agree with the bar formation criterion given by the simulations.
Image Enhancement and Object Recognition for Night Vision Surveillance

Object recognition is a critical part of any surveillance system. It is the matter of utmost concern to identify intruders and foreign objects in the area where surveillance is done. The performance of surveillance system using the traditional camera in daylight is vastly superior as compared to night. The main problem for surveillance during the night is the objects captured by traditional cameras have low contrast against the background because of the absence of ambient light in the visible spectrum. Due to that reason, the image is taken in low light condition using an Infrared Camera and the image is enhanced to obtain an image with higher contrast using different enhancing algorithms based on the spatial domain. The enhanced image is then sent to the classification process. The classification is done by using convolutional neural network followed by a fully connected layer of neurons. The accuracy of classification after implementing different enhancement algorithms is compared in this paper.
Continuous Action Reinforcement Learning From a Mixture of Interpretable Experts

Reinforcement learning (RL) has demonstrated its ability to solve high dimensional tasks by leveraging non-linear function approximators. However, these successes are mostly achieved by ’black-box’ policies in simulated domains. When deploying RL to the real world, several concerns regarding the use of a ’black-box’ policy might be raised. In order to make the learned policies more transparent, we propose in this paper a policy iteration scheme that retains a complex function approximator for its internal value predictions but constrains the policy to have a concise, hierarchical, and human-readable structure, based on a mixture of interpretable experts. Each expert selects a primitive action according to a distance to a prototypical state. A key design decision to keep such experts interpretable is to select the prototypical states from trajectory data. The main technical contribution of the paper is to address the challenges introduced by this non-differentiable prototypical state selection procedure. Experimentally, we show that our proposed algorithm can learn compelling policies on continuous action deep RL benchmarks, matching the performance of neural network based policies, but returning policies that are more amenable to human inspection than neural network or linear-in-feature policies.
On the phase transition in the sublattice TASEP with stochastic blockage

We revisit the defect-induced nonequilibrium phase transition from a largely homogeneous free-flow phase to a phase-separated congested phase in the sublattice totally asymmetric simple exclusion process (TASEP) with local deterministic bulk dynamics and a stochastic defect that mimicks a random blockage. Exact results are obtained for the compressibility and density correlations for a stationary grandcanonical ensemble given by the matrix product ansatz. At the critical density the static compressibility diverges while in the phase separated state above the critical point the compressibility vanishes due to strong non-local correlations. These correlations arise from a long range effective interaction between particles that appears in the stationary state despite the locality of the microscopic dynamics.
Hand-crafted Attention is All You Need? A Study of Attention on Self-supervised Audio Transformer

In this paper, we seek to reduce the computation complexity of transformer-based models for speech representation learning. We evaluate 10 attention mechanisms; then, we pre-train the transformer-based model with those attentions in a self-supervised fashion and use them as feature extractors on downstream tasks, including phoneme classification and speaker classification. We find that the proposed approach, which only uses hand-crafted and learnable attentions, is comparable with the full self-attention.
Regulation of L‐type Ca2+ channels in rabbit portal vein by G protein αs and βγ subunits

1 The effect of purified G protein subunits αs and βγ on L‐type Ca2+ channels in vascular smooth muscle and the possible pathways involved were investigated using freshly isolated smooth muscle cells from rabbit portal vein and the whole‐cell patch clamp technique. 2 Cells dialysed with either Gαs or Gβγ exhibited significant increases in peak Ba2+ current (IBa) density (148 % and 131 %, respectively) compared with control cells. The combination of Gαs and Gβγ further increased peak IBa density (181 %). Inactive Gαs and Gβγ did not have any effect on Ca2+ channels. 3 The stimulatory effect of Gαs on peak IBa was entirely abolished by the protein kinase A inhibitor Rp‐8‐Br‐cAMPS, or the adenylyl cyclase inhibitor SQ 22536. On the other hand, the stimulatory response of Ca2+ channels to Gβγ was not affected by the protein kinase A inhibitors Rp‐8‐Br‐cAMPS and KT 5720, or by the Ca2+‐dependent protein kinase C inhibitor bisindolylmaleimide 1, but was completely blocked by the protein kinase C inhibitor calphostin C. Pretreatment of cells with phorbol 12‐myristate 13‐acetate for over 18 h prevented the stimulatory effect of Gβγ on peak IBa. In addition, acute application of phorbol 12,13‐dibutyrate enhanced peak IBa density in control cells, which could be entirely blocked by calphostin C. 4 These data indicate that enhancement of Ba2+ currents by Gαs and Gβγ can be attributed to increased activity of protein kinase A and protein kinase C, respectively. No direct membrane‐delimited pathway for Ca2+ channel regulation by activated Gs proteins could be detected in vascular smooth muscle cells.
The weak B, B
 
 s
 and B
 
 c
 decays to radially excited states

Recently, many new excited states of heavy mesons, especially the radially excited states, are discovered. The study of the production processes of these states from the ground b-flavored mesons is of interest. In this paper, we use the improved Bethe-Salpeter method to study the semi-leptonic and non-leptonic decays of $B$, $B_s$, and $B_c$ mesons, where the final states are focused on the radial excited $2S$ and $3S$ states. We find that many channels have branching ratios up to $10^{-4}$, which are within the detection accuracy of current experiments.
A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent

This article characterizes the exact asymptotics of random Fourier feature (RFF) regression, in the realistic setting where the number of data samples n, their dimension p, and the dimension of feature space N are all large and comparable. In this regime, the random RFF Gram matrix no longer converges to the well-known limiting Gaussian kernel matrix (as it does when N → ∞ alone), but it still has a tractable behavior that is captured by our analysis. This analysis also provides accurate estimates of training and test regression errors for large n, p, N. Based on these estimates, a precise characterization of two qualitatively different phases of learning, including the phase transition between them, is provided; and the corresponding double descent test error curve is derived from this phase transition behavior. These results do not depend on strong assumptions on the data distribution, and they perfectly match empirical results on real-world data sets.
Combination of abstractive and extractive approaches for summarization of long scientific texts

In this research work, we present a method to generate summaries of long scientific documents that uses the advantages of both extractive and abstractive approaches. Before producing a summary in an abstractive manner, we perform the extractive step, which then is used for conditioning the abstractor module. We used pre-trained transformer-based language models, for both extractor and abstractor. Our experiments showed that using extractive and abstractive models jointly significantly improves summarization results and ROUGE scores.
How Do Soil Organisms Affect Total Organic Nitrogen Storage and Substrate Nitrogen to Carbon Ratio in Soils? A Theoretical Analysis

To examine how soil organisms affect soil organic nitrogen (N) storage and mineralization, we theoretically derive the soil substrate nitrogen to carbon ratio and total soil N storage for both donor-controlled (DC) and Lotka-Volterra (LV) soil systems with three trophic levels: substrate, microbes and grazers. We show that substrate N:C ratio and soil N storage are functions of the structure of the soil food web. of the properties of the organisms in the soil community, and of the properties of litter input from above ground. The carbon production-to-assimilation efficiency of soil organisms is the most important property determining how the substrate N:C ratio and soil N storage are affected by changes in the soil community. The qualitative effects of grazers depend on whether they have an efficiency larger or smaller than some critical efficiencies. In this paper, we derive these critical efficiencies mathematically. Within the normal range of efficiencies, substrate N:C ratio and soil N storage will usually decrease, and the nitrogen mineralization will increase, with addition of grazers, or as a result of a succession of species from a lower to a higher level of metabolic activity. Numerical analyses show that an increase in soil temperature has a pronounced negative effect on soil N storage through the effects on the metabolic activity of organisms at all trophic levels. N dilution in litter, because of increased CO 2 levels in the atmosphere, can lead to a moderate increase in soil N storage.