Miracle-dz/UniFAR-FTUs huggingface dataset:
Filter Sketch for Network Pruning

We propose a novel network pruning approach by information preserving of pretrained network weights (filters). Network pruning with the information preserving is formulated as a matrix sketch problem, which is efficiently solved by the off-the-shelf frequent direction method. Our approach, referred to as FilterSketch, encodes the second-order information of pretrained weights, which enables the representation capacity of pruned networks to be recovered with a simple fine-tuning procedure. FilterSketch requires neither training from scratch nor data-driven iterative optimization, leading to a several-orders-of-magnitude reduction of time cost in the optimization of pruning. Experiments on CIFAR-10 show that FilterSketch reduces 63.3% of floating-point operations (FLOPs) and prunes 59.9% of network parameters with negligible accuracy cost for ResNet-110. On ILSVRC-2012, it reduces 45.5% of FLOPs and removes 43.0% of parameters with only 0.69% accuracy drop for ResNet-50. Our code and pruned models can be found at https://github.com/lmbxmu/FilterSketch.
Scaling Laws for Neural Language Models

We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of magnitude. Other architectural details such as network width or depth have minimal effects within a wide range. Simple equations govern the dependence of overfitting on model/dataset size and the dependence of training speed on model size. These relationships allow us to determine the optimal allocation of a fixed compute budget. Larger models are significantly more sample-efficient, such that optimally compute-efficient training involves training very large models on a relatively modest amount of data and stopping significantly before convergence.
Bifurcation and stability of downflowing gyrotactic micro-organism suspensions in a vertical pipe

Abstract In the experiment that first demonstrated gyrotactic behaviour of bottom-heavy swimming microalgae (e.g. Chlamydomonas), Kessler (Nature, vol. 313, 1985, pp. 218–220) showed that a beam-like structure, often referred to as a gyrotactic plume, would spontaneously appear from a suspension of gyrotactic swimmers in a downflowing pipe. Such a plume is prone to an instability to form blips. This work models the gyrotactic plume as a steady parallel basic state and its subsequent breakdown into blips as an instability, employing both the generalized Taylor dispersion (GTD) theory and the Fokker–Planck model for comparison. Upon solving for the basic state, it is discovered that the steady plume solution undergoes sophisticated bifurcations. When there is no net flow, there exists a non-trivial solution of the plume structure other than the stationary uniform suspension, stemming from a transcritical bifurcation with the average cell concentration. When a net downflow is prescribed, there exists a cusp bifurcation. Furthermore, there is a critical concentration at which the cell concentration at the centre would blow up for the GTD model. The subsequent stability analysis using the steady plume solution shows that the Fokker–Planck model is inconsistent with what was experimentally observed, as it predicts stabilisation of axisymmetric blips at high concentration of the plume and destabilisation of the first non-axisymmetric mode at low flow rates.
Improving Speaker Discrimination of Target Speech Extraction With Time-Domain Speakerbeam

Target speech extraction, which extracts a single target source in a mixture given clues about the target speaker, has attracted increasing attention. We have recently proposed SpeakerBeam, which exploits an adaptation utterance of the target speaker to extract his/her voice characteristics that are then used to guide a neural network towards extracting speech of that speaker. SpeakerBeam presents a practical alternative to speech separation as it enables tracking speech of a target speaker across utterances, and achieves promising speech extraction performance. However, it sometimes fails when speakers have similar voice characteristics, such as in same-gender mixtures, because it is difficult to discriminate the target speaker from the interfering speakers. In this paper, we investigate strategies for improving the speaker discrimination capability of SpeakerBeam. First, we propose a time-domain implementation of SpeakerBeam similar to that proposed for a time-domain audio separation network (TasNet), which has achieved state-of-the-art performance for speech separation. Besides, we investigate (1) the use of spatial features to better discriminate speakers when microphone array recordings are available, (2) adding an auxiliary speaker identification loss for helping to learn more discriminative voice characteristics. We show experimentally that these strategies greatly improve speech extraction performance, especially for same-gender mixtures, and outperform TasNet in terms of target speech extraction.
Oracle Efficient Estimation of Structural Breaks in Cointegrating Regressions

In this article, we propose an adaptive group lasso procedure to efficiently estimate structural breaks in cointegrating regressions. It is well known that the group lasso estimator is not simultaneously estimation consistent and model selection consistent in structural break settings. Hence, we use a first step group lasso estimation of a diverging number of breakpoint candidates to produce weights for a second adaptive group lasso estimation. We prove that parameter changes are estimated consistently by group lasso and show that the number of estimated breaks is greater than the true number but still sufficiently close to it. Then, we use these results and prove that the adaptive group lasso has oracle properties if weights are obtained from our first step estimation. Simulation results show that the proposed estimator delivers the expected results. An economic application to the long‐run US money demand function demonstrates the practical importance of this methodology.
Stability of a non-local kinetic model for cell migration with density dependent orientation bias

The aim of the article is to study the stability of a non-local kinetic model proposed by Loy and Preziosi (2019a). We split the population in two subgroups and perform a linear stability analysis. We show that pattern formation results from modulation of one non-dimensional parameter that depends on the tumbling frequency, the sensing radius, the mean speed in a given direction, the uniform configuration density and the tactic response to the cell density. Numerical simulations show that our linear stability analysis predicts quite precisely the ranges of parameters determining instability and pattern formation. We also extend the stability analysis in the case of different mean speeds in different directions. In this case, for parameter values leading to instability travelling wave patterns develop.
Multilingual Denoising Pre-training for Neural Machine Translation

Abstract This paper demonstrates that multilingual denoising pre-training produces significant performance gains across a wide variety of machine translation (MT) tasks. We present mBART—a sequence-to-sequence denoising auto-encoder pre-trained on large-scale monolingual corpora in many languages using the BART objective (Lewis et al., 2019). mBART is the first method for pre-training a complete sequence-to-sequence model by denoising full texts in multiple languages, whereas previous approaches have focused only on the encoder, decoder, or reconstructing parts of the text. Pre-training a complete model allows it to be directly fine-tuned for supervised (both sentence-level and document-level) and unsupervised machine translation, with no task- specific modifications. We demonstrate that adding mBART initialization produces performance gains in all but the highest-resource settings, including up to 12 BLEU points for low resource MT and over 5 BLEU points for many document-level and unsupervised models. We also show that it enables transfer to language pairs with no bi-text or that were not in the pre-training corpus, and present extensive analysis of which factors contribute the most to effective pre-training.1
SPOQ $\ell _p$-Over-$\ell _q$ Regularization for Sparse Signal Recovery Applied to Mass Spectrometry

Underdetermined or ill-posed inverse problems require additional information for sound solutions with tractable optimization algorithms. Sparsity yields consequent heuristics to that matter, with numerous applications in signal restoration, image recovery, or machine learning. Since the <inline-formula><tex-math notation="LaTeX">$\ell _0$</tex-math></inline-formula> count measure is barely tractable, many statistical or learning approaches have invested in computable proxies, such as the <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula> norm. However, the latter does not exhibit the desirable property of scale invariance for sparse data. Extending the SOOT Euclidean/Taxicab <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula>-over-<inline-formula><tex-math notation="LaTeX">$\ell _2$</tex-math></inline-formula> norm-ratio initially introduced for blind deconvolution, we propose SPOQ, a family of smoothed (approximately) scale-invariant penalty functions. It consists of a Lipschitz-differentiable surrogate for <inline-formula><tex-math notation="LaTeX">$\ell _p$</tex-math></inline-formula>-over-<inline-formula><tex-math notation="LaTeX">$\ell _q$</tex-math></inline-formula> quasi-norm/norm ratios with <inline-formula><tex-math notation="LaTeX">$p\in \,]0,2[$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$q\geq 2$</tex-math></inline-formula>. This surrogate is embedded into a novel majorize-minimize trust-region approach, generalizing the variable metric forward-backward algorithm. For naturally sparse mass-spectrometry signals, we show that SPOQ significantly outperforms <inline-formula><tex-math notation="LaTeX">$\ell _0$</tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula>, Cauchy, Welsch, SCAD and <inline-formula><tex-math notation="LaTeX">$\mathrm{CEL}0$</tex-math></inline-formula> penalties on several performance measures. Guidelines on SPOQ hyperparameters tuning are also provided, suggesting simple data-driven choices.
Pruning CNN's with linear filter ensembles

Despite the promising results of convolutional neural networks (CNNs), their application on devices with limited resources is still a big challenge; this is mainly due to the huge memory and computation requirements of the CNN. To counter the limitation imposed by the network size, we use pruning to reduce the network size and -- implicitly -- the number of floating point operations (FLOPs). Contrary to the filter norm method -- used in ``conventional`` network pruning -- based on the assumption that a smaller norm implies ``less importance'' to its associated component, we develop a novel filter importance norm that is based on the change in the empirical loss caused by the presence or removal of a component from the network architecture. Since there are too many individual possibilities for filter configuration, we repeatedly sample from these architectural components and measure the system performance in the respective state of components being active or disabled. The result is a collection of filter ensembles -- filter masks -- and associated performance values. We rank the filters based on a linear and additive model and remove the least important ones such that the drop in network accuracy is minimal. We evaluate our method on a fully connected network, as well as on the ResNet architecture trained on the CIFAR-10 dataset. Using our pruning method, we managed to remove $60\%$ of the parameters and $64\%$ of the FLOPs from the ResNet with an accuracy drop of less than $0.6\%$.
Understanding the Incel Community on YouTube

YouTube is by far the largest host of user-generated video content worldwide. Alas, the platform also hosts inappropriate, toxic, and/or hateful content. One community that has come into the spotlight for sharing and publishing hateful content are the so-called Involuntary Celibates (Incels), a loosely defined movement ostensibly focusing on men's issues, who have often been linked to misogynistic views. In this paper, we set out to analyze the Incel community on YouTube by focusing on the evolution of this community over the last decade and understanding whether YouTube's recommendation algorithm steers users towards Incel-related videos. We collect videos shared on Incel-related communities within Reddit, and perform a data-driven characterization of the content posted on YouTube. Among other things, we find that the Incel community on YouTube is getting traction and that during the last decade the number of Incel-related videos and comments rose substantially. Also, we quantify the probability that a user will encounter an Incel-related video by virtue of YouTube's recommendation algorithm. Within five hops when starting from a non-Incel-related video, this probability is 1 in 5, which is alarmingly high as such content is likely to share toxic and misogynistic views.
Virtual Reality Analgesia With Interactive Eye Tracking During Brief Thermal Pain Stimuli: A Randomized Controlled Trial (Crossover Design)

In light of growing concerns about opioid analgesics, developing new non-pharmacologic pain control techniques has become a high priority. Adjunctive virtual reality can help reduce acute pain during painful medical procedures. However, for some especially painful medical procedures such as burn wound cleaning, clinical researchers recommend that more distracting versions of virtual reality are needed, to further amplify the potency of virtual reality analgesia. The current study with healthy volunteers explores for the first time whether interacting with virtual objects in Virtual Reality (VR) via “hands free” eye-tracking technology integrated into the VR helmet makes VR more effective/powerful than non-interactive/passive VR (no eye-tracking) for reducing pain during brief thermal pain stimuli. Method Forty eight healthy volunteers participated in the main study. Using a within-subject design, each participant received one brief thermal pain stimulus during interactive eye tracked virtual reality, and each participant received another thermal pain stimulus during non-interactive VR (treatment order randomized). After each pain stimulus, participants provided subjective 0–10 ratings of cognitive, sensory and affective components of pain, and rated the amount of fun they had during the pain stimulus. Results As predicted, interactive eye tracking increased the analgesic effectiveness of immersive virtual reality. Compared to the passive non-interactive VR condition, during the interactive eye tracked VR condition, participants reported significant reductions in worst pain (p < 0.001) and pain unpleasantness (p < 0.001). Participants reported a significantly stronger illusion of presence (p < 0.001), and significantly more fun in VR (p < 0.001) during the interactive condition compared to during passive VR. In summary, as predicted by our primary hypothesis, in the current laboratory acute pain analog study with healthy volunteers, increasing the immersiveness of the VR system via interactive eye tracking significantly increased how effectively VR reduced worst pain during a brief thermal pain stimulus. Although attention was not directly measured, the pattern of pain ratings, presence ratings, and fun ratings are consistent with an attentional mechanism for how VR reduces pain. Whether the current results generalize to clinical patient populations is another important topic for future research. Additional research and development is recommended.
Distinct Responses of Elasmobranchs and Ray-Finned Fishes to Long-Term Global Change

Both biotic and abiotic factors likely played a role in influencing the diversification patterns of clades. Although the role of environmental forcing on the long-term evolution of biodiversity has been explored for invertebrate clades, little is known about how vertebrate groups responded to environmental changes. Among vertebrates, fishes (ray-finned fishes and elasmobranchs) have a long, rich, and complex evolutionary history comprising numerous diversification and extinction events. Yet, knowledge on the causes for the diversity fluctuations of these most speciose aquatic vertebrate clades in modern marine and continental ecosystems were restricted to qualitative interpretations. Here we use multiple regression methods to quantitatively examine the role of six abiotic parameters over the long-term variations of elasmobranch and actinopterygian genus-level diversity. We find that marine actinopterygian diversity is mainly controlled by temperature while continental fragmentation is the primary driver of the diversity fluctuations of elasmobranchs. Sea-level variations correlate positively with the diversity variations of both marine groups, whereas none of the tested proxies explains the diversity variation of freshwater ray-finned fishes. Our results indicate that such contrasting responses are mainly due to ecological and life-history trait differences between these groups.
Ciconiiformes Nesting on Trees in Cereal-Dominated Farmlands: Importance of Scattered Trees for Heronries in Lowland Nepal

Abstract. Cereal farming is on the increase globally, and trees on these agricultural landscapes are selectively removed by farmers. Waterbird populations, supported on many cereal farmlands, need trees to place heronries and can be impacted by farmers' habits. In a cereal-dominated landscape in lowland Nepal, a landscape-scale field design was used to quantify characteristics of existing trees (density, species, girth, height), and contrasted against heronry trees to understand how choice of nest trees by select Ciconiiformes species was affected by farmers' habits. Tree density was patchy and tree species richness low, dominated by two tree species that had direct utility to farmers (Dalbergia sissoo – timber; Mangifera indica – fruits). Heronries were preferentially located on two wild tree species that were 8% of available trees and were either revered in local religion (Ficus religiosa; 36% of all heronries) or favored for agroforestry (Bombax ceiba; 42%). Heronries were preferentially located on larger trees. Availability of suitable trees for heronries was reduced by farmers' habits, but religious beliefs and agroforestry continued to support multi-functionality of cereal-dominated cropfields in lowland Nepal.
Anthropometric and Biochemical Correlates of PAPP-A2, Free IGF-I, and IGFBP-3 in Childhood.

OBJECTIVE Pregnancy-associated plasma protein-A2 (PAPP-A2) is a metalloproteinase that cleaves IGFBP-3 and -5. Human mutations in PAPPA2 result in short stature with a low percentage of free IGF-I. Little is known about PAPP-A2 levels and the regulation of free IGF-I throughout childhood. We examined PAPP-A2 and intact IGFBP-3 levels in childhood and explored associations between PAPP-A2, free and total IGF-I, and total and intact IGFBP-3 and their relationship to the percentage of free to total IGF-I and anthropometric factors. DESIGN Cross-sectional study at a single center. METHODS PAPP-A2, free IGF-I, and intact IGFBP-3 levels were measured in childhood (3-18 years old) and evaluation of the relationship between these proteins and anthropometric factors. RESULTS In 838 children, PAPP-A2 consistently decreased throughout childhood. In contrast, free IGF-I increased. A pubertal peak in free IGF-I was present in females but was less evident in males. Intact and total IGFBP-3 increased throughout childhood; however, intact IGFBP-3 had a more marked rise than total IGFBP-3. Percent free IGF-I decreased with no distinct pubertal peak. PAPP-A2 levels positively correlated with the percent free IGF-I (Male, Female; r = 0.18, 0.38; P < 0.001) and negatively with intact IGFBP-3 (Male, Female; r = -0.58, -0.65; P < 0.0001). CONCLUSIONS This is the first study to describe serum PAPP-A2 and intact IGFBP-3 in children between 3 and 18 years of age. Our correlative findings suggest that PAPP-A2 is an important regulator of the percent free IGF-I which can be a marker of perturbations in the GH/IGF-I axis.
Gaze Behavior in Social Fear Conditioning: An Eye-Tracking Study in Virtual Reality

The vigilance-avoidance hypothesis of selective attention assumes that socially anxious persons initially direct their attention toward fear-related stimuli and subsequently avoid these social stimuli to reduce emotional distress. New technical developments provide tools to implicit measure overt attention on fear-related stimuli via eye-tracking in ecological valid virtual environments presented via a head-mounted display. We examined in 27 low (LSA) and 26 high socially anxious (HSA) individuals fear ratings, physical behavior (duration of approach), hypervigilance (time to first fixation), and attentional avoidance (count of fixations) toward virtual female and male agents (CS) during social fear conditioning (SFC) and extinction in virtual reality (VR). As hypothesized, generally SFC was successfully induced and extinguished concerning the fear ratings. Our findings partly support the vigilance-avoidance hypothesis as HSA directed especially at the first half of the fear acquisition their initial attention more at CS+ than CS− agents, and avoided subsequently the CS+ more than the CS− agents during the fear acquisition. In contrast, in LSA participants initial and sustained attention did not differ between CS+ and CS− agents during fear acquisition. We conclude that HSA individuals guide their initial attention to emotionally threatening stimuli and subsequently avoid the threatening stimuli to possibly reduce their emotional distress, whereas LSA individuals regulate themselves less in their (fear) responses during SFC. Measuring implicit gaze behavior within a well-controlled virtual environment is an interesting innovative tool to in deeply investigate the impact of attention on emotional learning processes.
Phage Lytic Protein LysRODI Prevents Staphylococcal Mastitis in Mice

Phage lytic proteins are promising antimicrobials that could complement conventional antibiotics and help to combat multi-drug resistant bacteria that cause important human and animal infections. Here, we report the characterization of endolysin LysRODI (encoded by staphylophage phiIPLA-RODI) and its application as a prophylactic mastitis treatment. The main properties of LysRODI were compared with those of endolysin LysA72 (encoded by staphylophage phiIPLA35) and the chimeric protein CHAPSH3b (derived from the virion-associated peptidoglycan hydrolase HydH5 and lysostaphin). Time-kill experiments performed with Staphylococcus aureus and Staphylococcus epidermidis demonstrated that the killing rate of LysRODI and CHAPSH3b is higher than that of LysA72 (0.1 μM protein removed 107 CFU/ml of S. aureus in 30 min). Of note, all proteins failed to select resistant mutants as bacterial exposure to sub-lethal concentrations of the proteins did not alter the MIC values. Additionally, LysRODI and CHAPSH3b were non-toxic in a zebrafish embryo model at concentrations near the MIC (0.5 and 0.7 μM, respectively). Moreover, these two proteins significantly reduced mortality in a zebrafish model of systemic infection. In contrast to LysRODI, the efficacy of CHAPSH3b was dose-dependent in zebrafish, requiring higher-dose treatments to achieve the maximum survival rate. For this reason, LysRODI was selected for further analysis in mice, demonstrating great efficacy to prevent mammary infections by S. aureus and S. epidermidis. Our findings strongly support the use of phage lytic proteins as a new strategy to prevent staphylococcal mastitis.