Miracle-dz/UniFAR-FTUs huggingface dataset:
Identifying Depressive Symptoms from Tweets: Figurative Language Enabled Multitask Learning Framework

Existing studies on using social media for deriving mental health status of users focus on the depression detection task. However, for case management and referral to psychiatrists, health-care workers require practical and scalable depressive disorder screening and triage system. This study aims to design and evaluate a decision support system (DSS) to reliably determine the depressive triage level by capturing fine-grained depressive symptoms expressed in user tweets through the emulation of the Patient Health Questionnaire-9 (PHQ-9) that is routinely used in clinical practice. The reliable detection of depressive symptoms from tweets is challenging because the 280-character limit on tweets incentivizes the use of creative artifacts in the utterances and figurative usage contributes to effective expression. We propose a novel BERT based robust multi-task learning framework to accurately identify the depressive symptoms using the auxiliary task of figurative usage detection. Specifically, our proposed novel task sharing mechanism,co-task aware attention, enables automatic selection of optimal information across the BERT lay-ers and tasks by soft-sharing of parameters. Our results show that modeling figurative usage can demonstrably improve the model’s robustness and reliability for distinguishing the depression symptoms.
Towards Optimal Problem Dependent Generalization Error Bounds in Statistical Learning Theory

We study problem-dependent rates, that is, generalization errors that scale near-optimally with the variance, effective loss, or gradient norms evaluated at the “best hypothesis. ” We introduce a principled framework dubbed “uniform localized convergence” and characterize sharp problem-dependent rates for central statistical learning problems. From a methodological viewpoint, our framework resolves several fundamental limitations of existing uniform convergence and localization analysis approaches. It also provides improvements and some level of unification in the study of localized complexities, one-sided uniform inequalities, and sample-based iterative algorithms. In the so-called “slow rate” regime, we provide the first (moment-penalized) estimator that achieves the optimal variance-dependent rate for general “rich” classes; we also establish an improved loss-dependent rate for standard empirical risk minimization. In the “fast rate” regime, we establish finite-sample, problem-dependent bounds that are comparable to precise asymptotics. In addition, we show that iterative algorithms such as gradient descent and first order expectation maximization can achieve optimal generalization error in several representative problems across the areas of nonconvex learning, stochastic optimization, and learning with missing data. Supplemental Material: The online appendix is available at https://doi.org/10.1287/moor.2021. 0076 .
Resolvent-based estimation of turbulent channel flow using wall measurements

Abstract We employ a resolvent-based methodology to estimate velocity and pressure fluctuations within turbulent channel flows at friction Reynolds numbers of approximately 180, 550 and 1000 using measurements of shear stress and pressure at the walls, taken from direct numerical simulation (DNS) databases. Martini et al. (J. Fluid Mech., vol. 900, 2021, p. A2) showed that the resolvent-based estimator is optimal when the true space–time forcing statistics are utilised, thus providing an upper bound for the accuracy of any linear estimator. We use this framework to determine the flow structures that can be linearly estimated from wall measurements, and we characterise these structures and the estimation errors in both physical and wavenumber space. We also compare these results to those obtained using approximate forcing models – an eddy-viscosity model and white-noise forcing – and demonstrate the significant benefit of using true forcing statistics. All models lead to accurate results up to the buffer layer, but only using the true forcing statistics allows accurate estimation of large-scale logarithmic-layer structures, with significant correlation between the estimates and DNS results throughout the channel. The eddy-viscosity model displays an intermediate behaviour, which may be related to its ability to partially capture the forcing colour. Our results show that structures that leave a footprint on the channel walls can be accurately estimated using the linear resolvent-based methodology, and the presence of large-scale wall-attached structures enables accurate estimations through the logarithmic layer.
Dynamic allocation of limited memory resources in reinforcement learning

Biological brains are inherently limited in their capacity to process and store information, but are nevertheless capable of solving complex tasks with apparent ease. Intelligent behavior is related to these limitations, since resource constraints drive the need to generalize and assign importance differentially to features in the environment or memories of past experiences. Recently, there have been parallel efforts in reinforcement learning and neuroscience to understand strategies adopted by artificial and biological agents to circumvent limitations in information storage. However, the two threads have been largely separate. In this article, we propose a dynamical framework to maximize expected reward under constraints of limited resources, which we implement with a cost function that penalizes precise representations of action-values in memory, each of which may vary in its precision. We derive from first principles an algorithm, Dynamic Resource Allocator (DRA), which we apply to two standard tasks in reinforcement learning and a model-based planning task, and find that it allocates more resources to items in memory that have a higher impact on cumulative rewards. Moreover, DRA learns faster when starting with a higher resource budget than what it eventually allocates for performing well on tasks, which may explain why frontal cortical areas in biological brains appear more engaged in early stages of learning before settling to lower asymptotic levels of activity. Our work provides a normative solution to the problem of learning how to allocate costly resources to a collection of uncertain memories in a manner that is capable of adapting to changes in the environment.
Exploring the Value of Personalized Word Embeddings

In this paper, we introduce personalized word embeddings, and examine their value for language modeling. We compare the performance of our proposed prediction model when using personalized versus generic word representations, and study how these representations can be leveraged for improved performance. We provide insight into what types of words can be more accurately predicted when building personalized models. Our results show that a subset of words belonging to specific psycholinguistic categories tend to vary more in their representations across users and that combining generic and personalized word embeddings yields the best performance, with a 4.7% relative reduction in perplexity. Additionally, we show that a language model using personalized word embeddings can be effectively used for authorship attribution.
$\mathrm{\Lambda_{c}^{+}}$ production in pp and in p-Pb collisions at $\sqrt{s_{\rm {NN}}} = 5.02$ TeV

The production cross section of prompt $\mathrm{\Lambda_{c}^{+}}$ charm baryons was measured with the ALICE detector at the LHC at midrapidity in proton-proton (pp) and proton-lead (p-Pb) collisions at a centre-of-mass energy per nucleon pair of $\sqrt{s_\mathrm{NN}} = 5.02$ TeV. The $\mathrm{\Lambda_{c}^{+}}$ and $\rm {\overline{\Lambda}{}_c^-}$ baryons were reconstructed in the hadronic decay channels $\rm \Lambda_{c}^{+} \rightarrow p K^{-}\pi^{+}$ and $\rm \Lambda_{c}^{+}\to p K^{0}_{S}$ and respective charge conjugates. The measured differential cross sections as a function of transverse momentum ($p_{\rm T}$) and the $p_{\rm T}$-integrated $\mathrm{\Lambda_{c}^{+}}$ production cross section in pp and in p-Pb collisions are presented. The $\mathrm{\Lambda_{c}^{+}}$ nuclear modification factor ($R_\mathrm{pPb}$), calculated from the cross sections in pp and in p-Pb collisions, is presented and compared with the $R_\mathrm{pPb}$ of D mesons. The $\mathrm {\Lambda_{c}^{+}}/\mathrm {D^0}$ ratio is also presented and compared with the light-flavour baryon-to-meson ratios p$/\pi$ and $\Lambda /\mathrm {K^0_S}$, and measurements from other LHC experiments. The results are compared to predictions from model calculations and Monte Carlo event generators.
Sparse PCA: Algorithms, Adversarial Perturbations and Certificates

We study efficient algorithms for Sparse PCA in standard statistical models (spiked covariance in its Wishart form). Our goal is to achieve optimal recovery guarantees while being resilient to small perturbations. Despite a long history of prior works, including explicit studies of perturbation resilience, the best known algorithmic guarantees for Sparse PCA are fragile and break down under small adversarial perturbations. We observe a basic connection between perturbation resilience and certifying algorithms that are based on certificates of upper bounds on sparse eigenvalues of random matrices. In contrast to other techniques, such certifying algorithms, including the brute-force maximum likelihood estimator, are automatically robust against small adversarial perturbation. We use this connection to obtain the first polynomial-time algorithms for this problem that are resilient against additive adversarial perturbations by obtaining new efficient certificates for upper bounds on sparse eigenvalues of random matrices. Our algorithms are based either on basic semidefinite programming or on its low-degree sum-of-squares strengthening depending on the parameter regimes. Their guarantees either match or approach the best known guarantees of fragile algorithms in terms of sparsity of the unknown vector, number of samples and the ambient dimension. To complement our algorithmic results, we prove rigorous lower bounds matching the gap between fragile and robust polynomial-time algorithms in a natural computational model based on low-degree polynomials (closely related to the pseudo-calibration technique for sum-of-squares lower bounds) that is known to capture the best known guarantees for related statistical estimation problems. The combination of these results provides formal evidence of an inherent price to pay to achieve robustness. Beyond these issues of perturbation resilience, our analysis also leads to new algorithms for the fragile setting, whose guarantees improve over best previous results in some parameter regimes (e.g. if the sample size is polynomially smaller than the dimension).
On the asymptotics of the shifted sums of Hecke eigenvalue squares

Abstract The purpose of this paper is to obtain asymptotics of shifted sums of Hecke eigenvalue squares on average. We show that for X 2 3 + ϵ < H < X 1 - ϵ {X^{\frac{2}{3}+\epsilon}<H<X^{1-\epsilon}} there are constants B h {B_{h}} such that ∑ X ≤ n ≤ 2 ⁢ X λ f ⁢ ( n ) 2 ⁢ λ f ⁢ ( n + h ) 2 - B h ⁢ X = O f , A , ϵ ⁢ ( X ⁢ ( log ⁡ X ) - A ) \sum_{X\leq n\leq 2X}\lambda_{f}(n)^{2}\lambda_{f}(n+h)^{2}-B_{h}X=O_{f,A,% \epsilon}(X(\log X)^{-A}) for all but O f , A , ϵ ⁢ ( H ⁢ ( log ⁡ X ) - 3 ⁢ A ) {O_{f,A,\epsilon}(H(\log X)^{-3A})} integers h ∈ [ 1 , H ] {h\in[1,H]} where { λ f ⁢ ( n ) } n ≥ 1 {\{\lambda_{f}(n)\}_{n\geq 1}} are normalized Hecke eigenvalues of a fixed holomorphic cusp form f. Our method is based on the Hardy–Littlewood circle method. We divide the minor arcs into two parts m 1 {m_{1}} and m 2 {m_{2}} . In order to treat m 2 {m_{2}} , we use the Hecke relations, a bound of Miller to apply some arguments from a paper of Matomäki, Radziwiłł and Tao. In order to treat m 1 {m_{1}} , we apply Parseval’s identity and Gallagher’s lemma.
Characterization of glutamatergic VTA neural population responses to aversive and rewarding conditioning in freely-moving mice

The Ventral Tegmental Area (VTA) is a midbrain structure known to integrate aversive and rewarding stimuli, a function involving VTA Dopaminergic and GABAergic neurons. VTA also contains a less known population: glutamatergic (VGluT2) neurons. Direct activation of VGluT2 soma evokes rewarding behaviors, while stimulation of their axonal projections to the Nucleus Accumbens (NAc) and the Lateral Habenula (LHb) evokes aversive behaviors. Here, a systematic investigation of the VTAVGluT2+ population response to aversive or rewarding conditioning facilitated our understanding these conflicting properties. We recorded calcium signals from VTA glutamatergic population neurons using fiber photometry in VGluT2-cre mice to investigate how the VTA glutamatergic neuronal population was recruited by aversive and rewarding stimulation, both during unconditioned and conditioned protocols. Our results revealed that, as a population, VTAVGluT2+ neurons responded similarly to unconditioned-aversive and unconditioned-rewarding stimulation. During aversive and rewarding conditioning, the CS-evoked responses gradually increased across trials whilst the US-evoked response remained stable. Retrieval 24 h after conditioning, during which mice received only CS presentation, resulted in VTAVGluT2+ neurons strongly responding to CS presentation and to the expected-US but only for aversive conditioning. The inputs and outputs of VTAVGluT2+ neurons were then investigated using Cholera Toxin B (CTB) and rabies virus, and we propose based on all results that VTAVGluT2+ neurons specialized function may be partially due to their connectivity.
Transience and recurrence of sets for branching random walk via non-standard stochastic orders

We study how the recurrence and transience of space-time sets for a branching random walk on a graph depends on the offspring distribution. Here, we say that a space-time set $A$ is recurrent if it is visited infinitely often almost surely on the event that the branching random walk survives forever, and say that $A$ is transient if it is visited at most finitely often almost surely. We prove that if $\mu$ and $\nu$ are supercritical offspring distributions with means $\bar \mu < \bar \nu$ then every space-time set that is recurrent with respect to the offspring distribution $\mu$ is also recurrent with respect to the offspring distribution $\nu$ and similarly that every space-time set that is transient with respect to the offspring distribution $\nu$ is also transient with respect to the offspring distribution $\mu$. To prove this, we introduce a new order on probability measures that we call the germ order and prove more generally that the same result holds whenever $\mu$ is smaller than $\nu$ in the germ order. Our work is inspired by the work of Johnson and Junge (AIHP 2018), who used related stochastic orders to study the frog model.
Improving Model Accuracy for Imbalanced Image Classification Tasks by Adding a Final Batch Normalization Layer: An Empirical Study

Some real-world domains, such as Agriculture and Healthcare, comprise early-stage disease indications whose recording constitutes a rare event, and yet, whose precise detection at that stage is critical. In this type of highly imbalanced classification problems, which encompass complex features, deep learning (DL) is much needed because of its strong detection capabilities. At the same time, DL is observed in practice to favor majority over minority classes and consequently suffer from inaccurate detection of the targeted early-stage indications. To simulate such scenarios, we artificially generate skewness (99% vs. 1%) for certain plant types out of the PlantVillage dataset as a basis for classification of scarce visual cues through transfer learning. By randomly and unevenly picking healthy and unhealthy samples from certain plant types to form a training set, we consider a base experiment as fine-tuning ResNet34 and VGG19 architectures and then testing the model performance on a balanced dataset of healthy and unhealthy images. We empirically observe that the initial F1 test score jumps from 0.29 to 0.95 for the minority class upon adding a final Batch Normalization (BN) layer just before the output layer in VGG19. We demonstrate that utilizing an additional BN layer before the output layer in modern CNN architectures has a considerable impact in terms of minimizing the training time and testing error for minority classes in highly imbalanced data sets. Moreover, when the final BN is employed, minimizing the loss function may not be the best way to assure a high F1 test score for minority classes in such problems. That is, the network might perform better even if it is not ‘confident’ enough while making a prediction; leading to another discussion about why softmax output is not a good uncertainty measure for DL models. We also report on the corroboration of these findings on the ISIC Skin Cancer as well as the Wall Crack datasets.
CD36hi monocytes play immunoregulatory roles in human umbilical cord blood

The fetal and neonatal immune systems are uniquely poised to generate tolerance to self, maternal, and environmental antigens encountered in the womb and shortly after birth. The tolerogenic nature of fetal and neonatal immunity is a rising health concern with the spread of vertically transmitted viruses, such as the Zika virus. A variety of mechanisms contribute to fetal and neonatal tolerance, including a propensity to generate Foxp3+ regulatory T cells (Tregs). Here, we demonstrate that a subset of CD14+ monocytes expressing the scavenger molecule, CD36, is able to generate CD4+ and CD8+ T cells that express Foxp3 from umbilical cord blood (UCB). Monocyte-induced Foxp3+ T cells have potent suppressive functions on T cell proliferation and maintain Foxp3 expression over six weeks in vitro. Importantly, UCB-derived Foxp3+ T cells are distinguishable from adult peripheral blood (APB) CD4+CD25+ Tregs by surface antigen expression. While UCB-derived Foxp3+ T cells express prototypic Treg-associated surface antigens, such as CD25 and glucocorticoid-induced tumor necrosis factor-related receptor (GITR), only UCB-derived Foxp3+ T cells express CD26. In addition, most UCB-derived CD8+Foxp3+ T cells express CD31. Mechanistically, both APB and UCB-derived monocytes support the development of Foxp3+ T cells from naïve T cells, but APB naïve T cells are less efficient in expressing Foxp3 than UCB naïve T cells. These data suggest that antigen presentation by CD36hi monocytes in the fetus leads to the development of a group of T cells that share some but not all phenotypes of adult thymus-derived Tregs.
Heterogeneous Data-Aware Federated Learning

Federated learning (FL) is an appealing concept to perform distributed training of Neural Networks (NN) while keeping data private. With the industrialization of the FL framework, we identify several problems hampering its successful deployment, such as presence of non i.i. d data, disjoint classes, signal multi-modality across datasets. In this work, we address these problems by proposing a novel method that not only (1) aggregates generic model parameters (e.g. a common set of task generic NN layers) on server (e.g. in traditional FL), but also (2) keeps a set of parameters (e.g, a set of task specific NN layer) specific to each client. We validate our method on the traditionally used public benchmarks (e.g., Femnist) as well as on our proprietary collected dataset (i.e., traffic classification). Results show the benefit of our method, with significant advantage on extreme cases.
Domain Generalization in Biosignal Classification

Objective: When training machine learning models, we often assume that the training data and evaluation data are sampled from the same distribution. However, this assumption is violated when the model is evaluated on another unseen but similar database, even if that database contains the same classes. This problem is caused by domain-shift and can be solved using two approaches: domain adaptation and domain generalization. Simply, domain adaptation methods can access data from unseen domains during training; whereas in domain generalization, the unseen data is not available during training. Hence, domain generalization concerns models that perform well on inaccessible, domain-shifted data. Method: Our proposed domain generalization method represents an unseen domain using a set of known basis domains, afterwhich we classify the unseen domain using classifier fusion. To demonstrate our system, we employ a collection of heart sound databases that contain normal and abnormal sounds (classes). Results: Our proposed classifier fusion method achieves accuracy gains of up to 16% for four completely unseen domains. Conclusion: Recognizing the complexity induced by the inherent temporal nature of biosignal data, the two-stage method proposed in this study is able to effectively simplify the whole process of domain generalization while demonstrating good results on unseen domains and the adopted basis domains. Significance: To our best knowledge, this is the first study that investigates domain generalization for biosignal data. Our proposed learning strategy can be used to effectively learn domain-relevant features while being aware of the class differences in the data.
Susceptibility to biofilm formation on 3D-printed titanium fixation plates used in the mandible: a preliminary study

ABSTRACT Background: In the oral and maxillofacial surgery, fixation plates are commonly used for the stabilization of bone fragments. Additive manufacturing has enabled us to design and create personalized fixation devices that would ideally fit any given fracture. Aim: The aim of the present preliminary study was to assess the susceptibility of 3D-printed titanium fixation plates to biofilm formation. Methods: Plates were manufactured using selective laser melting (SLM) from Ti-6Al-4 V. Reference strains of Streptococcus mutans, Staphyloccocus epidermidis, Staphylococcus aureus, Lactobacillus rhamnosus, and Candida albicans, were tested to evaluate the material’s susceptibility to biofilm formation over 48 hours. Biofilm formations were quantified by a colorimetric method and colony-forming units (CFU) quantification. Scanning electron microscopy (SEM) visualized the structure of the biofilm. Results: Surface analysis revealed the average roughness of 102.75 nm and irregular topography of the tested plates. They were susceptible to biofilm formation by all tested strains. The average CFUs were as follows: S. mutans (11.91 x 107) > S.epidermidis (4.45 x 107) > S. aureus (2.3 x 107) > C.albicans (1.22 x 107) > L. rhamnosus (0.78 x 107). Conclusions: The present preliminary study showed that rough surfaces of additively manufactured titanium plates are susceptible to microbial adhesion. The research should be continued in order to compare additively manufactured plates with other commercially available osteotomy plates. Therefore, we suggest caution when using this type of material.