Miracle-dz/UniFAR-FTUs huggingface dataset:
Large Scale Structure of Neural Network Loss Landscapes

There are many surprising and perhaps counter-intuitive properties of optimization of deep neural networks. We propose and experimentally verify a unified phenomenological model of the loss landscape that incorporates many of them. High dimensionality plays a key role in our model. Our core idea is to model the loss landscape as a set of high dimensional \emph{wedges} that together form a large-scale, inter-connected structure and towards which optimization is drawn. We first show that hyperparameter choices such as learning rate, network width and $L_2$ regularization, affect the path optimizer takes through the landscape in a similar ways, influencing the large scale curvature of the regions the optimizer explores. Finally, we predict and demonstrate new counter-intuitive properties of the loss-landscape. We show an existence of low loss subspaces connecting a set (not only a pair) of solutions, and verify it experimentally. Finally, we analyze recently popular ensembling techniques for deep networks in the light of our model.
Conceptual Approaches to Modulating Antibody Effector Functions and Circulation Half-Life

Antibodies and Fc-fusion antibody-like proteins have become successful biologics developed for cancer treatment, passive immunity against infection, addiction, and autoimmune diseases. In general these biopharmaceuticals can be used for blocking protein:protein interactions, crosslinking host receptors to induce signaling, recruiting effector cells to targets, and fixing complement. With the vast capability of antibodies to affect infectious and genetic diseases much effort has been placed on improving and tailoring antibodies for specific functions. While antibody:antigen engagement is critical for an efficacious antibody biologic, equally as important are the hinge and constant domains of the heavy chain. It is the hinge and constant domains of the antibody that engage host receptors or complement protein to mediate a myriad of effector functions and regulate antibody circulation. Molecular and structural studies have provided insight into how the hinge and constant domains from antibodies across different species, isotypes, subclasses, and alleles are recognized by host cell receptors and complement protein C1q. The molecular details of these interactions have led to manipulation of the sequences and glycosylation of hinge and constant domains to enhance or reduce antibody effector functions and circulating half-life. This review will describe the concepts being applied to optimize the hinge and crystallizable fragment of antibodies, and it will detail how these interactions can be tuned up or down to mediate a biological function that confers a desired disease outcome.
Professionally- and Self-Trained Service Dogs: Benefits and Challenges for Partners With Disabilities

It has been widely reported that service dogs offer benefits to their human partners, however, it is unclear whether the expanding methods of training and roles of service dogs for their partners with various disabilities also provide similar benefits. This study aimed to investigate the self-reported experience of service dog partners to understand whether three different factors influence the benefits and drawbacks associated with partnering with a service dog: (1) different methods of training service dogs; (2) different severities of human partners' disabilities; (3) different roles of service dogs. Partners of service dogs were recruited to the web survey through service dog facilities and networking groups. Answers from 19 men and 147 women participants (91.8% living in the U.S.) were analyzed in this study. Participants experienced the expected benefits of service dogs, including increased independence, social relationships, self-esteem, and life satisfaction, and decreased anxiety, stress, and loneliness. However, the perceived benefits, concerns, and burdens differed depending on the partners' disabilities and the training history of the dogs. When first living with their service dogs, people who had self-trained their service dogs experienced more burdens than those living with professionally trained service dogs. No major reduction in expenses for assistance after acquiring a dog was reported. Personalized team training based on each person's disabilities and situation is required to optimize the benefits and minimize the burdens and concerns of living with service dogs.
Hypothesis Testing Under Subjective Priors and Costs as a Signaling Game

Many communication, sensor network, and networked control problems involve agents (decision makers) which have either misaligned objective functions or subjective probabilistic models. In the context of such setups, we consider binary signaling problems in which the decision makers (the transmitter and the receiver) have subjective priors and/or misaligned objective functions. Depending on the commitment nature of the transmitter to his policies, we formulate the binary signaling problem as a Bayesian game under either Nash or Stackelberg equilibrium concepts and establish equilibrium solutions and their properties. We show that there can be informative or non-informative equilibria in the binary signaling game under the Stackelberg and Nash assumptions, and derive the conditions under which an informative equilibrium exists for the Stackelberg and Nash setups. For the corresponding team setup, however, an equilibrium typically always exists and is always informative. Furthermore, we investigate the effects of small perturbations in priors and costs on equilibrium values around the team setup (with identical costs and priors), and show that the Stackelberg equilibrium behavior is not robust to small perturbations whereas the Nash equilibrium is.
Disparities in Diffuse Cortical White Matter Integrity Between Socioeconomic Groups

There is a growing literature demonstrating a link between lower socioeconomic status (SES) and poorer neuroanatomical health, such as smaller total and regional gray and white matter volumes, as well as greater white matter lesion volumes. Little is known, however, about the relation between SES and white matter integrity. Here we examined the relation between SES and white matter integrity of the brain’s primary cortical regions, and evaluated potential moderating influences of age and self-identified race. Participants were 192 neurologically intact, community-dwelling African American and White adults (mean age = 52 years; 44% male, 60% White, low SES = 52%) from the Healthy Aging in Neighborhoods of Diversity across the Life Span (HANDLS) SCAN study. Participants underwent 3.0-T cranial magnetic resonance imaging. Diffusion tensor imaging was used to estimate regional fractional anisotropy (FA) to quantify the brain’s white matter integrity and trace to capture diffusivity. Multiple regression analyses examined independent and interactive associations of SES, age, and race with FA of the frontal, temporal, parietal, and occipital lobes bilaterally. Sensitivity analyses assessed the influence of several biopsychosocial risk factors on these associations. Exploratory analyses examined these relations with trace and using additional SES indicators. Results indicated there were no significant interactions of SES, age, and race for any region. Individuals with low SES had lower FA in all regions, and higher trace in the right and left frontal, right and left temporal, and left occipital lobes. Findings remained largely unchanged after inclusion of sensitivity variables. Older age was associated with lower FA and greater trace for all regions, except for the right temporal lobe with FA. No main effects were found for race in FA, and Whites had higher trace values in the parietal lobes. Novel findings of this study indicate that relative to the high SES group, low SES was associated with poorer white matter integrity and greater diffusivity. These results may, in part, reflect exposures to various biopsychosocial risk factors experienced by those of lower SES across the lifespan, and may help explain the preponderance of cognitive and functional disparities between socioeconomic groups.
Efficiently escaping saddle points on manifolds

Smooth, non-convex optimization problems on Riemannian manifolds occur in machine learning as a result of orthonormality, rank or positivity constraints. First- and second-order necessary optimality conditions state that the Riemannian gradient must be zero, and the Riemannian Hessian must be positive semidefinite. Generalizing Jin et al.'s recent work on perturbed gradient descent (PGD) for optimization on linear spaces [How to Escape Saddle Points Efficiently (2017), Stochastic Gradient Descent Escapes Saddle Points Efficiently (2019)], we propose a version of perturbed Riemannian gradient descent (PRGD) to show that necessary optimality conditions can be met approximately with high probability, without evaluating the Hessian. Specifically, for an arbitrary Riemannian manifold $\mathcal{M}$ of dimension $d$, a sufficiently smooth (possibly non-convex) objective function $f$, and under weak conditions on the retraction chosen to move on the manifold, with high probability, our version of PRGD produces a point with gradient smaller than $\epsilon$ and Hessian within $\sqrt{\epsilon}$ of being positive semidefinite in $O((\log{d})^4 / \epsilon^{2})$ gradient queries. This matches the complexity of PGD in the Euclidean case. Crucially, the dependence on dimension is low. This matters for large-scale applications including PCA and low-rank matrix completion, which both admit natural formulations on manifolds. The key technical idea is to generalize PRGD with a distinction between two types of gradient steps: "steps on the manifold" and "perturbed steps in a tangent space of the manifold. " Ultimately, this distinction makes it possible to extend Jin et al.'s analysis seamlessly.
Fusing Deep Quick Response Code Representations Improves Malware Text Classification

Recently, multimodal processing for various text classification tasks such as emotion recognition, sentiment analysis, author profiling etc. have gained traction due to its potential to improve performance by leveraging complementary sources of information such as texts, images and speech. In this line, we focus on multimodal malware text classification. However unlike traditional tasks such as emotions recognition and sentiment analysis generating complementary domain information is difficult in cyber security, leading to little focus of such a multimodal idea in context of malware classification. As such, in this work we propose to address this gap by improving malware text classification task by leveraging Quick Response (QR) codes generated from the same as complementary information. With superior capacity of Convolutional Neural Network's (CNN) to process images, we fuse the representations from CNN's for both text and image data in multiple ways, where we show that using complementary information from QR codes improves the performance of the task of malware text classification thereby achieving new state-of-the-art and creating the very first multimodal benchmark on malware text classification.
The Impact of Parental Role Distributions, Work Participation, and Stress Factors on Family Health-Related Outcomes: Study Protocol of the Prospective Multi-Method Cohort “Dresden Study on Parenting, Work, and Mental Health” (DREAM)

The Dresden Study on Parenting, Work, and Mental Health (“DResdner Studie zu Elternschaft, Arbeit, und Mentaler Gesundheit”, DREAM) aims to prospectively investigate the relationship between parental work participation, role distribution, stress factors, and their effects on perinatal outcomes and long-term family mental and somatic health in a community sample targeting N = 4,000 individuals, i.e., 2,000 couples, expecting a child and residing in Dresden, Germany (interim sample of N = 1,410 participants, recruitment ongoing). Various questionnaires are completed at four measurement points from pregnancy to 2 years postpartum (prolongation into middle childhood planned). Applying a multi-method approach, long-term endocrinological data (analyses of hair cortisol concentrations and other endogenous hormones, “DREAMHAIR”) and qualitative interview data (regarding gender role attitudes and distribution of domestic work, child care, and paid employment; “DREAMTALK”) are obtained. In this study protocol, the theoretical background, methods, and preliminary results considering sociodemographic characteristics during pregnancy and birth-related factors at 8 weeks postpartum are presented. Additionally, there is a focus on our endocrinological sub-study DREAMHAIR. In this sub-study currently comprising N = 152 participants, i.e., 88 families (recruitment ongoing), we want to gain knowledge on the transgenerational processes of stress regulation and psychopathology in the whole family by analyzing hair cortisol concentrations in both parents and children during the course from pregnancy (or after birth regarding children) to at least 2 years postpartum. By comparing data of the community sample to a clinical sample of mothers with postpartum mental disorders, their children, and their partners during the period between admission and discharge from a mother-baby unit and post-treatment (“DREAMMBU”), the course of mothers' psychopathology, parent-infant interaction, and infant regulation disorders with special regard to long-term endocrine correlates will be examined. With previous studies neglecting the fathers or partners involved, a major advantage of DREAM is the use of a multi-method and multi-level approach by examining the whole family in a longitudinal design. Therefore, the DREAM study will contribute to a better understanding of the role of social, work, and stress factors for mental and somatic health and its long-term endocrine correlates in the natural course of becoming a family.
Bag of Color Features for Color Constancy

In this paper, we propose a novel color constancy approach, called Bag of Color Features (BoCF), building upon Bag-of-Features pooling. The proposed method substantially reduces the number of parameters needed for illumination estimation. At the same time, the proposed method is consistent with the color constancy assumption stating that global spatial information is not relevant for illumination estimation and local information (edges, etc.) is sufficient. Furthermore, BoCF is consistent with color constancy statistical approaches and can be interpreted as a learning-based extension of many statistical approaches. To further improve the illumination estimation accuracy, we propose a novel attention mechanism for the BoCF model with two variants based on self-attention. BoCF approach and its variants achieve competitive, compared to the state of the art, results while requiring much fewer parameters on three benchmark datasets: ColorChecker RECommended, INTEL-TUT version 2, and NUS8.
Cisplatin Eligibility Issues and Alternative Regimens in Locoregionally Advanced Head and Neck Cancer: Recommendations for Clinical Practice

Well-designed randomized trials provide the highest level of scientific evidence to guide clinical decision making. In chemoradiotherapy of locally advanced squamous cell carcinoma of the head and neck (SCCHN), data support the use of three cycles of 100 mg/m2 cisplatin given every 3 weeks concurrently with conventionally fractionated external beam radiotherapy, although a full compliance with all three cycles is reserved to only about two thirds of initially eligible cases. On an individual patient level, practicing oncologists have to determine whether the patient is a suitable candidate for this treatment or whether contraindications exist. In the latter case, an adequate alternative has to be offered. In this regard, to facilitate triaging of medical information, we reviewed available publications on this topic and prepared practice-oriented recommendations for systemic treatment concurrent to definitive and post-operative radiotherapy. Even if no contraindications for the standard-of-care cisplatin apply, clinicians may opt for alternative regimens by adjusting the peak dose, cumulative dose, or timing of cisplatin. Relative contraindications pose the major issue in clinical practice, as very limited data is available in the literature and final decisions are usually based on an expert opinion or retrospective cohort studies. In the case of absolute interdiction of cisplatin, several alternative regimens incorporating carboplatin, 5-fluorouracil, cetuximab, and docetaxel are available. At the same time, it should be kept in mind that radiotherapy alone represents a viable option with hyperfractionation being particularly beneficial in the definitive management of limited nodal disease. Ideally, all treatment propositions should be discussed within multidisciplinary tumor boards taking into account the patient- and disease-related characteristics as well as local logistics and reimbursement policies.
The Effects of Trait Emotional Intelligence on Adolescent Substance Use: Findings From a Hungarian Representative Survey

Previous research has emphasized the importance of emotions in the development of adult and adolescent substance use. There is substantial evidence for deficits in emotional processing among teenagers with substance use, but few studies have investigated the association between emotional intelligence and adolescent substance use. The aim of the present study was to examine the relationship between the use of tobacco, alcohol, and illicit drugs and level of emotional intelligence among adolescents. A representative sample of high school students participated in the study (N = 2,380). Substance use patterns were assessed using data from the European School Survey Project on Alcohol and Other Drugs (ESPAD) Survey, and emotional intelligence was assessed with the Bar-On Emotional Quotient Inventory Youth Version. Self-esteem and depressive symptomatology were also assessed to compare their effects on the frequency of substance use with the effect of emotional intelligence. Results demonstrated that greater difficulty in stress management and empathy predicted a higher frequency of tobacco, alcohol, and cannabis use. However, the level of emotional intelligence showed only a weak relationship to substance use habits. Latent profile analyses supported the hypothesis that different emotional patterns and problems underlie different types of psychoactive substances. Using a multiple linear regression model, the present study found that although emotional intelligence is not a key factor underlying substance use habits, it has an individual effect on substance use beyond depressive tendencies and self-esteem. These results can be applied to both drug prevention programs and interventions in substance abuse treatment.
The Formation of Bimodal Dust Species in Nova Ejecta

The formation of bimodal dust species (namely the silicate and amorphous carbon dust grains coexistent) in a nova eruption is an open problem. According to the nova model simulated by Modules for Experiments in Stellar Astrophysics code, we calculate the formation and growth of carbon (C) and forsterite (Mg2SiO4) dust grains purely in nova ejecta for the fee-expansion model and the radiative shock model by assuming spherical geometry of the nova ejecta. In these models, the chemical properties of pre-existing circumstellar medium are not taken into account. In the free-expansion model, the nova ejecta is not an idea environment for dust nucleation. However, it can efficiently produce dust in the radiative shock model. We estimate that every nova can produce C grains with an average mass of about 10−9 and 10−8 M⊙, and Mg2SiO4 grains with an average mass of about 10−8 and 10−7 M⊙. Based on the mass of ejected gas, the ratio of dust to gas is about 1%. The C grains form first after several or tens of days of nova eruption. After that, the Mg2SiO4 grains begin to grow in tens of days, which is consistent with observations.
Psycholinguistics Meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering

We study the issue of catastrophic forgetting in the context of neural multimodal approaches to Visual Question Answering (VQA). Motivated by evidence from psycholinguistics, we devise a set of linguistically-informed VQA tasks, which differ by the types of questions involved (Wh-questions and polar questions). We test what impact task difficulty has on continual learning, and whether the order in which a child acquires question types facilitates computational models. Our results show that dramatic forgetting is at play and that task difficulty and order matter. Two well-known current continual learning methods mitigate the problem only to a limiting degree.
Skyrmion-electronics: writing, deleting, reading and processing magnetic skyrmions toward spintronic applications

The field of magnetic skyrmions has been actively investigated across a wide range of topics during the last decades. In this topical review, we mainly review and discuss key results and findings in skyrmion research since the first experimental observation of magnetic skyrmions in 2009. We particularly focus on the theoretical, computational and experimental findings and advances that are directly relevant to the spintronic applications based on magnetic skyrmions, i.e. their writing, deleting, reading and processing driven by magnetic field, electric current and thermal energy. We then review several potential applications including information storage, logic computing gates and non-conventional devices such as neuromorphic computing devices. Finally, we discuss possible future research directions on magnetic skyrmions, which also cover rich topics on other topological textures such as antiskyrmions and bimerons in antiferromagnets and frustrated magnets.
Discrimination of Pancreatic Serous Cystadenomas From Mucinous Cystadenomas With CT Textural Features: Based on Machine Learning

Objectives: This study was designed to estimate the performance of textural features derived from contrast-enhanced CT in the differential diagnosis of pancreatic serous cystadenomas and pancreatic mucinous cystadenomas. Methods: Fifty-three patients with pancreatic serous cystadenoma and 25 patients with pancreatic mucinous cystadenoma were included. Textural parameters of the pancreatic neoplasms were extracted using the LIFEx software, and were analyzed using random forest and Least Absolute Shrinkage and Selection Operator (LASSO) methods. Patients were randomly divided into training and validation sets with a ratio of 4:1; random forest method was adopted to constructed a diagnostic prediction model. Scoring metrics included sensitivity, specificity, accuracy, and AUC. Results: Radiomics features extracted from contrast-enhanced CT were able to discriminate pancreatic mucinous cystadenomas from serous cystadenomas in both the training group (slice thickness of 2 mm, AUC 0.77, sensitivity 0.95, specificity 0.83, accuracy 0.85; slice thickness of 5 mm, AUC 0.72, sensitivity 0.90, specificity 0.84, accuracy 0.86) and the validation group (slice thickness of 2 mm, AUC 0.66, sensitivity 0.86, specificity 0.71, accuracy 0.74; slice thickness of 5 mm, AUC 0.75, sensitivity 0.85, specificity 0.83, accuracy 0.83). Conclusions: In conclusion, our study provided preliminary evidence that textural features derived from CT images were useful in differential diagnosis of pancreatic mucinous cystadenomas and serous cystadenomas, which may provide a non-invasive approach to determine whether surgery is needed in clinical practice. However, multicentre studies with larger sample size are needed to confirm these results.