Miracle-dz/UniFAR-FTUs huggingface dataset:
Bidirectional Representation Learning from Transformers using Multimodal Electronic Health Record Data for Chronic to Predict Depression

Advancements in machine learning algorithms have had a beneficial impact on representation learning, classification, and prediction models built using electronic health record (EHR) data. Effort has been put both on increasing models' overall performance as well as improving their interpretability, particularly regarding the decision-making process. In this study, we present a temporal deep learning model to perform bidirectional representation learning on EHR sequences with a transformer architecture to predict future diagnosis of depression. This model is able to aggregate five heterogenous and high-dimensional data sources from the EHR and process them in a temporal manner for chronic disease prediction at various prediction windows. We applied the current trend of pretraining and fine-tuning on EHR data to outperform the current state-of-the-art in chronic disease prediction, and to demonstrate the underlying relation between EHR codes in the sequence. The model generated the highest increases of precision-recall area under the curve (PRAUC) from 0.70 to 0.76 in depression prediction compared to the best baseline model. Furthermore, the self-attention weights in each sequence quantitatively demonstrated the inner relationship between various codes, which improved the model's interpretability. These results demonstrate the model's ability to utilize heterogeneous EHR data to predict depression while achieving high accuracy and interpretability, which may facilitate constructing clinical decision support systems in the future for chronic disease screening and early detection.
Fixed Points Theorems for Non-Transitive Relations

In this paper, we develop an Isabelle/HOL library of order-theoretic fixed-point theorems. We keep our formalization as general as possible: we reprove several well-known results about complete orders, often with only antisymmetry or attractivity, a mild condition implied by either antisymmetry or transitivity. In particular, we generalize various theorems ensuring the existence of a quasi-fixed point of monotone maps over complete relations, and show that the set of (quasi-)fixed points is itself complete. This result generalizes and strengthens theorems of Knaster-Tarski, Bourbaki-Witt, Kleene, Markowsky, Pataraia, Mashburn, Bhatta-George, and Stouti-Maaden.
Formation and Evolution of Ultraluminous X-Ray Pulsar Binaries to Pulsar–Neutron Star and Pulsar–White Dwarf Systems

Recent observational and theoretical results have suggested that some of ultraluminous X-ray (ULX) sources may contain neutron star (NS) accretors. However, the formation channel and properties of donor stars of NS ULXs remain uncertain. By adopting the nonconservative and rotation-dependent mass transfer model in the primordial binary evolution, we investigate the way to form pulsar ULXs like observed pulsar ULXs in a systematic way. Our simulation results indicate that pulsar ULXs with Be stars and intermediate or/and high-mass donors match observed apparent luminosities, orbital periods, and observationally indicated donor masses of known pulsar ULXs. ULXs with Be and intermediate donors are main contributors. The route of accretion-induced collapse of WDs has a 4.5% contribution to the NS ULXs, 4.0% of NSs in ULXs are formed through electron-capture supernovae (SNe), and 91.5% of NSs in ULXs are born with core-collapse SNe. We also studied the evolution of pulsar ULXs to double compact star systems. We do not find NS–black hole systems (merging in a Hubble time) that evolved from pulsar ULXs. Pulsar–white dwarf (WD) cases that evolve through pulsar ULXs have significant contributions to the whole NS–WD gravitational wave sources. Contributions of pulsar–WD and pulsar–NS cases that experienced pulsar ULXs are ∼40% and 11% among all LISA NS–WD and NS–NS sources, respectively. Monte Carlo simulation noise with different models give a nonnegligible uncertainty.
Reinforcement Learning-based N-ary Cross-Sentence Relation Extraction

The models of n-ary cross sentence relation extraction based on distant supervision assume that consecutive sentences mentioning n entities describe the relation of these n entities. However, on one hand, this assumption introduces noisy labeled data and harms the models' performance. On the other hand, some non-consecutive sentences also describe one relation and these sentences cannot be labeled under this assumption. In this paper, we relax this strong assumption by a weaker distant supervision assumption to address the second issue and propose a novel sentence distribution estimator model to address the first problem. This estimator selects correctly labeled sentences to alleviate the effect of noisy data is a two-level agent reinforcement learning model. In addition, a novel universal relation extractor with a hybrid approach of attention mechanism and PCNN is proposed such that it can be deployed in any tasks, including consecutive and nonconsecutive sentences. Experiments demonstrate that the proposed model can reduce the impact of noisy data and achieve better performance on general n-ary cross sentence relation extraction task compared to baseline models.
On-The-Fly Control of Unknown Smooth Systems from Limited Data

We investigate the problem of data-driven, on-the-fly control of systems with unknown nonlinear dynamics where data from only a single finite-horizon trajectory and possibly side information on the dynamics are available. Such side information may include knowledge of the regularity of the dynamics, monotonicity of the states, or decoupling in the dynamics between the states. Specifically, we develop two algorithms, DaTaReach and DaTaControl, to over-approximate the reachable set and design control signals for the system on the fly. DaTaReach constructs a differential inclusion that contains the unknown vector field. Then, it computes an over-approximation of the reachable set based on interval Taylor-based methods applied to systems with dynamics described as differential inclusions. DaTaControl enables convex-optimization-based, near-optimal control using the computed over-approximation and the receding-horizon control framework. We provide a bound on its suboptimality and show that more data and side information enable DaTaControl to achieve tighter suboptimality bounds. Finally, we demonstrate the efficacy of DaTaControl over existing approaches on the problems of controlling a unicycle and quadrotor systems.
What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties

Multilingual sentence encoders have seen much success in cross-lingual model transfer for downstream NLP tasks. Yet, we know relatively little about the properties of individual languages or the general patterns of linguistic variation that they encode. We propose methods for probing sentence representations from state-of-the-art multilingual encoders (LASER, M-BERT, XLM and XLM-R) with respect to a range of typological properties pertaining to lexical, morphological and syntactic structure. In addition, we investigate how this information is distributed across all layers of the models. Our results show interesting differences in encoding linguistic variation associated with different pretraining strategies.
On Artin’s conjecture for pairs of diagonal forms

Let [Formula: see text] be an odd prime and [Formula: see text]. In the spirit of Artin’s conjecture, consider the system of two diagonal forms of degree [Formula: see text] in [Formula: see text] variables given by [Formula: see text] [Formula: see text] with [Formula: see text]. For [Formula: see text], this paper shows that this system has a non-trivial [Formula: see text]-adic solution for every [Formula: see text], and for every [Formula: see text], where [Formula: see text]. Moreover, for [Formula: see text], this system will have a non-trivial [Formula: see text]-adic solution for every [Formula: see text].
Over-the-Air Federated Learning From Heterogeneous Data

We focus on over-the-air (OTA) Federated Learning (FL), which has been suggested recently to reduce the communication overhead of FL due to the repeated transmissions of the model updates by a large number of users over the wireless channel. In OTA FL, all users simultaneously transmit their updates as analog signals over a multiple access channel, and the server receives a superposition of the analog transmitted signals. However, this approach results in the channel noise directly affecting the optimization procedure, which may degrade the accuracy of the trained model. We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local stochastic gradient descent (SGD) FL algorithm, introducing precoding at the users and scaling at the server, which gradually mitigates the effect of noise. We analyze the convergence of COTAF to the loss minimizing model and quantify the effect of a statistically heterogeneous setup, i.e. when the training data of each user obeys a different distribution. Our analysis reveals the ability of COTAF to achieve a convergence rate similar to that achievable over error-free channels. Our simulations demonstrate the improved convergence of COTAF over vanilla OTA local SGD for training using non-synthetic datasets. Furthermore, we numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL.
Beyond COVID-19: network science and sustainable exit strategies

On May 28th and 29th, a two day workshop was held virtually, facilitated by the Beyond Center at ASU and Moogsoft Inc. The aim was to bring together leading scientists with an interest in network science and epidemiology to attempt to inform public policy in response to the COVID-19 pandemic. Epidemics are at their core a process that progresses dynamically upon a network, and are a key area of study in network science. In the course of the workshop a wide survey of the state of the subject was conducted. We summarize in this paper a series of perspectives of the subject, and where the authors believe fruitful areas for future research are to be found.
Efficient Scaling and Moving Techniques for Spectral Methods in Unbounded Domains

When using Laguerre and Hermite spectral methods to numerically solve PDEs in unbounded domains, the number of collocation points assigned inside the region of interest is often insufficient, particularly when the region is expanded or translated to safely capture the unknown solution. Simply increasing the number of collocation points cannot ensure a fast convergence to spectral accuracy. In this paper, we propose a scaling technique and a moving technique to adaptively cluster enough collocation points in a region of interest in order to achieve a fast spectral convergence. Our scaling algorithm employs an indicator in the frequency domain that is used to determine when scaling is needed and informs the tuning of a scaling factor to redistribute collocation points to adapt to the diffusive behavior of the solution. Our moving technique adopts an exterior-error indicator and moves the collocation points to capture the translation. Both frequency and exterior-error indicators are defined using only the numerical solutions. We apply our methods to a number of different models, including diffusive and moving Fermi-Dirac distributions and nonlinear Dirac solitary waves, and demonstrate recovery of spectral convergence for time-dependent simulations. Performance comparison in solving a linear parabolic problem shows that our frequency scaling algorithm outperforms the existing scaling approaches. We also show our frequency scaling technique is able to track the blowup of average cell sizes in a model for cell proliferation.
Interest-Behaviour Multiplicative Network for Resource-limited Recommendation

Resource constraints, e.g. limited product inventory or product categories, may affect consumers' choices or preferences in some recommendation tasks, but are usually ignored in previous recommendation methods. In this paper, we aim to mine the cue of user preferences in resource-limited recommendation tasks, for which purpose we specifically build a largely used car transaction dataset possessing resource-limitation characteristics. Accordingly, we propose an interest-behaviour multiplicative network to predict the user's future interaction based on dynamic connections between users and items. To describe the user-item connection dynamically, mutually-recursive recurrent neural networks (MRRNNs) are introduced to capture interactive long-term dependencies, and meantime effective representations of users and items are obtained. To further take the resource limitation into consideration, a resource-limited branch is built to specifically explore the influence of resource variation caused by user behaviour for user preferences. Finally, mutual information is introduced to measure the similarity between the user action and fused features to predict future interaction, where the fused features come from both MRRNNs and resource-limited branches. We test the performance on the built used car transaction dataset as well as the Tmall dataset, and the experimental results verify the effectiveness of our framework.
RoGAT: a robust GNN combined revised GAT with adjusted graphs

Graph Neural Networks(GNNs) are useful deep learning models to deal with the non-Euclid data. However, recent works show that GNNs are vulnerable to adversarial attacks. Small perturbations can lead to poor performance in many GNNs, such as Graph attention networks(GATs). Therefore, enhancing the robustness of GNNs is a critical problem. Robust GAT(RoGAT) is proposed to improve the robustness of GNNs in this paper, . Note that the original GAT uses the attention mechanism for different edges but is still sensitive to the perturbation, RoGAT adjusts the edges' weight to adjust the attention scores progressively. Firstly, RoGAT tunes the edges weight based on the assumption that the adjacent nodes should have similar nodes. Secondly, RoGAT further tunes the features to eliminate feature's noises since even for the clean graph, there exists some unreasonable data. Then, we trained the adjusted GAT model to defense the adversarial attacks. Different experiments against targeted and untargeted attacks demonstrate that RoGAT outperforms significantly than most the state-of-the-art defense methods. The implementation of RoGAT based on the DeepRobust repository for adversarial attacks.
Multi-timescale representation learning in LSTM Language Models

Although neural language models are effective at capturing statistics of natural language, their representations are challenging to interpret. In particular, it is unclear how these models retain information over multiple timescales. In this work, we construct explicitly multi-timescale language models by manipulating the input and forget gate biases in a long short-term memory (LSTM) network. The distribution of timescales is selected to approximate power law statistics of natural language through a combination of exponentially decaying memory cells. We then empirically analyze the timescale of information routed through each part of the model using word ablation experiments and forget gate visualizations. These experiments show that the multi-timescale model successfully learns representations at the desired timescales, and that the distribution includes longer timescales than a standard LSTM. Further, information about high-,mid-, and low-frequency words is routed preferentially through units with the appropriate timescales. Thus we show how to construct language models with interpretable representations of different information timescales.
The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances

Face recognition (FR) is one of the most popular and long-standing topics in computer vision. With the recent development of deep learning techniques and large-scale datasets, deep face recognition has made remarkable progress and has been widely used in many real-world applications. Given a natural image or video frame as input, an end-to-end deep face recognition system outputs the face feature for recognition. To achieve this, a typical end-to-end system is built with three key elements: face detection, face alignment, and face representation. Face detection locates faces in the image or frame. Then, the face alignment is proceeded to calibrate the faces to the canonical view and crop them with a normalized pixel size. Finally, in the stage of face representation, the discriminative features are extracted from the aligned face for recognition. Nowadays, all of the three elements are fulfilled by the technique of deep convolutional neural network. In this survey article, we present a comprehensive review about the recent advance of each element of the end-to-end deep face recognition, since the thriving deep learning techniques have greatly improved their capability of them. To start with, we present an overview of the end-to-end deep face recognition. Then, we review the advance of each element, respectively, covering many aspects such as the to-date algorithm designs, evaluation metrics, datasets, performance comparison, existing challenges, and promising directions for future research. Also, we provide a detailed discussion about the effect of each element on its subsequent elements and the holistic system. Through this survey, we wish to bring contributions in two aspects: first, readers can conveniently identify the methods which are quite strong-baseline style in the subcategory for further exploration; second, one can also employ suitable methods for establishing a state-of-the-art end-to-end face recognition system from scratch.
Antifungal, antitoxigenic, and antioxidant activities of the essential oil from laurel (Laurus nobilis L.): Potential use as wheat preservative

Abstract Essential oils (EOs) are widely used in the food industry as natural food preservatives to extend product shelf life and as flavoring agents. The aim of this work was to study the chemical profile of the EO from laurel (Laurus nobilis) and its antifungal, antitoxigenic, and antioxidant activities. The extractive yield of the EO from Algerian laurel was 1.13% being 1,8‐cineole the most dominant compound (35.5%) by gas chromatography–mass spectrometry analysis. The values of minimum inhibitory concentration and minimum fungicidal concentration (MFC) against Aspergillus flavus were 1.75 and 2 mg/ml, respectively. The production of aflatoxin B1 was inhibited by EO concentrations between 0.25 mg/ml (15% decrease) and 1.50 mg/ml (86% decrease), and it was totally inhibited at the MFC value. The EO showed a wide antifungal spectrum against other species in a dose‐dependent manner. In a food‐model study, the L. nobilis EO showed remarkable efficacy in fumigated wheat grains, providing from 51.5% to 76.7% protection against A. flavus during 6‐month storage. The L. nobilis EO showed good free radical scavenging activity by DPPH assay (IC50 value of 602 μg/ml) and moderate antioxidant activity in the β‐carotene bleaching assay (46% inhibition of linoleic acid oxidation). The conclusions of this study justify future research for the application of EO from laurel as a natural preservative to improve food safety and extend shelf life by controlling spoilage and toxigenic molds as well as oxidative damage.
Hybrid Fc‐fused interleukin‐7 induces an inflamed tumor microenvironment and improves the efficacy of cancer immunotherapy

Emerging oncotherapeutic strategies require the induction of an immunostimulatory tumor microenvironment (TME) containing numerous tumor‐reactive CD8+ T cells. Interleukin‐7 (IL‐7), a T‐cell homeostatic cytokine, induces an antitumor response; however, the detailed mechanisms underlying the contributions of the IL‐7 to TME remain unclear. Here, we aimed to investigate the mechanism underlying the induction of antitumor response by hybrid Fc‐fused long‐acting recombinant human IL‐7 (rhIL‐7‐hyFc) through regulation of both adaptive and innate immune cells in the TME.
Study on Feeding Natural Sediment of Perinereis aibuhitensis

Some Perinereis aibuhitensis were cultured in the laboratory,and the gut content experiments were carried out later.The result of particle selection experiment shows that P.aibuhitensis intendes to select smaller particles,and this result accords with the predictions of deposit-feeders optimal foraging models.The feeding rate of P.aibuhitensis on natural sedi-ment is 0.469±0.196,g/(g·d),and the weights are dry weights of sediment and P.aibuhitensis body.This rate was much lower than that of other polychaeta.The result of feeding rate experiment also shows that P.aibuhitensis will eat enough food after 2.5 hours,and its feeding cycle is about 7 hours.Compare with similar studiest,he reasons of these differences are also discussed in this paper.
Protective Effect of Astaxanthin on Primary Retinal Cells of the Gerbil Psammomys Obesus Cultured in Diabetic Milieu

Astaxanthin is a major marine carotenoid with powerful antioxidant and neuroprotective properties. The in vitro protective effect of astaxanthin in adult retinal cells of the type-2 diabetic model Psammomys obesus in hyperglycemic conditions was investigated. Primary retinal cells were cultured in normal (5 mM) or high concentrations of glucose (25 and 40 mM) for 5 days and treated with 1–20 µM astaxanthin for the last 48 h of culturing. Mitochondrial dehydrogenase activity and cell viability were assessed using MTT test and trypan blue exclusion dye. Retinal cells were characterized by immunohistochemistry. The results showed that mitochondrial function increased significantly (P < 0.05) with 5–20 µM astaxanthin in 25–40 mM glucose. At 10 µM astaxanthin, cell viability recovered to 78.51 ± 5.81% and 54.37 ± 9.64% of control in 25 and 40 mM, respectively. Neurons and glial cells expression were enhanced in elevated glucose conditions. These finding suggest that astaxanthin exerted neuroprotection against high glucose induced- retinal damage. Practical Applications The xanthophyll astaxanthin is found in many marine food sources such as shrimp. The cytoprotective and antioxidant capacity of astaxanthin in retinal disease has been pointed out and presented as a promising therapeutic strategy. The present study shows that astaxanthin can interfere with hyperglycemia-induced retinal cell death and that primary retinal culture of Psammomys obesus may represent an effective system to explore astaxanthin antioxidant mechanisms in diabetic retinopathy.