Miracle-dz/UniFAR-FTUs huggingface dataset:
NormFormer: Improved Transformer Pretraining with Extra Normalization

During pretraining, the Pre-LayerNorm transformer suffers from a gradient magnitude mismatch: gradients at early layers are much larger than at later layers. These issues can be alleviated by our proposed NormFormer architecture, which adds three normalization operations to each layer: a Layer Norm after self attention, head-wise scaling of self-attention outputs, and a Layer Norm after the first fully connected layer. The extra operations incur negligible compute cost (+0.4% parameter increase), but improve pretraining perplexity and downstream task performance for both causal and masked language models ranging from 125 Million to 2.7 Billion parameters. For example, adding NormFormer on top of our strongest 1.3B parameter baseline can reach equal perplexity 24% faster, or converge 0.27 perplexity better in the same compute budget. This model reaches GPT3-Large (1.3B) zero shot performance 60% faster. For masked language modeling, NormFormer improves fine-tuned GLUE performance by 1.9% on average. Code to train NormFormer models is available in fairseq https://github.com/pytorch/fairseq/tree/main/examples/normformer .
Probing as Quantifying the Inductive Bias of Pre-trained Representations

Pre-trained contextual representations have led to dramatic performance improvements on a range of downstream tasks. This has motivated researchers to quantify and understand the linguistic information encoded in them. In general, this is done by probing, which consists of training a supervised model to predict a linguistic property from said representations. Unfortunately, this deﬁnition of probing has been subject to extensive criticism, and can lead to paradoxical or counter-intuitive results. In this work, we present a novel framework for probing where the goal is to evaluate the inductive bias of representations for a particular task, and provide a practical avenue to do this using Bayesian inference. We apply our framework to a series of token-, arc-, and sentence-level tasks. Our results suggest that our framework solves problems of previous approaches and that fastText can offer a better inductive bias than BERT in certain situations.
Understanding Health Video Engagement: An Interpretable Deep Learning Approach

Predicting video engagement is a top priority for content creators and video-sharing platforms. Early assessment of video popularity enables content creators to maximize their influences and optimize video production budgets. Video sharing platforms also disclosed that they rely on video engagement prediction to promote credible videos and curb violative videos. Deep learning has been deployed to predict video engagement. While achieving state-of-the-art predictive performance, deep learning lacks interpretability. To remedy this gap, we propose a novel interpretable deep learning approach, Generative Adversarial Network-based Piecewise Wide and Attention Deep Learning (GAN-PiWAD), to predict video engagement. Improving upon state-of-the-art interpretable methods, GAN-PiWAD incorporates unstructured data, captures the interactions among multi-modal inputs, offers a precise estimation of the total effect, and models the dynamic total effect when feature value varies. GAN-PiWAD’s prediction outperforms strong benchmarks on two use cases: general video engagement prediction and misinformation viewership prediction. Results from a user study confirm that the interpretability of GANPiWAD is superior to state-of-the-art interpretable methods. This study contributes to IS with a novel interpretable deep learning framework that is generalizable to understand other human decision factors. Our findings provide direct implications for content creators and video sharing platforms to take proactive actions to improve video engagement and manage video credibility.
Prototype-based Incremental Few-Shot Semantic Segmentation

Semantic segmentation models have two fundamental weaknesses: i) they require large training sets with costly pixel-level annotations, and ii) they have a static output space, constrained to the classes of the training set. Toward addressing both problems, we introduce a new task, Incremental Few-Shot Segmentation (iFSS). The goal of iFSS is to extend a pretrained segmentation model with new classes from few annotated images and without access to old training data. To overcome the limitations of existing models iniFSS, we propose Prototype-based Incremental Few-Shot Segmentation (PIFS) that couples prototype learning and knowledge distillation. PIFS exploits prototypes to initialize the classifiers of new classes, fine-tuning the network to refine its features representation. We design a prototype-based distillation loss on the scores of both old and new class prototypes to avoid overfitting and forgetting, and batch-renormalization to cope with non-i.i. d.few-shot data. We create an extensive benchmark for iFSS showing that PIFS outperforms several few-shot and incremental learning methods in all scenarios.
Re-Evaluating Strengthened-IV Designs: Asymptotic Efficiency, Bias Formula, and the Validity and Power of Sensitivity Analyses

Instrumental variables (IVs) are extensively used to estimate treatment effects when the treatment and outcome are confounded by unmeasured confounders; however, weak IVs are often encountered in empirical studies and may cause problems. Many studies have considered building a stronger IV from the original, possibly weak, IV in the design stage of a matched study at the cost of not using some of the samples in the analysis. It is widely accepted that strengthening an IV tends to render nonparametric tests more powerful and will increase the power of sensitivity analyses in large samples. In this article, we re-evaluate this conventional wisdom to bring new insights into this topic. We consider matched observational studies from three perspectives. First, we evaluate the trade-off between IV strength and sample size on nonparametric tests assuming the IV is valid and exhibit conditions under which strengthening an IV increases power and conversely conditions under which it decreases power. Second, we derive a necessary condition for a valid sensitivity analysis model with continuous doses. We show that the Γ sensitivity analysis model, which has been previously used to come to the conclusion that strengthening an IV increases the power of sensitivity analyses in large samples, does not apply to the continuous IV setting and thus this previously reached conclusion may be invalid. Third, we quantify the bias of the Wald estimator with a possibly invalid IV under an oracle and leverage it to develop a valid sensitivity analysis framework; under this framework, we show
3D Human Pose Estimation for Free-form Activity Using WiFi Signals

WiFi human sensing has become increasingly attractive in enabling emerging human-computer interaction applications. The corresponding technique has gradually evolved from the classification of multiple activity types to more fine-grained tracking of 3D human poses. However, existing WiFi-based 3D human pose tracking is limited to a set of predefined activities. In this work, we present Winect, a 3D human pose tracking system for free-form activity using commodity WiFi devices. Our system tracks free-form activity by estimating a 3D skeleton pose that consists of a set of joints of the human body. In particular, we combine signal separation and joint movement modeling to achieve free-form activity tracking. Our system first identifies the moving limbs by leveraging the two-dimensional angle of arrival of the signals reflected off the human body and separates the entangled signals for each limb. Then, it tracks each limb and constructs a 3D skeleton of the body by modeling the inherent relationship between the movements of the limb and the corresponding joints. Our evaluation results show that Winect is environment-independent and achieves centimeter-level accuracy for free-form activity tracking under various challenging environments including the none-line-of-sight (NLoS) scenarios.
Boosting coherence of language models

Naturality of long-term information structure – coherence – remains a challenge in language generation. Large language models have insuf-ﬁciently learned such structure, as their long-form generations differ from natural text in measures of coherence. To alleviate this divergence, we propose coherence boosting , an inference procedure that increases the effect of distant context on next-token prediction. We show the beneﬁts of coherence boosting with pretrained models by distributional analyses of generated ordinary text and dialog responses. We also ﬁnd that coherence boosting with state-of-the-art models for various zero-shot NLP tasks yields performance gains with no additional training.
Aspect-Oriented Summarization through Query-Focused Extraction

A reader interested in a particular topic might be interested in summarizing documents on that subject with a particular focus, rather than simply seeing generic summaries produced by most summarization systems. While query-focused summarization has been explored in prior work, this is often approached from the standpoint of document-speciﬁc questions or on synthetic data. Real users’ needs often fall more closely into aspects, broad topics in a dataset the user is interested in rather than spe-ciﬁc queries. In this paper, we collect a dataset of realistic aspect-oriented test cases, A SPECT - N EWS , which covers different subtopics about articles in news sub-domains. We then investigate how query-focused methods, for which we can construct synthetic data, can handle this aspect-oriented setting: we benchmark ex-tractive query-focused training schemes, and propose a contrastive augmentation approach to train the model. We evaluate on two aspect-oriented datasets and ﬁnd this approach yields (a) focused summaries, better than those from a generic summarization system, which go beyond simple keyword matching; (b) a system sensitive to the choice of keywords. 1
Second-order cumulants ghost imaging

Ghost imaging (GI) is a technique to retrieve images by correlating intensity fluctuations. In this Letter, we present a novel scheme for GI referred to as second-order cumulants GI (SCGI). The image is retrieved from fluctuation information, and resolution may be enhanced compared to traditional GI. We experimentally performed SCGI image reconstruction, and the results are in agreement with theoretical predictions.
Neural Network Pruning Through Constrained Reinforcement Learning

Network pruning reduces the size of neural networks by removing neurons such that the performance drop is minimal. Traditional pruning approaches focus on designing metrics to quantify the usefulness of a neuron which is often tedious and sub-optimal. More recent methods have instead focused on training auxiliary networks to automatically learn how useful each neuron is however, they often do not take computational limitations into account. We propose a general methodology for pruning neural networks. Our approach can prune neural networks to respect pre-defined computational budgets on arbitrary, possibly non-differentiable, functions. Furthermore, we only assume the ability to be able to evaluate these functions for different inputs, and hence they do not need to be fully specified beforehand. We achieve this by proposing a novel pruning strategy via constrained reinforcement learning algorithms. We prove the effectiveness of our approach via comparison with state-of-the-art methods on standard image classification datasets. Specifically, we reduce 83−92.90% of total parameters on various variants of VGG while achieving comparable or better performance. We also achieved 75.09% reduction in parameters on ResNet18 without incurring any loss in accuracy. The code is also available at our github repository1.
A Bayesian Selection Model for Correcting Outcome Reporting Bias With Application to a Meta-analysis on Heart Failure Interventions

Multivariate meta-analysis (MMA) is a powerful tool for jointly estimating multiple outcomes’ treatment effects. However, the validity of results from MMA is potentially compromised by outcome reporting bias (ORB), or the tendency for studies to selectively report outcomes. Until recently, ORB has been understudied. Since ORB can lead to biased conclusions, it is crucial to correct the estimates of effect sizes and quantify their uncertainty in the presence of ORB. With this goal, we develop a Bayesian selection model to adjust for ORB in MMA. We further propose a measure for quantifying the impact of ORB on the results from MMA. We evaluate our approaches through a meta-evaluation of 748 bivariate meta-analyses from the Cochrane Database of Systematic Reviews. Our model is motivated by and applied to a meta-analysis of interventions on hospital readmission and quality of life for heart failure patients. In our analysis, the relative risk (RR) of hospital readmission for the intervention group changes from a significant decrease (RR: 0.931, 95% confidence interval [CI]: 0.862-0.993) to a statistically nonsignificant effect (RR: 0.955, 95% CI: 0.876–1.051) after adjusting for ORB. This study demonstrates that failing to account for ORB can lead to different conclusions in a meta-analysis.
Prospects of Fast Flavor Neutrino Conversion in Rotating Core-collapse Supernovae

There is mounting evidence that neutrinos undergo fast flavor conversion (FFC) in core-collapse supernova (CCSN). In this paper, we investigate the roles of stellar rotation on the occurrence of FFC by carrying out axisymmetric CCSN simulations with full Boltzmann neutrino transport. Our result suggests that electron neutrino lepton number (ELN) angular crossings, which are the necessary and sufficient condition to trigger FFC, preferably occur in the equatorial region for rotating CCSNe. By scrutinizing the neutrino–matter interaction and neutrino radiation field, we find some pieces of evidence that the stellar rotation facilitates the occurrence of FFC. The low-electron-fraction region in the post-shock layer expands by centrifugal force, enhancing the disparity of neutrino absorption between electron-type neutrinos (ν e) and their anti-particles ( ν¯e ). This has a significant impact on the angular distribution of neutrinos in momentum space, in which ν e tends to be more isotropic than ν¯e; consequently, ELN crossings emerge. The ELN crossing found in this study is clearly associated with rotation, which motivates further investigation on how the subsequent FFC influences explosion dynamics, nucleosynthesis, and neutrino signals in rotating CCSNe.
Nonequilibrium dynamics in pumped Mott insulators

We use time-evolution techniques for (infinite) matrix-product-states to calculate, directly in the thermodynamic limit, the time-dependent photoemission spectra and dynamic structure factors of the half-filled Hubbard chain after pulse irradiation. These quantities exhibit clear signatures of the photoinduced phase transition from insulator to metal that occurs because of the formation of so-called η pairs. In addition, the spin dynamic structure factor loses spectral weight in the whole momentum space, reflecting the suppression of antiferromagnetic correlations due to the buildup of η-pairing states. The numerical method demonstrated in this work can be readily applied to other one-dimensional models driven out of equilibrium by optical pumping.
HyperSeed: Unsupervised Learning with Vector Symbolic Architectures

Motivated by recent innovations in biologically inspired neuromorphic hardware, this article presents a novel unsupervised machine learning algorithm named Hyperseed that draws on the principles of vector symbolic architectures (VSAs) for fast learning of a topology preserving feature map of unlabeled data. It relies on two major operations of VSA, binding and bundling. The algorithmic part of Hyperseed is expressed within the Fourier holographic reduced representations (FHRR) model, which is specifically suited for implementation on spiking neuromorphic hardware. The two primary contributions of the Hyperseed algorithm are few-shot learning and a learning rule based on single vector operation. These properties are empirically evaluated on synthetic datasets and on illustrative benchmark use cases, IRIS classification, and a language identification task using the n -gram statistics. The results of these experiments confirm the capabilities of Hyperseed and its applications in neuromorphic hardware.
Phytochemical and Therapeutic Potential of Citrus grandis (L.) Osbeck: A Review

Citrus grandis or Citrus maxima, widely recognized as Pomelo is widely cultivated in many countries because of their large amounts of functional, nutraceutical and biological activities. In traditional medicine, various parts of this plant including leaf, pulp and peel are used for generations as they are scientifically proven to have therapeutic potentials and safe for human use. The main objective of this study was to review the different therapeutic applications of Citrus grandis and the phytochemicals associated with its medicinal values. In this article different pharmacological properties like antimicrobial, antitumor, antioxidant, anti-inflammatory, anticancer, antiepileptic, stomach tonic, cardiac stimulant, cytotoxic, hepatoprotective, nephroprotective, and anti-diabetic activities of the plant are highlighted. The enrichment of the fruit with flavonoids, polyphenols, coumarins, limonoids, acridone alkaloids, essential oils and vitamins mainly helps in exhibiting the pharmacological activities within the body. The vitamins enriched fruit is rich in nutritional value and also has minerals like calcium, phosphorous, sodium and potassium, which helps in maintaining the proper health and growth of the bones as well as the electrolyte balance of the body. To conclude, various potential therapeutic effects of Citrus grandis have been demonstrated in recent literature. Further studies on various parts of fruit, including pulp, peel, leaf, seed and it essential oil could unveil additional pharmacological activities which can be beneficial to the mankind.
M6A-mediated up-regulation of LncRNA LIFR-AS1 enhances the progression of pancreatic cancer via miRNA-150-5p/ VEGFA/Akt signaling

ABSTRACT N6-methyladenosine (m6A) modification, the most abundant internal methylation of eukaryotic RNA transcripts, is critically implicated in RNA processing. There is extensive evidence indicating that long non-coding RNAs (lncRNAs) serve as key regulators of oncogenesis and tumor progression in humans. Through prior study has assessed that LIFR-AS1 plays a key role in various kinds of malignant tumors. However, the exact role of m6A induced LIFR-AS1 in pancreatic cancer (PC) and its potential molecular mechanisms remain largely unknown. In this study, we determined that PC cell lines and tumors exhibit increased LIFR-AS1 expression that correlates with larger tumor size, lymph node metastasis, and more advanced TNM stage. Functionally, loss-of-function studies indicated that LIFR-AS1 knockdown decreased the proliferation, migration, and invasion of PC cells in vitro. Mechanistically, we found that METTL3 induced m6A hyper-methylation on the 3′ UTR of LIFR-AS1 to enhance its mRNA stability and LIFR-AS1 could directly interact with miR-150-5p, thereby indirectly up-regulating VEGFA expressions within cells. Through rescue experiments, we were able to confirm that the unfavorable impact of LIFR-AS1 knockdown on VEGFA /PI3K/Akt Signaling could be reversed via the inhibition of miR-150-5p expression. Together, these findings indicate that a noval m6A-LIFR-AS1 axis promotes PC progression at least in part via regulation of the miR-150-5p/VEGFA axis, indicating that this regulatory axis may be a viable clinical target for the treatment of PC.
What Does Patient-Centered Communication Look Like?: Linguistic Markers of Provider Compassionate Care and Shared Decision-Making and Their Impacts on Patient Outcomes

ABSTRACT Patient-centered communication promotes positive patient outcomes. This study examines the linguistic markers of two key dimensions of patient-centered communication (i.e., provider compassionate care and shared decision-making) and their mediating effects on patient perceived quality of and affective responses to the provider’s treatment recommendations. Transcripts (N = 343) of provider talk from provider-patient interactions in medical visits for upper respiratory infection symptoms where patients were not prescribed with antibiotics were analyzed with the Linguistic Inquiry and Word Count (LIWC) dictionary. Results showed that providers’ use of affiliation words positively predicted patients’ perceptions of their providers’ compassionate care. Providers’ use of insight words negatively predicted patients’ perceptions of provider shared decision-making. Meanwhile, providers’ use of first-person singular pronouns, causation and differentiation words, and clout words were positively related to perceived provider shared decision-making. Patient perceived compassionate care and shared decision-making further increased patients’ positive affect toward and perceived quality of non-antibiotic treatment recommendations. These perceptions also reduced their negative affect toward the recommendations. Implications of the findings are discussed with regard to patient-centered communication in relation to the promotion of antibiotic stewardship.
Incorporation of β‐Alanine in Cu(II) ATCUN Peptide Complexes Increases ROS Levels, DNA Cleavage and Antiproliferative Activity

Abstract Redox‐active Cu(II) complexes are able to form reactive oxygen species (ROS) in the presence of oxygen and reducing agents. Recently, Faller et al. reported that ROS generation by Cu(II) ATCUN complexes is not as high as assumed for decades. High complex stability results in silencing of the Cu(II)/Cu(I) redox cycle and therefore leads to low ROS generation. In this work, we demonstrate that an exchange of the α‐amino acid Gly with the β‐amino acid β‐Ala at position 2 (Gly2→β‐Ala2) of the ATCUN motif reinstates ROS production (•OH and H2O2). Potentiometry, cyclic voltammetry, EPR spectroscopy and DFT simulations were utilized to explain the increased ROS generation of these β‐Ala2‐containing ATCUN complexes. We also observed enhanced oxidative cleavage activity towards plasmid DNA for β‐Ala2 compared to the Gly2 complexes. Modifications with positively charged Lys residues increased the DNA affinity through electrostatic interactions as determined by UV/VIS, fluorescence, and CD spectroscopy, and consequently led to a further increase in nuclease activity. A similar trend was observed regarding the cytotoxic activity of the complexes against several human cancer cell lines where β‐Ala2 peptide complexes had lower IC50 values compared to Gly2. The higher cytotoxicity could be attributed to an increased cellular uptake as determined by ICP‐MS measurements.