Miracle-dz/UniFAR-FTUs huggingface dataset:
Clustering Hidden Markov Models With Variational Bayesian Hierarchical EM

The hidden Markov model (HMM) is a broadly applied generative model for representing time-series data, and clustering HMMs attract increased interest from machine learning researchers. However, the number of clusters (<inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula>) and the number of hidden states (<inline-formula> <tex-math notation="LaTeX">$S$ </tex-math></inline-formula>) for cluster centers are still difficult to determine. In this article, we propose a novel HMM-based clustering algorithm, the variational Bayesian hierarchical EM algorithm, which clusters HMMs through their densities and priors and simultaneously learns posteriors for the novel HMM cluster centers that compactly represent the structure of each cluster. The numbers <inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$S$ </tex-math></inline-formula> are automatically determined in two ways. First, we place a prior on the pair <inline-formula> <tex-math notation="LaTeX">$(K,S)$ </tex-math></inline-formula> and approximate their posterior probabilities, from which the values with the maximum posterior are selected. Second, some clusters and states are pruned out implicitly when no data samples are assigned to them, thereby leading to automatic selection of the model complexity. Experiments on synthetic and real data demonstrate that our algorithm performs better than using model selection techniques with maximum likelihood estimation.
What Have Been Learned & What Should Be Learned? An Empirical Study of How to Selectively Augment Text for Classification

Text augmentation techniques are widely used in text classification problems to improve the performance of classifiers, especially in low-resource scenarios. Whilst lots of creative text augmentation methods have been designed, they augment the text in a non-selective manner, which means the less important or noisy words have the same chances to be augmented as the informative words, and thereby limits the performance of augmentation. In this work, we systematically summarize three kinds of role keywords, which have different functions for text classification, and design effective methods to extract them from the text. Based on these extracted role keywords, we propose STA (Selective Text Augmentation) to selectively augment the text, where the informative, class-indicating words are emphasized but the irrelevant or noisy words are diminished. Extensive experiments on four English and Chinese text classification benchmark datasets demonstrate that STA can substantially outperform the non-selective text augmentation methods.
Halo concentration strengthens dark matter constraints in galaxy-galaxy strong lensing analyses

A deﬁning prediction of the cold dark matter cosmological model is the existence of a very large population of low-mass haloes. This population is absent in models in which the dark matter particle is warm (WDM). These alternatives can, in principle, be distinguished observationally because haloes along the line of sight can perturb g alaxy–g alaxy strong gravitational lenses. Furthermore, the WDM particle mass could be deduced because the cut-off in their halo mass function depends on the mass of the particle. We systematically explore the detectability of low-mass haloes in WDM models by simulating and ﬁtting mock lensed images. Contrary to previous studies, we ﬁnd that haloes are harder to detect when they are either behind or in front of the lens. Furthermore, we ﬁnd that the perturbing effect of haloes increases with their concentration: Detectable haloes are systematically high-concentration haloes, and accounting for the scatter in the mass–concentration relation boosts the expected number of detections by as much as an order of magnitude. Haloes have lower concentration for lower particle masses and this further suppresses the number of detectable haloes beyond the reduction arising from the lower halo abundances alone. Taking these effects into account can make lensing constraints on the value of the mass function cut-off at least an order of magnitude more stringent than previously appreciated.
Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis

In multimodal sentiment analysis (MSA), the performance of a model highly depends on the quality of synthesized embeddings. These embeddings are generated from the upstream process called multimodal fusion, which aims to extract and combine the input unimodal raw data to produce a richer multimodal representation. Previous work either back-propagates the task loss or manipulates the geometric property of feature spaces to produce favorable fusion results, which neglects the preservation of critical task-related information that flows from input to the fusion results. In this work, we propose a framework named MultiModal InfoMax (MMIM), which hierarchically maximizes the Mutual Information (MI) in unimodal input pairs (inter-modality) and between multimodal fusion result and unimodal input in order to maintain task-related information through multimodal fusion. The framework is jointly trained with the main task (MSA) to improve the performance of the downstream MSA task. To address the intractable issue of MI bounds, we further formulate a set of computationally simple parametric and non-parametric methods to approximate their truth value. Experimental results on the two widely used datasets demonstrate the efficacy of our approach.
Machine-learning media bias

We present an automated method for measuring media bias. Inferring which newspaper published a given article, based only on the frequencies with which it uses different phrases, leads to a conditional probability distribution whose analysis lets us automatically map newspapers and phrases into a bias space. By analyzing roughly a million articles from roughly a hundred newspapers for bias in dozens of news topics, our method maps newspapers into a two-dimensional bias landscape that agrees well with previous bias classifications based on human judgement. One dimension can be interpreted as traditional left-right bias, the other as establishment bias. This means that although news bias is inherently political, its measurement need not be.
Characteristics of patients with heart failure with preserved ejection fraction in primary care: a cross-sectional analysis

Background Many patients with heart failure with preserved ejection fraction (HFpEF) are undiagnosed, and UK general practice registers do not typically record heart failure (HF) subtype. Improvements in management of HFpEF is dependent on improved identification and characterisation of patients in primary care. Aim To describe a cohort of patients recruited from primary care with suspected HFpEF and compare patients in whom HFpEF was confirmed and refuted. Design & setting Baseline data from a longitudinal cohort study of patients with suspected HFpEF recruited from primary care in two areas of England. Method A screening algorithm and review were used to find patients on HF registers without a record of reduced ejection fraction (EF). Baseline evaluation included cardiac, mental and physical function, clinical characteristics, and patient reported outcomes. Confirmation of HFpEF was clinically adjudicated by a cardiologist. Results In total, 93 (61%) of 152 patients were confirmed HFpEF. The mean age of patients with HFpEF was 79 years, 46% were female, 80% had hypertension, and 37% took ≥10 medications. Patients with HFpEF were more likely to be obese, pre-frail or frail, report more dyspnoea and fatigue, were more functionally impaired, and less active than patients in whom HFpEF was refuted. Few had attended cardiac rehabilitation. Conclusion Patients with confirmed HFpEF had frequent multimorbidity, functional impairment, frailty, and polypharmacy. Although comorbid conditions were similar between people with and without HFpEF, the former had more obesity, symptoms, and worse physical function. These findings highlight the potential to optimise wellbeing through comorbidity management, medication rationalisation, rehabilitation, and supported self-management.
Downregulation of miR-23b by transcription factor c-Myc alleviates ischemic brain injury by upregulating Nrf2

Ischemic brain injury (IBI) is a common acute cerebral vessel disease that occurs secondary to blockage in arteries, mainly characterized by insufficient blood supply to the brain. The transcription factor c-Myc in IBI continues to be implicated in numerous studies. This study was conducted with emphasis placed on the underlying mechanism of c-Myc in IBI. Clinical samples were collected from IBI patients. Middle cerebral artery occlusion (MCAO) was induced in mice by inserting a suture from the external carotid artery to the anterior cerebral artery through the internal carotid artery to mechanically block the blood supply at the origin of the middle cerebral artery, and cortical neurons from mice were exposed to oxygen glucose deprivation (OGD) conditions for IBI model in vitro construction. RT-qPCR was performed to determine microRNA-23b (miR-23b) expression. TUNEL staining and Western blot analysis was conducted to detect apoptosis. The regulatory relationship was analyzed by dual-luciferase reporter gene assay. After loss- and gain-of-function assays, triphenyltetrazolium chloride staining was carried out to detect the area of cerebral infarction, after which the spatial memory in mice was evaluated with Morris water maze test. As per our findings, miR-23b was upregulated in the serum of IBI patients and OGD-treated murine primary neurons. Silencing of miR-23b resulted in reduced OGD-induced neuronal apoptosis. miR-23b inversely targeted nuclear factor erythroid 2-related factor 2 (Nrf2) and c-Myc negatively regulated miR-23b expression. Overexpression of c-Myc and inhibition of miR-23b led to reduced neurological scores, infarction area, neuronal apoptosis, shortened platform arrival time and significantly increased the time spent on the platform quadrant and the times of crossing the platform in vivo. Collectively, downregulated miR-23b by c-Myc might alleviate IBI by upregulating Nrf2.
Storing Multi-model Data in RDBMSs based on Reinforcement Learning

How to manage various data in a unified way is a significant research topic in the field of databases. To address this problem, researchers have proposed multi-model databases to support multiple data models in a uniform platform with a single unified query language. However, since relational databases are predominant in the current market, it is expensive to replace them with others. Besides, due to the theories and technologies of RDBMSs having been enhanced over decades, it is hard to use few years to develop a multi-model database that can be compared with existing RDBMSs in handling security, query optimization, transaction management, etc. In this paper, we reconsider employing relational databases to store and query multi-model data. Unfortunately, the mismatch between the complexity of multi-model data structure and the simplicity of flat relational tables makes this difficult. Against this challenge, we utilize the reinforcement learning (RL) method to learn a relational schema by interacting with an RDBMS. Instead of using the classic Q-learning algorithm, we propose a variant Q-learning algorithm, called Double Q-tables, to reduce the dimension of the original Q-table and improve learning efficiency. Experimental results show that our approach could learn a relational schema outperforming the existing multi-model storage schema in terms of query time and space consumption.
Scalable Spatiotemporally Varying Coefficient Modeling with Bayesian Kernelized Tensor Regression

As a regression technique in spatial statistics, the spatiotemporally varying coefficient model (STVC) is an important tool for discovering nonstationary and interpretable response-covariate associations over both space and time. However, it is difficult to apply STVC for large-scale spatiotemporal analyses due to its high computational cost. To address this challenge, we summarize the spatiotemporally varying coefficients using a third-order tensor structure and propose to reformulate the spatiotemporally varying coefficient model as a special low-rank tensor regression problem. The low-rank decomposition can effectively model the global patterns of the large data sets with a substantially reduced number of parameters. To further incorporate the local spatiotemporal dependencies, we use Gaussian process (GP) priors on the spatial and temporal factor matrices. We refer to the overall framework as Bayesian Kernelized Tensor Regression (BKTR). For model inference, we develop an efficient Markov chain Monte Carlo (MCMC) algorithm, which uses Gibbs sampling to update factor matrices and slice sampling to update kernel hyperparameters. We conduct extensive experiments on both synthetic and real-world data sets, and our results confirm the superior performance and efficiency of BKTR for model estimation and parameter inference.
Guarding Machine Learning Hardware Against Physical Side-channel Attacks

Machine learning (ML) models can be trade secrets due to their development cost. Hence, they need protection against malicious forms of reverse engineering (e.g., in IP piracy). With a growing shift of ML to the edge devices, in part for performance and in part for privacy benefits, the models have become susceptible to the so-called physical side-channel attacks. ML being a relatively new target compared to cryptography poses the problem of side-channel analysis in a context that lacks published literature. The gap between the burgeoning edge-based ML devices and the research on adequate defenses to provide side-channel security for them thus motivates our study. Our work develops and combines different flavors of side-channel defenses for ML models in the hardware blocks. We propose and optimize the first defense based on Boolean masking. We first implement all the masked hardware blocks. We then present an adder optimization to reduce the area and latency overheads. Finally, we couple it with a shuffle-based defense. We quantify that the area-delay overhead of masking ranges from 5.4× to 4.7× depending on the adder topology used and demonstrate a first-order side-channel security of millions of power traces. Additionally, the shuffle countermeasure impedes a straightforward second-order attack on our first-order masked implementation.
Segmentation approach for brain tumor detection

The task of detecting tumours is an important one as it can lead to many lives being saved as nearly 20k lives are lost every year due to brain tumour. The task is challenging as in few cases brain tumours are not detected in early stages. With the help of artificial intelligence this gap can be bridged. The task of detecting regions of tumour can be classified into the popular computer vision task of image segmentation. Given the actual MRI scans and the masks representing the region of tumour, the machine learning models can learn the functions which map the regions to presence or non presence of tumour. The models are trained using a pair of MRI images and corresponding masks and the given an MRI image, the model should detect the region of tumour if present. This paper explores different state of the art architectures to perform the task of image segmentation.
Evaluating Predictive Uncertainty under Distributional Shift on Dialogue Dataset

In open-domain dialogues, predictive uncertainties are mainly evaluated in a domain shift setting to cope with out-of-distribution inputs. However, in real-world conversations, there could be more extensive distributional shifted inputs than the out-of-distribution. To evaluate this, we first propose two methods, Unknown Word (UW) and Insufficient Context (IC), enabling gradual distributional shifts by corruption on the dialogue dataset. We then investigate the effect of distributional shifts on accuracy and calibration. Our experiments show that the performance of existing uncertainty estimation methods consistently degrades with intensifying the shift. The results suggest that the proposed methods could be useful for evaluating the calibration of dialogue systems under distributional shifts.
Exploring Deep Learning for recognizing rare diseases and their clinical manifestations from texts

Although rare diseases are characterized by low prevalence, approximately 300 million people are affected by a rare disease. The early and accurate diagnosis of these conditions is a major challenge for general practitioners, who do not have enough knowledge to identify them. In addition to this, rare diseases usually show a wide variety of manifestations, which might make the diagnosis even more difﬁcult. A delayed diagnosis can negatively affect the patient’s life. Therefore, there is an urgent need to increase the scientiﬁc and medical knowledge about rare diseases. Natural Language Processing (NLP) and Deep Learning can help to extract relevant information about rare diseases to facilitate their diagnosis and treatments. The paper proposes several deep learning techniques such as Bidirectional Long Short Term Memory (BiLSTM) networks or deep contextualized word representations based on Bidirectional Encoder Representations from Transformers (BERT) to recognize rare diseases and their clinical manifestations (signs and symptoms) in the RareDis corpus. This corpus contains more than 5,000 rare diseases and almost 6,000 clinical manifestations. BioBERT, a domain-speciﬁc language representation based on BERT and trained on biomedical corpora, obtains the best results. In particular, this model obtains an F1-score of 85.2% for rare diseases, outperforming all the other models.
Phylogenetic comparative morphological analysis of fern spores in subfamily Pteridoideae (Pteridaceae, Pteridophyta)

Scanning electron microscopy (SEM) was used to characterize spore morphology of 24 taxa of the subfamily Pteridoideae C. Chr. ex Crabbe, Jermy & Mickel of the family Pteridaceae E.D. M. Kirchn. The family is considered to be one of the most taxonomically confusing families due to its high level of polymorphism. The standardized data on spore morphology of the subfamily Pteridoideae were projected onto the final phylogenetic tree in the Mesquite program. This approach made it possible to carry out a comprehensive interdisciplinary analysis of the evolution of spore morphology characters of the subfamily Pteridoideae, as well as to assess the relationships in the family Pteridaceae. The equatorial ridge (cingulum, “flange”) has been proven as one of the key spore morphology features, which confirms the close relationship of “Onychium clade” with Pteris. The species‐specific characters of the subfamily are fold and tubercle along laesura, equatorial ridge on proximal and distal side, tubercle, and folds on proximal and distal side. The knowledge will help to solve the problems of taxonomy in the family Preridaceae and to supplement the information on the natural classification of the subfamily Pteridoideae.
Complete natural gradients for structured approximations in mixtures of exponential families

Stochastic gradient methods have enabled variational inference for high-dimensional models and large data sets. However, the direction of steepest ascent in the parameter space of a statistical model is given not by the commonly used Euclidean gradient, but the natural gradient which premultiplies the Euclidean gradient by the inverse of the Fisher information matrix. Use of natural gradients in optimization can improve convergence signiﬁcantly, but inverting the Fisher information matrix is daunting in high-dimensions. The contribution of this article is twofold. First, we derive the natural gradient updates of a Gaussian variational approximation in terms of the mean and Cholesky factor of the covariance matrix, and show that these updates depend only on the ﬁrst derivative of the variational objective function. Second, we derive complete natural gradient updates for structured variational approximations with a minimal conditional exponential family representation, which include highly ﬂexible mixture of exponential family distributions that can ﬁt skewed or multimodal posteriors. These updates, albeit more complex than those presented priorly, account fully for the dependence between the mixing distribution and the distributions of the components. Further experiments will be carried out to evaluate the performance of proposed methods.
Exploring the accretion-ejection geometry of GRS 1915+105 in the obscured state with future X-ray spectro-polarimetry

GRS 1915+105 has been in a bright flux state for more than 2 decades, but in 2018 a significant drop in flux was observed, partly due to changes in the central engine along with increased X-ray absorption. The aim of this work is to explore how X-ray spectro-polarimetry can be used to derive the basic geometrical properties of the absorbing and reflecting matter. In particular, the expected polarisation of the radiation reflected off the disc and the putative outflow is calculated. We use \textit{NuSTAR} data collected after the flux drop to derive the parameters of the system from hard X-ray spectroscopy. The spectroscopic parameters are then used to derive the expected polarimetric signal, using results from a MonteCarlo radiative transfer code both in the case of neutral and fully ionised matter. From the spectral analysis, we find that the continuum emission becomes softer with increasing flux, and that in all flux levels the obscuring matter is highly ionised. This analysis, on the other hand, confirms that spectroscopy alone is unable to put constraints on the geometry of the reflectors. Simulations show that X-ray polarimetric observations, like those that will be provided soon by the Imaging X-ray Polarimetry Explorer (IXPE), will help to determine the geometrical parameters which are left unconstrained by the spectroscopic analysis.
Direct-acting oral anticoagulants versus warfarin in relation to risk of gastrointestinal bleeding: a systematic review and meta-analysis of randomized controlled trials

Background Direct-acting oral anticoagulants (DOACs) are increasingly used, with studies showing a lower risk of gastrointestinal bleeding (GIB), but overall data for GIB risk remains debatable. The objective was to assess non-fatal and fatal GIB risk in patients on DOACs compared with warfarin from randomized clinical trials (RCTs). Methods RCTs comparing warfarin and DOACs for various indications (atrial fibrillation, thromboembolism, insertion of mechanical heart valves) were included. The primary endpoint was any GIB event. Other clinical events, such as fatal GIB, and effects of age (≤60 years or older), time in therapeutic range for warfarin, and choice of individual DOACs on GIB risk, were also assessed. Results 14 RCTs were included, comprising 87,407 participants (DOACs n=46,223, warfarin control n=41,184). The risk of GIB with DOACs was similar to that of warfarin (relative risk [RR] 1.04, 95% confidence interval [CI] 0.85-1.27). Compared with warfarin, rivaroxaban (RR 1.23, 95%CI 1.03-1.48) and dabigatran (RR 1.38, 95%CI 1.12-1.71) had a higher risk of any GIB, whereas fatal GIB risk was lower in the DOACs group (RR 0.36, 95%CI 0.15-0.82). The risk of DOAC-related fatal GIB was lower in patients aged ≤60 years and in those with poor coagulation control (RR 0.39, 95%CI 0.15-0.98). Conclusions DOACs compared with warfarin have a lower risk of fatal GIB, especially in those aged <60 years and those with poor coagulation control. However, the risk of GIB was comparable with warfarin and DOACs, except for rivaroxaban and dabigatran.