Miracle-dz/UniFAR-FTUs huggingface dataset:
How Well Self-Supervised Pre-Training Performs with Streaming Data?

The common self-supervised pre-training practice requires collecting massive unlabeled data together and then trains a representation model, dubbed joint training . However, in real-world scenarios where data are collected in a streaming fashion, the joint training scheme is usually storage-heavy and time-consuming. A more efﬁcient alternative is to train a model continually with streaming data, dubbed sequential training . Nevertheless, it is unclear how well sequential self-supervised pre-training performs with streaming data. In this paper, we conduct thorough experiments to investigate self-supervised pre-training with streaming data. Speciﬁcally, we evaluate the transfer performance of sequential self-supervised pre-training with four different data sequences on three different downstream tasks and make comparisons with joint self-supervised pre-training. Surprisingly, we ﬁnd sequential self-supervised learning exhibits almost the same performance as the joint training when the distribution shifts within streaming data are mild. Even for data sequences with large distribution shifts, sequential self-supervised training with simple techniques, e.g., parameter regularization or data replay, still performs comparably to joint training. Based on our ﬁndings, we recommend using sequential self-supervised training as a more efﬁcient yet performance-competitive representation learning practice for real-world applications.
Open Intent Discovery through Unsupervised Semantic Clustering and Dependency Parsing

Intent understanding plays an important role in dialog systems, and is typically formulated as a supervised learning problem. However, it is challenging and time-consuming to design the intents for a new domain from scratch, which usually requires a lot of manual effort of domain experts. This paper presents an unsupervised two-stage approach to discover intents and generate meaningful intent labels automatically from a collection of unlabeled utterances in a domain. In the first stage, we aim to generate a set of semantically coherent clusters where the utterances within each cluster convey the same intent. We obtain the utterance representation from various pre-trained sentence embeddings and present a metric of balanced score to determine the optimal number of clusters in K-means clustering for balanced datasets. In the second stage, the objective is to generate an intent label automatically for each cluster. We extract the ACTION-OBJECT pair from each utterance using a dependency parser and take the most frequent pair within each cluster, e.g., book-restaurant, as the generated intent label. We empirically show that the proposed unsupervised approach can generate meaningful intent labels automatically and achieve high precision and recall in utterance clustering and intent discovery.
Represent Items by Items: An Enhanced Representation of the Target Item for Recommendation

Item-based collaborative filtering (ICF) has been widely used in industrial applications such as recommender system and online advertising. It models users' preference on target items by the items they have interacted with. Recent models use methods such as attention mechanism and deep neural network to learn the user representation and scoring function more accurately. However, despite their effectiveness, such models still overlook a problem that performance of ICF methods heavily depends on the quality of item representation especially the target item representation. In fact, due to the long-tail distribution in the recommendation, most item embeddings can not represent the semantics of items accurately and thus degrade the performance of current ICF methods. In this paper, we propose an enhanced representation of the target item which distills relevant information from the co-occurrence items. We design sampling strategies to sample fix number of co-occurrence items for the sake of noise reduction and computational cost. Considering the different importance of sampled items to the target item, we apply attention mechanism to selectively adopt the semantic information of the sampled items. Our proposed Co-occurrence based Enhanced Representation model (CER) learns the scoring function by a deep neural network with the attentive user representation and fusion of raw representation and enhanced representation of target item as input. With the enhanced representation, CER has stronger representation power for the tail items compared to the state-of-the-art ICF methods. Extensive experiments on two public benchmarks demonstrate the effectiveness of CER.
Image Inpainting with Edge-guided Learnable Bidirectional Attention Maps

For image inpainting, the convolutional neural networks (CNN) in previous methods often adopt standard convolutional operator, which treats valid pixels and holes indistinguishably. As a result, they are limited in handling irregular holes and tend to produce color-discrepant and blurry inpainting result. Partial convolution (PConv) copes with this issue by conducting masked convolution and feature re-normalization conditioned only on valid pixels, but the mask-updating is handcrafted and independent with image structural information. In this paper, we present an edge-guided learnable bidirectional attention map (Edge-LBAM) for improving image inpainting of irregular holes with several distinct merits. Instead of using a hard 0-1 mask, a learnable attention map module is introduced for learning feature re-normalization and mask-updating in an end-to-end manner. Learnable reverse attention maps are further proposed in the decoder for emphasizing on filling in unknown pixels instead of reconstructing all pixels. Motivated by that the filling-in order is crucial to inpainting results and largely depends on image structures in exemplar-based methods, we further suggest a multi-scale edge completion network to predict coherent edges. Our Edge-LBAM method contains dual procedures,including structure-aware mask-updating guided by predict edges and attention maps generated by masks for feature re-normalization.Extensive experiments show that our Edge-LBAM is effective in generating coherent image structures and preventing color discrepancy and blurriness, and performs favorably against the state-of-the-art methods in terms of qualitative metrics and visual quality.
Exploiting Transitivity Constraints for Entity Matching in Knowledge Graphs

The goal of entity matching in knowledge graphs is to identify entities that refer to the same real-world objects using some similarity metric. The result of entity matching can be seen as a set of entity pairs interpreted as the same-as relation. However, the identified set of pairs may fail to satisfy some structural properties, in particular transitivity, that are expected from the same-as relation. In this work, we show that an ad-hoc enforcement of transitivity, i.e. taking the transitive closure, on the identified set of entity pairs may decrease precision dramatically. We therefore propose a methodology that starts with a given similarity measure, generates a set of entity pairs that are identified as referring to the same real-world objects, and applies the cluster editing algorithm to enforce transitivity without adding many spurious links, leading to overall improved performance.
A Survey Of Regression Algorithms And Connections With Deep Learning

Regression has attracted immense interest lately due to its effectiveness in tasks like predicting values. And Regression is of widespread use in multiple fields such as Economics, Finance, Business, Biology and so on. While considerable studies have proposed some impressive models, few of them have provided a whole picture regarding how and to what extent Regression has developed. With the aim of aiding beginners in understanding the relationships among different Regression algorithms, this paper characterizes a broad and thoughtful selection of recent regression algorithms, providing an organized and comprehensive overview of existing work and models utilized frequently. In this paper, the relationship between Regression and Deep Learning is also discussed and a conclusion can be drawn that Deep Learning can be more powerful as an combination with Regression models in the future.
Anomaly Detection for Solder Joints Using β-VAE

In the assembly process of printed circuit boards (PCBs), most of the errors are caused by solder joints in surface mount devices (SMDs). In the literature, traditional feature extraction-based methods require designing hand-crafted features and rely on the tiered red green blue (RGB) illumination to detect solder joint errors, whereas the supervised convolutional neural network (CNN)-based approaches require a lot of labeled abnormal samples (defective solder joints) to achieve high accuracy. To solve the optical inspection problem in unrestricted environments with no special lighting and without the existence of error-free reference boards, we propose a new beta-variational autoencoder (beta-VAE) architecture for anomaly detection that can work on both integrated circuit (IC) and non-IC components. We show that the proposed model learns disentangled representation of data, leading to more independent features and improved latent space representations. We compare the activation and gradient-based representations that are used to characterize anomalies and observe the effect of different beta parameters on accuracy and untwining the feature representations in beta-VAE. Finally, we show that anomalies on solder joints can be detected with high accuracy via a model trained directly on normal samples without designated hardware or feature engineering.
CFNet: LiDAR-Camera Registration Using Calibration Flow Network

As an essential procedure of data fusion, LiDAR-camera calibration is critical for autonomous vehicles and robot navigation. Most calibration methods require laborious manual work, complicated environmental settings, and specific calibration targets. The targetless methods are based on some complex optimization workflow, which is time-consuming and requires prior information. Convolutional neural networks (CNNs) can regress the six degrees of freedom (6-DOF) extrinsic parameters from raw LiDAR and image data. However, these CNN-based methods just learn the representations of the projected LiDAR and image and ignore the correspondences at different locations. The performances of these CNN-based methods are unsatisfactory and worse than those of non-CNN methods. In this paper, we propose a novel CNN-based LiDAR-camera extrinsic calibration algorithm named CFNet. We first decided that a correlation layer should be used to provide matching capabilities explicitly. Then, we innovatively defined calibration flow to illustrate the deviation of the initial projection from the ground truth. Instead of directly predicting the extrinsic parameters, we utilize CFNet to predict the calibration flow. The efficient Perspective-n-Point (EPnP) algorithm within the RANdom SAmple Consensus (RANSAC) scheme is applied to estimate the extrinsic parameters with 2D–3D correspondences constructed by the calibration flow. Due to its consideration of the geometric information, our proposed method performed better than the state-of-the-art CNN-based methods on the KITTI datasets. Furthermore, we also tested the flexibility of our approach on the KITTI360 datasets.
An Open-source Bayesian Atmospheric Radiative Transfer (BART) Code. I. Design, Tests, and Application to Exoplanet HD 189733b

We present the open-source Bayesian Atmospheric Radiative Transfer (BART) retrieval package, which produces estimates and uncertainties for an atmosphere’s thermal profile and chemical abundances from observations. Several BART components are also stand-alone packages, including the parallel Multi-Core Markov-chain Monte Carlo (MC3), which implements several Bayesian samplers; a line-by-line radiative-transfer model, transit; a code that calculates Thermochemical Equilibrium Abundances (TEA), and a test suite for verifying radiative-transfer and retrieval codes, BARTTest. The codes are in Python and C. BART and TEA are under a Reproducible Research (RR) license, which requires reviewed-paper authors to publish a compendium of all inputs, codes, and outputs supporting the paper’s scientific claims. BART and TEA produce the compendium’s content. Otherwise, these codes are under permissive open-source terms, as are MC3 and BARTTest, for any purpose. This paper presents an overview of the code, BARTTest, and an application to eclipse data for exoplanet HD 189733b. Appendices address RR methodology for accelerating science, a reporting checklist for retrieval papers, the spectral resolution required for synthetic tests, and a derivation of the effective sample size required to estimate any Bayesian posterior distribution to a given precision, which determines how many iterations to run. Paper II, by Cubillos et al., presents the underlying radiative-transfer scheme and an application to transit data for exoplanet HAT-P-11b. Paper III, by Blecic et al., discusses the initialization and post-processing routines, with an application to eclipse data for exoplanet WASP-43b. We invite the community to use and improve BART and its components at http://GitHub.com/ExOSPORTS/BART/.
Aligned Contrastive Predictive Coding

We investigate the possibility of forcing a self-supervised model trained using a contrastive predictive loss to extract slowly varying latent representations. Rather than producing individual predictions for each of the future representations, the model emits a sequence of predictions shorter than that of the upcoming representations to which they will be aligned. In this way, the prediction network solves a simpler task of predicting the next symbols, but not their exact timing, while the encoding network is trained to produce piece-wise constant latent codes. We evaluate the model on a speech coding task and demonstrate that the proposed Aligned Contrastive Predictive Coding (ACPC) leads to higher linear phone prediction accuracy and lower ABX error rates, while being slightly faster to train due to the reduced number of prediction heads.
Selecting a number of voters for a voting ensemble

For a voting ensemble that selects an odd-sized subset of the ensemble classifiers at random for each example, applies them to the example, and returns the majority vote, we show that any number of voters may minimize the error rate over an out-of-sample distribution. The optimal number of voters depends on the out-of-sample distribution of the number of classifiers in error. To select a number of voters to use, estimating that distribution then inferring error rates for numbers of voters gives lower-variance estimates than directly estimating those error rates.
On the joint volatility dynamics in international dairy commodity markets

The present study investigates the price (co)volatility of four dairy commodities—skim milk powder, whole milk powder, butter, and cheddar cheese—in three major dairy markets. It uses a multivariate factor stochastic volatility model for estimating the time‐varying covariance and correlation matrices by imposing a low‐dimensional latent dynamic factor structure. The empirical results support four factors representing the European Union and Oceania dairy sectors as well as the milk powder markets. Factor volatilities and marginal posterior volatilities of each dairy commodity increase after the 2006/07 global (food) crisis, which also coincides with the free trade agreements enacted from 2007 onward and EU and US liberalization policy changes. The model‐implied correlation matrices show increasing dependence during the second half of 2006, throughout the first half of 2007, as well as during 2008 and 2014, which can be attributed to various regional agricultural dairy policies. Furthermore, in‐sample value at risk measures (VaRs and CoVaRs) are provided for each dairy commodity under consideration.
Molecular Typing of Listeria monocytogenes IVb Serogroup Isolated from Food and Food Production Environments in Poland

Listeria monocytogenes is one of the most important foodborne pathogens that may be present in food and in food processing environments. In the present study, 91 L. monocytogenes isolates of serogroup IVb from raw meat, ready-to-eat food and food production environments in Poland were characterized by whole genome sequencing (WGS). The strains were also compared, using core genome multi-locus sequence typing (cgMLST) analysis, with 186 genomes of L. monocytogenes recovered worldwide from food, environments, and from humans with listeriosis. The L. monocytogenes examined belonged to three MLST clonal complexes: CC1 (10; 11.0% isolates), CC2 (70; 76.9%), and CC6 (11; 12.1%). CC1 comprised of two STs (ST1 and ST515) which could be divided into five cgMLST, CC2 covered two STs (ST2 and ST145) with a total of 20 cgMLST types, whereas CC6 consisted of only one ST (ST6) classified as one cgMLST. WGS sequences of the tested strains revealed that they had several pathogenic markers making them potentially hazardous for public health. Molecular comparison of L. monocytogenes strains tested in the present study with those isolated from food and human listeriosis showed a relationship between the isolates from Poland, but not from other countries.
Potential of Climate Change and Herbivory to Affect the Release and Atmospheric Reactions of BVOCs from Boreal and Subarctic Forests

Compared to most other forest ecosystems, circumpolar boreal and subarctic forests have few tree species, and are prone to mass outbreaks of herbivorous insects. A short growing season with long days allows rapid plant growth, which will be stimulated by predicted warming of polar areas. Emissions of biogenic volatile organic compounds (BVOC) from soil and vegetation could be substantial on sunny and warm days and biotic stress may accelerate emission rates. In the atmosphere, BVOCs are involved in various gas-phase chemical reactions within and above forest canopies. Importantly, the oxidation of BVOCs leads to secondary organic aerosol (SOA) formation. SOA particles scatter and absorb solar radiation and grow to form cloud condensation nuclei (CCN) and participate in cloud formation. Through BVOC and moisture release and SOA formation and condensation processes, vegetation has the capacity to affect the abiotic environment at the ecosystem scale. Recent BVOC literature indicates that both temperature and herbivory have a major impact on BVOC emissions released by woody species. Boreal conifer forest is the largest terrestrial biome and could be one of the largest sources of biogenic mono- and sesquiterpene emissions due to the capacity of conifer trees to store terpene-rich resins in resin canals above and belowground. Elevated temperature promotes increased diffusion of BVOCs from resin stores. Moreover, insect damage can break resin canals in needles, bark, and xylem and cause distinctive bursts of BVOCs during outbreaks. In the subarctic, mountain birch forests have cyclic outbreaks of Geometrid moths. During outbreaks, trees are often completely defoliated leading to an absence of BVOC-emitting foliage. However, in the years following an outbreak there is extended shoot growth, a greater number of leaves, and greater density of glandular trichomes that store BVOCs. This can lead to a delayed chemical defense response resulting in the highest BVOC emission rates from subarctic forest in the 1–3 years after an insect outbreak. Climate change is expected to increase insect outbreaks at high latitudes due to warmer seasons and arrivals of invasive herbivore species. Increased BVOC emission will affect tropospheric ozone (O3) formation and O3 induced oxidation of BVOCs. Herbivore-induced BVOC emissions from deciduous and coniferous trees are also likely to increase the formation rate of SOA and further growth of the particles in the atmosphere. Field experiments measuring the BVOC emission rates, SOA formation rate and particle concentrations within and above the herbivore attacked forest stands are still urgently needed.
Hemodynamic Forces Regulate Cardiac Regeneration-Responsive Enhancer Activity during Ventricle Regeneration

Cardiac regenerative capacity varies widely among vertebrates. Zebrafish can robustly regenerate injured hearts and are excellent models to study the mechanisms of heart regeneration. Recent studies have shown that enhancers are able to respond to injury and regulate the regeneration process. However, the mechanisms to activate these regeneration-responsive enhancers (RREs) remain poorly understood. Here, we utilized transient and transgenic analysis combined with a larval zebrafish ventricle ablation model to explore the activation and regulation of a representative RRE. lepb-linked enhancer sequence (LEN) directed enhanced green fluorescent protein (EGFP) expression in response to larval ventricle regeneration and such activation was attenuated by hemodynamic force alteration and mechanosensation pathway modulation. Further analysis revealed that Notch signaling influenced the endocardial LEN activity as well as endogenous lepb expression. Altogether, our work has established zebrafish models for rapid characterization of cardiac RREs in vivo and provides novel insights on the regulation of LEN by hemodynamic forces and other signaling pathways during heart regeneration.
Mitral Valve Prolapse, Arrhythmias, and Sudden Cardiac Death: The Role of Multimodality Imaging to Detect High-Risk Features

Mitral valve prolapse (MVP) was first described in the 1960s, and it is usually a benign condition. However, a subtype of patients are known to have a higher incidence of ventricular arrhythmias and sudden cardiac death, the so called “arrhythmic MVP. ” In recent years, several studies have been published to identify the most important clinical features to distinguish the benign form from the potentially lethal one in order to personalize patient’s treatment and follow-up. In this review, we specifically focused on red flags for increased arrhythmic risk to whom the cardiologist must be aware of while performing a cardiovascular imaging evaluation in patients with MVP.
A Systematic Review of Clinical Trials Assessing Sexuality in Hysterectomized Patients

In hysterectomized patients, even though there is still controversy, evidence indicates that in the short term, the vaginal approach shows benefits over the laparoscopic approach, as it is less invasive, faster and less costly. However, the quality of sexual life has not been systematically reviewed in terms of the approach adopted. Through a systematic review, we analyzed (CRD42020158465 in PROSPERO) the impact of hysterectomy on sexual quality and whether there are differences according to the surgical procedure (abdominal or vaginal) for noncancer patients. MEDLINE (through PubMed), Embase, Cochrane Central Register of Controlled Trials, ClinicalTrials.gov and Scopus were reviewed to find randomized clinical trials assessing sexuality in noncancer patients undergoing total hysterectomy, comparing vaginal and abdominal (laparoscopic and/or open) surgery. Three studies that assessed the issue under study were finally included. Two of these had a low risk of bias (Cochrane risk of bias tool); one was unclear. There was significant variability in how sexuality was measured, with no differences between the two approaches considered in the review. In conclusion, no evidence was found to support one procedure (abdominal or vaginal) over another for non-oncological hysterectomized patients regarding benefits in terms of sexuality.