text: 374,95,1405100087555104768,302547719,Craig Glastonbury,"Our new paper is now out on Arxiv! <LINK> Contrastive Mixture of Posteriors for Counterfactual Inference, Data Integration and Fairness. A project started by Adam Foster a fantastic intern last year at @benevolent_ai who continued to work with us on this. In the paper we demonstrate that counterfactual inference, batch correction, data integration and learning fair representations in a CVAE framework, can all be seen as learning representations that are conditionally independent of a covariate. Previous methods have tried to tackle this problem, such as the CVAE, FairVAE & TrVAE from @fabian_theis lab. Whilst the CVAE does condition on a known label (c), either the encoder or decoder can choose to ignore this, leading to a latent space that separates on condition (c) TrVAE and FairVAE do better than CVAE by introducing an MMD penalty. For CoMP, inspired by the VaMP prior and contrastive learning, we remove the need for any external discrepancy metric (e.g MMD) and use mixtures of the variational posterior alone, demonstrating better mixing! <LINK> We apply CoMP to three problems: 1. Aligning cancer cell lines with tumors (RNA-seq data - Celligner problem). CoMP can align latent representations of cancer cell lines and their tumour equivalents. CoMP preserves tumour type and subtype clustering and has better sensitivity. <LINK> 2. Counterfactual inference. We use PBMC scRNA-seq data treated and untreated with IFNg to ask: ""What would an untreated cell look like if perturbed with IFNg?"" CoMP successfully infers this - Whilst other methods overestimate underexpression or underestimate over expression. <LINK> 3. Fair representation learning Whilst solving these biological problems, we noticed parallels with fair representation learning and demonstrate that we can learn fair representations of biased income data that are invariant to Sex, yet still expressive (able to predict income). <LINK> The paper has many other neat findings including nice theoretical results demonstrating that CoMP is actually equivalent to an upper bound on a weighted sum of KL-divergences p(c)KL[q(z|c) | q(z|c')]. Check it out: <LINK> @jmtomczak VamP prior getting some love.",https://arxiv.org/abs/2106.08161,"Learning meaningful representations of data that can address challenges such as batch effect correction and counterfactual inference is a central problem in many domains including computational biology. Adopting a Conditional VAE framework, we show that marginal independence between the representation and a condition variable plays a key role in both of these challenges. We propose the Contrastive Mixture of Posteriors (CoMP) method that uses a novel misalignment penalty defined in terms of mixtures of the variational posteriors to enforce this independence in latent space. We show that CoMP has attractive theoretical properties compared to previous approaches and we prove counterfactual identifiability of CoMP under additional assumptions. We demonstrate state of the art performance on a set of challenging tasks including aligning human tumour samples with cancer cell-lines, predicting transcriptome-level perturbation responses, and batch correction on single-cell RNA sequencing data. We also find parallels to fair representation learning and demonstrate that CoMP is competitive on a common task in the field. ","Contrastive Mixture of Posteriors for Counterfactual Inference, Data