Original: E XPLORING
Revised: E XPLORING

Original: C ELLULAR
Revised: C ELLULAR

Original: P ROTEIN
Revised: P ROTEIN

Original: L OCALIZATION
Revised: L OCALIZATION

Original: T HROUGH S EMANTIC I MAGE S YNTHESIS
Revised: T HROUGH S EMANTIC I MAGE S YNTHESIS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Cell-cell interactions have an integral role in tumorigenesis as they are critical in governing immune responses.
Revised: Cell-cell interactions have an integral role in tumorigenesis as they are critical in governing immune responses.

Original: As such, investigating speciﬁc cell-cell interactions has the potential to not only expand upon the understanding of tumorigenesis, but also guide clinical management of patient responses to cancer immunotherapies.
Revised: As such, investigating speciﬁc cell-cell interactions has the potential to not only expand upon the understanding of tumorigenesis, but also guide clinical management of patient responses to cancer immunotherapies.

Original: A recent imaging technique for exploring cell-cell interactions, multiplexed ion beam imaging by time-of-ﬂight
Revised: A recent imaging technique for exploring cell-cell interactions, multiplexed ion beam imaging by time-of-ﬂight

Original: (MIBI-TOF), allows for cells to be quantiﬁed in 36 different protein markers at sub-cellular resolutions in situ as high resolution multiplexed images.
Revised: (MIBI-TOF), allows for cells to be quantiﬁed in 36 different protein markers at sub-cellular resolutions in situ as high resolution multiplexed images.

Original: To explore the MIBI images, we propose a GAN for multiplexed data with protein speciﬁc attention.
Revised: To explore the MIBI images, we propose a GAN for multiplexed data with protein speciﬁc attention.

Original: By conditioning image generation on cell types, sizes and neighborhoods through semantic segmentation maps, we are able to observe how these factors affect cell-cell interactions simultaneously in different protein channels.
Revised: By conditioning image generation on cell types, sizes and neighborhoods through semantic segmentation maps, we are able to observe how these factors affect cell-cell interactions simultaneously in different protein channels.

Original: Furthermore, we design a set of metrics and offer the ﬁrst insights towards cell spatial orientations, cell protein expressions and cell neighborhoods.
Revised: Furthermore, we design a set of metrics and offer the ﬁrst insights towards cell spatial orientations, cell protein expressions and cell neighborhoods.

Original: Our model, cell-cell interaction GAN (CCIGAN), outperforms or matches existing image synthesis methods on all conventional measures and signiﬁcantly outperforms on biologically motivated metrics.
Revised: Our model, cell-cell interaction GAN (CCIGAN), outperforms or matches existing image synthesis methods on all conventional measures and signiﬁcantly outperforms on biologically motivated metrics.

Original: To our knowledge, we are the ﬁrst to systematically model multiple cellular protein behaviors and interactions under simulated conditions through image synthesis.
Revised: To our knowledge, we are the ﬁrst to systematically model multiple cellular protein behaviors and interactions under simulated conditions through image synthesis.

Original:  I NTRODUCTION 1. B IOLOGICAL R OLE OF C ELLULAR P ROTEINS
Revised:  I NTRODUCTION 1. B IOLOGICAL R OLE OF C ELLULAR P ROTEINS

Original: Cell-cell interactions within the tumor microenvironment have been implicated in many facets of cancer pathogenesis and treatment.
Revised: Cell-cell interactions within the tumor microenvironment have been implicated in many facets of cancer pathogenesis and treatment.

Original: Most prominently, tumor cell evasion of immune surveillance (Jiang et al., 2019), tumor metastasis (Nishida-Aoki & Gujral, 2019), and efﬁcacy of cancer immunotherapies (Lau et al., 2017) have all been closely linked to the relationships between immune and cancer cells.
Revised: Most prominently, tumor cell evasion of immune surveillance (Jiang et al., 2019), tumor metastasis (Nishida-Aoki & Gujral, 2019), and efﬁcacy of cancer immunotherapies (Lau et al., 2017) have all been closely linked to the relationships between immune and cancer cells.

Original: These types of cell-cell relationships are generally governed by the interactions of cell surface proteins which drive cell behavior, gene expression, and survival.
Revised: These types of cell-cell relationships are generally governed by the interactions of cell surface proteins which drive cell behavior, gene expression, and survival.

Original: One of the most prominent examples of cellular proteins inﬂuencing disease progression is the case of PD-1/PD-L1.
Revised: One of the most prominent examples of cellular proteins inﬂuencing disease progression is the case of PD-1/PD-L1.

Original: PD-L1 is a protein often overexpressed on tumor cells and has the capacity to bind PD-1 on local T cells to downregulate their anti-tumor immune responses (Iwai et al., 2002).
Revised: PD-L1 is a protein often overexpressed on tumor cells and has the capacity to bind PD-1 on local T cells to downregulate their anti-tumor immune responses (Iwai et al., 2002).

Original: Antibodies which interrupt the interaction of PD-L1 and PD-1 and allow the immune system to attack tumor cells, have become clinically inﬂuential treatments for a variety of cancers (Pardoll, 2012).
Revised: Antibodies which interrupt the interaction of PD-L1 and PD-1 and allow the immune system to attack tumor cells, have become clinically inﬂuential treatments for a variety of cancers (Pardoll, 2012).

Original: This example highlights the value of accurately predicting cellular protein patterns which play key roles in disease processes.
Revised: This example highlights the value of accurately predicting cellular protein patterns which play key roles in disease processes.

Original: Exploring protein localizations in a multi-cellular system represents a challenge for which deep learning models are uniquely suited.
Revised: Exploring protein localizations in a multi-cellular system represents a challenge for which deep learning models are uniquely suited.

Original: However, to our knowledge no image-based deep generative models have utilized semantic image synthesis to produce accurate predictions of these biological phenomena.
Revised: However, to our knowledge no image-based deep generative models have utilized semantic image synthesis to produce accurate predictions of these biological phenomena.

Original: 1. F RAMEWORK AND N OVELTY OF A P REDICTIVE M ODEL
Revised: 1. F RAMEWORK AND N OVELTY OF A P REDICTIVE M ODEL

Original: A meaningful exploration of cell-cell interactions, particularly in the tumor microenvironment, requires a thorough understanding of the proteins expressed on and within a cell and its neighborhood.
Revised: A meaningful exploration of cell-cell interactions, particularly in the tumor microenvironment, requires a thorough understanding of the proteins expressed on and within a cell and its neighborhood.

Original: Multiplexed ion beam imaging by time-of-ﬂight
Revised: Multiplexed ion beam imaging by time-of-ﬂight

Original: (MIBI-TOF) represents a novel technology that can accurately quantify and spatially resolve cellular protein expressions at the single cell level within tissue samples.
Revised: (MIBI-TOF) represents a novel technology that can accurately quantify and spatially resolve cellular protein expressions at the single cell level within tissue samples.

Original: Given a tissue sample that is ﬁrst stained with protein-speciﬁc antibodies tethered to elemental metals, MIBI-TOF bombards the sample with atomic ions (i.e. O +2 ) from a primary ion beam.
Revised: Given a tissue sample that is ﬁrst stained with protein-speciﬁc antibodies tethered to elemental metals, MIBI-TOF bombards the sample with atomic ions (i.e. O +2 ) from a primary ion beam.

Original: This causes the release of elemental isotopes and tissue-speciﬁc material which can be quantiﬁed in a mass spectrometer (Angelo et al., 2014).
Revised: This causes the release of elemental isotopes and tissue-speciﬁc material which can be quantiﬁed in a mass spectrometer (Angelo et al., 2014).

Original: The cellular proteins characterized by this technique indicate speciﬁc cell types (i.e. immune cells, tumor cells), cell status (i.e. markers of proliferation), and immunomodulation.
Revised: The cellular proteins characterized by this technique indicate speciﬁc cell types (i.e. immune cells, tumor cells), cell status (i.e. markers of proliferation), and immunomodulation.

Original: Figure 1 (A) shows an example spatial orientation of cell types and some selected cellular protein expressions.
Revised: Figure 1 (A) shows an example spatial orientation of cell types and some selected cellular protein expressions.

Original: Here, we propose a novel protein based attention mechanism for a convolutional Generative Adversarial Network (GAN) with the capacity to provide accurate conditioned predictions of cellular protein localizations.
Revised: Here, we propose a novel protein based attention mechanism for a convolutional Generative Adversarial Network (GAN) with the capacity to provide accurate conditioned predictions of cellular protein localizations.

Original: The model, Cell-Cell Interaction
Revised: The model, Cell-Cell Interaction

Original: GAN (CCIGAN), learns a many to many mapping between different cell types and different protein markers.
Revised: GAN (CCIGAN), learns a many to many mapping between different cell types and different protein markers.

Original: CCIGAN is trained on semantic segmentation maps of cell tissue samples to identify and associate cell type, shape, and the identity of its cellular neighbors with cellular protein expressions for each cell.
Revised: CCIGAN is trained on semantic segmentation maps of cell tissue samples to identify and associate cell type, shape, and the identity of its cellular neighbors with cellular protein expressions for each cell.

Original: The network was then applied to simulated segmentation maps to understand how cells vary their protein localizations.
Revised: The network was then applied to simulated segmentation maps to understand how cells vary their protein localizations.

Original: CCIGAN demonstrates its ability to probe elements of oncologic processes by:
Revised: CCIGAN demonstrates its ability to probe elements of oncologic processes by:

Original: Independently recapitulating established patterns of biological phenomena, thereby demonstrating the accuracy of its predictive power.
Revised: Independently recapitulating established patterns of biological phenomena, thereby demonstrating the accuracy of its predictive power.

Original: Yielding quantitative and spatial information on speciﬁc cellular protein modulations as a result of immune cell - cancer cell interactions.
Revised: Yielding quantitative and spatial information on speciﬁc cellular protein modulations as a result of immune cell - cancer cell interactions.

Original: Allowing for biological data to be generalized beyond traditional in vitro scenarios where speciﬁc, rare, or hypothetical cell-cell events can be posed in counterfactual scenarios and the results of their interactions predicted.
Revised: Allowing for biological data to be generalized beyond traditional in vitro scenarios where speciﬁc, rare, or hypothetical cell-cell events can be posed in counterfactual scenarios and the results of their interactions predicted.

Original: The ability for CCIGAN to generate subcellular protein predictions represents a step forward in the ability to understand cellular relationships within a microenvironment.
Revised: The ability for CCIGAN to generate subcellular protein predictions represents a step forward in the ability to understand cellular relationships within a microenvironment.

Original: Rather than assessing in vitro incidence of cell interaction phenomena as is done by MIBI-TOF, CCIGAN allows for hypothetical biological situations to be generated.
Revised: Rather than assessing in vitro incidence of cell interaction phenomena as is done by MIBI-TOF, CCIGAN allows for hypothetical biological situations to be generated.

Original: Figure 1 (B) illustrates examples how a segmentation map can be directly manipulated to pose various biological scenarios.
Revised: Figure 1 (B) illustrates examples how a segmentation map can be directly manipulated to pose various biological scenarios.

Original: In other words, individual cellular responses to challenge or proximity of cells of any identity can be assessed without having to seek this speciﬁc occurrence within the available biological tissue sample.
Revised: In other words, individual cellular responses to challenge or proximity of cells of any identity can be assessed without having to seek this speciﬁc occurrence within the available biological tissue sample.

Original: These capabilities allow the model to provide insight on hypothetical interaction events which shed insight on disease pathogenesis and which cannot be feasibly achieved by biological investigation alone.
Revised: These capabilities allow the model to provide insight on hypothetical interaction events which shed insight on disease pathogenesis and which cannot be feasibly achieved by biological investigation alone.

Original: MIBI-TOF is a labor intensive process which can make the data collection process cumbersome.
Revised: MIBI-TOF is a labor intensive process which can make the data collection process cumbersome.

Original: Deep learning models such as CCIGAN have the potential to be useful tools which can extend on MIBI-TOF capabilities and result in more rapidly synthesized data.
Revised: Deep learning models such as CCIGAN have the potential to be useful tools which can extend on MIBI-TOF capabilities and result in more rapidly synthesized data.

Original: CCIGAN, once trained on
Revised: CCIGAN, once trained on

Original: MIBI-TOF data sets and properly replicating known cell-cell interaction patterns, can be used as a robust means to provide high throughput readouts on myriad single cell scenarios which may normally take many iterations of MIBI-TOF experiments to investigate.
Revised: MIBI-TOF data sets and properly replicating known cell-cell interaction patterns, can be used as a robust means to provide high throughput readouts on myriad single cell scenarios which may normally take many iterations of MIBI-TOF experiments to investigate.

Original: Recent work by (Wu et al., 2019) demonstrated the utility of conditional GANs for adding a depth dimension to images; we demonstrate learning funstional dimensitons, facilitating rapid hypothesis testing to assess biological environments.
Revised: Recent work by (Wu et al., 2019) demonstrated the utility of conditional GANs for adding a depth dimension to images; we demonstrate learning contextual cell-cell interactions, facilitating rapid hypothesis testing to assess biological environments.

Original: This provides valuable insight into a complex system that normally would have takena resource-intensive wet lab experiments to interrogate.
Revised: This provides valuable insight into a complex system that normally would have taken resource-intensive wet lab experiments to interrogate.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: We are interested in the task of generating biologically consistent expression patterns of cellular proteins given a segmentation map of cell neighborhoods.
Revised: We are interested in the task of generating biologically consistent expression patterns of cellular proteins given a segmentation map of cell neighborhoods.

Original: Speciﬁcally, we want to learn a generative model that simultaneously produces high quality maps of protein expression for individual cells that are probabilistically consistent when conditioned on the same factors, e.g. similar cell neighborhoods should produce similar expression patterns.
Revised: Speciﬁcally, we want to learn a generative model that simultaneously produces high quality maps of protein expression for individual cells that are probabilistically consistent when conditioned on the same factors, e.g. similar cell neighborhoods should produce similar expression patterns.

Original: Such generative models typically take the framework of a generative adversarial network (GAN) (Goodfellow et al., 2014; Brock et al., 2018) or a variational autoencoder (VAE) (Kingma & Welling, 2013).
Revised: Such generative models typically take the framework of a generative adversarial network (GAN) (Goodfellow et al., 2014; Brock et al., 2018) or a variational autoencoder (VAE) (Kingma & Welling, 2013).

Original: Within these generative modeling techniques, image translation focuses on learning a many to many mapping to transform data from one domain to another.
Revised: Within these generative modeling techniques, image translation focuses on learning a many to many mapping to transform data from one domain to another.

Original: One approach is to model the translation task, known as image synthesis, with conditioning such as Pix2Pix (Isola et al., 2016),
Revised: One approach is to model the translation task, known as image synthesis, with conditioning such as Pix2Pix (Isola et al., 2016),

Original: Pix2PixHD (Wang et al., 2018a) and CycleGAN (Zhu et al., 2017).
Revised: Pix2PixHD (Wang et al., 2018a) and CycleGAN (Zhu et al., 2017).

Original: By using image synthesis, a model is able to distill more information from a segmentation map to various protein channels.
Revised: By using image synthesis, a model is able to distill more information from a segmentation map to various protein channels.

Original: As
Revised: As

Original: MIBI-TOF measures information such as protein localizations at a subcellular resolution, framing the problem through an image synthesis framework allows a model to see how predictive neighborhoods and cell types are of a cell’s phenotype.
Revised: MIBI-TOF measures information such as protein localizations at a subcellular resolution, framing the problem through an image synthesis framework allows a model to see how predictive neighborhoods and cell types are of a cell’s phenotype.

Original: Recently, Park et al.
Revised: Recently, Park et al.

Original: (2019) proposed spatial adaptive normalization (SPADE) to synthesize images using segmentation maps by learning fully convolutional normalization parameters based on their segmentation conditioning.
Revised: (2019) proposed spatial adaptive normalization (SPADE) to synthesize images using segmentation maps by learning fully convolutional normalization parameters based on their segmentation conditioning.

Original: For each layer, semantic information is retained by allowing the network to learn from the segmentation map directly and modulate the current layer.
Revised: For each layer, semantic information is retained by allowing the network to learn from the segmentation map directly and modulate the current layer.

Original: They demonstrate impressive quality and diversity in generated images, especially modeling objects in context.
Revised: They demonstrate impressive quality and diversity in generated images, especially modeling objects in context.

Original: Using
Revised: Using

Original: SPADE normalization, CCIGAN is able to condition on surrounding cell neighbors and capture cellcell interactions in a local receptive ﬁeld.
Revised: SPADE normalization, CCIGAN is able to condition on surrounding cell neighbors and capture cellcell interactions in a local receptive ﬁeld.

Original: Despite image generation, there has been no work done towards understanding such generations in context, particularly in biological images.
Revised: Despite image generation, there has been no work done towards understanding such generations in context, particularly in biological images.

Original: Attention layers (Vaswani et al., 2017; Wang et al., 2018b; Zhang et al., 2018) have also been added to generative models with great success.
Revised: Attention layers (Vaswani et al., 2017; Wang et al., 2018b; Zhang et al., 2018) have also been added to generative models with great success.

Original: Self-attention was initially proposed in machine translation tasks to help model long distance dependencies that occur frequently in language (Sutskever et al., 2014), which led to the idea of external memories as a persistent state to model long range dependencies (Sukhbaatar et al., 2015).
Revised: Self-attention was initially proposed in machine translation tasks to help model long distance dependencies that occur frequently in language (Sutskever et al., 2014), which led to the idea of external memories as a persistent state to model long range dependencies (Sukhbaatar et al., 2015).

Original: While convolutional neural networks are apt at exploiting local structures of patches, they may struggle to model global structures.
Revised: While convolutional neural networks are apt at exploiting local structures of patches, they may struggle to model global structures.

Original: For this reason visual attention allows the network to enhance activations in interesting parts of an image.
Revised: For this reason visual attention allows the network to enhance activations in interesting parts of an image.

Original: Goodfellow et al.
Revised: Goodfellow et al.

Original: (2014) and Brock et al.
Revised: (2014) and Brock et al.

Original: both achieve state-of-the-art unconditional generative modeling using self attention GAN (Zhang et al., 2018).
Revised: both achieve state-of-the-art unconditional generative modeling using self attention GAN (Zhang et al., 2018).

Original: We propose a specialized attention module conditioned on different proteins to mimic real world protein markers.
Revised: We propose a specialized attention module conditioned on different proteins to mimic real world protein markers.

Original:  D ATASET 3. M ULTIPLEXED I ON B EAM I MAGES (MIBI)
Revised:  D ATASET 3. M ULTIPLEXED I ON B EAM I MAGES (MIBI)

Original: All experiments are performed on data obtained through MIBI-TOF characterized tissue samples collected from triple-negative breast cancer (TNBC) patients.
Revised: All experiments are performed on data obtained through MIBI-TOF characterized tissue samples collected from triple-negative breast cancer (TNBC) patients.

Original: By simultaneously imaging overprotein markers, MIBI-TOF is able to identify cell type as well as provide detailed information of sub-cellular structure, cell neighbors, and interactions in the tumor microenvironment across these different marker settings.
Revised: By simultaneously imaging overprotein markers, MIBI-TOF is able to identify cell type as well as provide detailed information of sub-cellular structure, cell neighbors, and interactions in the tumor microenvironment across these different marker settings.

Original: Each of these markers m ∈ { 1 , ..., M } , given as a channel taking on real values continuous in [0 , 1] at each ( x, y ) coordinate, demarcates a different cellular subtype and furthermore, is indicative of the functional properties of a cell.
Revised: Each of these markers m ∈ { 1 , ..., M } , given as a channel taking on real values continuous in [0 , 1] at each ( x, y ) coordinate, demarcates a different cellular subtype and furthermore, is indicative of the functional properties of a cell.

Original: While MIBI-TOF is capable ofdifferent markers, we chose markers exhibiting cellular protein localization and disregarded markers that were blank or used simply for cell-typing 1 resulting in M = 24 .
Revised: While MIBI-TOF is capable ofdifferent markers, we chose markers exhibiting cellular protein localization and disregarded markers that were blank or used simply for cell-typing 1 resulting in M = 24 .

Original: A full list of these markers is given in A.8.
Revised: A full list of these markers is given in A.8.

Original: The MIBI-TOF data collected from the TNBC tissue allows for the differentiation between a wide variety of cell types within the samples.
Revised: The MIBI-TOF data collected from the TNBC tissue allows for the differentiation between a wide variety of cell types within the samples.

Original: For instance, cells positive for the marker CD3 could be identiﬁed as T cells, and then subdivided into cytotoxic or helper lineages by the presence of markers CD8 or CD4, respectively.
Revised: For instance, cells positive for the marker CD3 could be identiﬁed as T cells, and then subdivided into cytotoxic or helper lineages by the presence of markers CD8 or CD4, respectively.

Original: Tumor cells could be identiﬁed by markers such as pan-keratin and overexpressed beta-catenin.
Revised: Tumor cells could be identiﬁed by markers such as pan-keratin and overexpressed beta-catenin.

Original: Along these lines, a wide variety of cellular proteins identiﬁed by
Revised: Along these lines, a wide variety of cellular proteins identiﬁed by

Original: MIBI-TOF could characterize cell interactions as well as immunomodulatory processes occurring within
Revised: MIBI-TOF could characterize cell interactions as well as immunomodulatory processes occurring within

Original: MIBI-TOF data is fundamentally different than typical RGB images.
Revised: MIBI-TOF data is fundamentally different than typical RGB images.

Original: This poses unique challenges in image modeling and characterization.
Revised: This poses unique challenges in image modeling and characterization.

Original: Such challenges stem from each marker being conditionally expressed on the cell type in its respective channel m .
Revised: Such challenges stem from each marker being conditionally expressed on the cell type in its respective channel m .

Original: Using a simpliﬁed 3 channel multiplex setting, a T cell expresses signals in the CD3, CD8 channels (indicators for immune cells) but not in a pankeratin channel (indicator for tumor cells).
Revised: Using a simpliﬁed 3 channel multiplex setting, a T cell expresses signals in the CD3, CD8 channels (indicators for immune cells) but not in a pankeratin channel (indicator for tumor cells).

Original: Another problem is the sparsity of the data, meaning either some expressions for rare cell types are rarely observed or have weak signals.
Revised: Another problem is the sparsity of the data, meaning either some expressions for rare cell types are rarely observed or have weak signals.

Original: For example,
Revised: For example,

Original: Figure 2 includes a segmentation of a CD8 T cell (orange, top right corner), where other models fail in generating correct CD3 expression, if at all 2 .
Revised: Figure 2 includes a segmentation of a CD8 T cell (orange, top right corner), where other models fail in generating correct CD3 expression, if at all 2 .

Original: Lastly, the noisy nature of the data leads to inaccurate cell type classiﬁcations, creating inconsistent pairs of labels and outputs during training.
Revised: Lastly, the noisy nature of the data leads to inaccurate cell type classiﬁcations, creating inconsistent pairs of labels and outputs during training.

Original: These issues make it especially difﬁcult for an RGB multihead decoder to output multiple channels in a biologically accurate manner.
Revised: These issues make it especially difﬁcult for an RGB multihead decoder to output multiple channels in a biologically accurate manner.

Original: Without addressing multiplexed data, a decoder would equally attend to every location of the current latent representation, even if it is irrelevant to the current protein.
Revised: Without addressing multiplexed data, a decoder would equally attend to every location of the current latent representation, even if it is irrelevant to the current protein.

Original: Furthermore each protein channel in the output has its own sensitivities to signal intensities and noise, suggesting each channel requires a unique prior and that equal attention would be problematic.
Revised: Furthermore each protein channel in the output has its own sensitivities to signal intensities and noise, suggesting each channel requires a unique prior and that equal attention would be problematic.

Original: It follows that special care must be given towards modeling speciﬁc channels and the multiplexed nature of MIBI-TOF images.
Revised: It follows that special care must be given towards modeling speciﬁc channels and the multiplexed nature of MIBI-TOF images.

Original: 3. D ATA P ROCESSING
Revised: 3. D ATA P ROCESSING

Original: MIBI-TOF images are represented as a high dimensional tensor T ∈ R ( M, 2048 , 2048) .
Revised: MIBI-TOF images are represented as a high dimensional tensor T ∈ R ( M, 2048 , 2048) .

Original: These images are then further processed at a cell by cell basis into Y ∈ R ( M, 64 , 64) patches, where a cell is at the center of the patch along with its neighbors.
Revised: These images are then further processed at a cell by cell basis into Y ∈ R ( M, 64 , 64) patches, where a cell is at the center of the patch along with its neighbors.

Original: Next, we construct semantic segmentation maps
Revised: Next, we construct semantic segmentation maps

Original: S ∈ R ( C +1 , 64 , 64) , where a vector S : ,i,j is one-hot encoded based on a cell type C = 17 , and the
Revised: S ∈ R ( C +1 , 64 , 64) , where a vector S : ,i,j is one-hot encoded based on a cell type C = 17 , and the

Original: C + 1 -th channel denotes empty segmentation space.
Revised: C + 1 -th channel denotes empty segmentation space.

Original: The data is train-test split at a 9:1 ratio at the
Revised: The data is train-test split at a 9:1 ratio at the

Original: MIBI-TOF image level to avoid cell neighborhood bias.
Revised: MIBI-TOF image level to avoid cell neighborhood bias.

Original: We also use a synthetic test set where cells and their neighbors are sequentially modiﬁed to observe how varying cell type, position, and size affects the progressive changes in protein localizations 3 .
Revised: We also use a synthetic test set where cells and their neighbors are sequentially modiﬁed to observe how varying cell type, position, and size affects the progressive changes in protein localizations 3 .

Original: [Figure]
Revised: [Figure]

Original: We use SPADE residual blocks (Park et al., 2019) as our generative backbone and DCGAN’s discriminator’s architecture (Figure 3, A.1)
Revised: We use SPADE residual blocks (Park et al., 2019) as our generative backbone and DCGAN’s discriminator’s architecture (Figure 3, A.1)

Original: (Radford et al., 2015).
Revised: (Radford et al., 2015).

Original: Park et al.
Revised: Park et al.

Original: (2019) have shown
Revised: (2019) have shown

Original: SPADE to be an effective way to inject conditioning into a generative model.
Revised: SPADE to be an effective way to inject conditioning into a generative model.

Original: The SPADE normalization layer serves as a replacement for previous layer normalization techniques.
Revised: The SPADE normalization layer serves as a replacement for previous layer normalization techniques.

Original: Instead of learning a universally shared per channel afﬁne transformation, like in Batch Normalization (Ioffe & Szegedy, 2015) or Instance Normalization (Ulyanov et al., 2016), SPADE learns to predict afﬁne transformations based on segmentation maps; each feature is uniquely transformed based on its cell type, size, and neighboring cells.
Revised: Instead of learning a universally shared per channel afﬁne transformation, like in Batch Normalization (Ioffe & Szegedy, 2015) or Instance Normalization (Ulyanov et al., 2016), SPADE learns to predict afﬁne transformations based on segmentation maps; each feature is uniquely transformed based on its cell type, size, and neighboring cells.

Original: The ability for SPADE to modulate activations based on the context of adjacent cell segmentations allows the network to effectively model the behaviors and interactions of cells.
Revised: The ability for SPADE to modulate activations based on the context of adjacent cell segmentations allows the network to effectively model the behaviors and interactions of cells.

Original: The input of CCIGAN is a noise vector z ∈ R 128 and a segmentation map S .
Revised: The input of CCIGAN is a noise vector z ∈ R 128 and a segmentation map S .

Original: f denotes a linear layer R 128 (cid:55)→ R 2048 .
Revised: f denotes a linear layer R 128 (cid:55)→ R 2048 .

Original: R i are feature map representations from SPADE resblocks and X denotes the ﬁnal output of M cell expressions.
Revised: R i are feature map representations from SPADE resblocks and X denotes the ﬁnal output of M cell expressions.

Original: Below, each layer’s output dimensions are given next to their respective equations.
Revised: Below, each layer’s output dimensions are given next to their respective equations.

Original: [Equation]
Revised: [Equation]

Original: A
Revised: A

Original: TTENTION M ODULE
Revised: TTENTION M ODULE

Original: Our architectural contribution is a protein marker dependent attention module in the ﬁnal output layer.
Revised: Our architectural contribution is a protein marker dependent attention module in the ﬁnal output layer.

Original: The goal of the attention module is to condition the ﬁnal output of a channel on a protein marker m and S ’s cell types.
Revised: The goal of the attention module is to condition the ﬁnal output of a channel on a protein marker m and S ’s cell types.

Original: For example the protein marker, pan-keratin m pk , is expressed exclusively in tumor cells but not in other cells.
Revised: For example the protein marker, pan-keratin m pk , is expressed exclusively in tumor cells but not in other cells.

Original: Appropriately, an attention mechanism should attend to tumor cells and ignore irrelevant cells in S for m pk .
Revised: Appropriately, an attention mechanism should attend to tumor cells and ignore irrelevant cells in S for m pk .

Original: To replicate a marker searching for speciﬁc cell types that express it, we deﬁne a learned persistent vector for each marker denoted by s m ∈ M ∈
Revised: To replicate a marker searching for speciﬁc cell types that express it, we deﬁne a learned persistent vector for each marker denoted by s m ∈ M ∈

Original: Rthat undergo a series of operations (Figure 4) with the ﬁnal feature map representation attending to m ’s speciﬁc cell types.
Revised: Rthat undergo a series of operations (Figure 4) with the ﬁnal feature map representation attending to m ’s speciﬁc cell types.

Original: It is also worthwhile to note that these persistent vectors s m offer a degree of model interpretability that mimic real world markers.
Revised: It is also worthwhile to note that these persistent vectors s m offer a degree of model interpretability that mimic real world markers.

Original: The current input dimensions to the attention module are R (128 , 64 , 64) following the last resblock R 4 and m indexes from 1 , .., M .
Revised: The current input dimensions to the attention module are R (128 , 64 , 64) following the last resblock R 4 and m indexes from 1 , .., M .

Original: [Equation]
Revised: [Equation]

Original: Shown in Figure 4, after R 4 , a bottleneck convolution is applied to match the original data’s dimension as O (step 1), which is used in a residual manner with the ﬁnal output.
Revised: Shown in Figure 4, after R 4 , a bottleneck convolution is applied to match the original data’s dimension as O (step 1), which is used in a residual manner with the ﬁnal output.

Original: Intuitively at this stage, O ’s feature maps resemble the target Y , but we wish to further reﬁne the output channels.
Revised: Intuitively at this stage, O ’s feature maps resemble the target Y , but we wish to further reﬁne the output channels.

Original: We convolve O into MK channeled features for each protein marker where K = 8 .
Revised: We convolve O into MK channeled features for each protein marker where K = 8 .

Original: Considering each C i where i ∈ { 1 , ..., M } as a group of K channels, the model spatially adaptive normalizes each C i and computes an outer product with the corresponding persistent vector s i and C i .
Revised: Considering each C i where i ∈ { 1 , ..., M } as a group of K channels, the model spatially adaptive normalizes each C i and computes an outer product with the corresponding persistent vector s i and C i .

Original: The resulting matrix is ﬂattened and convolved (with a kernel size of 1 on the pixel level) from
Revised: The resulting matrix is ﬂattened and convolved (with a kernel size of 1 on the pixel level) from

Original: A i ∈ R ( | s |× K, 64 , 64) (cid:55)→ R (1 , 64 , 64) followed by a sigmoid σ ( · ) activation.
Revised: A i ∈ R ( | s |× K, 64 , 64) (cid:55)→ R (1 , 64 , 64) followed by a sigmoid σ ( · ) activation.

Original: Lastly, the attentions
Revised: Lastly, the attentions

Original: B 1 ,...,M are added to O to obtain the output X .
Revised: B 1 ,...,M are added to O to obtain the output X .

Original: Initially, the model has no priors over the interaction of protein markers and cell types.
Revised: Initially, the model has no priors over the interaction of protein markers and cell types.

Original: The proposed outer product attention layer (outer product and 1 × 1 convolution) excels at modeling these relationships and interactions between speciﬁc markers and cell types.
Revised: The proposed outer product attention layer (outer product and 1 × 1 convolution) excels at modeling these relationships and interactions between speciﬁc markers and cell types.

Original: By using an outer product, the model forces attention at a pairwise pixel level comparison for all combinations of elements between s m and A i .
Revised: By using an outer product, the model forces attention at a pairwise pixel level comparison for all combinations of elements between s m and A i .

Original: As training progresses, both the learned features over segmentation patches and the learned persistent vectors
Revised: As training progresses, both the learned features over segmentation patches and the learned persistent vectors

Original: s m improve, in turn allowing the learned 1 × 1 convolution to reason about positive or negative relationships from the pairwise pixel combinations.
Revised: s m improve, in turn allowing the learned 1 × 1 convolution to reason about positive or negative relationships from the pairwise pixel combinations.

Original: 4. I MPLEMENTATION D ETAILS AND T RAINING R EGIMEN
Revised: 4. I MPLEMENTATION D ETAILS AND T RAINING R EGIMEN

Original: Our implementation (A.1) of the generator applies Spectral Norm to all layers (Miyato et al., 2018).
Revised: Our implementation (A.1) of the generator applies Spectral Norm to all layers (Miyato et al., 2018).

Original: The discriminator’s input is the output of the generator concatenated with the segmentation patch [ X , S ] and [ Y , S ] for the ground truth.
Revised: The discriminator’s input is the output of the generator concatenated with the segmentation patch [ X , S ] and [ Y , S ] for the ground truth.

Original: Finally CCIGAN uses ADAM ( lr G = 0 .
Revised: Finally CCIGAN uses ADAM ( lr G = 0 .

Original: 0004 , lr D = 0 . 0001 ) with GAN loss and feature matching loss.
Revised: 0004 , lr D = 0 . 0001 ) with GAN loss and feature matching loss.

Original: Full training details and loss functions are given in A.2.
Revised: Full training details and loss functions are given in A.2.

Original:  E VALUATION
Revised:  E VALUATION

Original: To conduct fair experiments, all models were optimized, tuned, and set with similar parameters.
Revised: To conduct fair experiments, all models were optimized, tuned, and set with similar parameters.

Original: They were also taken from their ofﬁcial online implementations and trained for 120 epochs or until convergence (max 150).
Revised: They were also taken from their ofﬁcial online implementations and trained for 120 epochs or until convergence (max 150).

Original: CCIGAN is identical to our designed SPADE comparison baseline with the exception of the attention module.
Revised: CCIGAN is identical to our designed SPADE comparison baseline with the exception of the attention module.

Original: Three experiments were conducted to validate the trained model’s utility in generating biologically meaningful cellular proteins in the tumor microenvironment and ability to recapitulate and quantify previously established biological phenomena.
Revised: Three experiments were conducted to validate the trained model’s utility in generating biologically meaningful cellular proteins in the tumor microenvironment and ability to recapitulate and quantify previously established biological phenomena.

Original: Each subsection describes the experiment and the relevant metrics used in evaluation.
Revised: Each subsection describes the experiment and the relevant metrics used in evaluation.

Original: Full mathematical deﬁnitions are given in section A.3.
Revised: Full mathematical deﬁnitions are given in section A.3.

Original: 5. I MAGE E VALUATION AND R ECONSTRUCTION
Revised: 5. I MAGE E VALUATION AND R ECONSTRUCTION

Original: First, we use the following evaluation metrics in order to compare with baseline results: adjusted Land MSE score, L 1 and MSE score, structural similarity (SSIM) index (Wang et al., 2004) and cell based mutual information (MI) shown in Table 1.
Revised: First, we use the following evaluation metrics in order to compare with baseline results: adjusted Land MSE score, L 1 and MSE score, structural similarity (SSIM) index (Wang et al., 2004) and cell based mutual information (MI) shown in Table 1.

Original: Equations and explanations are given in A.3.1.
Revised: Equations and explanations are given in A.3.1.

Original: Bolded scores indicate the best scores.
Revised: Bolded scores indicate the best scores.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: 5. PD-1/PD-L1 R ELATIONSHIPS BETWEEN T CELLS AND TUMOR CELLS
Revised: 5. PD-1/PD-L1 R ELATIONSHIPS BETWEEN T CELLS AND TUMOR CELLS

Original: In the ﬁrst step of biologically verifying the accuracy of CCIGAN, PD-1/PD-L1 expression relationships between CD8 T cells and tumor cells were assessed.
Revised: In the ﬁrst step of biologically verifying the accuracy of CCIGAN, PD-1/PD-L1 expression relationships between CD8 T cells and tumor cells were assessed.

Original: Many T cells located within the tumor microenvironment have upregulated expression of PD-1 suggesting that the tumor milieu exerts inﬂuence on the protein localization and expression of inﬁltrating T cells (Ahmadzadeh et al., 2009).
Revised: Many T cells located within the tumor microenvironment have upregulated expression of PD-1 suggesting that the tumor milieu exerts inﬂuence on the protein localization and expression of inﬁltrating T cells (Ahmadzadeh et al., 2009).

Original: We assessed if increased expression of PD-L1 on neighboring tumor cells would result in increased directional PD-1 expression in a CD8 T cell (Figure 5).
Revised: We assessed if increased expression of PD-L1 on neighboring tumor cells would result in increased directional PD-1 expression in a CD8 T cell (Figure 5).

Original: We also determined if there was a shift in the cell surface localization of the PD-1.
Revised: We also determined if there was a shift in the cell surface localization of the PD-1.

Original: To do so, we computed the Earth
Revised: To do so, we computed the Earth

Original: Mover’s Distance between a CD8 T cell’s PD-1 expression (represented as a histogram on a cell’s polar coordinates) and itself before and after in different tumor scenarios.
Revised: Mover’s Distance between a CD8 T cell’s PD-1 expression (represented as a histogram on a cell’s polar coordinates) and itself before and after in different tumor scenarios.

Original: In addition to the mass, we computed the expected center of mass (COM) of PDin a CD8 T cell with respect to the neighboring tumor PD-L1 expressions.
Revised: In addition to the mass, we computed the expected center of mass (COM) of PDin a CD8 T cell with respect to the neighboring tumor PD-L1 expressions.

Original: Equations and explanations are given in A.3.2 and A.3.3.
Revised: Equations and explanations are given in A.3.2 and A.3.3.

Original: Additional ﬁgures illustrating this process are given in A.4.
Revised: Additional ﬁgures illustrating this process are given in A.4.

Original: We expected that the properties of the PD-L1 expressing tumor cell would result in an increased directional PD-1 expression in a neighboring T cell.
Revised: We expected that the properties of the PD-L1 expressing tumor cell would result in an increased directional PD-1 expression in a neighboring T cell.

Original: As a control, a PD-1 expressing T Cell was surrounded with endothelial cells (a normal tissue lining cell).
Revised: As a control, a PD-1 expressing T Cell was surrounded with endothelial cells (a normal tissue lining cell).

Original: As shown in Table 2, our initial hypothesis was conﬁrmed in CCIGAN’s predictions, wherein the presence of a PD-L1 expressing tumor cell, PD-1 expression in the CD8 T cell increased and moved towards the PD-L1 COM.
Revised: As shown in Table 2, our initial hypothesis was conﬁrmed in CCIGAN’s predictions, wherein the presence of a PD-L1 expressing tumor cell, PD-1 expression in the CD8 T cell increased and moved towards the PD-L1 COM.

Original: On the other hand, the endothelial cell presence yielded no effect on the T cell’s PD-1 expression, a result which is biologically expected.
Revised: On the other hand, the endothelial cell presence yielded no effect on the T cell’s PD-1 expression, a result which is biologically expected.

Original: While both tumor and endothelial cells express PD-L1, the tumor microenvironment exercises more complex suppressive effects on CD8 T cells, which are more likely to modulate the T cell expression than endothelial cells.
Revised: While both tumor and endothelial cells express PD-L1, the tumor microenvironment exercises more complex suppressive effects on CD8 T cells, which are more likely to modulate the T cell expression than endothelial cells.

Original: These ﬁndings verify CCIGAN captures the biological relationship of tumor cells inducing immunomodulatory changes on a neighboring T cell.
Revised: These ﬁndings verify CCIGAN captures the biological relationship of tumor cells inducing immunomodulatory changes on a neighboring T cell.

Original: None of the other models succeeded in capturing this protein relationship.
Revised: None of the other models succeeded in capturing this protein relationship.

Original: This serves to highlight the capacity of CCIGAN to recapitulate established cell interaction phenomena within the tumor microenvironment.
Revised: This serves to highlight the capacity of CCIGAN to recapitulate established cell interaction phenomena within the tumor microenvironment.

Original: 5. P AN - KERATIN AND CD8 E XPRESSION
Revised: 5. P AN - KERATIN AND CD8 E XPRESSION

Original: Secondly, we assessed the effect of immune cell presence on tumor cell status markers.
Revised: Secondly, we assessed the effect of immune cell presence on tumor cell status markers.

Original: Keratins are a class of intracellular proteins that play an important role in ensuring cell structure.
Revised: Keratins are a class of intracellular proteins that play an important role in ensuring cell structure.

Original: Furthermore, when CD8 T cells mediate tumor killing, they release enzymes which cleave the tumor cell’s pan-keratin, disrupting the tumor cell structure (Oshima, 2002).
Revised: Furthermore, when CD8 T cells mediate tumor killing, they release enzymes which cleave the tumor cell’s pan-keratin, disrupting the tumor cell structure (Oshima, 2002).

Original: We explored how the presence of neighboring CD8 T cells to a tumor cell would affect pan-keratin levels within the tumor cell, hypothesizing that T cell mediated tumor killing would result in a drop in pan-keratin tumor expression.
Revised: We explored how the presence of neighboring CD8 T cells to a tumor cell would affect pan-keratin levels within the tumor cell, hypothesizing that T cell mediated tumor killing would result in a drop in pan-keratin tumor expression.

Original: In this experiment, we used a Student’s t -test as the statistical hypothesis test to evaluate the correlations between the pan-keratin expression in tumor cells and the area/number of CD8 T cells in contact with the tumor cell surface.
Revised: In this experiment, we used a Student’s t -test as the statistical hypothesis test to evaluate the correlations between the pan-keratin expression in tumor cells and the area/number of CD8 T cells in contact with the tumor cell surface.

Original: Given a generated pan-keratin channel X i ∈ R ( H,W ) and the segmentation map channel for CD8 T cells S i ∈
Revised: Given a generated pan-keratin channel X i ∈ R ( H,W ) and the segmentation map channel for CD8 T cells S i ∈

Original: R
Revised: R

Original: ( H,W ) , we compute the total area of the cells a i =
Revised: ( H,W ) , we compute the total area of the cells a i =

Original: (cid:80)
Revised: (cid:80)

Original: Hh =1 (cid:80) Ww =1 S i , and the total expression level of pan-keratin e i =
Revised: Hh =1 (cid:80) Ww =1 S i , and the total expression level of pan-keratin e i =

Original: (cid:80)
Revised: (cid:80)

Original: Hh =1 (cid:80) Ww =1 X i .
Revised: Hh =1 (cid:80) Ww =1 X i .

Original: We then regress { e i } Ni =1 on { a i } Ni =1 and assess signiﬁcance of the slope using a t -test against the null of no change in pan-keratin expression as a function of CD8 T cell-tumor contact.
Revised: We then regress { e i } Ni =1 on { a i } Ni =1 and assess signiﬁcance of the slope using a t -test against the null of no change in pan-keratin expression as a function of CD8 T cell-tumor contact.

Original: CD8 T cells were placed adjacent to a tumor cell in increasing number, similar to Figure 1 (B)’s ﬁrst row and Figure 5’s experiment.
Revised: CD8 T cells were placed adjacent to a tumor cell in increasing number, similar to Figure 1 (B)’s ﬁrst row and Figure 5’s experiment.

Original: It was observed that the presence of neighboring CD8 T cells to a tumor cell tended to decrease the pan-keratin expression in the tumor cell.
Revised: It was observed that the presence of neighboring CD8 T cells to a tumor cell tended to decrease the pan-keratin expression in the tumor cell.

Original: This effect became more dramatic as the number of surrounding CD8 T cells was increased (A.5).
Revised: This effect became more dramatic as the number of surrounding CD8 T cells was increased (A.5).

Original: As a control, when the tumor cell of interest was surrounded with adjacent tumor cell(s) instead of CD8 T cells, no change was noted in that tumor cell’s pan keratin for CCIGAN.
Revised: As a control, when the tumor cell of interest was surrounded with adjacent tumor cell(s) instead of CD8 T cells, no change was noted in that tumor cell’s pan keratin for CCIGAN.

Original: Table 2’s fourth row shows the result of the t -test where the slope of the linear regression of pan-keratin expression with respect to the number of surrounding cells is compared to a y = 0 ﬂat baseline.
Revised: Table 2’s fourth row shows the result of the t -test where the slope of the linear regression of pan-keratin expression with respect to the number of surrounding cells is compared to a y = 0 ﬂat baseline.

Original: A statistically signiﬁcant difference between the slope of the linear regression for the T cell scenario vs. the baseline was found in the main experiment, but was not present in the control scenario.
Revised: A statistically signiﬁcant difference between the slope of the linear regression for the T cell scenario vs. the baseline was found in the main experiment, but was not present in the control scenario.

Original: These results indicate that the T cell presence mitigates a decrease in pan-keratin expression in the tumor cell, which is suggestive of T cell mediated tumor killing.
Revised: These results indicate that the T cell presence mitigates a decrease in pan-keratin expression in the tumor cell, which is suggestive of T cell mediated tumor killing.

Original: Other models reported contrasting results where a signiﬁcant change in the tumor cell pan-keratin expression was incorrectly reported under the control conditions.
Revised: Other models reported contrasting results where a signiﬁcant change in the tumor cell pan-keratin expression was incorrectly reported under the control conditions.

Original: The resulting graphs are given in A.5.
Revised: The resulting graphs are given in A.5.

Original: Further protein interaction patterns identiﬁed by CCIGAN are also reported in A.7.
Revised: Further protein interaction patterns identiﬁed by CCIGAN are also reported in A.7.

Original: The ﬁndings from this experiment serve to highlight the robustness and accuracy of
Revised: The ﬁndings from this experiment serve to highlight the robustness and accuracy of

Original: CCIGAN compared to existing image synthesis techniques in probing cell-cell interactions.
Revised: CCIGAN compared to existing image synthesis techniques in probing cell-cell interactions.

Original: 5. T UMOR I NFILTRATED AND C OMPARTMENTALIZED M ICROENVIRONMENTS
Revised: 5. T UMOR I NFILTRATED AND C OMPARTMENTALIZED M ICROENVIRONMENTS

Original: As a ﬁnal biological evaluation, we consider the variability of PD-L1 expression in immune and tumor cells across TNBC patient groups.
Revised: As a ﬁnal biological evaluation, we consider the variability of PD-L1 expression in immune and tumor cells across TNBC patient groups.

Original: Keren et al.
Revised: Keren et al.

Original: (2018) determined that in situations of mixed tumor-immune architecture, where immune cells freely inﬁltrated the tumor, the tumor cells predominantly expressed PD-L1.
Revised: (2018) determined that in situations of mixed tumor-immune architecture, where immune cells freely inﬁltrated the tumor, the tumor cells predominantly expressed PD-L1.

Original: Conversely, in situations of compartmentalized tumors, where there is a greater degree of delineation between immune and tumor cells, macrophages were the predominant source of expressed PD-L1, particularly at the tumor boundary.
Revised: Conversely, in situations of compartmentalized tumors, where there is a greater degree of delineation between immune and tumor cells, macrophages were the predominant source of expressed PD-L1, particularly at the tumor boundary.

Original: Figure 1 (B)’s row 2, 3 show examples of how directly manipulated segmentation maps can simulate the two microenvironments.
Revised: Figure 1 (B)’s row 2, 3 show examples of how directly manipulated segmentation maps can simulate the two microenvironments.

Original: We used CCIGAN to predict on 200 directly manipulated mixed and non-mixed tumor environment segmentation patches.
Revised: We used CCIGAN to predict on 200 directly manipulated mixed and non-mixed tumor environment segmentation patches.

Original: Similar to Section 5.3’s experimental settings, we then compute the average expression of a speciﬁc marker for the cells of interest for all patches.
Revised: Similar to Section 5.3’s experimental settings, we then compute the average expression of a speciﬁc marker for the cells of interest for all patches.

Original: For each experiment, we use endothelial cells as control cells to show our result has biological signiﬁcance.
Revised: For each experiment, we use endothelial cells as control cells to show our result has biological signiﬁcance.

Original: These ﬁndings were recapitulated by CCIGAN in Table 3.
Revised: These ﬁndings were recapitulated by CCIGAN in Table 3.

Original: For a patient with a mixed tumor environment, when trained with mixed patient samples, CCIGAN reported increased PD-L1 expression on tumor cells.
Revised: For a patient with a mixed tumor environment, when trained with mixed patient samples, CCIGAN reported increased PD-L1 expression on tumor cells.

Original: Furthermore, CCIGAN was able to quantify this difference in expression at the single cell level, reporting a tumor to macrophage PD-L1 expression ratio (bolded) of approximately 3.
Revised: Furthermore, CCIGAN was able to quantify this difference in expression at the single cell level, reporting a tumor to macrophage PD-L1 expression ratio (bolded) of approximately 3.

Original: and 1.75 for patients A and B respectively
Revised: and 1.75 for patients A and B respectively

Original: [Table]
Revised: [Table]

Original: Experiment Microenvironments
Revised: Experiment Microenvironments

Original: [Table]
Revised: [Table]

Original: Conversely, when trained with compartmentalized patient samples, CCIGAN reported increased
Revised: Conversely, when trained with compartmentalized patient samples, CCIGAN reported increased

Original: PD-L1 expression on macrophages adjacent to tumor cells as compared to macrophages adjacent to normal endothelial (inert) cells for patients C and D.
Revised: PD-L1 expression on macrophages adjacent to tumor cells as compared to macrophages adjacent to normal endothelial (inert) cells for patients C and D.

Original: This difference was quantiﬁed as a ratio of PDL1 expression of tumor-adjacent macrophages to endothelial-adjacent macrophages, approximately 1.85 and 2.7 for patient C and patient D respectively.
Revised: This difference was quantiﬁed as a ratio of PDL1 expression of tumor-adjacent macrophages to endothelial-adjacent macrophages, approximately 1.85 and 2.7 for patient C and patient D respectively.

Original: Moreover, using the trained compartmentalized model to predict on mixed segmentation patches, CCIGAN still reports a 26% (patient C) and 19% (patient D) increase of macrophage PD-L1 expression when compared to mixed microenvironments (Table 4).
Revised: Moreover, using the trained compartmentalized model to predict on mixed segmentation patches, CCIGAN still reports a 26% (patient C) and 19% (patient D) increase of macrophage PD-L1 expression when compared to mixed microenvironments (Table 4).

Original: [Table]
Revised: [Table]

Original: The PD-L1 ratios for the above two scenarios indicate that CCIGAN has appropriately captured previously reported biological outcomes and is capable of quantifying these phenomena at single cell levels.
Revised: The PD-L1 ratios for the above two scenarios indicate that CCIGAN has appropriately captured previously reported biological outcomes and is capable of quantifying these phenomena at single cell levels.

Original: Furthermore, the model is adaptable to various different types of tumor architecture depending on its training set to produce different hypothesis testing environments.
Revised: Furthermore, the model is adaptable to various different types of tumor architecture depending on its training set to produce different hypothesis testing environments.

Original: The agreement between CCIGAN data and those reported in Keren et al.
Revised: The agreement between CCIGAN data and those reported in Keren et al.

Original: (2018) serve as an important control and demonstrate the ﬁdelity of the CCIGAN output towards true biological results.
Revised: (2018) serve as an important control and demonstrate the ﬁdelity of the CCIGAN output towards true biological results.

Original: More importantly, this provides strong evidence to support the accuracy of the predictions made by CCIGAN in the assessment of hypothetical cellular scenarios which cannot be tested via in vitro tissue study alone.
Revised: More importantly, this provides strong evidence to support the accuracy of the predictions made by CCIGAN in the assessment of hypothetical cellular scenarios which cannot be tested via in vitro tissue study alone.

Original: 5. M ODEL I NTERPRETABILITY
Revised: 5. M ODEL I NTERPRETABILITY

Original: Examining the model’s persistent vectors s m , we can try to understand if there is a match between real world protein markers and the representations of s m .
Revised: Examining the model’s persistent vectors s m , we can try to understand if there is a match between real world protein markers and the representations of s m .

Original: For example, the vector s pk for pankeratin attends to tumor cells and s CD8 attends to CD8 T cells at pixel pairwise levels.
Revised: For example, the vector s pk for pankeratin attends to tumor cells and s CD8 attends to CD8 T cells at pixel pairwise levels.

Original: It follows that in a simple experiment where corresponding s CD8 ↔ s pk vectors are exchanged internally in the attention module (Eq. 10, Figure 4 Step 3, outer product)
Revised: It follows that in a simple experiment where corresponding s CD8 ↔ s pk vectors are exchanged internally in the attention module (Eq. 10, Figure 4 Step 3, outer product)

Original: we may observe a lower expression for tumor cells in channel m pk and a lower expression for CD8 T cells in channel m CD8 since tumor cells do not express CD8 and CD8 T cells do not express pan-keratin.
Revised: we may observe a lower expression for tumor cells in channel m pk and a lower expression for CD8 T cells in channel m CD8 since tumor cells do not express CD8 and CD8 T cells do not express pan-keratin.

Original: As a control, we also switch surface membrane markers HLA Class 1 and dsDNA markers as they are present in all cells and have very similar average expression values ( s HLAc1 ↔ s dsDNA ).
Revised: As a control, we also switch surface membrane markers HLA Class 1 and dsDNA markers as they are present in all cells and have very similar average expression values ( s HLAc1 ↔ s dsDNA ).

Original: Accordingly, for our control, we expect to see negligble changes.
Revised: Accordingly, for our control, we expect to see negligble changes.

Original: We deﬁne the expression ratio as after before − 1 .
Revised: We deﬁne the expression ratio as after before − 1 .

Original: [Table]
Revised: [Table]

Original: In Table 5, we can see a larger magnitude decrease of the expression ratios in the s CD8 ↔ s pk experiment and a minute difference in the s HLAc1 ↔ s dsDNA .
Revised: In Table 5, we can see a larger magnitude decrease of the expression ratios in the s CD8 ↔ s pk experiment and a minute difference in the s HLAc1 ↔ s dsDNA .

Original: Further visualizations (Figure 13) and discussion (model generativeness, Figure 14) are given in A.6.
Revised: Further visualizations (Figure 13) and discussion (model generativeness, Figure 14) are given in A.6.

Original:  C ONCLUSION AND D ISCUSSION
Revised:  C ONCLUSION AND D ISCUSSION

Original: We introduced the idea of applying image synthesis to understanding and exploring cell-cell interactions in various and different contexts.
Revised: We introduced the idea of applying image synthesis to understanding and exploring cell-cell interactions in various and different contexts.

Original: To do so we use a protein attention based GAN, CCIGAN, which can provide accurate characterizations of cellular protein localization phenomena from conditioned counterfactual cell-cell scenarios.
Revised: To do so we use a protein attention based GAN, CCIGAN, which can provide accurate characterizations of cellular protein localization phenomena from conditioned counterfactual cell-cell scenarios.

Original: Additionally, the architecture of the attention module we propose can be generalized to other multiplexed datasets that require real world priors.
Revised: Additionally, the architecture of the attention module we propose can be generalized to other multiplexed datasets that require real world priors.

Original: Furthermore, CCIGAN outperforms a variety of current methods in biological modeling.
Revised: Furthermore, CCIGAN outperforms a variety of current methods in biological modeling.

Original: We demonstrate this through biological consistency where CCIGAN recapitulates, discovers, and quantiﬁes meaningful cellular interactions through 3 different experiments in a tumor environment unrecognized by other models.
Revised: We demonstrate this through biological consistency where CCIGAN recapitulates, discovers, and quantiﬁes meaningful cellular interactions through 3 different experiments in a tumor environment unrecognized by other models.

Original: This highlights the potential for CCIGAN to identify cellular protein interactions which account for variation in patient responses to cancer therapy, providing a framework for biological hypotheses which explain clinical outcomes on a cellular level
Revised: This highlights the potential for CCIGAN to identify cellular protein interactions which account for variation in patient responses to cancer therapy, providing a framework for biological hypotheses which explain clinical outcomes on a cellular level

Original: A A PPENDIX A. A RCHITECTURE D ETAILS
Revised: A A PPENDIX A. A RCHITECTURE D ETAILS

Original: The detailed architecture of our generator is shown on Table 6.
Revised: The detailed architecture of our generator is shown on Table 6.

Original: Layers
Revised: Layers

Original: Output Size
Revised: Output Size

Original: Generator
Revised: Generator

Original: Linear (128 , 4 , 4)
Revised: Linear (128 , 4 , 4)

Original: Linear 128 ×
Revised: Linear 128 ×

Original: Upsampling (128 , 8 , 8)
Revised: Upsampling (128 , 8 , 8)

Original: Upsampling 2 ×
Revised: Upsampling 2 ×

Original: SPADE ResBlk- (128 , 8 , 8)
Revised: SPADE ResBlk- (128 , 8 , 8)

Original: SPADE 128 , Leaky ReLU
Revised: SPADE 128 , Leaky ReLU

Original: Convolution 3 ×
Revised: Convolution 3 ×

Original: SPADE 128 , Leaky ReLU
Revised: SPADE 128 , Leaky ReLU

Original: Convolution 3 ×
Revised: Convolution 3 ×

Original: Upsampling (128 , 16 , 16)
Revised: Upsampling (128 , 16 , 16)

Original: Upsampling 2 ×
Revised: Upsampling 2 ×

Original: SPADE ResBlk- (128 , 16 , 16)
Revised: SPADE ResBlk- (128 , 16 , 16)

Original: SPADE 128 , Leaky ReLU
Revised: SPADE 128 , Leaky ReLU

Original: Convolution 3 ×
Revised: Convolution 3 ×

Original: SPADE 128 , Leaky ReLU
Revised: SPADE 128 , Leaky ReLU

Original: Convolution 3 ×
Revised: Convolution 3 ×

Original: Upsampling (128 , 32 , 32)
Revised: Upsampling (128 , 32 , 32)

Original: Upsampling 2 ×
Revised: Upsampling 2 ×

Original: SPADE ResBlk- (64 , 32 , 32)
Revised: SPADE ResBlk- (64 , 32 , 32)

Original: SPADE 128 , Leaky ReLU
Revised: SPADE 128 , Leaky ReLU

Original: Convolution 3 ×
Revised: Convolution 3 ×

Original: SPADE 64 , Leaky ReLU
Revised: SPADE 64 , Leaky ReLU

Original: Convolution 3 ×
Revised: Convolution 3 ×

Original: SPADE 64 , Leaky ReLU
Revised: SPADE 64 , Leaky ReLU

Original: Shortcut Convolution 3 ×
Revised: Shortcut Convolution 3 ×

Original: Upsampling (64 , 64 , 64)
Revised: Upsampling (64 , 64 , 64)

Original: Upsampling 2 ×
Revised: Upsampling 2 ×

Original: SPADE ResBlk- (64 , 64 , 64)
Revised: SPADE ResBlk- (64 , 64 , 64)

Original: SPADE 64 , Leaky ReLU
Revised: SPADE 64 , Leaky ReLU

Original: Convolution 3 ×
Revised: Convolution 3 ×

Original: SPADE 64 , Leaky ReLU
Revised: SPADE 64 , Leaky ReLU

Original: Convolution 3 ×
Revised: Convolution 3 ×

Original: Convolution (24 , 64 , 64)
Revised: Convolution (24 , 64 , 64)

Original: Leaky ReLU, Convolution 5 ×
Revised: Leaky ReLU, Convolution 5 ×

Original: Convolution (24 ∗ 8 , 64 , 64)
Revised: Convolution (24 ∗ 8 , 64 , 64)

Original: Leaky ReLU, Convolution 5 ×
Revised: Leaky ReLU, Convolution 5 ×

Original: Group SPADE (24 ∗ 8 , 64 , 64) [SPADE 8 ] *
Revised: Group SPADE (24 ∗ 8 , 64 , 64) [SPADE 8 ] *

Original: Modulation (24 ∗ 64 , 64 , 64) [Outer Product 8 ⊗ 8 ] *(24 , 64 , 64)
Revised: Modulation (24 ∗ 64 , 64 , 64) [Outer Product 8 ⊗ 8 ] *(24 , 64 , 64)

Original: Convolution 1 × 1 , Sigmoid
Revised: Convolution 1 × 1 , Sigmoid

Original: Output (24 , 64 , 64)
Revised: Output (24 , 64 , 64)

Original: Sum residual, Sigmoid where ResBlk is the residual block with skip connection used in ResNet (He et al., 2016), and
Revised: Sum residual, Sigmoid where ResBlk is the residual block with skip connection used in ResNet (He et al., 2016), and

Original: SPADE is the spatially-adaptive normalization layer.
Revised: SPADE is the spatially-adaptive normalization layer.

Original: The detailed architecture of our discriminator is shown on Table 7.
Revised: The detailed architecture of our discriminator is shown on Table 7.

Original: [Table]
Revised: [Table]

Original: Convolution
Revised: Convolution

Original: [Table]
Revised: [Table]

Original: Instance
Revised: Instance

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Convolution
Revised: Convolution

Original: [Table]
Revised: [Table]

Original: ×
Revised: ×

Original: [Table]
Revised: [Table]

Original: stride
Revised: stride

Original: [Table]
Revised: [Table]

Original: Instance
Revised: Instance

Original: [Table]
Revised: [Table]

Original: For all baseline models, we use the architecture based on the their original implementation.
Revised: For all baseline models, we use the architecture based on the their original implementation.

Original: Due to the size of the cell patch is (64 , 64) , we reduce the size of hidden layers to ﬁt our dataset.
Revised: Due to the size of the cell patch is (64 , 64) , we reduce the size of hidden layers to ﬁt our dataset.

Original: For fair comparison, we use the same reduction of hidden layers and the same discriminator architecture for
Revised: For fair comparison, we use the same reduction of hidden layers and the same discriminator architecture for

Original: SPADE, pix2pixHD, and CCIGAN.
Revised: SPADE, pix2pixHD, and CCIGAN.

Original: A. M ODEL T RAINING
Revised: A. M ODEL T RAINING

Original: G is the generator and D is the discriminator for CCIGAN.
Revised: G is the generator and D is the discriminator for CCIGAN.

Original: Given segmentation map S , ground truth
Revised: Given segmentation map S , ground truth

Original: Y and noise δ , the generated image is X =
Revised: Y and noise δ , the generated image is X =

Original: G ( S , δ ) .
Revised: G ( S , δ ) .

Original: The input of the discriminator is the cell image conditioned on the segmentation map S .
Revised: The input of the discriminator is the cell image conditioned on the segmentation map S .

Original: We use LSGAN loss (Mao et al., 2017) in CCIGAN, which is deﬁned as follows:
Revised: We use LSGAN loss (Mao et al., 2017) in CCIGAN, which is deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: In addition to GAN loss, we also use feature matching loss (Wang et al., 2018a) during training expressed as:
Revised: In addition to GAN loss, we also use feature matching loss (Wang et al., 2018a) during training expressed as:

Original: [Equation]
Revised: [Equation]

Original: where D j is j -th layer feature map of the discriminator for j ∈ { 1 , ..., J } , and N j is the number of elements in j -th layer.
Revised: where D j is j -th layer feature map of the discriminator for j ∈ { 1 , ..., J } , and N j is the number of elements in j -th layer.

Original: Consequently, the objective function for training is given as follows:
Revised: Consequently, the objective function for training is given as follows:

Original: [Equation]
Revised: [Equation]

Original: (15) where λ = 10 .
Revised: (15) where λ = 10 .

Original: Due to the size of cell patch is (64 , 64) , we do not use multi-scale discriminators and perceptual loss in CCIGAN and other baseline models e.g. SPADE and pix2pixHD.
Revised: Due to the size of cell patch is (64 , 64) , we do not use multi-scale discriminators and perceptual loss in CCIGAN and other baseline models e.g. SPADE and pix2pixHD.

Original: In training, we use ADAM as the optimizer.
Revised: In training, we use ADAM as the optimizer.

Original: The generator learning rate is lr G = 0 .
Revised: The generator learning rate is lr G = 0 .

Original: 0004 and
Revised: 0004 and

Original: the discriminator learning rate is lr D = 0 . 0001 .
Revised: the discriminator learning rate is lr D = 0 . 0001 .

Original: We train CCIGAN 120 epochs with a training set of 5648 cell patches.
Revised: We train CCIGAN 120 epochs with a training set of 5648 cell patches.

Original: We train other baseline models for 120 epochs or until they converge (max 150).
Revised: We train other baseline models for 120 epochs or until they converge (max 150).

Original: The full details of training of CCIGAN and baselines are shown as Table 8.
Revised: The full details of training of CCIGAN and baselines are shown as Table 8.

Original: The hyperparameters of each model are ﬁne-tuned to get better performance.
Revised: The hyperparameters of each model are ﬁne-tuned to get better performance.

Original: The training time was roughly equal for all models.
Revised: The training time was roughly equal for all models.

Original: In particular, CCIGAN was around 1.2 times slower than the SPADE baseline on a single
Revised: In particular, CCIGAN was around 1.2 times slower than the SPADE baseline on a single

Original: Tesla V100 GPU.
Revised: Tesla V100 GPU.

Original: [Table]
Revised: [Table]

Original: A. E VALUATION M ETRICS A.3. R ECONSTRUCTION M ETRICS
Revised: A. E VALUATION M ETRICS A.3. R ECONSTRUCTION M ETRICS

Original: Given the generated image set X = { X i } Ni =1 and the ground truth set Y =
Revised: Given the generated image set X = { X i } Ni =1 and the ground truth set Y =

Original: { Y i } Ni =1 with X i , Y i ∈
Revised: { Y i } Ni =1 with X i , Y i ∈

Original: R ( M,H,W ) , the L 1 /MSE score is deﬁned as follows,
Revised: R ( M,H,W ) , the L 1 /MSE score is deﬁned as follows,

Original: [Equation]
Revised: [Equation]

Original: (16) where (cid:107)·(cid:107) ∗ can be either L 1 or L 2 norm, (cid:12) is the element-wise product, X i,m and Y i,m are the m -th channel of the i -th cell patch, U i ∈ { 0 , 1 } ( H,W ) is the mask matrix which masks all the cells in i -th patch.
Revised: (16) where (cid:107)·(cid:107) ∗ can be either L 1 or L 2 norm, (cid:12) is the element-wise product, X i,m and Y i,m are the m -th channel of the i -th cell patch, U i ∈ { 0 , 1 } ( H,W ) is the mask matrix which masks all the cells in i -th patch.

Original: For any matrix A , sort ( A ) is the sort function that sorts all entries of A .
Revised: For any matrix A , sort ( A ) is the sort function that sorts all entries of A .

Original: The sorting function ensures our metrics are position independent and only measures the intensity of the generated image and ground truth.
Revised: The sorting function ensures our metrics are position independent and only measures the intensity of the generated image and ground truth.

Original: The score function L ( X , Y ) only computes the loss of sorted expression inside of the cells.
Revised: The score function L ( X , Y ) only computes the loss of sorted expression inside of the cells.

Original: Then we add penalization for expression outside of cells.
Revised: Then we add penalization for expression outside of cells.

Original: The adjusted L 1 /MSE score is introduced as follows,
Revised: The adjusted L 1 /MSE score is introduced as follows,

Original: [Equation]
Revised: [Equation]

Original: where 1 d is the matrix with all entries equal to 1.
Revised: where 1 d is the matrix with all entries equal to 1.

Original: The (adjusted) L 1 /MSE scores of CCIGAN and baseline models are shown on Table 1.
Revised: The (adjusted) L 1 /MSE scores of CCIGAN and baseline models are shown on Table 1.

Original: A smaller score means a better result.
Revised: A smaller score means a better result.

Original: For any two images X , Y ∈ [0 , 1] ( H,W ) , the SSIM and MI are deﬁned as:
Revised: For any two images X , Y ∈ [0 , 1] ( H,W ) , the SSIM and MI are deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where H ( · ) is entropy, µ X and σ X are the mean and standard deviation of X ,
Revised: where H ( · ) is entropy, µ X and σ X are the mean and standard deviation of X ,

Original: c 1 , c 2 are constants.
Revised: c 1 , c 2 are constants.

Original: Then the SSIM between X , Y is
Revised: Then the SSIM between X , Y is

Original: [Equation]
Revised: [Equation]

Original: In cell based MI, test patches are processed at a cell-cell basis where their mutual information is computed with the corresponding cell in the ground truth.
Revised: In cell based MI, test patches are processed at a cell-cell basis where their mutual information is computed with the corresponding cell in the ground truth.

Original: For the generated image X i of the i -th patch, we assume there are T i cells in the i -th patch.
Revised: For the generated image X i of the i -th patch, we assume there are T i cells in the i -th patch.

Original: Then for each cell t , the pixels of m -th channel of the t -th cell in the i -th patch can be expressed as a vector x ti,m .
Revised: Then for each cell t , the pixels of m -th channel of the t -th cell in the i -th patch can be expressed as a vector x ti,m .

Original: Hence, the cell based MI is formulated as:
Revised: Hence, the cell based MI is formulated as:

Original: [Equation]
Revised: [Equation]

Original: We report I ( X ; Y ) on Table 1.
Revised: We report I ( X ; Y ) on Table 1.

Original: The SSIM measures the similarity between the generated image and the ground truth.
Revised: The SSIM measures the similarity between the generated image and the ground truth.

Original: For SSIM, we use HLA Class 1 and dsDNA due to the their expressions in all cells.
Revised: For SSIM, we use HLA Class 1 and dsDNA due to the their expressions in all cells.

Original: If all channels were considered, the SSIM would be uninformative due to the majority of the channels being blank or sparse.
Revised: If all channels were considered, the SSIM would be uninformative due to the majority of the channels being blank or sparse.

Original: The MI measures the information shared between generated image and ground truth at a cell by cell basis where we consider all channels.
Revised: The MI measures the information shared between generated image and ground truth at a cell by cell basis where we consider all channels.

Original: Consider the example where a model generates no expression in marker m but the real data has expression in m , the MI would be 0 and vice versa.
Revised: Consider the example where a model generates no expression in marker m but the real data has expression in m , the MI would be 0 and vice versa.

Original: Higher SSIM and MI values mean better results.
Revised: Higher SSIM and MI values mean better results.

Original: Table 1 demonstrates that
Revised: Table 1 demonstrates that

Original: CCIGAN outperforms or matches all other baselines on all reconstruction metrics.
Revised: CCIGAN outperforms or matches all other baselines on all reconstruction metrics.

Original: A.3. W EIGHTED C ENTROID
Revised: A.3. W EIGHTED C ENTROID

Original: For a generated cell image, its centroid is the mean position of all the points in the cell.
Revised: For a generated cell image, its centroid is the mean position of all the points in the cell.

Original: We use pixel values as weights in computing the weighted centroid, now referred to as center of mass (COM).
Revised: We use pixel values as weights in computing the weighted centroid, now referred to as center of mass (COM).

Original: Given a cell image X ∈ R ( H,W ) , with indices of the segmented cell V ⊆ { 1 , . . .
Revised: Given a cell image X ∈ R ( H,W ) , with indices of the segmented cell V ⊆ { 1 , . . .

Original: , H }×{ 1 , . . .
Revised: , H }×{ 1 , . . .

Original: , W } , the COM ¯ p
Revised: , W } , the COM ¯ p

Original: = (¯ x, ¯ y ) is deﬁned as ¯ x = (cid:80) ( x,y ) ∈
Revised: = (¯ x, ¯ y ) is deﬁned as ¯ x = (cid:80) ( x,y ) ∈

Original: V x X x,y (cid:80) ( x,y ) ∈
Revised: V x X x,y (cid:80) ( x,y ) ∈

Original: V X x,y and ¯ y = (cid:80) ( x,y ) ∈
Revised: V X x,y and ¯ y = (cid:80) ( x,y ) ∈

Original: V y X x
Revised: V y X x

Original: ,y (cid:80) ( x,y ) ∈
Revised: ,y (cid:80) ( x,y ) ∈

Original: V X x,y .
Revised: V X x,y .

Original: In the PD-1/PD-L1 experiment, we compute the COM of the CD8 T cell (cell of interest) weighted by PD-1 expression, given as ¯ p CD8 , and the COM of all tumor cells weighted by PD-L1 expression, given as ¯ p Tumor .
Revised: In the PD-1/PD-L1 experiment, we compute the COM of the CD8 T cell (cell of interest) weighted by PD-1 expression, given as ¯ p CD8 , and the COM of all tumor cells weighted by PD-L1 expression, given as ¯ p Tumor .

Original: Since T cells located within the tumor microenvironment often have upregulated expression of PD-L1, we assume that ¯ p CD8 should have the same COM as all of its surrounding tumor cells ¯ p Tumor .
Revised: Since T cells located within the tumor microenvironment often have upregulated expression of PD-L1, we assume that ¯ p CD8 should have the same COM as all of its surrounding tumor cells ¯ p Tumor .

Original: The center of mass score is deﬁned below as the relative distance between ¯ p CD and ¯ p Tumor , where N is deﬁned as the number of patches:
Revised: The center of mass score is deﬁned below as the relative distance between ¯ p CD and ¯ p Tumor , where N is deﬁned as the number of patches:

Original: [Equation]
Revised: [Equation]

Original: The projection function Proj ( · ) is used to project ¯ p Tumor onto the CD8 T cell to ensure the expected
Revised: The projection function Proj ( · ) is used to project ¯ p Tumor onto the CD8 T cell to ensure the expected

Original: COM of the tumor cells is inside of the CD8 T cell.
Revised: COM of the tumor cells is inside of the CD8 T cell.

Original: As a reference we choose a random position ¯ p Random in the CD8 T cell (PD-1) which replaces ¯ p CD8 in Eq. 22 and compute the random COM score to show the effectiveness of the result.
Revised: As a reference we choose a random position ¯ p Random in the CD8 T cell (PD-1) which replaces ¯ p CD8 in Eq. 22 and compute the random COM score to show the effectiveness of the result.

Original: An example illustration is given in Figure 6 A.4.
Revised: An example illustration is given in Figure 6 A.4.

Original: A.3. E ARTH M OVER ’ S D ISTANCE
Revised: A.3. E ARTH M OVER ’ S D ISTANCE

Original: For each segmentation map i , we iteratively add its T i tumor cells around one CD8 T cell.
Revised: For each segmentation map i , we iteratively add its T i tumor cells around one CD8 T cell.

Original: The COM is deﬁned for the t -th tumor cell as ¯ p Tumor t,i for t ∈ { 1 , ..., T i } .
Revised: The COM is deﬁned for the t -th tumor cell as ¯ p Tumor t,i for t ∈ { 1 , ..., T i } .

Original: We omit the subscript i when it is clear from context.
Revised: We omit the subscript i when it is clear from context.

Original: The subsequent instances of the PD-1 COMs in the CD8 T cell by adding the t -th tumor are given by ¯ p CD8 t .
Revised: The subsequent instances of the PD-1 COMs in the CD8 T cell by adding the t -th tumor are given by ¯ p CD8 t .

Original: Initially when there are no tumor cells, we deﬁne ¯ p CD8as the centroid of the CD8 T cell.
Revised: Initially when there are no tumor cells, we deﬁne ¯ p CD8as the centroid of the CD8 T cell.

Original: Based on the above setting, we deﬁne a vector v t which points from the centroid of the CD8 T cell ¯ p CD8 0 , to the COM of the t -th tumor cell ¯ p Tumor t .
Revised: Based on the above setting, we deﬁne a vector v t which points from the centroid of the CD8 T cell ¯ p CD8 0 , to the COM of the t -th tumor cell ¯ p Tumor t .

Original: We deﬁne vector u t which points from the previous COM ¯ p CD8 t − 1 to the current COM ¯ p CD8 t of the CD8 T cell.
Revised: We deﬁne vector u t which points from the previous COM ¯ p CD8 t − 1 to the current COM ¯ p CD8 t of the CD8 T cell.

Original: We deﬁne θ t as the angle between v t , u t .
Revised: We deﬁne θ t as the angle between v t , u t .

Original: If cos θ t > 0 , that is to say if the cosine similarity is positive, the COM of a CD8 T cell ¯ p CD8 t , moves correctly towards the COM of the added tumor cell ¯ p Tumor t .
Revised: If cos θ t > 0 , that is to say if the cosine similarity is positive, the COM of a CD8 T cell ¯ p CD8 t , moves correctly towards the COM of the added tumor cell ¯ p Tumor t .

Original: An illustration of the points and vectors is given in Figure 8.
Revised: An illustration of the points and vectors is given in Figure 8.

Original: Formally:
Revised: Formally:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: After obtaining the directional information, we use Earth Mover’s Distance (EMD) (Rubner et al., 2000) to evaluate the changes in PD-1 expression of the CD8 T cell.
Revised: After obtaining the directional information, we use Earth Mover’s Distance (EMD) (Rubner et al., 2000) to evaluate the changes in PD-1 expression of the CD8 T cell.

Original: The EMD, which measures the dissimilarity of two distributions, is used in this context to measure the protein localization shifts in PD-1 before and after adding a tumor cell.
Revised: The EMD, which measures the dissimilarity of two distributions, is used in this context to measure the protein localization shifts in PD-1 before and after adding a tumor cell.

Original: We consider each cell X in polar coordinates ( r, θ ) with respect to its centroid, integrate its expression along the radius coordinates, and evaluate the resulting histogram hist ( X ) along the angle coordinate.
Revised: We consider each cell X in polar coordinates ( r, θ ) with respect to its centroid, integrate its expression along the radius coordinates, and evaluate the resulting histogram hist ( X ) along the angle coordinate.

Original: This allows for the deﬁnition of distance for moving one histogram to another, i.e. em ( X ti , X t − 1 i ) =
Revised: This allows for the deﬁnition of distance for moving one histogram to another, i.e. em ( X ti , X t − 1 i ) =

Original: d EM ( hist ( X ti ) , hist ( X t − 1 i )) , for the generated PD-1 expression of the CD8 T cell X ti when adding the t -th tumor cell.
Revised: d EM ( hist ( X ti ) , hist ( X t − 1 i )) , for the generated PD-1 expression of the CD8 T cell X ti when adding the t -th tumor cell.

Original: The ﬁnal EMD score is deﬁned as:
Revised: The ﬁnal EMD score is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where the indicator function 1 ( · ) = 1 if and only if (cid:107) X ti (cid:107) > (cid:107) X t − 1 i (cid:107) , otherwise 1 ( · ) = 0 .
Revised: where the indicator function 1 ( · ) = 1 if and only if (cid:107) X ti (cid:107) > (cid:107) X t − 1 i (cid:107) , otherwise 1 ( · ) = 0 .

Original: This ensures that the biological constraint of PD-1 expression increasing as a response to added tumor cells is met.
Revised: This ensures that the biological constraint of PD-1 expression increasing as a response to added tumor cells is met.

Original: Recall, if cos θ t > 0 , ¯ p CD8 t has moved in the direction of ¯ p Tumor t , implying the shift in
Revised: Recall, if cos θ t > 0 , ¯ p CD8 t has moved in the direction of ¯ p Tumor t , implying the shift in

Original: PD-1 expression is correct, and in turn increases the EMD.
Revised: PD-1 expression is correct, and in turn increases the EMD.

Original: By contrast, the EMD score decreases when ¯ p CD8 t moves in the opposite direction.
Revised: By contrast, the EMD score decreases when ¯ p CD8 t moves in the opposite direction.

Original: Example ﬁgures and illustrations showing this process are given in Figure 7 A.4.
Revised: Example ﬁgures and illustrations showing this process are given in Figure 7 A.4.

Original: Using the EMD we deﬁne a randomized search algorithm for discovering other cell-cell interactions; their results and discussion are given in A.7.
Revised: Using the EMD we deﬁne a randomized search algorithm for discovering other cell-cell interactions; their results and discussion are given in A.7.

Original: Based on the deﬁnition of EMD score, the positive EMD score is deﬁned as:
Revised: Based on the deﬁnition of EMD score, the positive EMD score is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: The positive EMD score only evaluates the change in PD-1 expression when the COM of a CD8 T cell moves towards the COM of the added tumor cell.
Revised: The positive EMD score only evaluates the change in PD-1 expression when the COM of a CD8 T cell moves towards the COM of the added tumor cell.

Original: The projected EMD score is deﬁned as:
Revised: The projected EMD score is deﬁned as:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: The projected EMD score is the EMD score weighted by (cid:107) u t,i (cid:107) , i.e. the shift from the previous
Revised: The projected EMD score is the EMD score weighted by (cid:107) u t,i (cid:107) , i.e. the shift from the previous

Original: COM to the current COM of the CD8 T cell.
Revised: COM to the current COM of the CD8 T cell.

Original: A. E VALUATION V ISUALIZATIONS
Revised: A. E VALUATION V ISUALIZATIONS

Original: Here we provide some example visualizations and illustrations center of mass nomenclature, mass movement of PD-1 as a function of neighboring PD-L1, and process of computing EM distance.
Revised: Here we provide some example visualizations and illustrations center of mass nomenclature, mass movement of PD-1 as a function of neighboring PD-L1, and process of computing EM distance.

Original: We can observe in Figure 7 that the mass inside of the T cell in the PD-1 channel shifts as a response to surrounding tumor cell expressions of PD-L1.
Revised: We can observe in Figure 7 that the mass inside of the T cell in the PD-1 channel shifts as a response to surrounding tumor cell expressions of PD-L1.

Original: The surrounding tumor PD-L1 expressions ¯ p Tumor t are shown in the third row on a cell by cell basis for t ∈ { 1 , ..., T i = 4 } .
Revised: The surrounding tumor PD-L1 expressions ¯ p Tumor t are shown in the third row on a cell by cell basis for t ∈ { 1 , ..., T i = 4 } .

Original: Note that the 3rd column in PD-L1 has sparse expression.
Revised: Note that the 3rd column in PD-L1 has sparse expression.

Original: Finally the last row shows the PD-1 and PD-L1 channels superimposed into one channel.
Revised: Finally the last row shows the PD-1 and PD-L1 channels superimposed into one channel.

Original: We give an example in Figure 8 to illustrate the vectors v t and u t after adding t -th tumor cell for t = 1 , 2 in computing EMD score, where v t and u t deﬁned in Eq. 23.
Revised: We give an example in Figure 8 to illustrate the vectors v t and u t after adding t -th tumor cell for t = 1 , 2 in computing EMD score, where v t and u t deﬁned in Eq. 23.

Original: A. R ESULTS G RAPHS
Revised: A. R ESULTS G RAPHS

Original: The pan-keratin/CD8 experiment is similar to Figure 7’s orientation except the center cell (cell of interest) is a tumor cell (red) and the adjacent neighboring cells are CD8 T cells (orange).
Revised: The pan-keratin/CD8 experiment is similar to Figure 7’s orientation except the center cell (cell of interest) is a tumor cell (red) and the adjacent neighboring cells are CD8 T cells (orange).

Original: Note the shift in expression of the PD-1 as a response to the added tumor’s PD-L1 expression.
Revised: Note the shift in expression of the PD-1 as a response to the added tumor’s PD-L1 expression.

Original: CCIGAN predicted a decrease in tumor cell pan-keratin expression with respect to increasing CD
Revised: CCIGAN predicted a decrease in tumor cell pan-keratin expression with respect to increasing CD

Original: T cell area/number (Figure 9).
Revised: T cell area/number (Figure 9).

Original: This is juxtaposed to the tumor cell control where there is no change in the pan-keratin level as the number of neighboring tumor cells is increased.
Revised: This is juxtaposed to the tumor cell control where there is no change in the pan-keratin level as the number of neighboring tumor cells is increased.

Original: SPADE does not predict a decrease in tumor cell pan-keratin expression with respect to increasing
Revised: SPADE does not predict a decrease in tumor cell pan-keratin expression with respect to increasing

Original: CD8 T cell area/number and shows no difference in pan-keratin expression trends between the T cell and control groups (Figure 10).
Revised: CD8 T cell area/number and shows no difference in pan-keratin expression trends between the T cell and control groups (Figure 10).

Original: pix2pixHD erroneously predicts an increase in tumor cell pan-keratin expression with respect to increasing CD8 T cell area/number and shows no difference in pan-keratin expression trends between the T cell and control groups (Figure 11).
Revised: pix2pixHD erroneously predicts an increase in tumor cell pan-keratin expression with respect to increasing CD8 T cell area/number and shows no difference in pan-keratin expression trends between the T cell and control groups (Figure 11).

Original: CycleGAN fails to predict a decrease in tumor cell pan-keratin expression with respect to increasing
Revised: CycleGAN fails to predict a decrease in tumor cell pan-keratin expression with respect to increasing

Original: CD8 T cell area/number and shows no difference in pan-keratin expression trends between the T cell and control groups (Figure 12).
Revised: CD8 T cell area/number and shows no difference in pan-keratin expression trends between the T cell and control groups (Figure 12).

Original: A. M ODEL I NTERPRETABILITY AND G ENERATIVENESS
Revised: A. M ODEL I NTERPRETABILITY AND G ENERATIVENESS

Original: Figure 13 shows the persistent vectors s i for all proteins.
Revised: Figure 13 shows the persistent vectors s i for all proteins.

Original: Note the similarity between CD3 and
Revised: Note the similarity between CD3 and

Original: CD8 T cell protein markers and the similarity between dsDNA and HLA Class 1 surface membrane proteins (expressed in all cells).
Revised: CD8 T cell protein markers and the similarity between dsDNA and HLA Class 1 surface membrane proteins (expressed in all cells).

Original: It is also important to make the distinction that sparse markers (while different) are similar in state.
Revised: It is also important to make the distinction that sparse markers (while different) are similar in state.

Original: This is due to the lack of training data for rare cell types, making it difﬁcult for the model to reason on such a small sample size.
Revised: This is due to the lack of training data for rare cell types, making it difﬁcult for the model to reason on such a small sample size.

Original: Figure 14 shows the generativeness of CCIGAN through an uncertainty map over 100 instances (random noise).
Revised: Figure 14 shows the generativeness of CCIGAN through an uncertainty map over 100 instances (random noise).

Original: An uncertainty map shows the differences per pixel ( x, y ) location.
Revised: An uncertainty map shows the differences per pixel ( x, y ) location.

Original: The higher intensities indicate a higher probability of changing at the speciﬁed ( x, y ) location.
Revised: The higher intensities indicate a higher probability of changing at the speciﬁed ( x, y ) location.

Original: A. S EARCH A LGORITHM
Revised: A. S EARCH A LGORITHM

Original: Here we provide a randomized search algorithm to try to discover further cell-cell interactions in other channels.
Revised: Here we provide a randomized search algorithm to try to discover further cell-cell interactions in other channels.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: The experimental ﬁndings in Indices 4, 5 and 6 support those reported in 6.2.2, demonstrating that immune cell presence adjacent to tumor cells causes a decrease in the tumor’s pan-keratin, regardless of immune cell identity.
Revised: The experimental ﬁndings in Indices 4, 5 and 6 support those reported in 6.2.2, demonstrating that immune cell presence adjacent to tumor cells causes a decrease in the tumor’s pan-keratin, regardless of immune cell identity.

Original: In addition to conﬁrming the results of 6.2.1, ﬁndings in indices 1 andalso indicate that tumor cells increase the expression of the T cell co-receptor, although this is of unclear functional signiﬁcance.
Revised: In addition to conﬁrming the results of 6.2.1, ﬁndings in indices 1 andalso indicate that tumor cells increase the expression of the T cell co-receptor, although this is of unclear functional signiﬁcance.

Original: Index 2 suggests T cell clustering reduces the expression of the immune suppressive PD-1 marker.
Revised: Index 2 suggests T cell clustering reduces the expression of the immune suppressive PD-1 marker.

Original: Lastly, Index 7 demonstrates an increase in macrophage expressed vimentin when macrophages are placed adjacent to tumor cells.
Revised: Lastly, Index 7 demonstrates an increase in macrophage expressed vimentin when macrophages are placed adjacent to tumor cells.

Original: Since vimentin is secreted as a pro-inﬂammatory marker in macrophages, this suggests an early macrophage inﬂammatory response to its tumor neighbor.
Revised: Since vimentin is secreted as a pro-inﬂammatory marker in macrophages, this suggests an early macrophage inﬂammatory response to its tumor neighbor.

Original: A. M ARKERS
Revised: A. M ARKERS

Original: The markers we used (total 24) in our experiments are: Pan-Keratin, EGFR, Beta catenin, dsDNA,
Revised: The markers we used (total 24) in our experiments are: Pan-Keratin, EGFR, Beta catenin, dsDNA,

Original: Ki67, CD3, CD8, CD4, FoxP3, MPO, HLA-DR, HLA-Class-1, CD209, CD11b, CD11c, CD68,
Revised: Ki67, CD3, CD8, CD4, FoxP3, MPO, HLA-DR, HLA-Class-1, CD209, CD11b, CD11c, CD68,

Original: CD63, Lag3, PD1, PD-L1, IDO, Vimentin, SMA, CD31.
Revised: CD63, Lag3, PD1, PD-L1, IDO, Vimentin, SMA, CD31.

Original: The markers we didn’t use (total 12) in our experiments are: CD16, B7H3, CD45, CD45RO, Keratin17, CD20, CD163, CD56, Keratin6, CSF-1R, p53, CD
Revised: The markers we didn’t use (total 12) in our experiments are: CD16, B7H3, CD45, CD45RO, Keratin17, CD20, CD163, CD56, Keratin6, CSF-1R, p53, CD

Original: Input: Cell segmentation list { S i } ni =1 , the channel index m , rotation angle ∆ θ , ﬁxed noise δ , threshold β and the generator G .
Revised: Input: Cell segmentation list { S i } ni =1 , the channel index m , rotation angle ∆ θ , ﬁxed noise δ , threshold β and the generator G .

Original: Randomly chose initial cell index i 0 ∈ { 1 , ..., n } ;
Revised: Randomly chose initial cell index i 0 ∈ { 1 , ..., n } ;

Original: Input segmentation S INPUT =
Revised: Input segmentation S INPUT =

Original: S i 0 ;
Revised: S i 0 ;

Original: Mask for the initial cell U =
Revised: Mask for the initial cell U =

Original: (cid:80) nj =1 S
Revised: (cid:80) nj =1 S

Original: i 

Revised: i

Original: [ j, : , :] ;
Revised: [ j, : , :] ;

Original: Generated image X 

Revised: Generated image X

Original: =
Revised: =

Original: G ( δ, S INPUT ) ;
Revised: G ( δ, S INPUT ) ;

Original: Speciﬁed channel M 

Revised: Speciﬁed channel M

Original: = U ∗ X 

Revised: = U ∗ X

Original: [ m ] ;
Revised: [ m ] ;

Original: Show S INPUT and M 0 ; for k = 1 to n − 1 do
Revised: Show S INPUT and M 0 ; for k = 1 to n − 1 do

Original: Random index i k ∈ { 1 , ..., n } / ∪ k − 1 j =

Revised: Random index i k ∈ { 1 , ..., n } / ∪ k − 1 j =

Original: { i j } ;
Revised: { i j } ;

Original: S INPUT =
Revised: S INPUT =

Original: S INPUT + S
Revised: S INPUT + S

Original: i
Revised: i

Original: k ;
Revised: k ;

Original: X k = G ( δ, S INPUT ) ;
Revised: X k = G ( δ, S INPUT ) ;

Original: M k
Revised: M k

Original: = U ∗ X k
Revised: = U ∗ X k

Original: [ m ] ;
Revised: [ m ] ;

Original: Show S INPUT and M k ;
Revised: Show S INPUT and M k ;

Original: E k
Revised: E k

Original: =
Revised: =

Original: d EM ( M k −
Revised: d EM ( M k −

Original: , M k ) ; if E k > β (cid:80) i,j U i,j then
Revised: , M k ) ; if E k > β (cid:80) i,j U i,j then

Original: Log Signiﬁcance; if (cid:80) i,j M k − 1 ,i,j < (cid:80) i,j M k,i,j then
Revised: Log Signiﬁcance; if (cid:80) i,j M k − 1 ,i,j < (cid:80) i,j M k,i,j then

Original: Log Increase; else
Revised: Log Increase; else

Original: Log Decrease;
Revised: Log Decrease;