Package: medicc2
MEDICC2 - Whole-genome doubling-aware copy number phylogenies for cancer evolutionFor more information see the accompanying publicationWhole-genome doubling-aware copy number phylogenies for cancer evolution with MEDICC2.InstallationInstall MEDICC2 via conda (recommended), pip or from source. MEDICC2 was developed and tested on unix-built systems (Linux and MacOS). For Windows users we recommended WSL2.Note that the notebooks and examples are not included when installing from conda or pip.For all installation methods you need to make sure to have a working version of the GNU Compiler Collection (gcc,gxxas well as related packages such aslibgcc-ng) installed.Installation via conda (recommended)It is best to use a dedicated conda environment for your MEDICC2 installation withconda create -n medicc_env.After activating the environment withconda activate medicc_envyou can install MEDICC2 viaconda install -c bioconda -c conda-forge medicc2.Installation via pipAs MEDICC2 relies on OpenFST version 1.8.2 which is not packaged on PyPi you have to first install it using conda withconda install -c conda-forge openfst=1.8.2. Next you can install MEDICC2 viapip install medicc2.Installation from sourceClone the MEDICC2 repository and its submodules usinggit clone --recursive https://bitbucket.org/schwarzlab/medicc2.git. It is important to use the--recursiveflag to also download the modified OpenFST submodule.All dependencies including OpenFST (v1.8.2) should be directly installable via conda. A yaml file with a suggested MEDICC2 conda environment is provided in 'doc/medicc2.yml'. You can create a new conda environment with all requirements usingconda env create -f doc/medicc2.yml -n medicc_env.Then, inside themedicc2folder, runpip install .to install MEDICC2 to your environment.Development modeIf you want to make changes to the MEDICC2 source code, install MEDICC2 as explained above in "Installation from source" but install in editable mode (withpip install -e .). Any changes to the local files will now be reflected in your environment.UsageAfter installing MEDICC2, you can use MEDICC2 functions in python scripts (throughimport medicc) and from the command line. General usage from the command line ismedicc2 path/to/input/file path/to/output/folder. Runmedicc2 --helpfor information on optional arguments.Logging settings can be changed using themedicc/logging_conf.yamlfile with the standard python logging syntax.Command line Flagsinput_file: path to the input fileoutput_dir: path to the output folder--version: Print version information--input-type,-i: Choose the type of input: f for FASTA, t for TSV. Default: 'TSV'--input-allele-columns,-a: Name of the CN columns (comma separated) if using TSV input format. This also adjusts the number of alleles considered (min. 1, max. 2). Default: 'cn_a, cn_b'--input-chr-separator: Character used to separate chromosomes in the input data (condensed FASTA only). Default: 'X'--tree: Do not reconstruct tree, use provided tree instead (in newick format) and only perform ancestral reconstruction. Default: None--topology-only,-s: Output only tree topology, without reconstructing ancestors. Default: False--normal-name,-n: ID of the sample to be treated as the normal sample. Trees are rooted at this sample for ancestral reconstruction. If the sample ID is not found, an artificial normal sample of the same name is created with CN states = 1 for each allele. Default: 'diploid'--exclude-samples,-x: Comma separated list of sample IDs to exclude. Default: None--filter-segment-length: Removes segments that are smaller than specified length. Default: None--bootstrap-method: Bootstrap method. Has to be either 'chr-wise' or 'segment-wise'. Default: 'chr-wise'--bootstrap-nr: Number of bootstrap runs to perform. Default: None--prefix, '-p': Output prefix to be used. None uses input filename. Default: None--no-wgd: Disable whole-genome doubling events. Default: False--plot: Type of copy-number plot to save. 'bars' is recommended for <50 samples, heatmap for more samples, 'auto' will decide based on the number of samples, 'both' will plot both and 'none' will plot neither. (default: auto).--total-copy-numbers: Run for total copy number data instead of allele-specific data. Default: False-j,--n-cores: Number of cores to run on. Default: None--events: Whether to infer copy-number events. See section "Event Reconstruction" below--chromosomes-bed: BED file for chromosome regions to compare copy-number events to--regions-bed: BED file for regions of interest to compare copy-number events to-v,--verbose: Enable verbose output. Default: False-vv,--debug: Enable more verbose output Default: False--maxcn: Expert option: maximum CN at which the input is capped. Does not change FST. Default: 8--prune-weight: Expert option: Prune weight in ancestor reconstruction. Values >0 might result in more accurate ancestors but will require more time and memory. Default: 0--fst: Expert option: path to an alternative FST. Default: None--fst-chr-separator: Expert option: character used to separate chromosomes in the FST. Default: 'X'--wgd-x2: Expert option: Treat WGD as a x2 operation. Default: FalseInput filesInput files can be either in fasta or tsv format:tsv (recommended):Files should have the following columns:sample_id,chrom,start,endas well as columns for the copy numbers. MEDICC expects the copy number columns to be calledcn_aandcn_b. Using the flag--input-allele-columnsyou can set your own copy number columns. If you want to use total copy numbers, make sure to use the flag--total-copy-numbers. Important: MEDICC2 does not create total copy numbers for you. You will have to calculate total copy numbers yourself and then specify the column using the--input-allele-columnsflag.fasta:A description file should be provided to MEDICC. This file should include one line per file with the name of the chromosome and the corresponding file names. If fasta files are provided you have to use the flag--input-type fasta.MEDICC2 follows the BED convention for segment coordinates, i.e. segment start is at 0 and the segment end is non-inclusive.The folderexamples/simple_examplecontains a simple example input both in fasta and tsv format.
The folderexamples/OV03-04contains a larger example consisting of multiple fasta files. If you want to run MEDICC on this data runmedicc2 examples/OV03-04/OV03-04_descr.txt path/to/output/folder --input-type fasta.Output filesMEDICC creates the following output files:_final_tree.new,_final_tree.xml,_final_tree.png: The final phylogenetic tree in Newick and XML format as well as an image_pairwise_distances.tsv: A NxN matrix (N being the number of samples) of pairwise distances calculated with the symmetric MEDICC2 distance_final_cn_profiles.tsv: Copy-number profiles of the input as well as the newly internal nodes. Also includes additional information such as whether a gain or loss has happened_cn_profiles.pdf: Combined plot of the phylogenetic tree as well as the copy-number profiles of all samples (including the internal nodes)_branch_lengths.tsv: List of all branches and their corresponding lenghts of the final tree_summary.tsv: Contains summary information about the created tree. If the--eventsflag was set, this includes the WGD status.optional (see "Event Reconstruction" below)_copynumber_events_df.tsv: List of all copy-number events detected. The entries for WGD events have non-meaningful values for chrom, cn_child, etc. Note that the events derived are not unambiguous._events_overlap.tsv: Overlap of copy-number events with regions of interestOutput plotsApart from the file_tree.pdfwhich contains the inferred phylogeny, the main plot created by MEDICC is the copy-number plots named either_cn_profiles.pdfor_cn_profiles_heatmap.pdf.
The left part consists of the inferred phylogenetic tree including the number of events in the branches. The right part is made up of the copy-number profiles of the samples (and potentially the reconstructed ancestral nodes).There are two kinds of copy-number plots: the bars and the heatmap version. The bars version is most suitable for fewer samples (<50) as more details are visible while the heatmap version is most suitable many samples expected for example in single-cell experiments.
You can toggle the kind of plot MEDICC2 creates with the--plotflag (see above).Example bars copy-number plotExample from patient PTX011 from the Gundem et al. Nature 2015. The data can be found inexample/gundem_et_al_2015/.LegendExample heatmap copy-number plotExample from patient PTX011 from the Gundem et al. Nature 2015. The data can be found inexample/gundem_et_al_2015/.Usage examplesFor first time users we recommend to have a look atexamples/simple_exampleto get an idea of how input data should look like. Then runmedicc2 examples/simple_example/simple_example.tsv path/to/output/folderas an example of a standard MEDICC run. Finally, the notebooknotebooks/example_workflows.pyshows how the individual functions in the workflow are used.The notebooknotebooks/bootstrap_demo.pydemonstrates how to use the bootstrapping routine andnotebooks/plot_demo.pyshows how to use the main plotting functions.Event ReconstructionMEDICC2 can create a list of copy-number events in the file_copynumber_events_df.tsvwhich are also displayed in the final copy-number barplot.
These are disabled by default and are enabled using the--eventsflag.Note, that the inferred events are not unambigous but just one possible solution. In some cases there are multiple possible paths that can result in the same final copy-number state in the same number of steps. Without additional information, MEDICC2 cannot determine which possible path is the right one and thus opts for a path that creates the longest consecutive gains.
Even though the events inferred by MEDICC2 are not unambigous they are minimal (as in there are no solutions with fewer number of steps) and deterministic (as in multiple runs of MEDICC2 will always return the same events).Minimal example:111 -> 232which can be explained bygain-gain-gain+x-gain-xorgain-gain-x+x-gain-gain. MEDICC2 would select the first option.
In order to infer the events thatRegions of interestMEDICCC2 can overlap the inferred events with regions of interest such as chromosome arms or oncogenes.
This process requires the installation ofpyrangeswhich might be incompatible with newer version of python and/or numpy.
The overlap is turned off by default. You can turn on the overlapping with the ``--chromosomes-bedand--regions-bed` flag by providing bed-files with regions of interest. By default MEDICC2 uses hg38 chromosome-arms and a list of genes taken from Davoli et al. Cell 2013. This data is present as BED files in the `medicc/objects` folder. Invoke these using the flags `--chromosomes-bed "default"` and/or `--regions-bed "default"`.
Users can specify regions of interest of their own in BED format by providing the `--chromosomes-bed` or `--regions-bed` flags.Single sample WGD detectionIf you are interested in the WGD status of individual samples in your data, have a look at the notebooknotebooks/WGD_detection.ipynb. By replacing the input data with your data you can easily calculate the WGD status of any copy-number input.IssuesIf you experience problems with MEDICC2 pleasefile an issue directly on Bitbucketorcontact us directly.Known IssuesPhasingWe do recommend to phase your input data before using MEDICC2. MEDICC2's own phasing algorithm is only to be used when looking at single samples and should not be used in the case of multiple samples! Here we recommend using Refphase (Bitbucket).
If phasing is not possible for you, working on major/minor configuration works reasonably well in most cases.
For very noisy data, where accurate phasing cannot be guaranteed, you can also try to create total copy numbers and run MEDICC2 with the--total-copy-numbersflag.long runtimeMEDICC2 tries to solve an NP-hard problem by inferring a symmetric distance between samples and therefore has a higher runtime than other tools than only compute the asymmetric (and less accurate) distance between samples.In order to improve runtime, you should first run MEDICC2 with multiple cores using the--n-coresflag. Using multiple cores will roughly lead to a decrease in runtime linear w.r.t. numbers of cores used (depending on your system architecture).Next, you can remove duplicate and diploid cells. Especially in the case of 100s to 1000s single-cell samples, there are oftentimes multiple copies of cells with the same copy-number profiles as well as cells that are (almost) purely diploid. Removing those will (in most cases) not alter the results but decrease runtime.Finally, MEDICC2's runtime scales with the number of segments used. If you use copy-number bins, try to increase the bin-size for decreased runtime as well as merge neighboring bins that are equal in copy-number states across all samples. (For example a chromosome that is purely diploid in all samples should be represented as a single segment rather than multiple bins).
If you instead create a minimum consistent segmentation, be aware of individual samples with many breakpoints that will drive up your total number of breakpoints and therefore number of segments. If individual samples have an excessive amount of breakpoints, it is best to remove them before creating a minimum consistent segmentation.Noisy segmentsSmall faulty or noisy segments can have a strong effect on the distances MEDICC2 calculates between samples and therefore the resulting tree.
This is because MEDICC2 counts all segments equally in order appropriatlely take focal events into account.
If the resulting and the inferred events look strange to you, you can replot the tree and copy-number profiles using the functionplot_cn_profilessettingignore_segment_lengths=True(see the notebooknotebooks/plot_demo.pyfor usage examples) in order to investigate small segments that might not have been visible in the original plot.
If you are unsure about the copy-number profiles we recommened to filter small segments.Taxon imbalanceIf your data contains 100s to 1000s samples with a few distinct subgroups, an imbalance in the number of samples per subgroups might lead to an incorrect tree (e.g. 50 samples of subclone A and 1000 samples each of subclone B and C).
This is a known problem in phylogeny calledtaxon imbalanceortaxon sampling. If you have multiple, clearly separable subgroups in your data we recommoned either subsampling over-represented groups or upsampling under-represented groups to gauge the effect of taxon imbalance.Running out of memory / bad_alloc errorIf MEDICC2 terminates with the following errorterminate called after throwing an instance of 'std::bad_alloc'or your machine runs out of memory this hints towards an issue with the FST.
Rerun MEDICC2 with the-vvflag to enable extended logging. If the error occurs during the ancestral reconstruction routine, the issue is related to OpenFST which is the FST library employed by MEDICC2 and cannot be easily solved by us.
This issue can be related to small bin sizes (and therefore a large number of segments). Increasing the binsize (although decreasing accuracy) solves this issue most of the time.
You can also try to remove the sample that led to the error (see the extended logs for this).The output plots are not like I expectedMaybe you need to set the--plotflag. By default,--plotis set to auto which means that it plots different figures depending on the number of samples in the data (threshold is 50); see above.Faulty event reconstructionSometimes MEDICC2 will pass out the following warning:Event recreation was faulty. This means that the events in the_cn_events_df.tsvfile will not be accurate. If you selected total copy number this will mainly be due to multiple WGDs
in a single node. Please get in contact with us if the problem prevails even without the--total-copy-numbersflag.missing segments / gaps in the segmentationMEDICC2 will assume that the segmentation is gap-less, i.e. that gaps between neighboring segments are neglible. If your data contains large gaps this might affect the performance of MEDICC2 as it might incorrectly jointly mutate segments that are actually separated.Bugs, feature requests and contactYou can report bugs and request features directly inBitbucketor contact us via attom.kaufmann@mdc-berlin.de.LicenseMEDICC2 is available underGPLv3. It contains modified code of thepywrapfstPython module fromOpenFSTas permitted by theApache 2license.Kaufmann et al. 2022 PuplicationThe MEDICC2 model has been published in 2022 with Genome Biology:MEDICC2: whole-genome doubling aware copy-number phylogenies for cancer evolution.FiguresBoth the required data and the scripts to create all Figures from the publication are stored in the commit version9b400efof the MEDICC2 repository available onZenodoin the folderFigures_Kaufmann_et_al_2021/.Simulation ValidationThe simulated copy-number profiles used in the MEDICC2 publication were created using our simulation frameworkSimphyni. The Simphyni repository contains a notebook to recreate the exact data used in the publication.Please citeKaufmann, T.L., Petkovic, M., Watkins, T.B.K. et al.MEDICC2: whole-genome doubling aware copy-number phylogenies for cancer evolution.Genome Biol 23, 241 (2022).https://doi.org/10.1186/s13059-022-02794-9Schwarz RF, Trinh A, Sipos B, Brenton JD, Goldman N, Markowetz F.Phylogenetic quantification of intra-tumour heterogeneity.PLoS Comput Biol. 2014 Apr 17;10(4):e1003535. doi: 10.1371/journal.pcbi.1003535.