TITLE: Genotype imputation workflow v3.0
AUTHORS: Kalle Pärn, Marita A. Isokallio, Javier Nunez Fontarnau, Aarno Palotie, Samuli Ripatti, Priit Palta

[DESCRIPTION]
<div class = "text-blocks"></div>

[STEPS]
?. [Requirements and preparatory steps]
The actual imputation protocol begins at step 2. All consecutive steps (commands given in 'cmd COMMAND' sections) must be run to ensure high-quality results. For a 'quick and dirty' genotype imputation 'run', you can jump straight to Steps 9-10 and only run these (assuming you already have all the required files in the correct format).Throughout the protocol we assume Bash shell.This Step 1 defines the requirements for the protocol (e.g. required software packages and reference genome files) and suggests example commands on how to process the files into suitable formats.In the protocol, we assume human reference genome build version GRCh38/hg38. To lift-over and update the build of your chip data, you can use our lift-over protocol: https://www.protocols.io/view/genotyping-chip-data-reference-genome-build-lift-o-nqtddwnPlease note, that build 38 compatible VCF files should and auxiliary files might have 'chr' prefixes in chromosome names. Therefore, some minor corresponding changes might be required in the related commands to parse/work with these files correctly.If you prefer GRCh37/hg19 or other genome build versions, please download the corresponding reference genome, map and reference panel files and modify the suggested commands accordingly.1.0 Docker images and preprepared filesWe have prepared docker images to ease up the preparatory steps depending on your application: genotype-liftover-imputation-protocols-lightgenotype-liftover-imputation-protocols-fullLight image contains the software packages installed (Step 1.1), the reference genome (Step 1.2.1), genetic map files (Steps 1.2.2 and 1.2.3) and example scripts to run commands in each step (Steps 2-13). Full image contains the software packages installed (Step 1.1), preprepared publically available reference files (as defined in Steps 1.2 and 1.3) and example scripts to run commands in each step (Steps 2-13).Additional to this protocol, the images contain software packages and files required for genotyping chip data lift-over protocol version 2:https://www.protocols.io/view/genotyping-chip-data-lift-over-to-reference-genome-xbhfij6The docker images are available at Docker hub To pull the image, use the command below and replace  with 'light' or 'full' according to which image you want: !! If you don't want to use Docker, you can install the software packages by yourself (Step 1.1), and either download preprepared files (below) or prepare the files by yourself (Step 1.2 and 1.3).
Preprepared reference data (included into the docker images as listed above) are also available for separate downloading at:https://console.cloud.google.com/storage/browser/fimm-public-data/Imputation/ And example scripts how to run each step at:https://console.cloud.google.com/storage/browser/fimm-public-data/Imputation/dockers/genotype-imputation-protocol/1.1 Software packages 1.1.1 Download and install the software packages Required software packages are listed below with the versions used in this protocol. However, using the latest versions is recommended.BCFtools v1.7 (or later version) http://www.htslib.org/download/R v3.4.1 (or later version) https://www.r-project.org/R package data.table https://github.com/Rdatatable/data.table/wiki/InstallationR package sm https://cran.r-project.org/package=smEagle v2.3.5 https://data.broadinstitute.org/alkesgroup/Eagle/Beagle v4.1 beagle.27Jan18.7e1.jar https://faculty.washington.edu/browning/beagle/b4_1.htmlBeagle bref bref.27Jan18.7e1.jar https://faculty.washington.edu/browning/beagle/bref.27Jan18.7e1.jar 1.1.2 Export the pathsOnce installed, export the correct paths to environment variable PATH: BCFtools plugin usage requires environment variable BCFTOOLS_PLUGINS exported, e.g: 1.1.3 Install the R packagesOnce R is installed, for instance the 'data.table' package can be installed in R, e.g.:   1.2. Reference genome and genetic map files 1.2.1 Fasta filesHomo Sapiens assembly hg38 version 0 is used and the required files are:Homo_sapiens_assembly38.fastaHomo_sapiens_assembly38.fasta.faiThe files are available for downloading at Broad Insitute storage in Google cloud at: https://console.cloud.google.com/storage/browser/broad-references/hg38/v0/?pli=1 If you prefer GRCh37/hg19, the reference genome files are available for downloading at Ensembl site athttps://grch37.ensembl.org/index.html   1.2.2. Genetic map files for phasing with EagleGenetic map file (all chromosomes in a single file) with recombination frequencies for GRCh38/hg38 are available for downloading at Eagle download page at:https://data.broadinstitute.org/alkesgroup/Eagle/downloads/tables/genetic_map_hg38_withX.txt.gzWe have processed the file according to the command below in order to split it per chromosome with correct headers.The resulting files are saved as:eagle_chr#_b38.txt (where # is the chromosome number) Note: Currently the chromosome notation in the Eagle genetic map files is only the chromosome number without 'chr' and chrX is '23'. Starting from Eagle v2.4, also chromosome notation with 'chr' tag is supported. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Eagle download page at https://data.broadinstitute.org/alkesgroup/Eagle/downloads/tables/genetic_map_hg19_withX.txt.gz  1.2.3 Genetic map files for imputation with BeagleGenetic map files fro Beagle are available for downloading at Beagle download page athttp://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh38.map.zipUnzip the files and change the chromosome notation from PLINK format (only number or X) to GRCh38/hg38 standard notation with 'chr' tag as follows: Note: For GRCh38/hg38, the chromosome notation in the Beagle genetic map files is 'chr#' and chromosome 23 is 'chrX'. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Beagle site:http://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip1.3. Imputation reference panel files 1.3.1 Obtain the reference panel filesFor increased imputation accuracy, we recommend using a population-specific imputation reference panel, if available. If population-specific reference data is not available, for instance 1000 Genomes Project (1000 GP) (www.nature.com/articles/nature15393) data can be used instead. GRCh38/hg38 files are available at EBI 1000 genomes ftp site: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/GRCh37/hg19 files are available at Beagle site already processed to be compatible with Beagle:http://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.bref/ The 1000 GP files can be downloaded for instance with command: NOTE: The reference panel files should contain: phased genotypes,chromosome names with 'chr' and chromosome X as 'chrX', all variants as biallelic records,only SNPs and INDELs,only unique variants, non-missing data, chrX as diploid genotypes, andonly unique IDs If your reference panel files are not in the correct format, see some suggested processing commands below.1.3.2 Minimum quality controlHere, we have piped most of the processing steps together in order to save significant amount of time by avoiding writing out multiple intermediate files. If your imputation reference panel does not require all the steps, modify the command accordingly.For 1000GP data:Generate a text file containing space-spearated old and new chromosome names. This is required to rename the numerical chromosome names with 'chr' tag. Apply the new chromosome names with 'bcftools annotate'.Remove the rare variants, here singletons and doubletons by setting AC threshold with 'bcftools view'.Split multiallelic sites to biallelic records with 'bcftools norm'.Keep only SNPs and INDELs with 'bcftools view'. Here, the 1000GP data included a tag VT in the INFO field and data contain also structural variants which should be excluded.Align the variants to reference genome with 'bcftools norm' in order to have the REF and ALT alleles in the shortest possible representation and to confirm that the REF allele matches the reference genome, additionally remove duplicate variants (-d none).After alignment, remove multiallelic records with 'bcftools view', since these are formed during the alignment if the REF does not match with the reference genome.Finally, remove sites containing missing data with 'bcftools view'. 1.3.3 Convert haploid genotypes to homozygous diploidsOften chrX is represented as haploid genotypes for males, however, Beagle can only handle diploid genotypes. The command here produces unphased diploid genotypes. But since the haploid genotypes are in diploid format as REF/REF or ALT/ALT, we can simply set the phase for those alleles with a simple sed replacement. The chrX ploidy can be corrected as follows: 1.3.4 Duplicate ID removalRemove duplicate IDs. If you wish to preserve all multiallelic sites, replace the ID column with a unique ID e.g. CHR_POS_REF_ALT (as indicated in Step 1.3.2). Here, 1000GP did not contain multiallelic sites after AC filtering, and thus, RSIDs were preserved in the ID column. And since RSIDs are not always unique, duplicates should be removed.  1.3.5 Reference panel allele frequenciesGenerate a tab-delimited file of the reference panel allele frequencies, one variant per line, with columns CHR, SNP (in format CHR_POS_REF_ALT), REF, ALT, AF (including the header line).First, update (or add) AF values in the INFO field, calculate it with BCFtools plugin +fill-tags: Extract the wanted fields from each VCF file and combine as a single output file with the header: Note: Chromosome notation in the panel.frq file should follow the GRCh38/hg38 notations ('chr#' for autosomal chromosomes and 'chrX' for chromosome 23). 1.3.6 Create binary reference panel filesThe phased reference panel files per chromosome are required in bref format (bref = binary reference). For more information, see Beagle documentation at Beagle site:https://faculty.washington.edu/browning/beagle/bref.16Dec15.pdf. The required bref.*.jar is downloaded from Beagle site:  Use the processed imputation reference panel VCFs as inputs for the example command below. The output files have the suffix '.bref' instead of '.vcf.gz'. 1.3.7 Generate a list of the reference panel sample IDsList of sample IDs present in the reference panel, one line per sample ID can be generated from any of the VCF files as in the example below (assuming that all chromosomes contain the same set of samples): 1.4. You are ready to start! As the last prepatory step, let's go over the required input data file(s) and also expected final output files! 1.4.1 Input file: Post-QC chip genotype data in VCFv4.2 format and chrX genotypes as diploid genotypes:DATASET.vcf.gz Note: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.  1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdfLight image contains the software packages installed (Step 1.1), the reference genome (Step 1.2.1), genetic map files (Steps 1.2.2 and 1.2.3) and example scripts to run commands in each step (Steps 2-13). Full image contains the software packages installed (Step 1.1), preprepared publically available reference files (as defined in Steps 1.2 and 1.3) and example scripts to run commands in each step (Steps 2-13).Additional to this protocol, the images contain software packages and files required for genotyping chip data lift-over protocol version 2:https://www.protocols.io/view/genotyping-chip-data-lift-over-to-reference-genome-xbhfij6The docker images are available at Docker hub To pull the image, use the command below and replace  with 'light' or 'full' according to which image you want:!! If you don't want to use Docker, you can install the software packages by yourself (Step 1.1), and either download preprepared files (below) or prepare the files by yourself (Step 1.2 and 1.3).
Preprepared reference data (included into the docker images as listed above) are also available for separate downloading at:https://console.cloud.google.com/storage/browser/fimm-public-data/Imputation/And example scripts how to run each step at:https://console.cloud.google.com/storage/browser/fimm-public-data/Imputation/dockers/genotype-imputation-protocol/1.1 Software packages1.1.1 Download and install the software packages Required software packages are listed below with the versions used in this protocol. However, using the latest versions is recommended.BCFtools v1.7 (or later version) http://www.htslib.org/download/R v3.4.1 (or later version) https://www.r-project.org/R package data.table https://github.com/Rdatatable/data.table/wiki/InstallationR package sm https://cran.r-project.org/package=smEagle v2.3.5 https://data.broadinstitute.org/alkesgroup/Eagle/Beagle v4.1 beagle.27Jan18.7e1.jar https://faculty.washington.edu/browning/beagle/b4_1.htmlBeagle bref bref.27Jan18.7e1.jar https://faculty.washington.edu/browning/beagle/bref.27Jan18.7e1.jar1.1.2 Export the pathsOnce installed, export the correct paths to environment variable PATH:BCFtools plugin usage requires environment variable BCFTOOLS_PLUGINS exported, e.g:1.1.3 Install the R packagesOnce R is installed, for instance the 'data.table' package can be installed in R, e.g.:1.2. Reference genome and genetic map files1.2.1 Fasta filesHomo Sapiens assembly hg38 version 0 is used and the required files are:Homo_sapiens_assembly38.fastaHomo_sapiens_assembly38.fasta.faiThe files are available for downloading at Broad Insitute storage in Google cloud at: https://console.cloud.google.com/storage/browser/broad-references/hg38/v0/?pli=1 If you prefer GRCh37/hg19, the reference genome files are available for downloading at Ensembl site athttps://grch37.ensembl.org/index.html   1.2.2. Genetic map files for phasing with EagleGenetic map file (all chromosomes in a single file) with recombination frequencies for GRCh38/hg38 are available for downloading at Eagle download page at:https://data.broadinstitute.org/alkesgroup/Eagle/downloads/tables/genetic_map_hg38_withX.txt.gzWe have processed the file according to the command below in order to split it per chromosome with correct headers.The resulting files are saved as:eagle_chr#_b38.txt (where # is the chromosome number) Note: Currently the chromosome notation in the Eagle genetic map files is only the chromosome number without 'chr' and chrX is '23'. Starting from Eagle v2.4, also chromosome notation with 'chr' tag is supported. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Eagle download page at https://data.broadinstitute.org/alkesgroup/Eagle/downloads/tables/genetic_map_hg19_withX.txt.gz  1.2.3 Genetic map files for imputation with BeagleGenetic map files fro Beagle are available for downloading at Beagle download page athttp://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh38.map.zipUnzip the files and change the chromosome notation from PLINK format (only number or X) to GRCh38/hg38 standard notation with 'chr' tag as follows: Note: For GRCh38/hg38, the chromosome notation in the Beagle genetic map files is 'chr#' and chromosome 23 is 'chrX'. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Beagle site:http://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip1.3. Imputation reference panel files 1.3.1 Obtain the reference panel filesFor increased imputation accuracy, we recommend using a population-specific imputation reference panel, if available. If population-specific reference data is not available, for instance 1000 Genomes Project (1000 GP) (www.nature.com/articles/nature15393) data can be used instead. GRCh38/hg38 files are available at EBI 1000 genomes ftp site: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/GRCh37/hg19 files are available at Beagle site already processed to be compatible with Beagle:http://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.bref/ The 1000 GP files can be downloaded for instance with command: NOTE: The reference panel files should contain: phased genotypes,chromosome names with 'chr' and chromosome X as 'chrX', all variants as biallelic records,only SNPs and INDELs,only unique variants, non-missing data, chrX as diploid genotypes, andonly unique IDs If your reference panel files are not in the correct format, see some suggested processing commands below.1.3.2 Minimum quality controlHere, we have piped most of the processing steps together in order to save significant amount of time by avoiding writing out multiple intermediate files. If your imputation reference panel does not require all the steps, modify the command accordingly.For 1000GP data:Generate a text file containing space-spearated old and new chromosome names. This is required to rename the numerical chromosome names with 'chr' tag. Apply the new chromosome names with 'bcftools annotate'.Remove the rare variants, here singletons and doubletons by setting AC threshold with 'bcftools view'.Split multiallelic sites to biallelic records with 'bcftools norm'.Keep only SNPs and INDELs with 'bcftools view'. Here, the 1000GP data included a tag VT in the INFO field and data contain also structural variants which should be excluded.Align the variants to reference genome with 'bcftools norm' in order to have the REF and ALT alleles in the shortest possible representation and to confirm that the REF allele matches the reference genome, additionally remove duplicate variants (-d none).After alignment, remove multiallelic records with 'bcftools view', since these are formed during the alignment if the REF does not match with the reference genome.Finally, remove sites containing missing data with 'bcftools view'. 1.3.3 Convert haploid genotypes to homozygous diploidsOften chrX is represented as haploid genotypes for males, however, Beagle can only handle diploid genotypes. The command here produces unphased diploid genotypes. But since the haploid genotypes are in diploid format as REF/REF or ALT/ALT, we can simply set the phase for those alleles with a simple sed replacement. The chrX ploidy can be corrected as follows: 1.3.4 Duplicate ID removalRemove duplicate IDs. If you wish to preserve all multiallelic sites, replace the ID column with a unique ID e.g. CHR_POS_REF_ALT (as indicated in Step 1.3.2). Here, 1000GP did not contain multiallelic sites after AC filtering, and thus, RSIDs were preserved in the ID column. And since RSIDs are not always unique, duplicates should be removed.  1.3.5 Reference panel allele frequenciesGenerate a tab-delimited file of the reference panel allele frequencies, one variant per line, with columns CHR, SNP (in format CHR_POS_REF_ALT), REF, ALT, AF (including the header line).First, update (or add) AF values in the INFO field, calculate it with BCFtools plugin +fill-tags: Extract the wanted fields from each VCF file and combine as a single output file with the header: Note: Chromosome notation in the panel.frq file should follow the GRCh38/hg38 notations ('chr#' for autosomal chromosomes and 'chrX' for chromosome 23). 1.3.6 Create binary reference panel filesThe phased reference panel files per chromosome are required in bref format (bref = binary reference). For more information, see Beagle documentation at Beagle site:https://faculty.washington.edu/browning/beagle/bref.16Dec15.pdf. The required bref.*.jar is downloaded from Beagle site:  Use the processed imputation reference panel VCFs as inputs for the example command below. The output files have the suffix '.bref' instead of '.vcf.gz'. 1.3.7 Generate a list of the reference panel sample IDsList of sample IDs present in the reference panel, one line per sample ID can be generated from any of the VCF files as in the example below (assuming that all chromosomes contain the same set of samples): 1.4. You are ready to start! As the last prepatory step, let's go over the required input data file(s) and also expected final output files! 1.4.1 Input file: Post-QC chip genotype data in VCFv4.2 format and chrX genotypes as diploid genotypes:DATASET.vcf.gz Note: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.  1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdfThe files are available for downloading at Broad Insitute storage in Google cloud at: https://console.cloud.google.com/storage/browser/broad-references/hg38/v0/?pli=1If you prefer GRCh37/hg19, the reference genome files are available for downloading at Ensembl site athttps://grch37.ensembl.org/index.html1.2.2. Genetic map files for phasing with EagleGenetic map file (all chromosomes in a single file) with recombination frequencies for GRCh38/hg38 are available for downloading at Eagle download page at:https://data.broadinstitute.org/alkesgroup/Eagle/downloads/tables/genetic_map_hg38_withX.txt.gzWe have processed the file according to the command below in order to split it per chromosome with correct headers.The resulting files are saved as:eagle_chr#_b38.txt (where # is the chromosome number) Note: Currently the chromosome notation in the Eagle genetic map files is only the chromosome number without 'chr' and chrX is '23'. Starting from Eagle v2.4, also chromosome notation with 'chr' tag is supported. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Eagle download page at https://data.broadinstitute.org/alkesgroup/Eagle/downloads/tables/genetic_map_hg19_withX.txt.gz  1.2.3 Genetic map files for imputation with BeagleGenetic map files fro Beagle are available for downloading at Beagle download page athttp://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh38.map.zipUnzip the files and change the chromosome notation from PLINK format (only number or X) to GRCh38/hg38 standard notation with 'chr' tag as follows: Note: For GRCh38/hg38, the chromosome notation in the Beagle genetic map files is 'chr#' and chromosome 23 is 'chrX'. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Beagle site:http://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip1.3. Imputation reference panel files 1.3.1 Obtain the reference panel filesFor increased imputation accuracy, we recommend using a population-specific imputation reference panel, if available. If population-specific reference data is not available, for instance 1000 Genomes Project (1000 GP) (www.nature.com/articles/nature15393) data can be used instead. GRCh38/hg38 files are available at EBI 1000 genomes ftp site: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/GRCh37/hg19 files are available at Beagle site already processed to be compatible with Beagle:http://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.bref/ The 1000 GP files can be downloaded for instance with command: NOTE: The reference panel files should contain: phased genotypes,chromosome names with 'chr' and chromosome X as 'chrX', all variants as biallelic records,only SNPs and INDELs,only unique variants, non-missing data, chrX as diploid genotypes, andonly unique IDs If your reference panel files are not in the correct format, see some suggested processing commands below.1.3.2 Minimum quality controlHere, we have piped most of the processing steps together in order to save significant amount of time by avoiding writing out multiple intermediate files. If your imputation reference panel does not require all the steps, modify the command accordingly.For 1000GP data:Generate a text file containing space-spearated old and new chromosome names. This is required to rename the numerical chromosome names with 'chr' tag. Apply the new chromosome names with 'bcftools annotate'.Remove the rare variants, here singletons and doubletons by setting AC threshold with 'bcftools view'.Split multiallelic sites to biallelic records with 'bcftools norm'.Keep only SNPs and INDELs with 'bcftools view'. Here, the 1000GP data included a tag VT in the INFO field and data contain also structural variants which should be excluded.Align the variants to reference genome with 'bcftools norm' in order to have the REF and ALT alleles in the shortest possible representation and to confirm that the REF allele matches the reference genome, additionally remove duplicate variants (-d none).After alignment, remove multiallelic records with 'bcftools view', since these are formed during the alignment if the REF does not match with the reference genome.Finally, remove sites containing missing data with 'bcftools view'. 1.3.3 Convert haploid genotypes to homozygous diploidsOften chrX is represented as haploid genotypes for males, however, Beagle can only handle diploid genotypes. The command here produces unphased diploid genotypes. But since the haploid genotypes are in diploid format as REF/REF or ALT/ALT, we can simply set the phase for those alleles with a simple sed replacement. The chrX ploidy can be corrected as follows: 1.3.4 Duplicate ID removalRemove duplicate IDs. If you wish to preserve all multiallelic sites, replace the ID column with a unique ID e.g. CHR_POS_REF_ALT (as indicated in Step 1.3.2). Here, 1000GP did not contain multiallelic sites after AC filtering, and thus, RSIDs were preserved in the ID column. And since RSIDs are not always unique, duplicates should be removed.  1.3.5 Reference panel allele frequenciesGenerate a tab-delimited file of the reference panel allele frequencies, one variant per line, with columns CHR, SNP (in format CHR_POS_REF_ALT), REF, ALT, AF (including the header line).First, update (or add) AF values in the INFO field, calculate it with BCFtools plugin +fill-tags: Extract the wanted fields from each VCF file and combine as a single output file with the header: Note: Chromosome notation in the panel.frq file should follow the GRCh38/hg38 notations ('chr#' for autosomal chromosomes and 'chrX' for chromosome 23). 1.3.6 Create binary reference panel filesThe phased reference panel files per chromosome are required in bref format (bref = binary reference). For more information, see Beagle documentation at Beagle site:https://faculty.washington.edu/browning/beagle/bref.16Dec15.pdf. The required bref.*.jar is downloaded from Beagle site:  Use the processed imputation reference panel VCFs as inputs for the example command below. The output files have the suffix '.bref' instead of '.vcf.gz'. 1.3.7 Generate a list of the reference panel sample IDsList of sample IDs present in the reference panel, one line per sample ID can be generated from any of the VCF files as in the example below (assuming that all chromosomes contain the same set of samples): 1.4. You are ready to start! As the last prepatory step, let's go over the required input data file(s) and also expected final output files! 1.4.1 Input file: Post-QC chip genotype data in VCFv4.2 format and chrX genotypes as diploid genotypes:DATASET.vcf.gz Note: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.  1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdfWe have processed the file according to the command below in order to split it per chromosome with correct headers.The resulting files are saved as:eagle_chr#_b38.txt (where # is the chromosome number) Note: Currently the chromosome notation in the Eagle genetic map files is only the chromosome number without 'chr' and chrX is '23'. Starting from Eagle v2.4, also chromosome notation with 'chr' tag is supported. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Eagle download page at https://data.broadinstitute.org/alkesgroup/Eagle/downloads/tables/genetic_map_hg19_withX.txt.gz  1.2.3 Genetic map files for imputation with BeagleGenetic map files fro Beagle are available for downloading at Beagle download page athttp://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh38.map.zipUnzip the files and change the chromosome notation from PLINK format (only number or X) to GRCh38/hg38 standard notation with 'chr' tag as follows: Note: For GRCh38/hg38, the chromosome notation in the Beagle genetic map files is 'chr#' and chromosome 23 is 'chrX'. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Beagle site:http://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip1.3. Imputation reference panel files 1.3.1 Obtain the reference panel filesFor increased imputation accuracy, we recommend using a population-specific imputation reference panel, if available. If population-specific reference data is not available, for instance 1000 Genomes Project (1000 GP) (www.nature.com/articles/nature15393) data can be used instead. GRCh38/hg38 files are available at EBI 1000 genomes ftp site: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/GRCh37/hg19 files are available at Beagle site already processed to be compatible with Beagle:http://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.bref/ The 1000 GP files can be downloaded for instance with command: NOTE: The reference panel files should contain: phased genotypes,chromosome names with 'chr' and chromosome X as 'chrX', all variants as biallelic records,only SNPs and INDELs,only unique variants, non-missing data, chrX as diploid genotypes, andonly unique IDs If your reference panel files are not in the correct format, see some suggested processing commands below.1.3.2 Minimum quality controlHere, we have piped most of the processing steps together in order to save significant amount of time by avoiding writing out multiple intermediate files. If your imputation reference panel does not require all the steps, modify the command accordingly.For 1000GP data:Generate a text file containing space-spearated old and new chromosome names. This is required to rename the numerical chromosome names with 'chr' tag. Apply the new chromosome names with 'bcftools annotate'.Remove the rare variants, here singletons and doubletons by setting AC threshold with 'bcftools view'.Split multiallelic sites to biallelic records with 'bcftools norm'.Keep only SNPs and INDELs with 'bcftools view'. Here, the 1000GP data included a tag VT in the INFO field and data contain also structural variants which should be excluded.Align the variants to reference genome with 'bcftools norm' in order to have the REF and ALT alleles in the shortest possible representation and to confirm that the REF allele matches the reference genome, additionally remove duplicate variants (-d none).After alignment, remove multiallelic records with 'bcftools view', since these are formed during the alignment if the REF does not match with the reference genome.Finally, remove sites containing missing data with 'bcftools view'. 1.3.3 Convert haploid genotypes to homozygous diploidsOften chrX is represented as haploid genotypes for males, however, Beagle can only handle diploid genotypes. The command here produces unphased diploid genotypes. But since the haploid genotypes are in diploid format as REF/REF or ALT/ALT, we can simply set the phase for those alleles with a simple sed replacement. The chrX ploidy can be corrected as follows: 1.3.4 Duplicate ID removalRemove duplicate IDs. If you wish to preserve all multiallelic sites, replace the ID column with a unique ID e.g. CHR_POS_REF_ALT (as indicated in Step 1.3.2). Here, 1000GP did not contain multiallelic sites after AC filtering, and thus, RSIDs were preserved in the ID column. And since RSIDs are not always unique, duplicates should be removed.  1.3.5 Reference panel allele frequenciesGenerate a tab-delimited file of the reference panel allele frequencies, one variant per line, with columns CHR, SNP (in format CHR_POS_REF_ALT), REF, ALT, AF (including the header line).First, update (or add) AF values in the INFO field, calculate it with BCFtools plugin +fill-tags: Extract the wanted fields from each VCF file and combine as a single output file with the header: Note: Chromosome notation in the panel.frq file should follow the GRCh38/hg38 notations ('chr#' for autosomal chromosomes and 'chrX' for chromosome 23). 1.3.6 Create binary reference panel filesThe phased reference panel files per chromosome are required in bref format (bref = binary reference). For more information, see Beagle documentation at Beagle site:https://faculty.washington.edu/browning/beagle/bref.16Dec15.pdf. The required bref.*.jar is downloaded from Beagle site:  Use the processed imputation reference panel VCFs as inputs for the example command below. The output files have the suffix '.bref' instead of '.vcf.gz'. 1.3.7 Generate a list of the reference panel sample IDsList of sample IDs present in the reference panel, one line per sample ID can be generated from any of the VCF files as in the example below (assuming that all chromosomes contain the same set of samples): 1.4. You are ready to start! As the last prepatory step, let's go over the required input data file(s) and also expected final output files! 1.4.1 Input file: Post-QC chip genotype data in VCFv4.2 format and chrX genotypes as diploid genotypes:DATASET.vcf.gz Note: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.  1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdfNote: Currently the chromosome notation in the Eagle genetic map files is only the chromosome number without 'chr' and chrX is '23'. Starting from Eagle v2.4, also chromosome notation with 'chr' tag is supported.If you prefer GRCh37/hg19, the genetic map file is available for downloading at Eagle download page at https://data.broadinstitute.org/alkesgroup/Eagle/downloads/tables/genetic_map_hg19_withX.txt.gz  1.2.3 Genetic map files for imputation with BeagleGenetic map files fro Beagle are available for downloading at Beagle download page athttp://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh38.map.zipUnzip the files and change the chromosome notation from PLINK format (only number or X) to GRCh38/hg38 standard notation with 'chr' tag as follows: Note: For GRCh38/hg38, the chromosome notation in the Beagle genetic map files is 'chr#' and chromosome 23 is 'chrX'. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Beagle site:http://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip1.3. Imputation reference panel files 1.3.1 Obtain the reference panel filesFor increased imputation accuracy, we recommend using a population-specific imputation reference panel, if available. If population-specific reference data is not available, for instance 1000 Genomes Project (1000 GP) (www.nature.com/articles/nature15393) data can be used instead. GRCh38/hg38 files are available at EBI 1000 genomes ftp site: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/GRCh37/hg19 files are available at Beagle site already processed to be compatible with Beagle:http://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.bref/ The 1000 GP files can be downloaded for instance with command: NOTE: The reference panel files should contain: phased genotypes,chromosome names with 'chr' and chromosome X as 'chrX', all variants as biallelic records,only SNPs and INDELs,only unique variants, non-missing data, chrX as diploid genotypes, andonly unique IDs If your reference panel files are not in the correct format, see some suggested processing commands below.1.3.2 Minimum quality controlHere, we have piped most of the processing steps together in order to save significant amount of time by avoiding writing out multiple intermediate files. If your imputation reference panel does not require all the steps, modify the command accordingly.For 1000GP data:Generate a text file containing space-spearated old and new chromosome names. This is required to rename the numerical chromosome names with 'chr' tag. Apply the new chromosome names with 'bcftools annotate'.Remove the rare variants, here singletons and doubletons by setting AC threshold with 'bcftools view'.Split multiallelic sites to biallelic records with 'bcftools norm'.Keep only SNPs and INDELs with 'bcftools view'. Here, the 1000GP data included a tag VT in the INFO field and data contain also structural variants which should be excluded.Align the variants to reference genome with 'bcftools norm' in order to have the REF and ALT alleles in the shortest possible representation and to confirm that the REF allele matches the reference genome, additionally remove duplicate variants (-d none).After alignment, remove multiallelic records with 'bcftools view', since these are formed during the alignment if the REF does not match with the reference genome.Finally, remove sites containing missing data with 'bcftools view'. 1.3.3 Convert haploid genotypes to homozygous diploidsOften chrX is represented as haploid genotypes for males, however, Beagle can only handle diploid genotypes. The command here produces unphased diploid genotypes. But since the haploid genotypes are in diploid format as REF/REF or ALT/ALT, we can simply set the phase for those alleles with a simple sed replacement. The chrX ploidy can be corrected as follows: 1.3.4 Duplicate ID removalRemove duplicate IDs. If you wish to preserve all multiallelic sites, replace the ID column with a unique ID e.g. CHR_POS_REF_ALT (as indicated in Step 1.3.2). Here, 1000GP did not contain multiallelic sites after AC filtering, and thus, RSIDs were preserved in the ID column. And since RSIDs are not always unique, duplicates should be removed.  1.3.5 Reference panel allele frequenciesGenerate a tab-delimited file of the reference panel allele frequencies, one variant per line, with columns CHR, SNP (in format CHR_POS_REF_ALT), REF, ALT, AF (including the header line).First, update (or add) AF values in the INFO field, calculate it with BCFtools plugin +fill-tags: Extract the wanted fields from each VCF file and combine as a single output file with the header: Note: Chromosome notation in the panel.frq file should follow the GRCh38/hg38 notations ('chr#' for autosomal chromosomes and 'chrX' for chromosome 23). 1.3.6 Create binary reference panel filesThe phased reference panel files per chromosome are required in bref format (bref = binary reference). For more information, see Beagle documentation at Beagle site:https://faculty.washington.edu/browning/beagle/bref.16Dec15.pdf. The required bref.*.jar is downloaded from Beagle site:  Use the processed imputation reference panel VCFs as inputs for the example command below. The output files have the suffix '.bref' instead of '.vcf.gz'. 1.3.7 Generate a list of the reference panel sample IDsList of sample IDs present in the reference panel, one line per sample ID can be generated from any of the VCF files as in the example below (assuming that all chromosomes contain the same set of samples): 1.4. You are ready to start! As the last prepatory step, let's go over the required input data file(s) and also expected final output files! 1.4.1 Input file: Post-QC chip genotype data in VCFv4.2 format and chrX genotypes as diploid genotypes:DATASET.vcf.gz Note: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.  1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdf1.2.3 Genetic map files for imputation with BeagleGenetic map files fro Beagle are available for downloading at Beagle download page athttp://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh38.map.zipUnzip the files and change the chromosome notation from PLINK format (only number or X) to GRCh38/hg38 standard notation with 'chr' tag as follows: Note: For GRCh38/hg38, the chromosome notation in the Beagle genetic map files is 'chr#' and chromosome 23 is 'chrX'. If you prefer GRCh37/hg19, the genetic map file is available for downloading at Beagle site:http://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip1.3. Imputation reference panel files 1.3.1 Obtain the reference panel filesFor increased imputation accuracy, we recommend using a population-specific imputation reference panel, if available. If population-specific reference data is not available, for instance 1000 Genomes Project (1000 GP) (www.nature.com/articles/nature15393) data can be used instead. GRCh38/hg38 files are available at EBI 1000 genomes ftp site: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/GRCh37/hg19 files are available at Beagle site already processed to be compatible with Beagle:http://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.bref/ The 1000 GP files can be downloaded for instance with command: NOTE: The reference panel files should contain: phased genotypes,chromosome names with 'chr' and chromosome X as 'chrX', all variants as biallelic records,only SNPs and INDELs,only unique variants, non-missing data, chrX as diploid genotypes, andonly unique IDs If your reference panel files are not in the correct format, see some suggested processing commands below.1.3.2 Minimum quality controlHere, we have piped most of the processing steps together in order to save significant amount of time by avoiding writing out multiple intermediate files. If your imputation reference panel does not require all the steps, modify the command accordingly.For 1000GP data:Generate a text file containing space-spearated old and new chromosome names. This is required to rename the numerical chromosome names with 'chr' tag. Apply the new chromosome names with 'bcftools annotate'.Remove the rare variants, here singletons and doubletons by setting AC threshold with 'bcftools view'.Split multiallelic sites to biallelic records with 'bcftools norm'.Keep only SNPs and INDELs with 'bcftools view'. Here, the 1000GP data included a tag VT in the INFO field and data contain also structural variants which should be excluded.Align the variants to reference genome with 'bcftools norm' in order to have the REF and ALT alleles in the shortest possible representation and to confirm that the REF allele matches the reference genome, additionally remove duplicate variants (-d none).After alignment, remove multiallelic records with 'bcftools view', since these are formed during the alignment if the REF does not match with the reference genome.Finally, remove sites containing missing data with 'bcftools view'. 1.3.3 Convert haploid genotypes to homozygous diploidsOften chrX is represented as haploid genotypes for males, however, Beagle can only handle diploid genotypes. The command here produces unphased diploid genotypes. But since the haploid genotypes are in diploid format as REF/REF or ALT/ALT, we can simply set the phase for those alleles with a simple sed replacement. The chrX ploidy can be corrected as follows: 1.3.4 Duplicate ID removalRemove duplicate IDs. If you wish to preserve all multiallelic sites, replace the ID column with a unique ID e.g. CHR_POS_REF_ALT (as indicated in Step 1.3.2). Here, 1000GP did not contain multiallelic sites after AC filtering, and thus, RSIDs were preserved in the ID column. And since RSIDs are not always unique, duplicates should be removed.  1.3.5 Reference panel allele frequenciesGenerate a tab-delimited file of the reference panel allele frequencies, one variant per line, with columns CHR, SNP (in format CHR_POS_REF_ALT), REF, ALT, AF (including the header line).First, update (or add) AF values in the INFO field, calculate it with BCFtools plugin +fill-tags: Extract the wanted fields from each VCF file and combine as a single output file with the header: Note: Chromosome notation in the panel.frq file should follow the GRCh38/hg38 notations ('chr#' for autosomal chromosomes and 'chrX' for chromosome 23). 1.3.6 Create binary reference panel filesThe phased reference panel files per chromosome are required in bref format (bref = binary reference). For more information, see Beagle documentation at Beagle site:https://faculty.washington.edu/browning/beagle/bref.16Dec15.pdf. The required bref.*.jar is downloaded from Beagle site:  Use the processed imputation reference panel VCFs as inputs for the example command below. The output files have the suffix '.bref' instead of '.vcf.gz'. 1.3.7 Generate a list of the reference panel sample IDsList of sample IDs present in the reference panel, one line per sample ID can be generated from any of the VCF files as in the example below (assuming that all chromosomes contain the same set of samples): 1.4. You are ready to start! As the last prepatory step, let's go over the required input data file(s) and also expected final output files! 1.4.1 Input file: Post-QC chip genotype data in VCFv4.2 format and chrX genotypes as diploid genotypes:DATASET.vcf.gz Note: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.  1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdfUnzip the files and change the chromosome notation from PLINK format (only number or X) to GRCh38/hg38 standard notation with 'chr' tag as follows:Note: For GRCh38/hg38, the chromosome notation in the Beagle genetic map files is 'chr#' and chromosome 23 is 'chrX'.If you prefer GRCh37/hg19, the genetic map file is available for downloading at Beagle site:http://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip1.3. Imputation reference panel files 1.3.1 Obtain the reference panel filesFor increased imputation accuracy, we recommend using a population-specific imputation reference panel, if available. If population-specific reference data is not available, for instance 1000 Genomes Project (1000 GP) (www.nature.com/articles/nature15393) data can be used instead. GRCh38/hg38 files are available at EBI 1000 genomes ftp site: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/GRCh37/hg19 files are available at Beagle site already processed to be compatible with Beagle:http://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.bref/ The 1000 GP files can be downloaded for instance with command: NOTE: The reference panel files should contain: phased genotypes,chromosome names with 'chr' and chromosome X as 'chrX', all variants as biallelic records,only SNPs and INDELs,only unique variants, non-missing data, chrX as diploid genotypes, andonly unique IDs If your reference panel files are not in the correct format, see some suggested processing commands below.1.3.2 Minimum quality controlHere, we have piped most of the processing steps together in order to save significant amount of time by avoiding writing out multiple intermediate files. If your imputation reference panel does not require all the steps, modify the command accordingly.For 1000GP data:Generate a text file containing space-spearated old and new chromosome names. This is required to rename the numerical chromosome names with 'chr' tag. Apply the new chromosome names with 'bcftools annotate'.Remove the rare variants, here singletons and doubletons by setting AC threshold with 'bcftools view'.Split multiallelic sites to biallelic records with 'bcftools norm'.Keep only SNPs and INDELs with 'bcftools view'. Here, the 1000GP data included a tag VT in the INFO field and data contain also structural variants which should be excluded.Align the variants to reference genome with 'bcftools norm' in order to have the REF and ALT alleles in the shortest possible representation and to confirm that the REF allele matches the reference genome, additionally remove duplicate variants (-d none).After alignment, remove multiallelic records with 'bcftools view', since these are formed during the alignment if the REF does not match with the reference genome.Finally, remove sites containing missing data with 'bcftools view'. 1.3.3 Convert haploid genotypes to homozygous diploidsOften chrX is represented as haploid genotypes for males, however, Beagle can only handle diploid genotypes. The command here produces unphased diploid genotypes. But since the haploid genotypes are in diploid format as REF/REF or ALT/ALT, we can simply set the phase for those alleles with a simple sed replacement. The chrX ploidy can be corrected as follows: 1.3.4 Duplicate ID removalRemove duplicate IDs. If you wish to preserve all multiallelic sites, replace the ID column with a unique ID e.g. CHR_POS_REF_ALT (as indicated in Step 1.3.2). Here, 1000GP did not contain multiallelic sites after AC filtering, and thus, RSIDs were preserved in the ID column. And since RSIDs are not always unique, duplicates should be removed.  1.3.5 Reference panel allele frequenciesGenerate a tab-delimited file of the reference panel allele frequencies, one variant per line, with columns CHR, SNP (in format CHR_POS_REF_ALT), REF, ALT, AF (including the header line).First, update (or add) AF values in the INFO field, calculate it with BCFtools plugin +fill-tags: Extract the wanted fields from each VCF file and combine as a single output file with the header: Note: Chromosome notation in the panel.frq file should follow the GRCh38/hg38 notations ('chr#' for autosomal chromosomes and 'chrX' for chromosome 23). 1.3.6 Create binary reference panel filesThe phased reference panel files per chromosome are required in bref format (bref = binary reference). For more information, see Beagle documentation at Beagle site:https://faculty.washington.edu/browning/beagle/bref.16Dec15.pdf. The required bref.*.jar is downloaded from Beagle site:  Use the processed imputation reference panel VCFs as inputs for the example command below. The output files have the suffix '.bref' instead of '.vcf.gz'. 1.3.7 Generate a list of the reference panel sample IDsList of sample IDs present in the reference panel, one line per sample ID can be generated from any of the VCF files as in the example below (assuming that all chromosomes contain the same set of samples): 1.4. You are ready to start! As the last prepatory step, let's go over the required input data file(s) and also expected final output files! 1.4.1 Input file: Post-QC chip genotype data in VCFv4.2 format and chrX genotypes as diploid genotypes:DATASET.vcf.gz Note: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.  1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdf1.3. Imputation reference panel files1.3.1 Obtain the reference panel filesFor increased imputation accuracy, we recommend using a population-specific imputation reference panel, if available.If population-specific reference data is not available, for instance 1000 Genomes Project (1000 GP) (www.nature.com/articles/nature15393) data can be used instead.GRCh38/hg38 files are available at EBI 1000 genomes ftp site: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/GRCh37/hg19 files are available at Beagle site already processed to be compatible with Beagle:http://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.bref/The 1000 GP files can be downloaded for instance with command:NOTE: The reference panel files should contain: phased genotypes,chromosome names with 'chr' and chromosome X as 'chrX', all variants as biallelic records,only SNPs and INDELs,only unique variants, non-missing data, chrX as diploid genotypes, andonly unique IDs If your reference panel files are not in the correct format, see some suggested processing commands below.1.3.2 Minimum quality controlHere, we have piped most of the processing steps together in order to save significant amount of time by avoiding writing out multiple intermediate files. If your imputation reference panel does not require all the steps, modify the command accordingly.For 1000GP data:Generate a text file containing space-spearated old and new chromosome names. This is required to rename the numerical chromosome names with 'chr' tag. Apply the new chromosome names with 'bcftools annotate'.Remove the rare variants, here singletons and doubletons by setting AC threshold with 'bcftools view'.Split multiallelic sites to biallelic records with 'bcftools norm'.Keep only SNPs and INDELs with 'bcftools view'. Here, the 1000GP data included a tag VT in the INFO field and data contain also structural variants which should be excluded.Align the variants to reference genome with 'bcftools norm' in order to have the REF and ALT alleles in the shortest possible representation and to confirm that the REF allele matches the reference genome, additionally remove duplicate variants (-d none).After alignment, remove multiallelic records with 'bcftools view', since these are formed during the alignment if the REF does not match with the reference genome.Finally, remove sites containing missing data with 'bcftools view'. 1.3.3 Convert haploid genotypes to homozygous diploidsOften chrX is represented as haploid genotypes for males, however, Beagle can only handle diploid genotypes. The command here produces unphased diploid genotypes. But since the haploid genotypes are in diploid format as REF/REF or ALT/ALT, we can simply set the phase for those alleles with a simple sed replacement. The chrX ploidy can be corrected as follows: 1.3.4 Duplicate ID removalRemove duplicate IDs. If you wish to preserve all multiallelic sites, replace the ID column with a unique ID e.g. CHR_POS_REF_ALT (as indicated in Step 1.3.2). Here, 1000GP did not contain multiallelic sites after AC filtering, and thus, RSIDs were preserved in the ID column. And since RSIDs are not always unique, duplicates should be removed.  1.3.5 Reference panel allele frequenciesGenerate a tab-delimited file of the reference panel allele frequencies, one variant per line, with columns CHR, SNP (in format CHR_POS_REF_ALT), REF, ALT, AF (including the header line).First, update (or add) AF values in the INFO field, calculate it with BCFtools plugin +fill-tags: Extract the wanted fields from each VCF file and combine as a single output file with the header: Note: Chromosome notation in the panel.frq file should follow the GRCh38/hg38 notations ('chr#' for autosomal chromosomes and 'chrX' for chromosome 23). 1.3.6 Create binary reference panel filesThe phased reference panel files per chromosome are required in bref format (bref = binary reference). For more information, see Beagle documentation at Beagle site:https://faculty.washington.edu/browning/beagle/bref.16Dec15.pdf. The required bref.*.jar is downloaded from Beagle site:  Use the processed imputation reference panel VCFs as inputs for the example command below. The output files have the suffix '.bref' instead of '.vcf.gz'. 1.3.7 Generate a list of the reference panel sample IDsList of sample IDs present in the reference panel, one line per sample ID can be generated from any of the VCF files as in the example below (assuming that all chromosomes contain the same set of samples): 1.4. You are ready to start! As the last prepatory step, let's go over the required input data file(s) and also expected final output files! 1.4.1 Input file: Post-QC chip genotype data in VCFv4.2 format and chrX genotypes as diploid genotypes:DATASET.vcf.gz Note: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.  1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdfIf your reference panel files are not in the correct format, see some suggested processing commands below.1.3.2 Minimum quality controlHere, we have piped most of the processing steps together in order to save significant amount of time by avoiding writing out multiple intermediate files. If your imputation reference panel does not require all the steps, modify the command accordingly.For 1000GP data:Generate a text file containing space-spearated old and new chromosome names. This is required to rename the numerical chromosome names with 'chr' tag. Apply the new chromosome names with 'bcftools annotate'.Remove the rare variants, here singletons and doubletons by setting AC threshold with 'bcftools view'.Split multiallelic sites to biallelic records with 'bcftools norm'.Keep only SNPs and INDELs with 'bcftools view'. Here, the 1000GP data included a tag VT in the INFO field and data contain also structural variants which should be excluded.Align the variants to reference genome with 'bcftools norm' in order to have the REF and ALT alleles in the shortest possible representation and to confirm that the REF allele matches the reference genome, additionally remove duplicate variants (-d none).After alignment, remove multiallelic records with 'bcftools view', since these are formed during the alignment if the REF does not match with the reference genome.Finally, remove sites containing missing data with 'bcftools view'.If multiallelic sites are present in your data, in order to preserve them throughout the protocol, set ID field with unique IDs e.g. in format CHR_POS_REF_ALT. (RSIDs might contain duplicates, when the multiallelic sites are decomposed.)1.3.3 Convert haploid genotypes to homozygous diploidsOften chrX is represented as haploid genotypes for males, however, Beagle can only handle diploid genotypes. The command here produces unphased diploid genotypes. But since the haploid genotypes are in diploid format as REF/REF or ALT/ALT, we can simply set the phase for those alleles with a simple sed replacement. The chrX ploidy can be corrected as follows:1.3.4 Duplicate ID removalRemove duplicate IDs. If you wish to preserve all multiallelic sites, replace the ID column with a unique ID e.g. CHR_POS_REF_ALT (as indicated in Step 1.3.2). Here, 1000GP did not contain multiallelic sites after AC filtering, and thus, RSIDs were preserved in the ID column. And since RSIDs are not always unique, duplicates should be removed.1.3.5 Reference panel allele frequenciesGenerate a tab-delimited file of the reference panel allele frequencies, one variant per line, with columns CHR, SNP (in format CHR_POS_REF_ALT), REF, ALT, AF (including the header line).First, update (or add) AF values in the INFO field, calculate it with BCFtools plugin +fill-tags:Extract the wanted fields from each VCF file and combine as a single output file with the header:Note: Chromosome notation in the panel.frq file should follow the GRCh38/hg38 notations ('chr#' for autosomal chromosomes and 'chrX' for chromosome 23).1.3.6 Create binary reference panel filesThe phased reference panel files per chromosome are required in bref format (bref = binary reference). For more information, see Beagle documentation at Beagle site:https://faculty.washington.edu/browning/beagle/bref.16Dec15.pdf.The required bref.*.jar is downloaded from Beagle site:Use the processed imputation reference panel VCFs as inputs for the example command below. The output files have the suffix '.bref' instead of '.vcf.gz'.1.3.7 Generate a list of the reference panel sample IDsList of sample IDs present in the reference panel, one line per sample ID can be generated from any of the VCF files as in the example below (assuming that all chromosomes contain the same set of samples):1.4. You are ready to start! As the last prepatory step, let's go over the required input data file(s) and also expected final output files!1.4.1 Input file:Post-QC chip genotype data in VCFv4.2 format and chrX genotypes as diploid genotypes:DATASET.vcf.gz Note: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.  1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdfNote: Chromosome notation should follow the GRCh38/hg38 notations (e.g. 'chr#' for autosomal chromosomes, 'chrX', 'chrY' and 'chrM').Note: If the input data was lifted over from an older genome build to build version 38, cautious inspection of the data is highly recommended before proceeding with the protocol.Note: If chrX is represented as haploid genotypes, follow step 1.3.3 first 'bcftools +fixploidy' command (not the other two piped commands) to convert to diploid genotypes.1.4.2 Final output files:DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number)DATASET_postimputation_summary_plots.pdfNote: Several intermediate files are created during the protocol. Those files can be used for troubleshooting and deleted once the successful imputation is confirmed.
?. [Chip data validation and VCF formatting]
Align the variant alleles to human reference genome to correct for any dataset-specific REF/ALT flips.

Ensure that only biallelic sites are kept in the chip data, as 'bcftools norm' may introduce false multiallelic sites if the REF does not match to the reference genome.

Finally, replace the ID column with an 'SNP ID' in format CHR_POS_REF_ALT ie. chromosome_position__.Input files:• DATASET_chrfiltered.vcf.gz (Step 2)• Homo_sapiens_assembly38.fasta or another wanted reference genome file (Step 1.2.1)Outpt file:• DATASET_SNPID.vcf.gzNote: Confirm the results for example by comparing the input and output file sizes and line counts.
?. [Chip data validation and VCF formatting]
Only autosomal and X chromosomes are kept for the imputation.Input file:• DATASET.vcf.gz (your QC'ed genotyping chip data in VCFv4.2 format, chrX as diploid genotypes)Outpt file:• DATASET_chrfiltered.vcf.gzNote: Confirm the results for example by comparing the input and output file sizes and line counts.
?. [Chip data validation and VCF formatting]
Ensure that duplicate individuals do not exist in the chip data or between the chip and the imputation reference panel. Furthermore, ensure that there are no duplicate variants (these might appear in some chip genotype datasets and after reference genome alignment). Both sample and variant duplicates would compromise the imputation.Input files:• DATASET_SNPID.vcf.gz (Step 3)• panel_sample_IDs.txt (Step 1.3.7)Outpt file:• DATASET_noduplicate_variants.vcf.gzNote: Confirm the results for example by comparing the input and output file sizes and line counts.
?. [Chip data validation and VCF formatting]
Exclude rare variants (redundant step if they are already removed in quality control steps taken before this protocol).Input file:• DATASET_noduplicate_variants.vcf.gz (Step 4)Outpt file:• DATASET_AF.vcf.gzNote: Confirm the results for example by comparing the input and output file sizes and line counts.
?. [Chip data validation and VCF formatting]
Generate a frequency file for the chip data and the imputation reference panel allele frequency comparison.Input file:• DATASET_AF.vcf.gz (Step 5)Outpt file:• DATASET_chip.frq
?. [Chip data validation and VCF formatting]
Compare the chip data and the imputation reference panel allele frequencies and generate an exclusion list of those variants where AF values differ more than 10 pp or their log2 fold change is >5 or Variants showing highly discrepant AF frequency will be excluded (in the next step). Confirm from the plot that number of concordant variants is high (preferably >300k) to ensure successful phasing (in Step 9). Copy and save the R script below as 'plot_AF.R'.Set the script as executable for instance with 'chmod +x plot_AF.R' before running it for the first time.Run the script as indicated in the example command below. Here, we use 0.1 as a threshold for AF difference and 5 for the log2 fold change e.g. if chip data AF differs more than 10 pp from the reference data AF, it is marked as discordant. However, less conservative values may be more suitable if a population specific reference data is not available.Input files:• DATASET_chip.frq (Step 6)• panel.frq (Step 1.3.5)Outpt files:• DATASET_AFs.png• DATASET_exclude.txt• DATASET_nonpanel_exclude.txtAdditional output files:• DATASET_panel_isec.txt• DATASET_discrepant_AF.txt!! Inspect the generated plots before proceeding to the next step:See in the expected results section at the end of this step, for examples of plots from a high-quality dataset.Left plotComparison of the chip AF values to the panel AF values. Ideally, it should show a quite tight, uniform diagonal line of increasing slope 1. Variants with AF values differing more than 10 pp or log2 fold change >5 or Right plotDistribution of the AF values. Ideally the histogram should be smooth and skewed towards the smallest AF values.
?. [Chip data validation and VCF formatting]
Exclude the variants showing highly discordant allele frequencies or not present in the imputation reference panel as listed in the previous step.Input files:• DATASET_exclude.txt (Step7)• DATASET_nonpanel_exclude.txt (Step 7)• DATASET_AF.vcf.gz  (Step 5)Outpt files:• DATASET_exclusion1.txt• DATASET_exclusion2.txt• DATASET_for_phasing.vcf.gzNote: Confirm the results for example by comparing the input and output file sizes and line counts.
?. [Chip data pre-phasing]
Chip data pre-phasing will speed up the genotype imputation step.Phase haplotypes for each chromosome separately before imputation. Here, adjust the number of threads and parallelization of the phasing for each chromosomes according to your computing environment. We generally use 8 threads per chromosome and run all chromosomes in parallel distributed on 23 cores.Input files:• DATASET_for_phasing.vcf.gz (Step 8)• eagle_chr#_b38.map (where # is chromosome number) (Step 1.2.2)Outpt file:• DATASET_for_imputation_chr#.vcf.gz (where # is chromosome number)Note: Confirm the results for example by comparing the input and output file sizes and line counts.
?. [Genotype imputation]
Run genotype imputation for each chromosome separately. Here, adjust the number of threads and parallelization of the imputation for each chromosomes according to your computing environment. We have observed the fastest runs by using 16 threads pre chromosome and running all chromosomes in parallel distributed on 23 cores.
NOTE: Chromosome X should be represented as diploid genotypes. If this is not the case, fix the ploidy with the example command from Step 1.3.3.Input files:• DATASET_for_imputation_chr#.vcf.gz (Step 9)• panel_phased_AF_chr#.bref (where # is chromosome number) (Step 1.3.6)• beagle_chr#_b38.map (where # is chromosome number) (Step 1.2.3)Outpt file:• DATASET_imputed_chr#.vcf.gz (where # is chromosome number)
?. [Post-imputation processing and quality assurance]
Re-calculate/add INFO field values and compute Impute2-like INFO scores for each variant.Here, we use bcftools plugin +fill-tags to only add AF value. However, the plugin is able to add also other values, such as Hardy-Weinberg equilibrium p-value, if required. See documentation for details at: http://samtools.github.io/bcftools/howtos/plugin.fill-tags.htmlInput file:• DATASET_imputed_chr#.vcf.gz (where # is chromosome number) (Step 10)Outpt files:• DATASET_imputed_chr#.vcf.gz (where # is chromosome number)• DATASET_imputed_info_chr#.vcf.gz.tbi (where # is chromosome number)
?. [Post-imputation processing and quality assurance]
Extract allele frequencies (AF) and INFO scores from the imputed VCF file.Group the variants by their AF into either of the following categories,1: AF >= 5% or AF 2: 0.5%  3: AF  99.5% (rare variants)Input file:• DATASET_imputed_info_chr#.vcf.gz (where # is chromosome number) (Step 11)Outpt file:• DATASET_INFO_group_chr#.txt (where # is chromosome number)
?. [Post-imputation processing and quality assurance]
Produce multiple plots to illustrate the distribution of Impute2-like INFO scores and allele frequencies of the imputed data as well as the comparison of allele frequencies between the imputed data and the imputation reference panel.Copy and store the R script below as 'plot_INFO_and_AF_for_imputed_chrs.R'.Set the script as executable for instance with 'chmod +x plot_INFO_and_AF_for_imputed_chrs.R' before running it for the first time.Run the script as indicated in the example command below.Input files:• DATASET_INFO_group_chr#.txt (where # is chromosome number) (Step 12)• panel.frq (Step 1.3.5)Outpt files:• DATASET_chr#_postimputation_summary_plots.png (where # is chromosome number)• DATASET_postimputation_summary_plots.pdf!! Inspect the plots:See example of the produced plots in expected results section a the end of this step.Top left:Impute2-like INFO score density for each AF group.Bottom left:AF distribution of the imputed chip data.Top right:AF comparison between the panel and imputed chip data.Bottom right:Absolute AF difference along the chromosome positions between the panel and imputed chip data.
Example of chromosome 2 post-imputation summary plots with a population-specific imputation reference panel.