TITLE: Creating Differential Transcript Expression Results with DESeq2
AUTHORS: David A. Eccles

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block">Differential expression analysis of transcript count tables using DESeq2</div></div>

[STEPS]
?. [Collating count data]
Combine the transcript count data into a single data structure. To reduce confusion when this protocol is run multiple times, we declare an analysisDate variable to be used for output file names:countDate We also load libraries that will be used in the protocol:## steps 1-2library(dplyr);    # data table manipulationlibrary(tidyr);    # data table cleaninglibrary(readr);    # reading csv files into tbllibrary(magrittr); # additional pipe operators## steps 3-8library(DESeq2);   # differential expression analysislibrary(ashr);     # for log fold change shrinking## step 9options(java.parameters = "-Xmx10G"); # may be needed to avoid Excel errorslibrary(xlsx);                        # writing to an Excel file
?. [Collating count data]
Collect a list of the count data files:data.files names(data.files) CHECK - print out the file names to make sure they're correct:data.files## example output#                                  CGAug18_004#"CGAug18_004_wide_transcript_counts_LAST.csv" #                                      CGDec17 #    "CGDec17_wide_transcript_counts_LAST.csv" #                                  CGJan19_006 #"CGJan19_006_wide_transcript_counts_LAST.csv" #                                  CGMar19_009 #"CGMar19_009_wide_transcript_counts_LAST.csv" #                                  CGNov18_005 #"CGNov18_005_wide_transcript_counts_LAST.csv"
?. [Collating count data]
Set up the skeleton structures for creating the combined table. This is created in two parts:A gene lookup table, containing gene metadataA count table, containing transcript countsgeneLookup.tbl counts.raw.tbl
?. [Collating count data]
The skeleton structures are then populated with the data from individual library files:for(dfi in seq_along(data.files)){  data.files[dfi] %>%    read_csv %>%    mutate(tdir = paste0(transcript, "_", dir)) ->      counts.sub.tbl;  ## append columns without barcode names to gene table  geneLookup.tbl %%      rbind(select(counts.sub.tbl,                   -starts_with("BC"),                   -starts_with("RB")));  ## append columns *with* barcode names to count table  counts.sub.tbl %%      select("tdir", starts_with("BC"), starts_with("RB"));  ## add file label to barcode name column  bcCols   colnames(counts.sub.tbl)[bcCols] %%    paste0(names(data.files)[dfi], ".", .);  counts.raw.tbl %%      full_join(counts.sub.tbl, by="tdir");}## Remove duplicates from the gene tablegeneLookup.tbl %% unique;CHECK - make sure that the column headings of the aggregated count table match the expected names:colnames(counts.raw.tbl)## Example output# [1] "tdir"             "CGAug18_004.BC04" "CGAug18_004.BC05" "CGAug18_004.BC06"# [5] "CGDec17.BC06"     "CGDec17.BC07"     "CGJan19_006.BC03" "CGJan19_006.BC04"# [9] "CGJan19_006.BC05" "CGJan19_006.BC06" "CGJan19_006.BC07" "CGJan19_006.BC08"#[13] "CGMar19_009.BC07" "CGMar19_009.BC08" "CGMar19_009.BC09" "CGMar19_009.BC10"#[17] "CGNov18_005.BC01" "CGNov18_005.BC02" "CGNov18_005.BC03" "CGNov18_005.BC04"#[21] "CGNov18_005.BC05" "CGNov18_005.BC06"
?. [Collating count data]
Do some cleaning / reordering of the data, then create an intermediate aggregate count table
?. [Collating count data]
The sample metadata file is read in, mostly as factors; the sample ID is converted to a character vector:read.csv("metadata.csv") %>%    mutate(SampleID = as.character(SampleID)) ->    meta.df;Metadata rows are subset and re-ordered to match the order of the count table:meta.df
?. [Collating count data]
Missing values for genes are set to counts of zero, and the count table is appended to the genes table:counts.raw.tbl %>%    replace(is.na(.), 0) %>%    left_join(geneLookup.tbl, ., by="tdir") ->    counts.withGenes.tbl;
?. [Collating count data]
The combined table is output to an intermediate file, using the analysis date as a file name:counts.withGenes.tbl %>%    write_csv(sprintf("raw_counts_%s.csv", countDate));
?. [Differential Expression]
Set up variables to change output file names and behaviour:Note: the countDate is not "today" because different explorations of differential expression could be done on the same count data.countDate l2FCShrink analysisDate resultSource excluded.factors Read in the intermediate aggregated count file and the metadata file:sprintf("raw_counts_%s.csv", countDate) %>%    read_csv ->    count.tbl;read.csv("metadata.csv") %>%    mutate(SampleID = as.character(SampleID)) %>%    ## Make sure metadata information only includes samples in the count table    filter(SampleID %in% colnames(count.tbl)) ->    meta.df;
?. [Differential Expression]
Carry out metadata filtering (i.e. sample exclusion) and count filtering (e.g. gene / sample QC)
?. [Differential Expression]
Make sure the count table containing only genes with a total count across all samples of at least minCount, have at least nonZeroThreshold genes with nonzero counts, and filter to choose only Sample IDs that are in the metadata table:count.tbl %%    pivot_longer(cols=c(-transcript, -Chr, -Strand, -Start, -End,                 -Description, -Gene, -dir, -tdir),                 names_to = "SampleID", values_to = "count") %>%    ## only keep transcripts with a total count of 2 or more    group_by(SampleID) %>%    filter(sum(count) >= 2) %>%    filter(sum(count > 0) > (length(count) * 0.25)) %>%    ## convert to integer    mutate(count = as.integer(count)) %>%    ## restore wide format    ungroup %>%    pivot_wider(names_from = SampleID, values_from = count) %>%    ## reorder to match metadata table    select(transcript, Chr, Strand, Start, End, Description, Gene, dir, tdir,           match(meta.df$SampleID, colnames(.)))
?. [Differential Expression]
Filter the metadata file to match the count table (i.e. removing any samples filtered out in the previous step), and exclude any columns that have single values:Refactor the metadata structure to remove missing values:for(x in colnames(meta.df)){    if(is.factor(meta.df[[x]])){        meta.df[[x]] %% factor;    }}CHECK - make sure the SampleIDs in the column names match the exact order of the metadata table:colnames(count.tbl)[-(1:9)] == meta.df$SampleID## Example output# [1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE                    #[16] TRUE TRUE
?. [Differential Expression]
Filter the metadata table to only include the desired samples:Note: this step will be situation specificmeta.df %%    ## Only keep 4T1 strain data    filter(Strain == "4T1") %>%    ## Sort by cell line, then replicate    arrange(Line, Replicate);
?. [Differential Expression]
Create DESeq2 data structure, and run a differential expression analysis
?. [Differential Expression]
Identify factors for the statistical model from the metadata file:setdiff(colnames(metasub.df),        c(c("SampleID","Label","Replicate","Notes"), excluded.factors)) ->    factorNames;
?. [Differential Expression]
Create the transcript count matrix:count.mat rownames(count.mat)
?. [Differential Expression]
Convert to DESeq2 structure and run DESeq2:DESeqDataSetFromMatrix(count.mat, meta.df,        as.formula(paste0("~ ", paste(factorNames, collapse=" + ")))) %>%    DESeq ->    dds;CHECK - make sure dispersion and result names look reasonable:Note: The result names will not indicate every possible comparison; just a subset from which all comparisons can be derived from.plotDispEsts(dds);resultsNames(dds);## Example output# [1] "Intercept"                   "Experiment_CG005_vs_CG004"                                      # [3] "Experiment_CG006_vs_CG004"   "Experiment_CG009_vs_CG004"                                      # [5] "Experiment_CGDec17_vs_CG004" "Line_p0D25_vs_p0"                                               # [7] "Line_p0SC_vs_p0"             "Line_p0SCL_vs_p0"                                               # [9] "Line_WT_vs_p0"
?. [Result Collation]
Prepare results table skeletons for DESeq2 results
?. [Result Collation]
Collect up comparisons to make:resultList for(fi in factorNames){    vn     vn     if(length(vn) == 1){        next;    }    for(fai in seq(1, length(vn)-1)){        for(fbi in seq(fai+1, length(vn))){            cat(sprintf("%s: %s vs %s\n", fi, vn[fai], vn[fbi]));            resultList                             list(c(fi, as.character(vn[fai]), as.character(vn[fbi]))));        }    }}
?. [Result Collation]
Generate base count table:dds.withCounts.tbl ## replace column names with metadata labelscolnames(dds.withCounts.tbl)[match(meta.df$SampleID,                                   colnames(dds.withCounts.tbl))]     paste0("raw.",sub("^ +", "", as.character(meta.df$Label)));Tack on VST matrix:dds.withCounts.tbl[,colnames(dds.counts)] Pre-populate wth minimum p-value column:dds.withCounts.tbl$min.p.val
?. [Result Collation]
Create variance-stabilised [log2] count matrix from DESeq2 structure:dds.counts Rescale VST values to have the same 99th percentile, but a minimum value of zero. This makes the scaled counts resemble more closely the actual read counts:dds.quantile99  min(dds.counts)], 0.99);((dds.counts - min(dds.counts)) / (dds.quantile99 - min(dds.counts))) * (dds.quantile99) ->    dds.counts;Replace column names in the VST matrix with labels from the metadata:Note: the substution removes any initial whitespace from the labelcolnames(dds.counts)[match(meta.df$SampleID,                           colnames(dds.counts))]     paste0("adj.",sub("^ +", "", as.character(meta.df$Label)));
?. [Result Collation]
Fetch DESeq2 results for each comparison from the DESeq2 data structure and add to the base table:for(rn in resultList){    print(rn);    results.df         if(l2FCShrink){            as.data.frame(lfcShrink(dds, contrast=rn, type = "ashr"));        } else {           as.data.frame(results(dds, contrast=rn));        }    results.df$log2FoldChange     results.df$pvalue     results.df$padj     rn.label     results.tbl     colnames(results.tbl)     results.tbl$tdir     dds.withCounts.tbl         left_join(dds.withCounts.tbl, results.tbl, by="tdir");}
?. [Result Collation]
Write results out to a CSV file:dds.withCounts.tbl$min.p.val     apply(dds.withCounts.tbl[,grep("^padj\\.",colnames(dds.withCounts.tbl))],          1, min, na.rm=TRUE);write.csv(dds.withCounts.tbl, row.names=FALSE,          gzfile(sprintf("DE_%s_VST_%s_%s.csv.gz",                         if(l2FCShrink){"shrunk"} else {"orig"},                         resultSource, analysisDate)));
?. [Excel Worksheet Output]
Separate results and put into an Excel fileNote: this step will be situation specific
?. [Excel Worksheet Output]
Split out mitochondrial genes:filtered.dds.tbl MT.dds.tbl
?. [Excel Worksheet Output]
Write split datasets out to the Excel file:write.xlsx2(as.data.frame(filtered.dds.tbl),            sprintf("DE_%s_VST_%s_%s.xlsx",                    if(l2FCShrink){"shrunk"} else {"orig"},                    resultSource, analysisDate),            sheetName="Genome Data", row.names=FALSE);write.xlsx2(as.data.frame(MT.dds.tbl),            sprintf("DE_%s_VST_%s_%s.xlsx",                    if(l2FCShrink){"shrunk"} else {"orig"},                    resultSource, analysisDate),            sheetName="MT Data", append=TRUE, row.names=FALSE);
?. [Excel Worksheet Output]
Add worksheets for differentially-expressed pairs:for(rn in resultList){    print(rn);    if(rn[1] == "Experiment"){        next;    }    sheetName     meta.df     cnames     cnames     rn.label     res.tbl                                    cnames,paste0(c("L2FC.","pval.", "padj."), rn.label))];    res.tbl     res.tbl     write.xlsx2(as.data.frame(res.tbl),                sprintf("DE_%s_VST_%s_%s.xlsx",                        if(l2FCShrink){"shrunk"} else {"orig"},                        resultSource, analysisDate),               sheetName=sheetName, append=TRUE, row.names=FALSE);}