TITLE: Preparing Reads for Stranded Mapping
AUTHORS: David A. Eccles

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block">This protocol is for preparing long reads for stranded mapping, as an intermediate step for additional protocols:</div><div class = "text-block"><ul style = "list-style-type:disc;"><li style = "counter-reset:ol0;list-style-type:disc;">Aligning strand-oriented sequences to a transcriptome for transcript / gene counting</li><li style = "counter-reset:ol0;list-style-type:disc;">Aligning strand-oriented sequences to a genome for confirmatory QC</li></ul></div><div class = "text-block"><span style = "font-weight:bold;">Input(s)</span><span>: demultiplexed fastq files (see protocol </span><a href="https://dx.doi.org/10.17504/protocols.io.55dg826" style = "text-decoration:underline;color:blue;cursor:pointer;"><span style = ":;">Demultiplexing Nanopore reads with LAST</span></a><span>), adapter file (containing strand-sensitive adapter sequences)</span></div><div class = "text-block"><span style = "font-weight:bold;">Output(s):</span><span> oriented read files, as gzipped fastq files</span></div></div>

[STEPS]
?. [Index Preparation]
Prepare transcript index (see Guidelines for data sources)lastdb Mus_musculus.GRCm38.cds.all.fa
?. [Transcriptome Mapping]
Reads are mapped to the transcriptome with LAST.The results of that mapping can be piped through last-map-probs to exclude unlikely hits, then through 'maf-convert -n tab' to convert to a one-line-per-mapping CSV format. This CSV format is further processed to make sure that there is only one mapping per transcript-read pair, and then aggregated to sum up counts per transcript.mkdir -p mappedfor bc in $(awk '{print $2}' barcode_counts.txt);  do echo "** ${bc} **";  lastal -Q 1 -P 10 Mus_musculus.GRCm38.cds.all.fa     last-map-probs | maf-convert -n tab | cut -f 2,7,10 | sort | uniq | \    awk -F'\t' -v "bc=${bc}" '{print bc,$1,$3}' | sort | uniq -c | gzip > mapped/trnCounts_LAST_${bc}_vs_Mmus_transcriptome.txt.gz;done
?. [Annotation and Result generation]
Transcript counts are merged with ensembl gene annotation, then converted into wide format (one line per transcript) using an R script.The transcript annotation in this case is from ensembl BioMart (see Guidelines for more details).barcode_counts.txt#!/usr/bin/env Rscriptlibrary(dplyr);library(tidyr);## load ensemble transcript metadata (including gene name)ensembl.df   col.names=c('transcript','Description','Start','End',              'Strand','Gene','Chr'),  stringsAsFactors=FALSE));ensembl.df$Description ensembl.df$Description ensembl.df[,1:7] colnames(ensembl.df)[1:7] options(scipen=15); ## don't show scientific notation for large positions## load used barcode identifiersbcNames ## load count data into 'narrow' array (one line per count)trn.counts   trn.counts     bind_rows(trn.counts,      as.tbl(read.table(        sprintf('mapped/trnCounts_LAST_%s_vs_Mmus_transcriptome.txt.gz', bc),        col.names=c('count','barcode','transcript','dir'),        stringsAsFactors=FALSE)));}## remove revision number from transcript names (if present)trn.counts$transcript ## convert to wide format (one line per transcript)trn.counts.wide %  mutate(dir = c('+'='fwd', '-'='rev')[dir]);for(bd in colnames(trn.counts.wide[,-1])){  trn.counts.wide[[bd]] }## merge ensembl metadata with transcript countsgene.counts.wide gene.counts.wide ## write result out to a filewrite.csv(gene.counts.wide, file='wide_transcript_counts_LAST.csv',   row.names=FALSE);
?. [Index Preparation]
Prepare cDNA adapter indexlastdb -uNEAR -R01 adapter_seqs.fa adapter_seqs.fa
?. [Read Orientation]
Map demultiplexed reads to adapter sequences to generate tab-separated file of adapter assignments:mkdir mapped(for bc in $(awk '{print $2}' barcode_counts.txt);  do lastal -Q 1 -P 10 adapter_seqs.fa     maf-convert -n tab | cut -f 2,7,10 | perl -pe "s/^/${bc}\t/"; done) | gzip > adapter_assignments_all.tsv.gz
?. [Read Orientation]
Create 'wide' table indicating adapter assignments. This R script creates files 'adapter_assignments_ideal.csv.gz' and 'adapter_assignments_valid.csv.gz'.#!/usr/bin/env Rscriptad.df                     header=FALSE, col.names = c("bc","target","query","dir"),                    stringsAsFactors=FALSE);library(dplyr);library(tidyr);## Create table of adapter additionsad.tbl %     summarise() %>%     unite(tdir, target, dir, sep='.') %>% mutate(present=TRUE) %>%    spread(tdir, present, fill=FALSE);## Clean up adapter namescolnames(ad.tbl) colnames(ad.tbl) colnames(ad.tbl) ## read is considered 'valid' if there is only one SSP primer matchad.valid.tbl ## read is considered 'ideal' if VNP is in the reverse direction as SSPad.ideal.tbl write.csv(ad.ideal.tbl, row.names=FALSE,  file=gzfile('barcode-adapter_assignments_ideal.csv.gz'), quote=FALSE);write.csv(ad.valid.tbl, row.names=FALSE,  file=gzfile('barcode-adapter_assignments_valid.csv.gz'), quote=FALSE);
?. [Read Orientation]
Demultiplex barcode-demultiplexed reads by SSP direction.Note that the last four values in the 'wide' table refer to the reverse and forward mappings of the SSP and VNP primers respectively). The reverse reads are reverse-complemented with fastx-rc.pl, followed by a final concatenation to simplify the subsequent alignment steps.for bc in $(awk '{print $2}' barcode_counts.txt);   do echo "** ${bc}/fwd **";   mkdir -p demultiplexed/${bc};   pv reads_${bc}.fastq.gz | \     ~/scripts/fastx-fetch.pl -i   gzip > demultiplexed/${bc}/${bc}_reads_fwd.fastq.gz;   echo "** ${bc}/rev **";   pv reads_${bc}.fastq.gz | \     ~/scripts/fastx-fetch.pl -i   fastx-rc.pl | gzip > demultiplexed/${bc}/${bc}_reads_rev.fastq.gz;   pv demultiplexed/${bc}/${bc}_reads_fwd.fastq.gz demultiplexed/${bc}/${bc}_reads_rev.fastq.gz | zcat | \    gzip > demultiplexed/${bc}/${bc}_reads_dirAdjusted.fastq.gz;done
?. [Barcode Demultiplexing]
Demultiplex reads as per protocol Demultiplexing Nanopore reads with LAST.If this has been done, then the following command should produce output without errors:for bc in $(awk '{print $2}' barcode_counts.txt);  do ls demultiplexed/reads_${bc}.fq.gz;doneExample output:demultiplexed/reads_BC03.fq.gzdemultiplexed/reads_BC04.fq.gzdemultiplexed/reads_BC05.fq.gzdemultiplexed/reads_BC06.fq.gzdemultiplexed/reads_BC07.fq.gzdemultiplexed/reads_BC08.fq.gzIf the barcode_counts.txt file is missing, the output will look like this:awk: fatal: cannot open file `barcode_counts.txt' for reading (No such file or directory)If one or more of the barcode-demultiplexed files are missing, the output will look something like this:demultiplexed/reads_BC03.fq.gzdemultiplexed/reads_BC04.fq.gzdemultiplexed/reads_BC05.fq.gzls: cannot access 'demultiplexed/reads_BC06.fq.gz': No such file or directoryls: cannot access 'demultiplexed/reads_BC07.fq.gz': No such file or directorydemultiplexed/reads_BC08.fq.gz
?. [Demultiplex Reads    ]
Demultiplex reads as per the protocol Demultiplexing Nanopore reads with LAST. It is expected that these demultiplexed reads will be split up in the current directory, and coupled with a 'barcode_counts.txt' file. If that's the case, the following should work:for bc in $(awk '{print $2}' barcode_counts.txt);  do ls reads_${bc}.fastq.gz;doneExample expected output:reads_BC03.fastq.gzreads_BC04.fastq.gzreads_BC05.fastq.gzreads_BC06.fastq.gzreads_BC07.fastq.gzreads_BC08.fastq.gzIf the 'barcode_counts.txt' file is not present, this error will appear:awk: fatal: cannot open file `barcode_counts.txt' for reading (No such file or directory)If one or more of the demultiplexed read files is missing, it will look something like this:reads_BC03.fastq.gzreads_BC04.fastq.gzls: cannot access 'reads_BC05.fastq.gz': No such file or directoryls: cannot access 'reads_BC06.fastq.gz': No such file or directoryreads_BC07.fastq.gzreads_BC08.fastq.gz
?. [Orienting Reads]
Map the reads to the adapter sequences using the previously defined substitution matrix. In this case it's important that the direction of mapping is also recorded, so the cut command selects three fields (query name [7], target name [2], mapping direction [10]):for bc in $(awk '{print $2}' barcode_counts.txt);  do echo "** ${bc} **";  lastal -Q 1 -P10 -p bc.mat adapter_seqs.fa     maf-convert -n tab | cut -f 2,7,10 | uniq | \    gzip > demultiplexed/adapter_assignments_${bc}.txt.gzdone
?. [Index Preparation]
Prepare a FASTA file containing adapter sequences (see attached FASTA file).
?. [Index Preparation]
Prepare the LAST index for the adapter file. Following Martin Frith's recommendation, the '-uNEAR' seeding scheme is used to slightly increase sensitivity. This will generate seven additional files of the form .XXX:lastdb -uNEAR adapter_seqs.fa adapter_seqs.fa
?. [Orienting Reads]
The adapter assignments are filtered through uniq in order to catch (and exclude) any reads with the strand-switch primer matching multiple times. To unpack the uniq pipe a little bit more, it skips the first field (adapter name), then matches up to 36 characters, retaining only lines that don't match any others. This catches a few more chimeric reads that were missed by the unique barcode filter in the previous protocol.Reads are filtered into two groups (and one group-by-omission) based on the mapped direction of the strand-switch primer, then reverse-complemented (if necessary) to match the orientation of the original RNA strand. I use my fastx-fetch.pl and fastx-rc.pl scripts for this.mkdir -p orientedfor bc in $(awk '{print $2}' barcode_counts.txt); do echo "** ${bc} **"; fastx-fetch.pl -i      sort | uniq -f 1 -w 36 -u | \     awk '{if($3 == "+"){print $2}}')    gzip > oriented/${bc}_reads_fwd.fq.gz fastx-fetch.pl -i      sort | uniq -f 1 -w 36 -u | \     awk '{if($3 == "-"){print $2}}')    fastx-rc.pl | gzip > oriented/${bc}_reads_rev.fq.gzdone
?. [Orienting Reads]
Forward and reverse-oriented sequences are combined together to form a single group of RNA-oriented reads.for bc in $(awk '{print $2}' barcode_counts.txt);  do echo "** ${bc} **";  pv oriented/${bc}_reads_fwd.fq.gz oriented/${bc}_reads_rev.fq.gz | \    zcat | gzip > oriented/${bc}_reads_dirAdjusted.fq.gzdone
?. [Downstream Workflows]
Following on from here, the oriented reads can be mapped to a genome (e.g. for visual confirmation of mapping), or to a transcriptome (e.g. for read counting):Stranded Mapping from Oriented Long ReadsStranded Transcript Count Table Generation from Long Reads
?. [Index Preparation]
Prepare a substitution matrix for barcode mapping. The default substitution matrix is swayed too much by INDELs in the barcode sequences, so here's one that I've developed using a combination of trial & error and last-train:#last -Q 0
#last -a 10
#last -A 10
#last -b 5
#last -B 5
#last -S 1
# score matrix (query letters = columns, reference letters = rows):
       A      C      G      T
A      4    -24     -9    -24
C    -24      5    -24    -14
G     -9    -24      7    -24
T    -24    -14    -24      8 
[note: this is the same matrix as used for demultiplexing]