Package: phip-stat
phip-stat: tools for analyzing PhIP-seq dataNOTE: This project is no longer being maintained. Please see the phippery, phip-flow, and related projects maintained by Erick Matsen's group:https://github.com/matsengrp/phipperyhttps://github.com/matsengrp/phip-flowThe PhIP-seq assay was first described inLarman et.
al.. This repo contains code for
processing raw PhIP-seq data into analysis-ready enrichment scores.This code also implements multiple statistical models for processing PhIP-seq
data, including the model described in the original Larman et al paper
(generalized-poisson-model).  We currently recommend using one of the newer
models implemented here (e.g.,gamma-poisson-model).Please submitissuesto
report any problems.Installationphip-stat runs on Python 3.6+ and minimally depends on click, tqdm, numpy, scipy,
and pandas. The matrix factorization model also requires tensorflow.pipinstallphip-stator to install the latest development version from GitHubpipinstallgit+https://github.com/lasersonlab/phip-stat.gitUsageThe overall flow of the pipeline isalign — for each sample count the number of reads derived from each
possible library membermerge — combine the count values from all samples into a single count
matrixmodel — normalize counts and train a model to compute enrichment
scores/hitsAn entire NextSeq run with 500M reads can be processed in <30 min on a 4-core
laptop (if aligning with a tool like kallisto).Command-line interfaceAll the pipeline tools are accessed through thephipexecutable. All
(sub)command usage/options can be obtained by passing-h.$ phip -h
Usage: phip [OPTIONS] COMMAND [ARGS]...

  phip -- PhIP-seq analysis tools

Options:
  -h, --help  Show this message and exit.

Commands:
  align-parts     align fastq files to peptide reference
  compute-counts  compute counts from aligned bam file
  compute-pvals   compute p-values from counts
  groupby-sample  group alignments by sample
  join-barcodes   annotate Illumina reads with barcodes Some...
  merge-columns   merge tab-delim files
  split-fastq     split fastq files into smaller chunksExample pipeline 1: kallisto alignment followed by Gamma-Poisson modelThis pipeline will use kallisto to pseudoalign the reads to the reference.
Because the output of each alignment step is a directory, the merge step uses a
CLI tool designed for this directory structure. The counts are also
pre-normalized.# 1. alignkallistoquant--single--plaintext--fr-stranded-l75-s0.1-t4\-ireference.idx-osample_counts/sample1sample1.fastq.gz# ...kallistoquant--single--plaintext--fr-stranded-l75-s0.1-t4\-ireference.idx-osample_counts/sampleNsampleN.fastq.gz# 2. mergephipmerge-kallisto-tpm-isample_counts-ocpm.tsv# 3. modelphipgamma-poisson-model-t99.9-icpm.tsv-ogamma-poissonExample pipeline 2: exact-matching reads followed by matrix factorizationThis pipeline will match each read to the reference exactly (or a chosen subset
of the read) followed by merging into a single matrix. The matrix is then
factored with a low-rank approximation (allowing for clipping) and "hits" are
called with a heuristic.# 1. alignphipcount-exact-matches-rreference.fasta-l75-osample_counts/sample1.counts.tsvsample1.fastq.gz# ...phipcount-exact-matches-rreference.fasta-l75-osample_counts/sampleN.counts.tsvsampleN.fastq.gz# 2. mergephipmerge-columns-miter-isample_counts-ocounts.tsv# 3. modelphipclipped-factorization-model--rank2-icounts.tsv-oresiduals.tsv
phipcall-hits-iresiduals.tsv-ohits.tsv--beads-regex".*BEADS_ONLY.*"Example pipeline 3: bowtie2 alignment followed by normalization and Gamma-PoissonThis example uses bowtie2, which should give the maximum sensitivity at the
expense of speed. The main bowtie2 command accomplishes the following: align
reads to reference, sort and convert to BAM, compute coverage depth at each
position of each clone, for each clone take only the largest number observed,
finally sort by clone identifier.# 1. alignecho"id\tsample1">sample_counts/sample1.tsv
bowtie2-p4-xreference_index-Usample1.fastq.gz\|samtoolssort-OBAM\|samtoolsdepth-aa-m100000000-\|awk'BEGIN {OFS="\t"} {counts[$1] = ($3 < counts[$1]) ? counts[$1] : $3} END {for (c in counts) {print c, counts[c]}}'\|sort-k1\>>sample_counts/sample1.tsv# ...echo"id\tsampleN">sample_counts/sampleN.tsv
bowtie2-p4-xreference_index-UsampleN.fastq.gz\|samtoolssort-OBAM\|samtoolsdepth-aa-m100000000-\|awk'BEGIN {OFS="\t"} {counts[$1] = ($3 < counts[$1]) ? counts[$1] : $3} END {for (c in counts) {print c, counts[c]}}'\|sort-k1\>>sample_counts/sampleN.tsv# 2. merge -- NOTE: this performs a pandas outer join and loads all counts into memoryphipmerge-columns-mouter-isample_counts-ocounts.tsv# 3. modelphipnormalize-counts-msize-factors-icounts.tsv-onormalized_counts.tsv
phipgamma-poisson-model-t99.9-inormalized_counts.tsv-ogamma-poissonSnakemake recipesWe include several example Snakemake recipes for easily processing large sets of
samples at once, e.g.,workflows/example-kallisto-GamPois-factorization.snakefile. In general the
configuration section must be edited to specify the location of the raw
sequencing data.Running unit testsUnit tests use thenosepackage and can be run with:$ pip install nose  # if not already installed
$ nosetests -sv test/