Package: pycroquet
pyCROQUETpython Crispr Read to Oligo QUantification and Evaluation ToolpyCROQUETPublicationsGeneralSubcommandsOptionsguidelibquerieschunksrulesOutput filesCRAMReads mapping to asgrna_idReads assigned to a guideDual guideStatistic file extensionBoundary mode detailsexactTinQ- target in queryQinT- query in targetanyViewing alignmentsInstallationPypiDocker and SingularityDevelopmentLinuxMacTestingLocalvenvtestingLocalpre-commithooksDocker testingCI testsUpdating licence headersLICENSEPublicationsPlease contact the following for appropriate referencing methods:Keiran Raine (kr2@sanger.ac.uk)Emre Karakoc (ek11@sanger.ac.uk)Victoria Offord (vo1@sanger.ac.uk)GeneralCode in place to support read input from any of the following formats:fastq (also gzip compressed)sambamcramSubcommandssingle-guideShort single end read quantification.dual-guidePaired end read quantification.long-readLong single end read quantificationguides-to-faConvert guides to fasta for use withsamtools tviewOptionsAll options are not applicable to all subcommands, however the majority are common.guidelibPlease see theGuide library formatfor a description of this file.queriesCurrently thedual-guidemode only supports SAM/BAM/CRAM as input.  Convert fastq to unmapped CRAM with:# if data has casava read barcode/qc (text after space in read name) please add "-i"
samtools import  --output-fmt CRAM,no_ref=1 -@ 4 -1 $READ1 -2 $READ2 -o $OUTFILE.cramchunksChunks should be set to a value that allows all CPUs to be utilized.  The value is multiplied by the number of
CPUs requested and this give the number of unique read sequences held in memory during the mapping phase.This has a direct impact on memory. The value is automatically reduced when too large to allow full use of requested CPUs.rulesFor single-guide--rules MM(allow 2 mismatches in alignment) is a sensible value.  For other subcommands the decision
is dependent on the library protocol.Rules have a direct impact on run time as they increase the time taken to abort an alignment, individual costs are as follows:M= 1I= 2 (single b.p.)D= 2 (single b.p.)Performance is only impacted by the maximum penalty you allow.Be aware if you with to allow up to2 mismatchor1 mismatch + 1 b.p. insertyou must specify:pycroquet ... --rules MM --rules MIOutput filesCRAMFor single-guide you have the option to suppress it via the--no-alignment(-n) option.  In dual-guide it is tightly
linked to the pairing code so not possible to disable.Reads that map uniquely are written withMAPQ>0(score calculations have not been refined at this time).  There are some
differences in how to interpret the data depending on if you are processing single-guide, dual-guide or long-read.Reads mapping to asgrna_idTo get the reads that map uniquely to a guide element (sgrna_seq) use thesgrna_id.  This is primarily of use forsingle-guideandlong-read:samtoolsview-F4-q1result.cram$SGRNA_IDTo get a single instance of reads that map to a guide element but map equally well to others select for theSAtag (requires samtools>=1.12):samtoolsview-F4-F256-dSAresult.cram$SGRNA_IDTo get reads that failed to map:samtoolsview-f4result.cramReads assigned to a guideThis is only applicable todual-guide.You can pull reads by the guide id usingsamtools view, this example counts R1 mapping to a guide (equivalent to the*.counts.tsvresult), replace/set$GUIDE_IDas required:samtoolsview-F4-f64-c-dYG:$GUIDE_IDresult.cramTo select all the reads mapped to this guide grouped by readname:samtoolsview-u-F4-dYG:$GUIDE_IDresult.cram|samtoolssort-n-|samtoolsview-b->$GUIDE_ID.bamDual guideFASTQ(.gz) input is not currently supported for dual guide, please prepare your data appropriately withsamtools import:samtoolsimport-@4-1R1.fastq.gz-2R2.fastq.gz-OBAM-oOUTPUT.bamPlease review theimportoptions as casava information can be interpreted where appropriate.Statistic file extensionThe dual guide output extends the standard json statistics file addingpair_classifications:ClassificationDescriptionmatchsame vector F/Raberrant_matchsame vector pair, aberrant orientationf_multi_3p5p mapped F, 3p multihitf_multi_5p3p mapped F, 5p multihitr_multi_3p5p mapped R, 3p multihitr_multi_5p3p mapped R, 5p multihitf_open_3p5p mapped F, 3p open (unmapped)f_open_5p3p mapped F, 5p open (unmapped)r_open_3p5p mapped R, 3p open (unmapped)r_open_5p3p mapped R, 5p open (unmapped)swapmulti vector, uniq mappedambiguousboth ends multi hitno_matchmulti/unmapped either endBoundary mode detailsThe-b/--boundary-modeoption controls how the guide and read are allowed to overlap.  Each section shows the types of
alignment allowed, to be valid they still need to pass rules and any minimum score.In all casesXXXindicates original sequence.exactBoundary of sequence must be equal between target (guide) and query (read)T: XXXXXX
Q: XXXXXXTinQ- target in queryLike the name suggests, valid alignments include those viaexactand:T: XXXXX
Q: XXXXXX

T:  XXXX
Q: XXXXXX

T:  XXXXX
Q: XXXXXXQinT- query in targetReverse ofTinQ, valid alignments include those viaexactand:T: XXXXXX
Q: XXXXX

T: XXXXXX
Q:  XXXX

T: XXXXXX
Q:  XXXXXanyNo boundary checks are performed, this allows more complex events, all alignments fromexact,TinQ,QinTplus:T:   XXXXXXX
Q: XXXXXXX

T: XXXXXXX
Q:   XXXXXXXViewing alignmentsYou can use samtools tview to view the cram file, this is mainly useful when checking fuzzy matching or allowing all boundary types.To make this more informative generate the fasta file for the sgrna elements:pycroquet guides-to-fa --guidelib guide_library.tsv --fasta sgrna.faNOTE: a contig is the individual sgrna sequence, not the pairNow use this withsamtool tviewto view your alignments, see command line help to jump directly to a contig of interest
or?when using interactively.samtools tview result.cram sgrna.faWill give a full screen output like this (N is just padded to screen width for short contigs):1         11
CTAGTTCAGATAAAACAACNNNNNN
...................
...................
...................
...................
...................
................C..
...................
...................InstallationPypipip install Cython
pip install pycroquetDocker and SingularityThere are pre-built images containing this codebase onquay.io.  When pulling an image you must specify
the version there is nolatest.The docker images are known to work correctly after import into a singularity image.DevelopmentPython 3.9 or better required.Linuxgitclonegit@github.com:cancerit/pycroquet.gitcdpycroquet
python3-mvenvvenvsourcevenv/bin/activate
pipinstall-rrequirements-dev.txt
python3./setup.pydevelop# dynamic build# add/activate pre-commitpipinstallpre-commit
pre-comomitinstallMacbrewupdate
brewinstallpython@3.9
brewinstalllibmagic
gitclonegit@github.com:cancerit/pycroquet.gitcdpycroquet
python3.9-mvenvvenvsourcevenv/bin/activate
pipinstall-rrequirements-dev.txt
python3setup.shdevelop# add/activate pre-commitpipinstallpre-commit
pre-comomitinstallTestingThere are 4 layers to testing and standards:LocalvenvtestingLocalpre-commithooksTests embedded indocker buildCItestsLocalvenvtesting./tests/scripts/run_unit_tests.shAlso confirm the distribution can be installed by building and installing it into a different venv:rm-rfdist/
python3setup.pysdist# new terminalpython3-mvenvtmp-pycroquet-venvsourcetmp-pycroquet-venv/bin/activate
pipinstall~/pycroquet/dist/pycroquet-*.tar.gz
deactivate
rm-rftmp-pycroquet-venvLocalpre-commithooksThis project additionally uses git pre-commit hooks via thepre-commit tool.  These are concerned
with file formats and standards, not the actual execution of code.  See./.pre-commit-config.yaml.Docker testingThe Docker build includes the unit tests, but removes many of the libraries before the final build stage.  Mainly for CI tests.CI testsCI includes 2 additional tests, each based on the 2 datasets in the./examplesdirectory.Updating licence headersPlease useskywalking-eyes.Expected workflow:Check state before modifying.licenserc.yaml:docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header checkYou should get some 'valid' here, those without a header as 'invalid'Modify.licenserc.yamlApply the changes:docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header fixAdd/commit changesThis is executed in the CI pipeline.DO NOTedit the header in the files, please modify the date component ofcontentin.licenserc.yaml.  The only exception being:README.mdIf you need to make more extensive changes to the license carefully test the pattern is functional.LICENSECopyright (c) 2021-2022

Author: CASM/Cancer IT <cgphelp@sanger.ac.uk>

This file is part of pycroquet.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

1. The usage of a range of years within a copyright statement contained within
this distribution should be interpreted as being equivalent to a list of years
including the first and last year specified and all consecutive years between
them. For example, a copyright statement that reads ‘Copyright (c) 2005, 2007-
2009, 2011-2012’ should be interpreted as being identical to a statement that
reads ‘Copyright (c) 2005, 2007, 2008, 2009, 2011, 2012’ and a copyright
statement that reads ‘Copyright (c) 2005-2012’ should be interpreted as being
identical to a statement that reads ‘Copyright (c) 2005, 2006, 2007, 2008,
2009, 2010, 2011, 2012’.