Repository: HobnobMancer/cazy_webscraper (language: python)

Issue:
Add retrieving data from GenBank and PDB
Update the expand module to bulk insert data from GenBank and PDB into the CAZyme db.


Discussion / hints:

Fix patch:
diff --git a/README.md b/README.md
index 5a00be08..7672eaf2 100644
--- a/README.md
+++ b/README.md
@@ -28,7 +28,7 @@ Using the `expand` subcommand, a user can retrieve:
 
 ## Citation
 
-If you use `cazy_webscraper, please cite the following publication:
+If you use `cazy_webscraper`, please cite the following publication:
 
 > Hobbs, Emma E. M.; Pritchard, Leighton; Chapman, Sean; Gloster, Tracey M. (2021): cazy_webscraper Microbiology Society Annual Conference 2021 poster. FigShare. Poster. [https://doi.org/10.6084/m9.figshare.14370860.v7](https://doi.org/10.6084/m9.figshare.14370860.v7)
 
@@ -40,9 +40,21 @@ If you use `cazy_webscraper, please cite the following publication:
 - [Documentation](#documentation)
     - [Installation](#installation)
     - [Quick start](#quick-start)
-    - [Creating a local CAZyme database](#creating-a-local-cazyme-database)
+- [Creating a local CAZyme database](#creating-a-local-cazyme-database)
+    - [Combining configuration filters](#combining-configuration-filters)
     - [Default CAZy class synonyms](#default-cazy-class-synonyms)
-    - [Retrieving protein sequences and structure files](#retrieving-protein-sequences-and-structure-files)
+- [Retrieve data from UniProt](#retrieve-data-from-uniprot)
+    - [Configuring UniProt data retrieval](#configuring-uniprot-data-retrieval)
+- [Retrieving protein sequences from GenBank](#retrieving-protein-sequences-from-genbank)
+    - [Configuring GenBank protein sequence data retrieval](#configuring-genbank-protein-sequence-retrieval)
+- [Extracting protein sequences from the local CAZyme database and building a BLAST database](#extracting-protein-sequences-from-the-local-cazyme-database-and-building-a-blast-database)
+    - [Configuring extracting sequences from a local CAZyme db](#configuring-extracting-sequences-from-a-local-cazyme-db)
+- [Retrieving protein structure files from PDB](#retrieving-protein-structure-files-from-pdb)
+    - [Configuring PDB protein structure file retrieval](#configuring-pdb-protein-structure-file-retrieval)
+- [The `cazy_webscraper` API or Interrogating the local CAZyme database](#the_cazy_webscraper_api_or_interrogating_the_local_cazyme_database)
+- [Configuring `cazy_webscraper` using a YAML file](#configuring-using-a-yaml-file)
+- [CAZy coverage of GenBank](#cazy-coverage-of-genbank)
+    - [Configure calculating CAZy coverage of GenBank](#configure-calculating-cazy-coverage-of-genbank)
 - [Contributions](#contributions)
 - [License and copyright](#license-and-copyright)
 <!-- /TOC -->
@@ -57,7 +69,7 @@ Please see the [full documentation at ReadTheDocs](https://cazy-webscraper.readt
 
 ### Installation
 
-`cazy_webscraper` can be installed *via* `conda` or `pip`:
+`cazy_webscraper` can be installed via `conda` or `pip`:
 
 ```bash
 conda install -c bioconda cazy_webscraper
@@ -88,22 +100,25 @@ Options are written in alphabetical order.
 
 `--cache_dir` - Path to cache dir to be used instead of default cache dir path.
 
+`--cazy_data` - Path to a txt file downloaded from CAZy containing a CAZy db data dump.
+
 `--cazy_synonyms` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
 
+`--classes` - list of classes from which all families are to be scrape.
+
 `--config`, `-c` - Path to a configuration YAML file. Default: None.
 
 `--citation`, `-C` - Print the `cazy_webscraper` citation. When called, the program terminates after printng the citation and CAZy is **not** scraped.
 
-`--classes` - list of classes from which all families are to be scrape.
-
-`--database`, `-D` - Path to an **existings** local CAZyme database to add newly scraped too. Default: None.
+`--database`, `-d` - Path to an **existings** local CAZyme database to add newly scraped too. Default: None.
 
-`--db_output`, `-d` - Path to write out a **new** local CAZyme database to. Include the name of the new database, including the `.db` extension. Default: None.
 
 _Do not use `--db_output` **and** `--database` at the same time._
 
 _If `--db_output` **and** `--database` are **not** called, `cazy_webscraper` write out a local CAZyme database to the cwd with the standardised name `cazy_webscraper_<date>_<time>.db`_
 
+`--delete_old_relationships` - Detele old CAZy family annotations of GenBank accessions. These are CAZy family annotations of a given GenBank accession are in the local database but the accession is not longer associated with those CAZy families, so delete old accession-family relationships.
+
 `--families` - List of CAZy (sub)families to scrape.
 
 `--force`, `-f` - force overwriting existing output file. Default: False.
@@ -134,6 +149,15 @@ _If `--db_output` **and** `--database` are **not** called, `cazy_webscraper` wri
 
 `--version`, `-V` - Print `cazy_webscraper` version number. When called and the version number is printed, `cazy_webscraper` is immediately terminated.
 
+### Combining configuration filters
+
+`cazy_webscraper` applies filters in a successive and layered structure.
+
+CAZy class and family filters are applied first.
+
+Kingdom filters are applied second.
+
+Lastly, taxonomy (genus, species and strain) filters are applied.
 
 ### Default CAZy class synonyms
 
@@ -145,9 +169,432 @@ Spaces, hythens, underscores and no space or extract character can be used in th
 
 Class names can be written in all upper case, all lower case, or mixed case, such as GLYCOSIDE-HYDROLASES, glycoside hydrolases and Glycoside Hydrolases. All lower or all upper case CAZy class name abbreviations (such as GH and gh) are accepted.
 
-## Retrieving protein sequences and structure files
+## Retrieve data from UniProt
+
+[UniProtKB] is one of the largest protein database, incorporating data from the [PDB] structure database and other protein annotation databases.
+
+`cazy_webscraper` can retrieve protein data from UniProt for proteins catalogued in a local CAZyme database created using `cazy_webscraper`. Specifically, for each protein, `cazy_webscraper` can retrieve:
+- The UniProt accession
+- PDB accessions of associated structure files from the PDB database
+- EC number annnotations
+- Protein sequence from the UniProt
+
+`cazy_webscraper` always retrieves the UniProt accession, but the retrieval of PDB accession, EC numbers and protein sequences is optional.
+
+Data can be retrieived for all proteins in the local CAZyme database, or a specific subset. CAZy class, CAZy family, genus, species, strains, kingdom and EC number filters can be defined in order to define a dataset to retrieve protein data from UniProt for.
+
+To retrieve all UniProt data for all proteins in a local CAZyme datbase, using the following command:
+```bash
+cw_get_uniprot_data <path_to_local_CAZyme_db> --ec --pdb --seq
+```
+
+### Configuring UniProt data retrieval
+
+Below are listed the command-line flags for configuring the retrieval of UniProt data.
+
+`database` - \[REQUIRED\] Path to a local CAZyme database to add UniProt data to.
+
+`--bioservices_batch_size` - Change the query batch size submitted via [`bioservices`](https://bioservices.readthedocs.io/en/master/) to UniProt to retrieve protein data. Default is 150. `bioservices` recommands queries not larger than 200 objects.
+
+`--cache_dir` - Path to cache dir to be used instead of default cache dir path.
+
+`--cazy_synonyms` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
+
+`--classes` - List of classes to retrieve UniProt data for.
+
+`--config`, `-c` - Path to a configuration YAML file. Default: None.
+
+`--ec`, `-e` - Enable retrieval of EC number annotations from UniProt
+
+`--ec_filter` - Limist retrieval of protein data to proteins annotated with a provided list of EC numbers. Separate the EC numbers bu single commas without spaces. Recommend to wrap the entire str in quotation marks, for example:
+```bash
+cw_get_uniprot_data my_cazyme_db/cazyme_db.db --ec_filter 'EC1.2.3.4,EC2.3.1.-'
+```
+
+`--families` - List of CAZy (sub)families to scrape.
+
+`--genera` - List of genera to restrict the scrape to. Default: None, filter not applied to scrape.
+
+`--log`, `-l` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
+
+`--nodelete_cache` - When called, content in the existing cache dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--nodelete_log` - When called, content in the existing log dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--retries`, `-r` - Define the number of times to retry making a connection to CAZy if the connection should fail. Default: 10.
+
+`--sequence`, `-s` - Retrieve protein amino acid sequences from UniProt
+
+`--sql_echo` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
+
+`--species` - List of species written as Genus Species) to restrict the scraping of CAZymes to. CAZymes will be retrieved for **all** strains of each given species.
+
+`--strains` - List of specific species strains to restrict the scraping of CAZymes to.
+
+`--timeout`, `-t` - Connection timout limit (seconds). Default: 45.
+
+`--uniprot_batch_size` - Size of an individual batch query submitted to the [UniProt REST API](https://www.uniprot.org/help/programmatic_access) to retrieve the UniProt accessions of proteins identified by the GenBank accession. Default is 150. The UniProt API documentation recommands batch sizes of less than 20,000 but batch sizes of 1,000 often result in HTTP 400 errors. It is recommend to keep batch sizes less than 1,000, and ideally less than 200.
+
+`--seq_update` - If a newer version of the protein sequence is available, overwrite the existing sequence for the protein in the database. Default is false, the protein sequence is **not** overwritten and updated.
+
+`--verbose`, `-v` - Enable verbose logging. This does not set the SQLite engine `echo` parameter to True. Default: False.
+
+## Retrieveing protein seqences from GenBank
+
+Protein amino acid sequences can be retrieved for proteins in a local CAZyme database using `cazy_webscraper`. Protein sequences can be retrieved for a specific subset of proteins, identified through the use of CAZy class, CAZy family, taxonomy (kingdom, genus, species and strain) filters, and EC number filters. The retrieved protein sequences are written to the local CAZyme database.
+
+_Extracting protein sequences from the local CAZyme database and writing them to a BLAST database and/or FASTA file(s) is covered in the next section._
+
+To retrieve all GenBank protein seuqneces for all proteins in a local CAZyme datbase, using the following command:
+```bash
+cw_get_genbank_seqs <path_to_local_CAZyme_db>
+```
+
+`cazy_webscraper` produces to cache files, which are written to the cache dir:
+1. `no_seq_retrieved.txt` which lists the GenBank accessions for which no sequence could be retrieved from GenBank
+2. `seq_retrieved.txt` which list GenBank accessiosn for which a sequence was retrieved from GenBank
+
+### Configuring GenBank protein sequence retrieval
+
+Below are listed the command-line flags for configuring the retrieval of protein sequences from GenBank.
+
+`database` - \[REQUIRED\] Path to a local CAZyme database to add UniProt data to.
+
+`email` - \[REQUIRED\] User email address, required by NCBI Entrez.
+
+`--cache_dir` - Path to cache dir to be used instead of default cache dir path.
+
+`--cazy_synonyms` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
+
+`--config`, `-c` - Path to a configuration YAML file. Default: None.
+
+`--classes` - List of classes from which all families are to be scrape.
+
+`--ec_filter` - Limist retrieval of protein data to proteins annotated with a provided list of EC numbers. Separate the EC numbers bu single commas without spaces. Recommend to wrap the entire str in quotation marks, for example:
+```bash
+cw_get_uniprot_data my_cazyme_db/cazyme_db.db --ec_filter 'EC1.2.3.4,EC2.3.1.-'
+```
+
+`--entrez_batch_size` - Change the query batch size submitted via [`Entrez`]() to retrieve protein sequences from GenBank data. Default is 150. `Entrez` recommands queries not larger than XXX objects in length.
+
+`--families` - List of CAZy (sub)families to scrape.#
+
+`--kingdoms` - List of taxonomy kingdoms to retrieve UniProt data for.
+
+`--genera` - List of genera to restrict the scrape to. Default: None, filter not applied to scrape.
+
+`--log`, `-l` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
+
+`--nodelete_cache` - When called, content in the existing cache dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--retries`, `-r` - Define the number of times to retry making a connection to CAZy if the connection should fail. Default: 10.
+
+`--seq_update` - If a newer version of the protein sequence is available, overwrite the existing sequence for the protein in the database. Default is false, the protein sequence is **not** overwritten and updated.
+
+`--sql_echo` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
+
+`--species` - List of species written as Genus Species) to restrict the scraping of CAZymes to. CAZymes will be retrieved for **all** strains of each given species.
+
+`--strains` - List of specific species strains to restrict the scraping of CAZymes to.
+
+`--verbose`, `-v` - Enable verbose logging. This does not set the SQLite engine `echo` parameter to True. Default: False.
+
+## Extracting protein sequences from the local CAZyme database and building a BLAST database
+
+Protein sequences from GenBank and UniProt that are stored in the local CAZyme database can be extracted using `cazy_webscraper`, and written to any combination of:
+- 1 FASTA file per unique protein
+- A single FASTA file containing all extracted seqences
+- A BLAST database
+
+**FASTA file format:** Protein sequences extracted from a local CAZyme database are written out with the GenBank/UniProt accession as the protein ID, and the name of the source database ('GenBank' or 'UniProt') as the description.
+
+To extract all protein seqeunces from the local CAZyme database using the following command structure:
+```bash
+cw_extract_sequences <path_to_local_CAZyme_db> --genbank --uniprot
+```
+
+### Configuring extracting sequences from a local CAZyme db
+
+Below are listed the command-line flags for configuring the extraction of protein sequences from the local CAZyme db.
+
+`database` - \[REQUIRED\] Path to a local CAZyme database to add UniProt data to.
+
+`-g`, `--genbank` - Extract protein sequences retrieved from GenBank
+
+`-u`, `--uniprot` - Extract protein sequences retrieved from UniProt.
+
+_Note: At least one of `--genbank` and `--uniprot` must be called, otherwise not protein sequences will be extracted._
+
+`-b`, `--blastdb` - Create BLAST database of extracted protein sequences. Provide the path to the directory to store the BLAST database in.
+
+`--fasta_dir` - Write out each extracted sequence to a separate FASTA file in the provided dir. Provide a path to a directory to write out the FASTA files.
+
+`--fasta_file` - Write out all extracted sequences to a single FASTA file. Provide a path to write out the FASTA file.
+
+_Note: at least one of `--blastdb`, `--fasta_dir`, and `--fasta_file` must be called to inform `cazy_webscraper` where to write the output to. If none are called sequences will be extracted._
+
+`--cache_dir` - Path to cache dir to be used instead of default cache dir path.
+
+`--cazy_synonyms` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
+
+`--config`, `-c` - Path to a configuration YAML file. Default: None.
+
+`--classes` - List of classes from which all families are to be scrape.
+
+`--ec_filter` - Limist retrieval of protein data to proteins annotated with a provided list of EC numbers. Separate the EC numbers bu single commas without spaces. Recommend to wrap the entire str in quotation marks, for example:
+```bash
+cw_get_uniprot_data my_cazyme_db/cazyme_db.db --ec_filter 'EC1.2.3.4,EC2.3.1.-'
+```
+
+`--families` - List of CAZy (sub)families to scrape.#
+
+`--kingdoms` - List of taxonomy kingdoms to retrieve UniProt data for.
+
+`--genera` - List of genera to restrict the scrape to. Default: None, filter not applied to scrape.
+
+`--log`, `-l` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
+
+`--nodelete` - When called, content in the existing output dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--nodelete_cache` - When called, content in the existing cache dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--sql_echo` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
+
+`--species` - List of species written as Genus Species) to restrict the scraping of CAZymes to. CAZymes will be retrieved for **all** strains of each given species.
+
+`--strains` - List of specific species strains to restrict the scraping of CAZymes to.
+
+`--verbose`, `-v` - Enable verbose logging. This does not set the SQLite engine `echo` parameter to True. Default: False.
+
+
+## Retrieving protein structure files from PDB
+
+`cazy_webscraper` can retrieve protein structure files for proteins catalogued in a local CAZyme database. Structure files can be retrieved for all proteins in the database or a subset of proteins, chosen by defining CAZy class, CAZy family, taxonomy (kingdom, genus, species and strain) filters, and EC number filters.
+
+Retrieval of structure files from PDB is performed by the `BioPython` module `PDB` [Cock _et al._, 2009], which writes the downloaded structure files to the local disk. Therefore, the downloaded structure files are **not** stored in the local CAZyme database at the present.
+
+> Cock, P. J. A, Antao, T., Chang, J. T., Chapman, B. A., Cox, C. J., Dalke, A. _et al._ (2009) 'Biopython: freely available Python tools for computaitonal molecular biology and bioinformatics', _Bioinformatics_, 25(11), pp. 1422-3.
+
+To retrieve structure files for all proteins in a local CAZyme database in `mmCif` and `pdb` format, use the following command:
+```bash
+cw_get_pdb_structures <path_to_local_CAZyme_db> mmcif,pdb
+```
+
+### Configuring PDB protein structure file retrieval
+
+Below are listed the command-line flags for configuring the retrieval of protein structure files from PDB.
+
+`database` - \[REQUIRED\] Path to a local CAZyme database to add UniProt data to.
+
+`pdb` \[REQUIRED\] The file types to be retrieved from PDB. The following file types are supported:  
+- `mmCif`
+- `pdb`
+- `xml`
+- `mmft`
+- `bundle`
+To chose multiple file types, list all desired file types, separting the files using a single comma. For example:
+```bash
+cw_get_genbank_seq my_cazyme_db/cazyme_db.db mmcif,pdb,xml
+```
+Providing the file types is **not** case sensitive, and the order the file types are listed does **not** matter.
+
+`--cache_dir` - Path to cache dir to be used instead of default cache dir path.
+
+`--cazy_synonyms` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
+
+`--config`, `-c` - Path to a configuration YAML file. Default: None.
+
+`--classes` - List of classes from which all families are to be scrape.
+
+`--ec_filter` - Limist retrieval of protein data to proteins annotated with a provided list of EC numbers. Separate the EC numbers bu single commas without spaces. Recommend to wrap the entire str in quotation marks, for example:
+```bash
+cw_get_uniprot_data my_cazyme_db/cazyme_db.db --ec_filter 'EC1.2.3.4,EC2.3.1.-'
+```
+
+`--families` - List of CAZy (sub)families to scrape.
+
+`--genera` - List of genera to restrict the scrape to. Default: None, filter not applied to scrape.
+
+`--log`, `-l` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
+
+`--nodelete` - When called, content in the existing output dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--nodelete_cache` - When called, content in the existing cache dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--outdir`, `-o` - Output directory to write out downloaded protein structure files to. Default is to write out the downloaded structure files to the current working directory.
+
+`--retries`, `-r` - Define the number of times to retry making a connection to CAZy if the connection should fail. Default: 10.
+
+`--sql_echo` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
+
+`--species` - List of species written as Genus Species) to restrict the scraping of CAZymes to. CAZymes will be retrieved for **all** strains of each given species.
+
+`--strains` - List of specific species strains to restrict the scraping of CAZymes to.
+
+`--timeout`, `-t` - Connection timout limit (seconds). Default: 45.
+
+`--verbose`, `-v` - Enable verbose logging. This does not set the SQLite engine `echo` parameter to True. Default: False.
+
+
+## The `cazy_webscraper` API or Interrogating the local CAZyme database
+
+The SQLite3 database compiled by `cazy_webscraper` can be interrogated in the native interface (i.e. queries written in SQL can be used to interrogate the database). This can be achieved via the command-line or via an SQL database browser (such as [DB Browser for SQLite](https://sqlitebrowser.org/)).
+
+`cazy_webscraper` also provides its own API (Application Programming Interface) for interrogating the local CAZyme database: `cw_query_database`. The API faciliates the intergration of the dataset in the local CAZyme database into downstream bioinformatic pipelines, _and_ provides a method of interrograting the dataset for those who do not use SQL.
+
+`cw_query_database` is the command that can be used to interrogate the dataset in the local CAZyme database, and extract protein data of interest for the proteins matching the user's cirteria of 
+interest.
+
+By default `cw_query_database` retrieves only the GenBank accessions of proteins matching the user's criteria of interest. If not criteria of interest are provided, all GenBank accessions are retrieved. Optional flags can be applied to retrieve additional data about CAZymes that match the user's criteria of interest.
+
+`cw_query_database` currently supports writing the output in two file formats:
+- `json`, by default this is written to the current working directory and with name `<database_name>_<retrieved_data>_YYYY-MM-DD_HH-MM-SS.json`
+- `csv`, by default this is written to the current working directory and with name `<database_name>_<retrieved_data>_YYYY-MM-DD_HH-MM-SS.csv`
 
-The `expand` subcommand is used to update a local CAZy database. It manages retrieval of CAZyme protein sequences from GenBank and protein structure files from RCSB/PDB.
+`cw_query_database` takes two positional arguments:
+1. The path to the local CAZyme database
+2. The file formats of the output files, presented as a list with each file type separated by as single comma (e.g. `json,csv`). This is **not** case sensitive and the order does **not** matter.
+For example, to retrieve all GenBank accessions for all proteins in the local CAZyme database, and 
+write them to a `json` file, the following command could be used for a database called `cazy.db`:
+```bash
+cw_query_database cazy.db json
+```
+
+### Configuring interrogating the local CAZyme database
+
+Below are listed the command-line flags for configuring the interrogation of the local CAZyme database.
+
+`database` - \[REQUIRED\] Path to a local CAZyme database to add UniProt data to.
+
+`file_types` - \[REQUIRED\] file types to write the interrogation output to. Accepted file types are
+`JSON` and `CSV`. These are *not* case sensitive, and the order does not matter.
+
+`--cazy_synonyms` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
+
+`--config`, `-c` - Path to a configuration YAML file. Default: None.
+
+`--class` - Include a 'Class' column in the output `csv` file, listing the CAZy class of all retrieved CAZymes
+
+`--classes` - List of classes from which all families are to be retrieval.
+
+`--ec` - Included EC number annotations in the output
+
+`--ec_filter` - Limist retrieval of protein data to proteins annotated with a provided list of EC numbers. Separate the EC numbers bu single commas without spaces. Recommend to wrap the entire str in quotation marks, for example:
+```bash
+cw_get_uniprot_data my_cazyme_db/cazyme_db.db --ec_filter 'EC1.2.3.4,EC2.3.1.-'
+```
+
+`--family` - Include CAZy family annotations in the output.
+
+`--families` - List of CAZy (sub)families to retrieve CAZymes from. This includes families and SUBfamilies.
+
+`--genus` - Include a 'Genus' column in the output `csv` file, listing the genus for each retrieved CAZymes
+
+`--genera` - List of genera to restrict the retrieval to. Default: None, filter not applied to scrape.
+
+`--log`, `-l` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
+
+`--nodelete` - When called, content in the existing output dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--organism` - Include the scientific (organism) name of the source organis
+
+`--output`, `-o` - Output path to write the compiled `csv` file. Default is to write out the `csv` file to the current working directory.
+
+`--seq_genbank` - Add the protein sequences retrieved from GenBank to the query output
+
+`--seq_uniprot` - Add the protein sequences retrieved from GenBank to the query output
+
+`--sql_echo` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
+
+`--species` - List of species written as (Genus Species) to restrict the retrieval of CAZymes to. CAZymes will be retrieved for **all** strains of each given species.
+
+`--strains` - List of specific species strains to restrict the retrieval of CAZymes to.
+
+`--subfamily` - Include the CAZy subfamily annotations for proteins matching the criteria of interest.
+
+`--uniprot` - Include the UniProt protein accession and protein name in the output.
+
+`--verbose`, `-v` - Enable verbose logging. This does not set the SQLite engine `echo` parameter to True. Default: False.
+
+
+## Configuring `cazy_webscraper` using a YAML file
+
+The retrieval of data from CAZy, UniProt, GenBank and PDB can be configured at the command-line **and** via a YAML file.
+
+The YAML file must have the following structure, specifically the YAML file must have the exact keys presented below and the values can be customised to configure the behaviour of `cazy_webscraper`:
+```yaml
+classes:  # classes from which all proteins will be retrieved
+  - "GH"
+  - "CE"
+Glycoside Hydrolases (GHs):
+GlycosylTransferases (GTs):
+Polysaccharide Lyases (PLs):
+  - "GT1"
+  - "GT5"
+  - "GT6"
+Carbohydrate Esterases (CEs):
+Auxiliary Activities (AAs):
+Carbohydrate-Binding Modules (CBMs):
+genera:  # list genera to be scraped
+ - "Trichoderma"
+ - "Aspergillus"
+species:  # list species, this will scrape all strains under the species
+- "Pythium ultimum"
+strains:  # list specific strains to be scraped
+kingdoms:  # Archaea, Bacteria, Eukaryota, Viruses, Unclassified
+```
+
+For configuring the retrieval of data from UniProt, GenBank and PDB (_but not CAZy) the additional `ec` tag can be included to limit the retrieval of data to proteins annotated with specific EC numbers.
+
+When listing EC numbers, the 'EC' prefix can be included or excluded. For example, 'EC1.2.3.4' and '1.2.3.4' are accepted. Additionally, both dashes ('-') and astrixes ('*') can be used to represent missing digits, both '1.2.3.-' and '1.2.3.\*' are accepted.
+
+`cazy_webscraper` performs a direct EC number comparison. Therefore, supplying `cazy_webscraper` with the EC number EC1.2.3.- will only retrieve protein specifically annotated with EC1.2.3.-. `cazy_webscraper` will **not** retrieve proteins will all completed EC numbers under EC1.2.3.-, thus, `cazy_webscraper` will **not** retrieve data for proteins annotated with EC1.2.3.1, EC1.2.3.2, EC1.2.3.3, etc.
+
+Example configuration files, and an empty configuraiton file template are located in the [`config_files`]() directory of this repo.
+
+
+## CAZy coverage of GenBank
+
+The number of genomes represented by the local CAZyme database per taxonomy Kingdom can be compared to the number 
+GenBank genomic assemblies in NCBI. This done using the command `cw_cazy_genbank_coverage`, which requires two 
+positional arguments:
+1. Path to the local CAZyme database
+2. The users email address
+For example:
+```bash
+cw_cazy_genbank_coverage cazymes.db my_email@domain.com
+```
+
+This produces several output files (where time stamp is the date and time the command was invoked):
+1. `cazy_genomic_accessions_<time_stamp>.json`, a JSON file contained a multi-layer dictionary: {kingdom: {genus: {species: {genomic_accession: {proteins: {protein_accessions}, count=int(# of proteins)}}}}}
+2. `genomic_accessions_<time_stamp>.csv`, containing the columns 'Kingdom','Genus','Species','Genomic_accession','#ofProteins', where the number of proteins represents the number of proteins from the genome which are catalogued in CAZy
+3. `protein_genomic_accessions_<time_stamp>.csv`, containing the 'Kingdom', 'Genus', 'Species', 'Genomic_accession', 'Protein_accession'. A unique protein accession is listed on each row, and lists which protein accesison is derived from each genomic assembly.
+4. `cazy_genbank_genome_coverage_<time_stamp>.csv`, containing the columns 'Kingdom', 'NCBI_genomes', 'CAZy_genomes', 'Coverage_percent'. The dataframe lists the number of genomes catalogued in GenBank (NCBI) and the local CAZyme database per taxonomy Kingdom.
+5. `gbk_cazy_genomes_plot_<time_stamp>.png`, a plot of a stacked bar chart with the number of genomes in GenBank (NCBI) and CAZy (the local CAZyme database) per taxonomy Kingdom.
+
+### Configure calculating CAZy coverage of GenBank
+
+Optional cmd-line arguments for `cw_cazy_genbank_coverage` are listed below:
+
+`--batch_size` - The number of accessions posted to NCBI per epost, advice to be max 200. Default=150.
+
+``--force``, ``-f`` - Force writing in existing output directory.
+
+``--force_cache`` - Force writing in existing cache directory.
+
+`--log`, `-l` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
+
+`--nodelete` - When called, content in the existing output dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--nodelete_cache` - When called, content in the existing cache dir will **not** be deleted. Default: False (existing content is deleted).
+
+`--output_dir`, `-o` - Path to output directory.
+
+`--retries` - Number of times to retry connection to NCBI Entrez if connection fails.
+
+`--sql_echo` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
+
+`--verbose`, `-v` - Enable verbose logging. This does not set the SQLite engine `echo` parameter to True. Default: False.
 
 ## Contributions
 
@@ -160,6 +607,6 @@ We welcome contributions and suggestions. You can raise issues at this repositor
 
 MIT License
 
-Copyright (c) 2020-2021 University of St Andrews
-Copyright (c) 2020-2021 University of Strathclyde
-Copyright (c) 2020-2021 UJames Hutton Institute
+Copyright (c) 2020-2021 University of St Andrews  
+Copyright (c) 2020-2021 University of Strathclyde  
+Copyright (c) 2020-2021 James Hutton Institute  
diff --git a/cazy_webscraper/__init__.py b/cazy_webscraper/__init__.py
new file mode 100644
index 00000000..3173c873
--- /dev/null
+++ b/cazy_webscraper/__init__.py
@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+
+# Contact
+# eemh1@st-andrews.ac.uk
+
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Web scraper to scrape the CAZy website."""
+
+
+import logging
+
+import pandas as pd
+
+from datetime import datetime
+
+
+__version__ = "2.0.0-beta"
+
+VERSION_INFO = f"cazy_webscraper version: {__version__}"
+
+CITATION_INFO = (
+    "If you use cazy_webscraper in your work, please cite the following publication:\n"
+    "\tHobbs, E. E. M., Pritchard, L., Chapman, S., Gloster, T. M.,\n"
+    "\t(2021) cazy_webscraper Microbiology Society Annual Conference 2021 poster.\n"
+    "\tFigShare. Poster.\n"
+    "\thttps://doi.org/10.6084/m9.figshare.14370860.v7"
+)
+
+
+def closing_message(job, start_time, args):
+    """Write closing messsage to terminal"""
+    logger = logging.getLogger(__name__)
+
+    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+    end_time = pd.to_datetime(end_time)
+    total_time = end_time - start_time
+
+    if args.verbose:
+        logger.info(
+            "Finished scraping CAZy. Terminating program.\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {VERSION_INFO}\n"
+            f"Citation: {CITATION_INFO}"
+        )
+    else:
+        print(
+            f"====================={job}=====================\n"
+            "Finished scraping CAZy\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {VERSION_INFO}\n"
+            f"Citation: {CITATION_INFO}"
+        )
+
+    return
diff --git a/scraper/__init__.py b/cazy_webscraper/api/__init__.py
similarity index 96%
rename from scraper/__init__.py
rename to cazy_webscraper/api/__init__.py
index 5d3cc0e4..6f9e5a17 100644
--- a/scraper/__init__.py
+++ b/cazy_webscraper/api/__init__.py
@@ -36,4 +36,4 @@
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
-"""Web scraper to scrape the CAZy website."""
+"""Interrogate the local CAZyme database"""
diff --git a/cazy_webscraper/api/cw_query_database.py b/cazy_webscraper/api/cw_query_database.py
new file mode 100644
index 00000000..4ba6ce31
--- /dev/null
+++ b/cazy_webscraper/api/cw_query_database.py
@@ -0,0 +1,308 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2021
+# (c) University of Strathclyde 2021
+# (c) James Hutton Institute 20201
+# Author:
+# Emma E. M. Hobbs
+
+# Contact
+# eemh1@st-andrews.ac.uk
+
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Interrogate the local CAZyme database"""
+
+
+import logging
+import json
+
+import pandas as pd
+
+from datetime import datetime
+from typing import List, Optional
+
+from saintBioutils.utilities.logger import config_logger
+from saintBioutils.utilities import file_io
+from tqdm import tqdm
+
+from cazy_webscraper import cazy_scraper
+from cazy_webscraper.sql.sql_interface import get_selected_gbks, get_api_data
+from cazy_webscraper.utilities.parsers import query_db_parser
+from cazy_webscraper.utilities.parse_configuration import get_expansion_configuration
+
+
+def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = None):
+    """Set up parser, logger and coordinate overal scrapping of CAZy."""
+    time_stamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")  # used in naming files
+    start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
+    start_time = pd.to_datetime(start_time)
+
+    # Program preparation
+    if argv is None:
+        parser = query_db_parser.build_parser()
+        args = parser.parse_args()
+    else:
+        parser = query_db_parser.build_parser(argv)
+        args = parser.parse_args()
+
+    if logger is None:
+        logger = logging.getLogger(__name__)
+        config_logger(args)
+
+    connection, logger_name, cache_dir = cazy_scraper.connect_existing_db(args, time_stamp)
+
+    if args.cache_dir is not None:  # use user defined cache dir
+        cache_dir = args.cache_dir
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+    else:
+        cache_dir = cache_dir / "uniprot_data_retrieval"
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+
+    (
+        config_dict,
+        class_filters,
+        family_filters,
+        kingdom_filters,
+        taxonomy_filter_dict,
+        ec_filters,
+    ) = get_expansion_configuration(args)
+
+    # get the records of GenBank accessions matching the criteria of interest
+    # {gbk_acc: gbk_id}
+    gbk_dict = get_selected_gbks.get_genbank_accessions(
+        class_filters,
+        family_filters,
+        taxonomy_filter_dict,
+        kingdom_filters,
+        ec_filters,
+        connection,
+    )
+
+    query_data = get_query_data(gbk_dict)
+
+    write_output(query_data, args)
+
+
+def get_query_data(gbk_dict, connection, args):
+    """Retrieve additional data as requested per retrieved GenBank accession
+    
+    :param gbk_dict: dict, {gbk_acc: gbk_id} GenBank accessions matching user criteria
+    :param connection: open sqlaclchemy connection for an SQLite db
+    :param args: cmd-line args parser
+
+    Structure of dict if all data is retrievied:
+    query_data = {
+        genbank_accession: 
+            'class': {CAZy classes},
+            'family': {CAZy families},
+            'subfamily': {CAZy subfamilies},
+            'kingdom': kingdom,
+            'genus': genus,
+            'organism': genus species strain,
+            'ec_numbers': {EC number annotations},
+            'pdb_accessions': {PDB accessions},
+            'uniprot_accession': UniProt protein accession,
+            'uniprot_name': Name of the protein from UniProt,
+            'uniprot_sequence': Protein Aa seq from UniProt,
+            'uniprot_sequence_date': Date the seq was last modified in UniProt,
+            'gbk_sequence': Protein Aa seq from GenBank
+            'gbk_sequence_date': Date the seq was last modified in Gbk,
+    }
+    
+    Retrun dict of retrieved data, keyed by GenBank accession and valued by dict containing
+        requested data.
+    """
+    # create dict to store all data retrieved from the databsae query
+    query_data = {}
+    
+    # add the GenBank accessions of interest to the query dict
+    for gbk_acc in gbk_dict:
+        query_data[gbk_acc] = {}
+
+    if args.cazy_class or args.cazy_family or args.cazy_subfamily:
+        # retrieve the CAZy family annotations from the local CAZyme database
+        query_data = get_api_data.get_class_fam_annotations(gbk_dict, query_data, connection, args)
+
+    if args.kingdom or args.genus or args.organism:
+        # retrieve the taxonomy data from the local CAZyme database
+        query_data = get_api_data.get_tax_annotations(gbk_dict, query_data, connection, args)
+
+    if args.ec:
+       # retrieve the ec numbers from the local CAZyme database
+       query_data = get_api_data.get_ec_annotations(gbk_dict, query_data, connection)
+
+    if args.pdb:
+        # retrieve the PDB accessions from the local CAZyme database
+        query_data = get_api_data.get_pdb_accessions(gbk_dict, query_data, connection)
+
+    if args.uniprot or args.seq_uniprot:
+        # retrieve the UniProt data from the local CAZyme database
+        query_data = get_api_data.get_uniprot_data(gbk_dict, query_data, connection, args)
+
+    if args.seq_genbank:
+        # retrieve GenBank protein sequences from the local CAZyme database
+        query_data = get_api_data.get_gbk_seq(gbk_dict, query_data, connection)
+
+
+def write_output(query_data, args, time_stamp):
+    """Write out query data to disk.
+    
+    :param query_data: dict containing db query data.
+    :param args: cmd-line args parser
+    
+    Return nothing"""
+    output_path = compile_output_name(args, time_stamp)
+
+    if 'json' in args.file_types:
+        output_path += ".json"
+        with open(output_path, 'w') as fh:
+            json.dump(query_data, fh)
+
+    if 'csv' in args.file_types:
+        output_path += ".csv"
+
+        # compile pandas df of the data
+        column_names = get_column_names(args)
+        
+        df_data = []  # list of nested lists, one nested list per df row
+
+        for gbk_acc in tqdm(query_data, desc="Compiling output dataframe"):
+            new_rows = []
+
+            # create one row for each CAZy class, CAZy family and CAZy subfamily annotation
+            # link the parent-child relationships between CAZy class, family and subfamily
+            if args.cazy_class is False and args.cazy_family is False and args.cazy_subfamily is False:
+                new_rows.append([gbk_acc])  # don't need to create multiple rows to separate the class/fam/subfam annotations
+
+            if args.kingdom:
+                for row in new_rows:
+                    row.append(query_data[gbk_acc]["kingdom"])
+            if args.genus:
+                for row in new_rows:
+                    row.append(query_data[gbk_acc]["genus"])
+            if args.organism:
+                for row in new_rows:
+                    row.append(query_data[gbk_acc]["organism"])
+            if args.ec:
+                for row in new_rows:
+                    row.append(" ".join(list(query_data[gbk_acc]["ec_numbers"])))
+            if args.pdb:
+                for row in new_rows:
+                    row.append(" ".join(list(query_data[gbk_acc]["pdb_accessions"])))
+            if args.uniprot:
+                for row in new_rows:
+                    row.append(query_data[gbk_acc]["uniprot_accession"])
+                    row.append(query_data[gbk_acc]["uniprot_name"])
+            if args.seq_uniprot:
+                for row in new_rows:
+                    row.append(query_data[gbk_acc]["uniprot_sequence"])
+                    row.append(query_data[gbk_acc]["uniprot_sequence_date"])
+            if args.seq_genbank:
+                for row in new_rows:
+                    row.append(query_data[gbk_acc]["gbk_sequence"])
+                    row.append(query_data[gbk_acc]["gbk_sequence_date"])
+
+            for row in new_rows:
+                df_data.append(row)
+
+        query_df = pd.DataFrame(df_data, columns=column_names)
+
+        query_df.to_csv(output_path)
+
+
+def compile_output_name(args, time_stamp):
+    db_name = args.database.name.replace(".db","")
+    output_path = args.output_dir / f"{db_name}_{time_stamp}"
+
+    if args.cazy_class:
+        output_path += "_classes"
+    if args.cazy_family:
+        output_path += "_fams"
+    if args.cazy_subfamily:
+        output_path += "_subfams"
+    if args.kingdom:
+        output_path += "_kngdm"
+    if args.genus:
+        output_path += "_genus"
+    if args.organism:
+        output_path += "_orgnsm"
+    if args.ec:
+        output_path += "_ec"
+    if args.pdb:
+        output_path += "_pdb"
+    if args.uniprot:
+        output_path += "_uniprot"
+    if args.seq_uniprot:
+        output_path += "_uniprotSeq"
+    if args.seq_genbank:
+        output_path += "_gbkSeq"
+
+    return output_path
+
+
+def get_column_names(args):
+    """Compile the column names for the output df.
+    
+    :param args: cmd-line args parser
+    
+    Return list of column names"""
+    column_names = ["genbank_accession"]
+
+    if args.cazy_class:
+        column_names.append("class")
+    if args.cazy_family:
+        column_names.append("family")
+    if args.cazy_subfamily:
+        column_names.append("subfamily")
+    if args.kingdom:
+        column_names.append("kingdom")
+    if args.genus:
+        column_names.append("genus")
+    if args.organism:
+        column_names.append("source_organism")
+    if args.ec:
+        column_names.append("ec_number")
+    if args.pdb:
+        column_names.append("pdb_accession")
+    if args.uniprot:
+        column_names.append("uniprot_accession")
+        column_names.append("uniprot_name")
+    if args.seq_uniprot:
+        column_names.append("uniprot_sequence")
+        column_names.append("uniprot_sequence_date")
+    if args.seq_genbank:
+        column_names.append("genbank_sequence")
+        column_names.append("genbank_sequence_date")
+
+    return column_names
+
+
+if __name__ == "__main__":
+    main()
diff --git a/scraper/cazy/__init__.py b/cazy_webscraper/cazy/__init__.py
similarity index 58%
rename from scraper/cazy/__init__.py
rename to cazy_webscraper/cazy/__init__.py
index f3b820d4..f9048007 100644
--- a/scraper/cazy/__init__.py
+++ b/cazy_webscraper/cazy/__init__.py
@@ -37,36 +37,139 @@
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
-"""Parse data retrieved from CAZy."""
+"""Parse data retrieved from CAZy and build a dict of data matching the user's criteria."""
 
 
 import logging
 import re
-import time
+import sys
 
+from saintBioutils.genbank import entrez_retry
 from tqdm import tqdm
 from zipfile import ZipFile
 
 from Bio import Entrez
 
+from cazy_webscraper import crawler
 
-def extract_cazy_file_data(cazy_txt_path):
-    """Retrieve data from the text file downloaded from CAZy.
+
+def get_cazy_txt_file_data(cache_dir, time_stamp, args):
+    """Retrieve txt file of CAZy db dump from CAZy or the local disk.
     
-    :param cazy_txt_path: Path, location where CAZy text file was downloaded
+    :param cache_dir: Path(), path to directory where cache is written to
+    :param time_stamp: str, date and time cazy_webscraper was intiated
+    :param args: cmd-line args parser
     
-    Return list of lines from the CAZy text file.
-    """
-    with ZipFile(cazy_txt_path) as zip_handle:
-        cazy_filepath = zip_handle.namelist()[0]
+    Return list of lines from CAZy txt file, one line is one item in the list"""
+    logger = logging.getLogger(__name__)
+
+    if args.cazy_data is not None:   # retrieve lines from predownloaded CAZy txt file
+        with open(args.cazy_data, 'r') as fh:
+            cazy_txt_lines = fh.read().splitlines()
     
-        with zip_handle.open(cazy_filepath) as fh:
-            lines = fh.read().splitlines()
+    else:
+        # download CAZy database txt file
+        cazy_txt_path = cache_dir / f"cazy_db_{time_stamp}.zip"
+
+        tries, retries, success = 0, (args.retries + 1), False
+
+        err_message = None
+        while (tries <= retries) and (not success):
+            err_message = crawler.get_cazy_file(cazy_txt_path, args, max_tries=(args.retries + 1))
+
+            if err_message is None:
+                break
+
+            else:
+                tries += 1
+        
+        if err_message is not None:
+            logger.error(
+                "Could not connect to CAZy to download the CAZy db txt file after "
+                f"{(args.retries + 1)*(args.retries + 1)}\n"
+                f"The following error was raised:\n{err_message}"
+                f"File would have been written to {cazy_txt_path}"
+                "Terminating program"
+            )
+            sys.exit(1)
+        
+        with ZipFile(cazy_txt_path) as zip_handle:
+            cazy_filepath = zip_handle.namelist()[0]
+        
+            with zip_handle.open(cazy_filepath) as fh:
+                cazy_txt_lines = fh.read().splitlines()
+
+    return cazy_txt_lines
+
+
+def parse_all_cazy_data(lines, cazy_fam_populations):
+    """Extract ALL GenBank accession, taxonomy data and CAZy (sub)family annotations from CAZy txt file.
+    
+    This is when no filters are applied.
     
-    return lines
+    :param lines: list of str, lines from CAZy txt file, one unqiue line is one item in the list
+    :param cazy_fam_populations: None if args.validate is False, or dict of CAZy listed CAZy 
+        (sub)fam population sizes if args.validate is True
+    
+    Return cazy_data: dict, {gbk: {"organism": set(str), "families": {'fam': set (subfam)}}}
+    """
+
+    logger = logging.getLogger(__name__)
+
+    # define dicts CAZy data will be stored in
+    cazy_data = {} # {genbank_accession: {organism,}, families: {(fam, subfam,),} }
+    
+    # used for verbose logging
+    gbk_accessions = set()
+    fam_annotations = set()
+    organisms = set()
+    kingdoms = set()
+
+    for line in tqdm(lines, 'Parsing CAZy txt file'):
+        try:
+            line_str = line.decode()  # used when parsing zipped CAZy file
+        except AttributeError:
+            line_str = line   # used when parsing local CAZy file
+        line_data = line_str.split('\t')
+        
+        gbk_accession = line_data[3]
+
+        # retrieve CAZyme data
+        cazy_fam = line_data[0]
+        if cazy_fam.find("_") != -1:
+            cazy_subfam = cazy_fam
+            cazy_fam = cazy_fam[:cazy_fam.find("_")]
+        else:
+            cazy_subfam = None
+        
+        fam_annotations.add( (cazy_fam, cazy_subfam) )
+       
+        kingdom = line_data[1]
+        kingdoms.add(kingdom)
+        
+        organism = line_data[2]
+        organisms.add(organism)
+        
+        gbk_accession = line_data[3]
+        gbk_accessions.add(gbk_accession)
+        
+        add_protein_to_dict(cazy_data, gbk_accession, cazy_fam, cazy_subfam, organism, kingdom)
+
+    if cazy_fam_populations is not None:
+        validate_data_retrieval(cazy_data, cazy_fam_populations)
+        
+    logger.info(
+        "CAZy txt file contained:\n"
+        f"{len(list(gbk_accessions))} unique GenBank accessions\n"
+        f"{len(list(fam_annotations))} unique Fam-Subfam annotations\n"
+        f"{len(list(organisms))} unique source organisms\n"
+        f"{len(list(kingdoms))} unique tax kingdoms\n"
+    )
+
+    return cazy_data
 
 
-def parse_cazy_data(
+def parse_cazy_data_with_filters(
     lines,
     class_filter,
     fam_filter,
@@ -86,10 +189,11 @@ def parse_cazy_data(
     
     Return cazy_data: dict, {gbk: {"organism": set(str), "families": {'fam': set (subfam)}}}
     """
+
     logger = logging.getLogger(__name__)
 
     # define dicts CAZy data will be stored in
-    cazy_data = {} # {genbank_accession: {organism,}, families: {(fam, subfam,),} }
+    cazy_data = {} # {genbank_accession: {organism,}, families: {fam: {subfams}} }
     taxa_data = {} # {kingdom: organism}
     
     # used for verbose logging
@@ -97,10 +201,20 @@ def parse_cazy_data(
     fam_annotations = set()
     organisms = set()
     kingdoms = set()
+    
+    gbks_to_scrape = set()  # gbks matching filter criteria are added to enable retrieving ALL
+    # family annotations for each protein
 
     for line in tqdm(lines, 'Parsing CAZy txt file'):
-        line_str = line.decode()
+        try:  # decode line if retrieved from downloaded and zipped CAZy txt file
+            line_str = line.decode()
+        except AttributeError:
+            line_str = line
         line_data = line_str.split('\t')
+        
+        kingdom = line_data[1]
+        organism = line_data[2]
+        gbk_accession = line_data[3]
 
         # retrieve CAZyme data
         cazy_fam = line_data[0]
@@ -109,20 +223,11 @@ def parse_cazy_data(
             cazy_fam = cazy_fam[:cazy_fam.find("_")]
         else:
             cazy_subfam = None
-        fam_annotations.add( (cazy_fam, cazy_subfam) )
-       
-        kingdom = line_data[1]
-        kingdoms.add(kingdom)
-        
-        organism = line_data[2]
-        organisms.add(organism)
-        
-        gbk_accession = line_data[3]
-        gbk_accessions.add(gbk_accession)
-
-        # Apply filters
-        if (len(class_filter) == 0) and (len(fam_filter) == 0):
-            cazy_data = apply_kingdom_tax_filters(
+            
+        # check if protein previously matched filter criteria, and additional CAZy (sub)fam annotations
+        # are to be added to the db
+        if gbk_accession in gbks_to_scrape:
+            cazy_data, stored_gbk_accession = apply_kingdom_tax_filters(
                 cazy_data,
                 kingdom_filter,
                 tax_filter,
@@ -132,13 +237,12 @@ def parse_cazy_data(
                 organism,
                 kingdom,
             )
-
-        # check if scraping the class the family belongs to
-        if len(class_filter) != 0:
-            fam_class = re.match(r"\D{2,3}", cazy_fam).group()
+        
+        else:  # check if protein matches the filter criteria
             
-            if fam_class in class_filter:
-                cazy_data = apply_kingdom_tax_filters(
+            # only apply tax filters
+            if (len(class_filter) == 0) and (len(fam_filter) == 0):
+                cazy_data, stored_gbk_accession = apply_kingdom_tax_filters(
                     cazy_data,
                     kingdom_filter,
                     tax_filter,
@@ -149,24 +253,57 @@ def parse_cazy_data(
                     kingdom,
                 )
 
-                continue
-            
-        if (cazy_fam in fam_filter) or (cazy_subfam in fam_filter):
-            cazy_data = apply_kingdom_tax_filters(
-                cazy_data,
-                kingdom_filter,
-                tax_filter,
-                gbk_accession,
-                cazy_fam,
-                cazy_subfam,
-                organism,
-                kingdom,
-            )
+                if stored_gbk_accession:  # protein matched tax criteria and added to db
+                    gbks_to_scrape.add(gbk_accession)
 
-            continue
+                    fam_annotations.add( (cazy_fam, cazy_subfam) )
+                    kingdoms.add(kingdom)
+                    organisms.add(organism)
+                    gbk_accessions.add(gbk_accession)
+
+                
+            else:
+                # check against CAZy class and family filters
+                cazy_class = re.match(r'\D{2,3}\d', cazy_fam).group()[:-1]
+
+                scrape = False  # mark whether to scrape the protein or not
+
+                # check if another propety of the protein matches the filter criteria previously
+                if gbk_accession in gbks_to_scrape:
+                    scrape = True
+
+                # apply class filter
+                if cazy_class in class_filter:
+                    scrape = True
+
+                if cazy_fam in fam_filter:
+                    scrape = True
+
+                if cazy_subfam in fam_filter:
+                    scrape = True
+
+                if scrape:
+                    cazy_data, stored_gbk_accession = apply_kingdom_tax_filters(
+                        cazy_data,
+                        kingdom_filter,
+                        tax_filter,
+                        gbk_accession,
+                        cazy_fam,
+                        cazy_subfam,
+                        organism,
+                        kingdom,
+                    )
+
+                    if stored_gbk_accession:  # protein matched tax criteria and added to db
+                        gbks_to_scrape.add(gbk_accession)
+
+                        fam_annotations.add( (cazy_fam, cazy_subfam) )
+                        kingdoms.add(kingdom)
+                        organisms.add(organism)
+                        gbk_accessions.add(gbk_accession)
 
     if cazy_fam_populations is not None:
-        validate_data_retrieval(cazy_data, cazy_fam_populations)
+       validate_data_retrieval(cazy_data, cazy_fam_populations)
         
     logger.info(
         "CAZy txt file contained:\n"
@@ -176,7 +313,7 @@ def parse_cazy_data(
         f"{len(list(kingdoms))} unique tax kingdoms\n"
     )
 
-    return cazy_data, taxa_data
+    return cazy_data
 
 
 def apply_kingdom_tax_filters(
@@ -200,8 +337,12 @@ def apply_kingdom_tax_filters(
     :param cazy_subfam: str, CAZy subfamily annotation, or None if protein is not a CAZy subfamily
     :param kingdom: str, taxonomy kingdom of the source organism of the protein
 
-    Return nothing.
+    Return
+    - cazy_data ({gbk_acc: {kingdom: str, organism: str, families: {(fam, subfam, )}}})
+    - boolean if data for the given protein was (True) or was not (False) added to the db
     """
+    stored_gbk_accession = False
+    
     if (len(kingdom_filter) == 0) and (len(tax_filter) == 0):  # kingdom and tax filters NOT enabled
         cazy_data = add_protein_to_dict(
             cazy_data,
@@ -211,30 +352,31 @@ def apply_kingdom_tax_filters(
             organism,
             kingdom,
         )
+        stored_gbk_accession = True
 
-    if len(kingdom_filter) != 0:  # user enabled kingdom filter
-        if kingdom in kingdom_filter:
-            cazy_data = add_protein_to_dict(
-                cazy_data,
-                gbk_accession,
-                cazy_fam,
-                cazy_subfam,
-                organism,
-                kingdom,
-            )
+    elif kingdom in kingdom_filter:
+        cazy_data = add_protein_to_dict(
+            cazy_data,
+            gbk_accession,
+            cazy_fam,
+            cazy_subfam,
+            organism,
+            kingdom,
+        )
+        stored_gbk_accession = True
 
-    if len(tax_filter) != 0:  # user enabled tax filter
-        if any(filter in organism for filter in tax_filter):
-            cazy_data = add_protein_to_dict(
-                cazy_data,
-                gbk_accession,
-                cazy_fam,
-                cazy_subfam,
-                organism,
-                kingdom,
-            )
+    elif any(filter in organism for filter in tax_filter):
+        cazy_data = add_protein_to_dict(
+            cazy_data,
+            gbk_accession,
+            cazy_fam,
+            cazy_subfam,
+            organism,
+            kingdom,
+        )
+        stored_gbk_accession = True
     
-    return cazy_data
+    return cazy_data, stored_gbk_accession
 
 
 def add_protein_to_dict(cazy_data, gbk_accession, cazy_fam, cazy_subfam, organism, kingdom):
@@ -355,128 +497,22 @@ def validate_data_retrieval(cazy_data, cazy_fam_populations):
     return
 
 
-def entrez_retry(retries, entrez_func, *func_args, **func_kwargs):
-    """Call to NCBI using Entrez.
-    
-    Maximum number of retries is 10, retry initated when network error encountered.
+def build_taxa_dict(cazy_data):
+    """Createa a dict of taxa data extracted from the CAZy plain text file.
     
-    :param logger: logger object
-    :param retries: parser argument, maximum number of retries excepted if network error encountered
-    :param entrez_func: function, call method to NCBI
-    :param *func_args: tuple, arguments passed to Entrez function
-    :param ** func_kwargs: dictionary, keyword arguments passed to Entrez function
+    :param cazy_dict: dict of CAZy data
     
-    Returns record.
+    Return taxa_dict, dict of taxonomy data {kingdom: set(organism)}
     """
-    logger = logging.getLogger(__name__)
-    record, retries, tries = None, retries, 0
-
-    while record is None and tries < retries:
-        try:
-            record = entrez_func(*func_args, **func_kwargs)
-
-        except IOError:
-            # log retry attempt
-            if tries < retries:
-                logger.warning(
-                    f"Network error encountered during try no.{tries}.\nRetrying in 10s",
-                    exc_info=1,
-                )
-                time.sleep(10)
-            tries += 1
-
-    if record is None:
-        logger.error(
-            "Network error encountered too many times. Exiting attempt to call to NCBI"
-        )
-        return
-
-    return record
-
-
-def replace_multiple_tax(cazy_data, args):
-    """
-    Identify GenBank accessions which have multiple source organisms listedi in CAZy. Replace with
-    the latest source organism from NCBI.
-
-    :param cazy_data: dict of CAZy data
-    :param args: cmd-line args parser
-
-    Return dict of proteins with multiple source organisms listed in CAZy
-    {genbank_accession: {"families": {fam: set(subfam)} }}
-    """
-    logger = logging.getLogger(__name__)
-
-    Entrez.email = args.email
-
-    multi_taxa_gbk = {}
-
-    for genbank_accession in tqdm(cazy_data, total=len(list(cazy_data.keys())), desc='Searching for multiple taxa annotations'):
-        gbk_organisms = cazy_data[genbank_accession]["organism"]
-        if len(gbk_organisms) > 1:
-            multi_taxa_gbk[genbank_accession] = {"families": cazy_data[genbank_accession]["families"]}
-            logger.warning(
-                f"{genbank_accession} annotated with multiple taxa:\n"
-                f"{cazy_data[genbank_accession]['organism']}"
-            )
-    
-    logger.warning(
-        f"{len(list(multi_taxa_gbk.keys()))} proteins found with multiple source organisms in CAZy\n"
-        "Querying NCBI to retrieve the latest source organisms"
-    )
-
-    if len(list(multi_taxa_gbk.keys())) == 0:
-        return cazy_data
-
-    id_post_list = str(",".join(list(multi_taxa_gbk.keys())))
-    try:
-        epost_results = Entrez.read(
-            entrez_retry(
-                args.retries, Entrez.epost, "Protein", id=id_post_list
-            )
-        )
-    # if no record is returned from call to Entrez
-    except (TypeError, AttributeError):
-        logger.error(
-                f"Entrez failed to post assembly IDs.\n"
-                "Exiting retrieval of accession numbers, and returning null value 'NA'"
-        )
-
-    # Retrieve web environment and query key from Entrez epost
-    epost_webenv = epost_results["WebEnv"]
-    epost_query_key = epost_results["QueryKey"]
-
-    try:
-        with entrez_retry(
-            args.retries,
-            Entrez.efetch,
-            db="Protein",
-            query_key=epost_query_key,
-            WebEnv=epost_webenv,
-            retmode="xml",
-        ) as record_handle:
-            protein_records = Entrez.read(record_handle, validate=False)
-    # if no record is returned from call to Entrez
-    except (TypeError, AttributeError) as error:
-        logger.error(
-            f"Entrez failed to retireve accession numbers."
-            "Exiting retrieval of accession numbers, and returning null value 'NA'"
-        )
+    taxa_dict = {}  # {kingdom: {organism,}}
     
-    for protein in tqdm(protein_records, desc="Retrieving organism from NCBI"):
-        accession = protein['GBSeq_accession-version']
-        organism = protein['GBSeq_organism']
-        kingdom = protein['GBSeq_taxonomy'].split(';')[0]
+    for genbank_accession in tqdm(cazy_data, "Compiling taxa data"):
+        kingdom = list(cazy_data[genbank_accession]['kingdom'])[0]
+        organism = list(cazy_data[genbank_accession]['organism'])[0]
         
         try:
-            cazy_data[accession] = {
-                "kingdom": (kingdom,),
-                "organism": (organism,),
-                "families": multi_taxa_gbk[accession]["families"],
-            }
+            taxa_dict[kingdom].add(organism)
         except KeyError:
-            logger.error(
-                f'GenBank accession {accession} retrieved from NCBI, but it is not present in CAZy'
-            )
-
-    return cazy_data
+            taxa_dict[kingdom] = {organism}
+    
+    return taxa_dict
diff --git a/scraper/cazy_webscraper.py b/cazy_webscraper/cazy_scraper.py
similarity index 75%
rename from scraper/cazy_webscraper.py
rename to cazy_webscraper/cazy_scraper.py
index 5f2cdb0c..855ad205 100644
--- a/scraper/cazy_webscraper.py
+++ b/cazy_webscraper/cazy_scraper.py
@@ -77,64 +77,22 @@
 from pathlib import Path
 from typing import List, Optional
 
-from tqdm import tqdm
-
-from scraper import crawler
-from scraper import cazy
-from scraper.sql import sql_orm, sql_interface
-from scraper.sql.sql_interface import add_cazyme_data
-from scraper.utilities import (
+from Bio import Entrez
+from cazy_webscraper import cazy, crawler, taxonomy, closing_message, CITATION_INFO, VERSION_INFO
+from cazy_webscraper.sql import sql_orm, sql_interface
+from cazy_webscraper.sql.sql_interface import add_cazyme_data
+from cazy_webscraper.utilities import (
     build_logger,
     config_logger,
     file_io,
     parse_configuration,
     termcolour,
 )
-from scraper.utilities.parsers import cazy_webscraper_parser
-
-# Define constants
-
-__version__ = "2.0.0-beta"
-
-VERSION_INFO = [
-    termcolour(
-        f"cazy_webscraper version: {__version__}",
-        "cyan",
-    ),
-]
-
-CITATION_INFO = [
-    termcolour(
-        "If you use cazy_webscraper in your work, please cite the following publication:",
-        "green",
-    ),
-    termcolour(
-        "\tHobbs, E. E. M., Pritchard, L., Chapman, S., Gloster, T. M.,",
-        "yellow",
-    ),
-    termcolour(
-        "\t(2021) cazy_webscraper Microbiology Society Annual Conference 2021 poster. ",
-        "yellow",
-    ),
-    termcolour(
-        "\tFigShare. Poster.",
-        "yellow",
-    ),
-    termcolour(
-        "\thttps://doi.org/10.6084/m9.figshare.14370860.v7",
-        "yellow",
-    ),
-]
+from cazy_webscraper.utilities.parsers import cazy_webscraper_parser
 
 
 def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = None):
-    """Set up parser, logger and coordinate overal scrapping of CAZy.
-
-    The collected data can be stored as a singel dataframe containing (not split), split into
-    separate dataframes by class or by family. Excluded classes are CAZy classes not specified in
-    the configuration file and thus, will not be scraped. User_cazy_families is the list of CAZy
-    families specified to be scraped in the configration file.
-    """
+    """Set up parser, logger and coordinate overal scrapping of CAZy."""
     cazy_home_url = "http://www.cazy.org"
 
     time_stamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")  # used in naming files
@@ -163,7 +121,7 @@ def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = No
         return
 
     # check correct output was provided, exit if not operable
-    if args.database and args.db_output:
+    if args.database is not None and args.db_output is not None:
         warning_message = (
             "Target path for a NEW database (--db_output, -d) and\n"
             "a path to an EXISTING database (--database, -D) were provided.\n"
@@ -171,8 +129,29 @@ def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = No
             "Terminating program."
         )
         logger.warning(termcolour(warning_message, "red"))
+        closing_message("cazy_webscraper", start_time, args)
         return
 
+    if args.db_output is not None and args.db_output.exists():
+        if args.force:
+            logger.warning(
+                f"Local db {args.database} already exists\n"
+                "Force is True\n"
+                "Ovewriting existing database."
+            )
+            os.remove(args.db_output)
+        else:
+            logger.warning(
+                f"Local db {args.database} already exists\n"
+                "Force is False\n"
+                "Not ovewriting existing database\n"
+                "Termianting program"
+            )
+            closing_message("cazy_webscraper", start_time, args)
+            return
+
+    Entrez.email = args.email
+
     logger.info("Parsing configuration")
     (
         excluded_classes,
@@ -206,10 +185,10 @@ def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = No
     logger.info(termcolour(scrape_config_message, "cyan"))
 
     if args.database:  # adding data to an EXISTING database
-        connection, logger_name, cache_dir = connect_existing_db(args, time_stamp)
+        connection, logger_name, cache_dir = connect_existing_db(args, time_stamp, start_time)
     
     else:  # build a new database
-        connection, logger_name, cache_dir = connect_to_new_db(args, time_stamp)
+        connection, logger_name, cache_dir = connect_to_new_db(args, time_stamp, start_time)
 
     logger.info("Adding log of scrape to the local CAZyme database")
     with sql_orm.Session(bind=connection) as session:
@@ -218,6 +197,9 @@ def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = No
             config_dict,
             kingdom_filters,
             taxonomy_filter_dict,
+            set(),  # ec_filters not applied when scraping CAZy
+            'CAZy',
+            'CAZy annotations',
             session,
             args,
         )
@@ -244,6 +226,9 @@ def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = No
 
     logger.info("Starting retrieval of data from CAZy")
 
+    if args.cazy_data is not None:
+        logger.warning(f"Retrieving CAZy data from predownloaded CAZy db dump at:\n{args.cazy_data}")
+
     get_cazy_data(
         cazy_home_url,
         excluded_classes,
@@ -260,29 +245,7 @@ def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = No
         args,
     )
 
-    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
-    end_time = pd.to_datetime(end_time)
-    total_time = end_time - start_time
-
-    if args.verbose:
-        logger.info(
-            "Finished scraping CAZy. Terminating program.\n"
-            f"Scrape initated at {start_time}\n"
-            f"Scrape finished at {end_time}\n"
-            f"Total run time: {total_time}"
-            f"Version: {VERSION_INFO}\n"
-            f"Citation: {CITATION_INFO}"
-        )
-    else:
-        print(
-            "=====================cazy_webscraper=====================\n"
-            "Finished scraping CAZy\n"
-            f"Scrape initated at {start_time}\n"
-            f"Scrape finished at {end_time}\n"
-            f"Total run time: {total_time}"
-            f"Version: {VERSION_INFO}\n"
-            f"Citation: {CITATION_INFO}"
-        )
+    closing_message("cazy_webscraper", start_time, args)
 
 
 def get_cazy_data(
@@ -323,9 +286,10 @@ def get_cazy_data(
     logger = logging.getLogger(__name__)
 
     # define paths for additional logs files
+    # unless specifed they are added to the logs dir in the cache dir
     connection_failures_logger = build_logger(Path(f"{logger_name}_{time_stamp}_connection_failures.log"))
-    sql_failures_logger = build_logger(Path(f"{logger_name}_{time_stamp}_SQL_errors.log"))
-    format_failures_logger = build_logger(Path(f"{logger_name}_{time_stamp}_format_and_parsing_errors.log"))
+    multiple_taxa_logger = build_logger(Path(f"{logger_name}_{time_stamp}_multiple_taxa.log"))
+    replaced_taxa_logger = build_logger(Path(f"{logger_name}_{time_stamp}_replaced_taxa.log"))
 
     if args.validate:  # retrieve CAZy family population sizes for validating all data was retrieved
         # {fam (str): pop size (int)}
@@ -342,75 +306,65 @@ def get_cazy_data(
     else:
         cazy_fam_populations = None
 
-    # download CAZy database txt file
-    cazy_txt_path = cache_dir / f"cazy_db_{time_stamp}.zip"
-
-    tries, retries, success = 0, (args.retries + 1), False
-
-    err_message = None
-    while (tries <= retries) and (not success):
-        err_message = crawler.get_cazy_file(cazy_txt_path, args, max_tries=(args.retries + 1))
+    cazy_txt_lines = cazy.get_cazy_txt_file_data(cache_dir, time_stamp, args)
+    
+    logger.info(f"Retrieved {len(cazy_txt_lines)} lines from the CAZy db txt file")
 
-        if err_message is None:
-            break
+    if (len(class_filters) == 0) and \
+        (len(fam_filters) == 0) and \
+            (len(kingdom_filters) == 0) and \
+                (len(taxonomy_filters) == 0):
+        cazy_data = cazy.parse_all_cazy_data(cazy_txt_lines, cazy_fam_populations)
 
-        else:
-            tries += 1
-    
-    if err_message is not None:
-        logger.error(
-            "Could not connect to CAZy to download the CAZy db txt file after "
-            f"{(args.retries + 1)*(args.retries + 1)}\n"
-            f"The following error was raised:\n{err_message}"
-            f"File would have been written to {cazy_txt_path}"
-            "Terminating program"
+    else:
+        cazy_data = cazy.parse_cazy_data_with_filters(
+            cazy_txt_lines,
+            class_filters,
+            fam_filters,
+            kingdom_filters,
+            taxonomy_filters,
+            cazy_fam_populations,
         )
-        sys.exit(1)
-    
-    # extract the CAZy family data and add to the local CAZyme database
-    cazy_txt_lines = cazy.extract_cazy_file_data(cazy_txt_path)
-    logger.info(f"Retrieved {len(cazy_txt_lines)} lines from the CAZy db txt file")
 
-    cazy_data, taxa_data = cazy.parse_cazy_data(
-        cazy_txt_lines,
-        class_filters,
-        fam_filters,
-        kingdom_filters,
-        taxonomy_filters,
-        cazy_fam_populations,
-    )
     logger.info(
         f"Retrieved {len((list(cazy_data.keys())))} proteins from the CAZy txt file "
         "matching the scraping criteria"
     )
 
-    cazy_data = cazy.replace_multiple_tax(cazy_data, args)
+    # check for GenBank accessions with multiple source organisms in the CAZy data
+    multiple_taxa_gbks = taxonomy.identify_multiple_taxa(cazy_data, multiple_taxa_logger)
+
+    if len(multiple_taxa_gbks) != 0:
+        # remove the multiple taxa, and retrieve the latest taxa from NCBI
+        cazy_data, successful_replacement = taxonomy.replace_multiple_tax(
+            cazy_data,
+            multiple_taxa_gbks,
+            replaced_taxa_logger,
+            args,
+            invalid_ids=False,
+        )
+
+    taxa_dict = cazy.build_taxa_dict(cazy_data)  # {kingdom: {organisms}}
 
-    # add kingdoms to the db
-    # create dict of taxa data {kingdom: set(organism)}
-    taxa_data = add_cazyme_data.add_kingdoms(cazy_data, connection)
+    add_cazyme_data.add_kingdoms(taxa_dict, connection)
 
-    # add taxonomy source organisms to the db
-    add_cazyme_data.add_source_organisms(taxa_data, connection)
+    add_cazyme_data.add_source_organisms(taxa_dict, connection)
 
     add_cazyme_data.add_cazy_families(cazy_data, connection)
 
-    # load taxonomy and cazy families from the db into memory for adding
-    # relationships between GenBank accessions, CAZy (sub)fams and taxa
-    db_tax_dict, db_fam_dict = add_cazyme_data.load_taxa_fam_data(connection)
+    add_cazyme_data.add_genbanks(cazy_data, connection)
 
-    gbk_fam_values = add_cazyme_data.add_genbanks(cazy_data, db_tax_dict, db_fam_dict, connection)
-
-    add_cazyme_data.add_genbank_fam_relationships(gbk_fam_values, connection)
+    add_cazyme_data.add_genbank_fam_relationships(cazy_data, connection, args)
     
     return
 
 
-def connect_existing_db(args, time_stamp):
+def connect_existing_db(args, time_stamp, start_time):
     """Coordinate connecting to an existing local CAZyme database, define logger name and cache dir
     
     :param args: cmd-line args parser
     :param time_stamp: str, time cazy_webscraper was invoked
+    :param start_time: pd date-time obj, time cazy_webscraper was invoked
 
     Return connection to local CAZyme database, logger file name, and path to cache dir
     """
@@ -424,6 +378,7 @@ def connect_existing_db(args, time_stamp):
             "Check path is correct.\n"
             "Terminating programme."
         )
+        closing_message("cazy_webscraper", start_time, args)
         sys.exit(1)
 
     try:
@@ -435,6 +390,7 @@ def connect_existing_db(args, time_stamp):
             "Terminating program\n",
             exc_info=True,
         )
+        closing_message("cazy_webscraper", start_time, args)
         sys.exit(1)
     
     # used for naming additional log files
@@ -446,11 +402,12 @@ def connect_existing_db(args, time_stamp):
     return connection, logger_name, cache_dir
     
 
-def connect_to_new_db(args, time_stamp):
+def connect_to_new_db(args, time_stamp, start_time):
     """Build and connect to a new local CAZyme database.
     
     :param args: cmd-line args parser
-    :param time_stamp: str, time cazy_Webscraper was invoked
+    :param time_stamp: str, time cazy_webscraper was invoked
+    :param start_time: pd date-time obj, time cazy_webscraper was invoked
     
     Return connection to the database, name of the logger, and path to the cache dir
     """
@@ -471,6 +428,7 @@ def connect_to_new_db(args, time_stamp):
                     "Either enable forced overwriting (-f) or add data this data (-D).\n"
                     "Terminating program."
                 )
+                closing_message("cazy_webscraper", start_time, args)
                 sys.exit(1)
         
         else:  # may need to build dirs
@@ -481,7 +439,7 @@ def connect_to_new_db(args, time_stamp):
             )
 
         if str((args.db_output).parent) != '.':  # dirs defined in output put
-            file_io.make_target_directory(args.db_output, args.force)
+            file_io.make_output_directory(args.db_output, args.force, args.nodelete)
             cache_dir = Path(f"{str(args.db_output.parent)}/.cazy_webscraper_{time_stamp}/cache")
             
         else:  # writing to cwd
@@ -492,7 +450,7 @@ def connect_to_new_db(args, time_stamp):
     
     else:
         logger.info("Using default database name and writing to cwd")
-        db_path = Path(f"cazy_webscraper_{time_stamp}")
+        db_path = Path(f"cazy_webscraper_{time_stamp}.db")
         cache_dir = Path(f".cazy_webscraper_{time_stamp}/cache")
         logger_name = f'cazy_webscraper_{time_stamp}'
     
@@ -505,6 +463,7 @@ def connect_to_new_db(args, time_stamp):
             "Terminating program",
             exc_info=True,
         )
+        closing_message("cazy_webscraper", start_time, args)
         sys.exit(1)
 
     return connection, logger_name, cache_dir
diff --git a/scraper/crawler/__init__.py b/cazy_webscraper/crawler/__init__.py
similarity index 100%
rename from scraper/crawler/__init__.py
rename to cazy_webscraper/crawler/__init__.py
diff --git a/scraper/crawler/get_validation_data.py b/cazy_webscraper/crawler/get_validation_data.py
similarity index 99%
rename from scraper/crawler/get_validation_data.py
rename to cazy_webscraper/crawler/get_validation_data.py
index b9f000e6..d7989ba0 100644
--- a/scraper/crawler/get_validation_data.py
+++ b/cazy_webscraper/crawler/get_validation_data.py
@@ -43,13 +43,13 @@
 
 import logging
 import re
-from scraper.utilities import file_io
 import time
 
 from urllib.error import HTTPError
 
 from tqdm import tqdm
 from requests.exceptions import ConnectionError, MissingSchema
+from saintBioutils.utilities import file_io
 from urllib3.exceptions import HTTPError, RequestError
 
 import mechanicalsoup
diff --git a/cazy_webscraper/expand/__init__.py b/cazy_webscraper/expand/__init__.py
new file mode 100644
index 00000000..f12bbcb6
--- /dev/null
+++ b/cazy_webscraper/expand/__init__.py
@@ -0,0 +1,65 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+
+# Contact
+# eemh1@st-andrews.ac.uk
+
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""A module for expanding the the local CAZY database beyond what is provided within CAZy."""
+
+
+def get_chunks_gen(lst, chunk_length):
+    """Separate the long list into separate chunks.
+
+    :param lst: list to be separated into smaller lists (or chunks)
+    :param chunk_length: int, the length of the lists the longer list is to be split up into
+
+    Return a generator object containing lists.
+    """
+    for i in range(0, len(lst), chunk_length):
+        yield lst[i:i + chunk_length]
+
+
+def get_chunks_list(lst, chunk_length):
+    """Separate the long list into separate chunks.
+
+    :param lst: list to be separated into smaller lists (or chunks)
+    :param chunk_length: int, the length of the lists the longer list is to be split up into
+
+    Return a list of nested lists.
+    """
+    chunks = []
+    for i in range(0, len(lst), chunk_length):
+        chunks.append(lst[i:i + chunk_length])
+    return chunks
diff --git a/cazy_webscraper/expand/extract_seqs/__init__.py b/cazy_webscraper/expand/extract_seqs/__init__.py
new file mode 100644
index 00000000..9abcf387
--- /dev/null
+++ b/cazy_webscraper/expand/extract_seqs/__init__.py
@@ -0,0 +1,43 @@
+a#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+# Bio.PDB reference:
+# Hamelryck, T., Manderick, B. (2003) PDB parser and structure class 
+# implemented in Python. Bioinformatics 19: 2308–2310
+"""Extract protein sequences from local CAZyme database, write to FASTA files and/or BLAST db"""
diff --git a/cazy_webscraper/expand/extract_seqs/extract_sequences.py b/cazy_webscraper/expand/extract_seqs/extract_sequences.py
new file mode 100644
index 00000000..786cec91
--- /dev/null
+++ b/cazy_webscraper/expand/extract_seqs/extract_sequences.py
@@ -0,0 +1,287 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+# Bio.PDB reference:
+# Hamelryck, T., Manderick, B. (2003) PDB parser and structure class 
+# implemented in Python. Bioinformatics 19: 2308–2310
+"""Extract protein sequences from local CAZyme database, write to FASTA files and/or BLAST db"""
+
+
+import logging
+import sys
+
+import pandas as pd
+
+from datetime import datetime
+from typing import List, Optional
+
+from tqdm import tqdm
+from Bio import SeqIO
+from Bio.Blast.Applications import NcbimakeblastdbCommandline
+from Bio.SeqRecord import SeqRecord
+
+
+from cazy_webscraper import cazy_webscraper
+from cazy_webscraper.expand import get_chunks_gen
+from cazy_webscraper.sql.sql_interface import get_selected_gbks, get_table_dicts
+from cazy_webscraper.utilities import config_logger, file_io, parse_configuration
+from cazy_webscraper.utilities.parsers import pdb_strctre_parser
+
+
+def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = None):
+    """Set up programme and initate run."""
+    time_stamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
+    start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
+    start_time = pd.to_datetime(start_time)
+    # parse cmd-line arguments
+    if argv is None:
+        parser = pdb_strctre_parser.build_parser()
+        args = parser.parse_args()
+    else:
+        args = pdb_strctre_parser.build_parser(argv).parse_args()
+
+    if logger is None:
+        logger = logging.getLogger(__package__)
+        config_logger(args)
+
+    validate_user_options(args)
+
+    # make output directories
+    if args.fasta_file:
+        target_dir = args.fasta_file.parent
+        file_io.make_output_directory(target_dir, args.force, args.nodelete)
+    if args.fasta_dir:
+        file_io.make_output_directory(args.fasta_dir, args.force, args.nodelete)
+
+
+    connection, logger_name, cache_dir = cazy_webscraper.connect_existing_db(args, time_stamp)
+
+    if args.cache_dir is not None:  # use user defined cache dir
+        cache_dir = args.cache_dir
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+    else:
+        cache_dir = cache_dir / "sequence_extraction"
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+
+    (
+        config_dict,
+        class_filters,
+        family_filters,
+        kingdom_filters,
+        taxonomy_filter_dict,
+        ec_filters,
+    ) = parse_configuration.get_expansion_configuration(args)
+
+    gbk_dict = get_selected_gbks.get_genbank_accessions(
+        class_filters,
+        family_filters,
+        taxonomy_filter_dict,
+        kingdom_filters,
+        ec_filters,
+        connection,
+    )
+    genbank_accessions = list(gbk_dict.keys())
+
+    extracted_sequences = {}  # {accession: {'db': str, 'seq': str}}
+    if args.genbank:
+        extracted_sequences.update(get_genbank_sequences(gbk_dict, connection))
+    if args.uniprot:
+        extracted_sequences.update(get_uniprot_sequences(gbk_dict, connection))
+
+    protein_records = []
+    
+    for protein_accession in tqdm(extracted_sequences, "Compiling SeqRecords"):
+        new_record = SeqRecord(
+            extracted_sequences[protein_accession]['seq'],
+            id=protein_accession,
+            description=extracted_sequences[protein_accession]['db']
+        )
+        protein_records.append(new_record)
+
+    write_output(protein_records, args)
+    
+
+def validate_user_options(args):
+    """Check the user has provided suitable operational options
+    
+    :param args: cmd-line args parser
+    
+    Return nothing
+    """
+    logger = logging.getLogger(__name__)
+
+    if args.genbank is False and args.uniprot is False:
+        logger.error(
+            "No external sequence sources provided\n"
+            "Call at least one of --genbank and --uniprot\n"
+            "Terminating program."
+        )
+        sys.exit(1)
+
+    if args.genbank:
+        logger.info("Extract GenBank protein sequences")
+    
+    if args.uniprot:
+        logger.info("Extracting UniProt protein sequences")
+
+    if args.blastdb is False and args.fasta_dir is False and args.fasta_file is False:
+        logger.error(
+            "No output option provided.\n"
+            "No path to create a BLAST db, output dir or output file provided\n"
+            "Call at least one of --blastdb, --fasta_dir, --fasta_file\n"
+            "Terminating program."
+        )
+        sys.exit(1)
+
+    if args.blastdb:
+        logger.info(f"Enabled building a BLAST db at:\n{args.blastdb}")
+    
+    if args.fasta_dir:
+        logger.info(
+            f"Enabled writing a unqiue sequence to a separate FASTA file in:\n{args.fasta_dir}"
+        )
+    
+    if args.fasta_file:
+        logger.info(
+            f"Enabled writing all extracted sequences to a single FASTA file at:{args.fasta_file}"
+        )
+
+    return
+
+
+def get_genbank_sequences(gbk_dict, connection):
+    """Extract protein seqeunces from the database
+    
+    :param gbk_dict: dict of selected GenBank record {acc: id}
+    :param connection: open sqlalchemy connection to an SQLite db
+    
+    Return dict {gbk_acc: {'db': 'genbank', 'seq': str}}
+    """
+    gbk_seq_dict = get_table_dicts.get_gbk_table_seq_dict(connection)
+
+    extracted_sequences = {}  # {accession: {'db': str, 'seq': str}}
+
+    for gbk_accession in gbk_dict:
+        extracted_sequences[gbk_accession] = {
+            'db': 'GenBank',
+            'seq': gbk_seq_dict[gbk_accession]['sequence'],
+        }
+
+    return extracted_sequences
+
+
+def get_uniprot_sequences(gbk_dict, connection):
+    """Extract protein seqeunces from the database
+    
+    :param gbk_dict: dict of selected GenBank record {acc: id}
+    :param connection: open sqlalchemy connection to an SQLite db
+    
+    Return dict {gbk_acc: {'db': 'genbank', 'seq': str}}
+    """
+    uniprot_table_dict = get_table_dicts.get_uniprot_table_dict(connection)
+    # dict = {acc: {name: str, gbk_id: int, seq: str, seq_date:str } }
+
+    gbk_uniprot_dict = {}  # {gbk_id: {uniprot_acc: str, seq: str}}
+    
+    for uniprot_accession in uniprot_table_dict:
+        gbk_id = uniprot_table_dict[uniprot_accession]['gbk_id']
+        seq = uniprot_table_dict[uniprot_accession]['seq']
+        gbk_uniprot_dict[gbk_id] = {'uniprot_accession': uniprot_accession, 'seq': seq}
+
+    extracted_sequences = {}  # {accession: {'db': str, 'seq': str}}
+
+    for gbk_accession in gbk_dict:
+        gbk_id = gbk_dict[gbk_accession]['gbk_id']
+
+        extracted_sequences[gbk_accession] = {
+            'db': 'UniProt',
+            'seq': gbk_uniprot_dict[gbk_accession][gbk_id]['seq'],
+        }
+
+    return extracted_sequences
+
+
+def write_output(protein_records, cache_dir, args):
+    """Write out the extract protein sequences to the specified output(s)
+    
+    :param protein_records: list of SeqRecords
+    :param cache_dir: Path, cache directory
+    :param args: cmd-line args parser
+    
+    Return nothing.
+    """
+    logger = logging.getLogger(__name__)
+
+    if args.fasta_file:
+    
+        SeqIO.write(protein_records, args.fasta_file, "fasta")
+
+    if args.fasta_dir:
+        for record in protein_records:
+            accession = record.id
+            target_path = args.fasta_dir / f'{accession}.fasta'
+
+            SeqIO.write([record], target_path, "fasta")
+    
+    if args.blastdb:
+        fasta_name = args.blastdb / 'blastdb.fasta'
+        SeqIO.write(protein_records, target_path, "fasta")
+
+        cmd_makedb = NcbimakeblastdbCommandline(
+            cmd='makeblastdb',
+            dbtype='prot',
+            input_file=fasta_name,
+        )
+        stdout, stderr = cmd_makedb()
+
+        # check the command was successfully exectured
+        if len(stderr) != 0:
+            logger.warning()
+            print(f"Could not build non-CAZyme db.\nstdout={stdout}\nstderr={stderr}")
+
+    cache_path = cache_dir / 'extracted_sequences.txt'
+    with open(cache_path, 'a') as fh:
+        for record in protein_records:
+            fh.write(f"{record.id}\n")
+
+    return
+
+
+if __name__ == "__main__":
+    main()
diff --git a/scraper/expand/genbank_sequences/__init__.py b/cazy_webscraper/expand/genbank/__init__.py
similarity index 100%
rename from scraper/expand/genbank_sequences/__init__.py
rename to cazy_webscraper/expand/genbank/__init__.py
diff --git a/cazy_webscraper/expand/genbank/get_genbank_sequences.py b/cazy_webscraper/expand/genbank/get_genbank_sequences.py
new file mode 100644
index 00000000..bbc32330
--- /dev/null
+++ b/cazy_webscraper/expand/genbank/get_genbank_sequences.py
@@ -0,0 +1,296 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+
+# Contact
+# eemh1@st-andrews.ac.uk
+
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Retrieve proteins sequences from GenBank and populate the local database"""
+
+
+import logging
+import re
+
+import pandas as pd
+
+from datetime import datetime
+from typing import List, Optional
+
+from Bio import Entrez, SeqIO
+from saintBioutils.genbank import entrez_retry
+from saintBioutils.utilities.logger import config_logger, file_io
+
+from cazy_webscraper import cazy_webscraper
+from cazy_webscraper.sql import sql_orm, sql_interface
+from cazy_webscraper.sql.sql_interface import get_selected_gbks, add_genbank_data
+from cazy_webscraper.utilities.parsers import gbk_seq_parser
+from cazy_webscraper.utilities.parse_configuration import get_expansion_configuration
+
+
+def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = None):
+    """Set up programme and initate run."""
+    time_stamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")  # used in naming files
+    start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
+    start_time = pd.to_datetime(start_time)
+    date_today = datetime.now().strftime("%Y-%m-%d")  # used as seq_update_date in the db
+
+    # parse cmd-line arguments
+    if argv is None:
+        parser = gbk_seq_parser.build_parser()
+        args = parser.parse_args()
+    else:
+        args = gbk_seq_parser.build_parser(argv).parse_args()
+
+    if logger is None:
+        logger = logging.getLogger(__package__)
+        config_logger(args)
+    
+    logger.info("Providing user email address to NCBI.Entrez")
+    Entrez.email = args.email
+
+    if args.update_seq:
+        logger.warning("Enabled updating sequences")
+
+    connection, logger_name, cache_dir = cazy_webscraper.connect_existing_db(args, time_stamp)
+
+    if args.cache_dir is not None:  # use user defined cache dir
+        cache_dir = args.cache_dir
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+    else:
+        cache_dir = cache_dir / "uniprot_data_retrieval"
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+
+    (
+        config_dict,
+        class_filters,
+        family_filters,
+        kingdom_filters,
+        taxonomy_filter_dict,
+        ec_filters,
+    ) = get_expansion_configuration(args)
+
+    # add log to the local CAZyme database
+    logger.info("Adding log of scrape to the local CAZyme database")
+    with sql_orm.Session(bind=connection) as session:
+        retrieved_annotations = "UniProt accessions"
+        if len(config_dict['ec']) != 0:
+            retrieved_annotations = f"{retrieved_annotations}, EC numbers"
+        if len(config_dict['pdb']) != 0:
+            retrieved_annotations = f"{retrieved_annotations}, PDB accessions"
+        if len(config_dict['seq']) != 0:
+            retrieved_annotations = f"{retrieved_annotations}, Protein sequence"
+        if args.update_seq_seq:
+            retrieved_annotations = f"{retrieved_annotations}, Updated UniProt protein sequences"
+        sql_interface.log_scrape_in_db(
+            time_stamp,
+            config_dict,
+            kingdom_filters,
+            taxonomy_filter_dict,
+            ec_filters,
+            'UniProt',
+            retrieved_annotations,
+            session,
+            args,
+        )
+    
+    # retrieve dict of genbank accession and genbank accession ids from the local CAZyme db
+    gbk_dict = get_selected_gbks.get_genbank_accessions(
+        class_filters,
+        family_filters,
+        taxonomy_filter_dict,
+        kingdom_filters,
+        ec_filters,
+        connection,
+    )
+    genbank_accessions = list(gbk_dict.keys())
+
+    seq_dict = get_sequences(genbank_accessions, cache_dir, args)  # {gbk_accession: seq}
+
+    add_genbank_data.add_gbk_seqs_to_db(seq_dict, date_today, connection, args)
+
+    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+    end_time = pd.to_datetime(end_time)
+    total_time = end_time - start_time
+
+    if args.verbose:
+        logger.info(
+            "Finished getting sequences from GenBank\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {cazy_webscraper.VERSION_INFO}\n"
+            f"Citation: {cazy_webscraper.CITATION_INFO}"
+        )
+    else:
+        print(
+            "=====================cazy_webscraper=====================\n"
+            "Finished getting sequences from GenBank\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {cazy_webscraper.VERSION_INFO}\n"
+            f"Citation: {cazy_webscraper.CITATION_INFO}"
+        )
+
+
+def get_sequences(genbank_accessions, cache_dir, args):
+    """Retrieve protein sequences from Entrez.
+
+    :param genbank_accessions: list, GenBank accessions
+    :param cache_dir: Path, cache directory
+    :param args: cmb-line args parser
+
+    Return nothing.
+    """
+    logger = logging.getLogger(__name__)
+
+    seq_dict = {}  # {gbk_accession: seq}
+
+    epost_webenv, epost_query_key = bulk_query_ncbi(genbank_accessions, args)
+
+    success_accessions = set()  # accessions for which seqs were retrieved
+
+    # retrieve the protein sequences
+    with entrez_retry(
+        Entrez.efetch,
+        args,
+        db="Protein",
+        query_key=epost_query_key,
+        WebEnv=epost_webenv,
+        rettype="fasta",
+        retmode="text",
+    ) as seq_handle:
+        for record in SeqIO.parse(seq_handle, "fasta"):
+            temp_accession = record.id
+
+            # check if multiple items returned in ID
+            temp_accession = temp_accession.split("|")
+            index, success = 0, False
+            
+            while (index < len(temp_accession)) and (success is False):
+                acc = temp_accession[index]
+                try:
+                    re.match(
+                        (
+                            r"(\D{3}\d{5,7}\.\d+)|"
+                            r"(\D\d(\D|\d){3}\d)|"
+                            r"((\D\d(\D|\d){3}\d)\.\d)|"
+                            r"(\D\d(\D|\d){3}\d\D(\D|\d){2}\d)|"
+                            r"((\D\d(\D|\d){3}\d\D(\D|\d){2}\d)\.\d)|"
+                            r"(\D\D_\d{9}\.\d)"
+                        ),
+                        acc,
+                    ).group()
+                    seq_accession = acc
+                    success = True
+                except AttributeError:
+                    index += 1
+                    pass
+
+            if success is False:  # if could not retrieve GenBank accession from the record
+                logger.error(
+                    "Could not retrieve a GenBank protein accession from the record with the id:\n"
+                    f"{record.id}\n"
+                    "The sequence from this record will not be added to the db"    
+                )
+                continue
+
+            if seq_accession not in genbank_accessions:  # if the retrieved Gbk acc does not match an acc in the db
+                logger.warning(
+                    f"Retrieved the accession {temp_accession} from the record with the id:\n"
+                    f"{record.id},\n"
+                    "Not adding the protein seq from this record to the db"
+                )
+                continue
+                
+            seq_dict[seq_accession] = record.sequence
+
+            success_accessions.add(seq_accession)
+
+            # remove the accession from the list
+            try:
+                genbank_accessions.remove(temp_accession)
+            except ValueError:
+                logger.warning(
+                    f"Tried to remove {temp_accession} from list of accessions, "
+                    "but it was not in the list of accessions.\n"
+                    "The returned accession and the one present in CAZy do not match."
+                )
+    
+    if len(genbank_accessions) != 0:
+        logger.warning("Protein sequences for the following CAZymes were not retrieved:")
+        for acc in genbank_accessions:
+            logger.warning(f"GenBank accession: {acc}")
+
+        cache_path = cache_dir / "no_seq_retrieved.txt"
+        with open(cache_path, "a") as fh:
+            for acc in genbank_accessions:
+                fh.write(f"{acc}\n")
+    
+    cache_path = cache_dir / "seq_retrieved.txt"
+    with open(cache_path, "a") as fh:
+        for acc in success_accessions:
+            fh.write(f"{acc}\n")
+
+    return
+
+
+def bulk_query_ncbi(accessions, args):
+    """Bulk query NCBI and retrieve webenvironment and history tags
+    :param accessions: list of GenBank protein accessions
+    :param args: cmd-line args parser
+    Return webenv and query key
+    """
+    # perform batch query of Entrez
+    accessions_string = ",".join(accessions)
+
+    # Runtime error captured by try/except function call
+    epost_result = Entrez.read(
+        entrez_retry(
+            Entrez.epost,
+            args,
+            db="Protein",
+            id=accessions_string,
+        )
+    )
+
+    # retrieve the web environment and query key from the Entrez post
+    epost_webenv = epost_result["WebEnv"]
+    epost_query_key = epost_result["QueryKey"]
+
+    return epost_webenv, epost_query_key
+
+
+if __name__ == "__main__":
+    main()
diff --git a/cazy_webscraper/expand/pdb/__init__.py b/cazy_webscraper/expand/pdb/__init__.py
new file mode 100644
index 00000000..16b7b61e
--- /dev/null
+++ b/cazy_webscraper/expand/pdb/__init__.py
@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+# Bio.PDB reference:
+# Hamelryck, T., Manderick, B. (2003) PDB parser and structure class 
+# implemented in Python. Bioinformatics 19: 2308–2310
+"""Retrieve PDB structures from RSCB PDB and write to disk"""
diff --git a/cazy_webscraper/expand/pdb/get_pdb_structures.py b/cazy_webscraper/expand/pdb/get_pdb_structures.py
new file mode 100644
index 00000000..d5183424
--- /dev/null
+++ b/cazy_webscraper/expand/pdb/get_pdb_structures.py
@@ -0,0 +1,220 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+# Bio.PDB reference:
+# Hamelryck, T., Manderick, B. (2003) PDB parser and structure class 
+# implemented in Python. Bioinformatics 19: 2308–2310
+"""Retrieve PDB structures from RSCB PDB and write to disk"""
+
+
+import logging
+import os
+import sys
+
+import pandas as pd
+
+from datetime import datetime
+from typing import List, Optional
+
+import Bio.PDB
+
+from tqdm import tqdm
+
+from cazy_webscraper import cazy_webscraper
+from cazy_webscraper.expand import get_chunks_gen
+from cazy_webscraper.sql.sql_interface import get_selected_pdbs
+from cazy_webscraper.utilities import config_logger, file_io, parse_configuration
+from cazy_webscraper.utilities.parsers import pdb_strctre_parser
+
+
+def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = None):
+    """Set up programme and initate run."""
+    time_stamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
+    start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
+    start_time = pd.to_datetime(start_time)
+    # parse cmd-line arguments
+    if argv is None:
+        parser = pdb_strctre_parser.build_parser()
+        args = parser.parse_args()
+    else:
+        args = pdb_strctre_parser.build_parser(argv).parse_args()
+
+    if logger is None:
+        logger = logging.getLogger(__package__)
+        config_logger(args)
+
+    # validate PDB file choices
+    logger.info("Checking valid file types were provided")
+    validate_pdb_file_types(args)
+
+    connection, logger_name, cache_dir = cazy_webscraper.connect_existing_db(args, time_stamp)
+
+    (
+        config_dict,
+        class_filters,
+        family_filters,
+        kingdom_filters,
+        taxonomy_filter_dict,
+        ec_filters,
+    ) = parse_configuration.get_expansion_configuration(args)
+
+    pdb_accessions = get_selected_pdbs.get_pdb_accessions(
+        class_filters,
+        family_filters,
+        taxonomy_filter_dict,
+        kingdom_filters,
+        ec_filters,
+        connection,
+    )
+
+    if len(pdb_accessions) == 0:
+        logger.warning(
+            "No PDB accessions matched the criteria provided.\n"
+            "Retrieving no protein structure files from PDB"
+        )
+    else:
+        logger.warning(f"Retrieving {len(pdb_accessions)} structure files from PDB")
+
+    # make output and cache dirs
+    if args.cache_dir is not None:  # use user defined cache dir
+        cache_dir = args.cache_dir
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+    else:
+        cache_dir = cache_dir / "uniprot_data_retrieval"
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+
+    download_pdb_structures(pdb_accessions, cache_dir, args)
+
+    cache_path = cache_dir / f"pdb_retrieval_{time_stamp}.txt"
+    with open(cache_path, 'a') as fh:
+        for acc in pdb_accessions:
+            fh.write(f"{acc}\n")
+
+    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+    end_time = pd.to_datetime(end_time)
+    total_time = end_time - start_time
+
+    if args.verbose:
+        logger.info(
+            "Finished getting structure files from PDB\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {cazy_webscraper.VERSION_INFO}\n"
+            f"Citation: {cazy_webscraper.CITATION_INFO}"
+        )
+    else:
+        print(
+            "=====================cazy_webscraper=====================\n"
+            "Finished getting structure files from PDB\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {cazy_webscraper.VERSION_INFO}\n"
+            f"Citation: {cazy_webscraper.CITATION_INFO}"
+        )
+
+
+def validate_pdb_file_types(args):
+    """Check valid file types for PDB were specified by the user.
+    :param args: cmd-line args parser
+    Return nothing.
+    """
+    logger = logging.getLogger(__name__)
+
+    valid_choices = ['mmCif', 'pdb', 'xml', 'mmtf', 'bundle']
+
+    invalid_choices = []
+
+    user_choices = (args.pdb).split(",")
+
+    for choice in user_choices:
+        if choice not in valid_choices:
+            invalid_choices.append(choice)
+
+    if len(invalid_choices) != 0:
+        logger.error(
+            f"Invalid file option selected: {invalid_choices}.\n"
+            f"The valid choices are: {valid_choices}.\n"
+            "Terminating program."
+        )
+        sys.exit(1)
+
+    return
+
+
+def download_pdb_structures(pdb_accessions, args):
+    """Download protein structure from the RSCB PDB database
+
+    :param pdb_accession: list of PDB accessions
+    :param args: cmd-line args parser
+
+    Return nothing.
+    """
+    pdbl = Bio.PDB.PDBList()
+
+    logger = logging.getLogger(__name__)
+    logger.warning("Starting downloading of structure files from PDB")
+
+    if args.outdir is None:
+        logger.warning("Downloading to current working directory")
+        for accession_list in get_chunks_gen(pdb_accessions, args.batch_limit):
+            for file_type in tqdm((args.pdb).split(","), desc="Downloading"):
+                pdbl.download_pdb_files(
+                    pdb_codes=accession_list,
+                    file_format=file_type,
+                    overwrite=args.overwrite,
+                )
+
+    else:
+        logger.warning(f"Downloading structures to {args.outdir}")
+        for accession_list in get_chunks_gen(pdb_accessions, args.batch_limit):
+            for file_type in tqdm((args.pdb).split(","), desc="Downloading"):
+                pdbl.download_pdb_files(
+                    pdb_codes=accession_list,
+                    file_format=file_type,
+                    overwrite=args.overwrite,
+                    pdir=args.outdir,
+                )
+
+    return
+
+
+if __name__ == "__main__":
+    main()
diff --git a/cazy_webscraper/expand/uniprot/__init__.py b/cazy_webscraper/expand/uniprot/__init__.py
new file mode 100644
index 00000000..a7342e76
--- /dev/null
+++ b/cazy_webscraper/expand/uniprot/__init__.py
@@ -0,0 +1,41 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Submodule for retrieving data from UniProtKB and adding it to a local CAZyme db"""
diff --git a/cazy_webscraper/expand/uniprot/get_uniprot_data.py b/cazy_webscraper/expand/uniprot/get_uniprot_data.py
new file mode 100644
index 00000000..51f9ebfc
--- /dev/null
+++ b/cazy_webscraper/expand/uniprot/get_uniprot_data.py
@@ -0,0 +1,391 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Retrieve data from UniProtKB and adding it to a local CAZyme db"""
+
+
+import logging
+import urllib.parse
+import urllib.request
+
+import pandas as pd
+
+from datetime import datetime
+from typing import List, Optional
+from urllib.error import HTTPError
+
+from bioservices import UniProt
+from tqdm import tqdm
+
+from cazy_webscraper import cazy_scraper
+from cazy_webscraper.expand import get_chunks_list
+from cazy_webscraper.sql import sql_interface
+from cazy_webscraper.sql.sql_interface import get_selected_gbks
+from cazy_webscraper.sql.sql_interface.add_uniprot_data import (
+    add_ec_numbers,
+    add_pdb_accessions,
+    add_uniprot_accessions,
+)
+from cazy_webscraper.sql import sql_orm
+from cazy_webscraper.utilities import config_logger, file_io
+from cazy_webscraper.utilities.parsers import uniprot_parser
+from cazy_webscraper.utilities.parse_configuration import get_expansion_configuration
+
+
+def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = None):
+    time_stamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")  # used in naming files
+    start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
+    start_time = pd.to_datetime(start_time)
+
+    # Program preparation
+    if argv is None:
+        parser = uniprot_parser.build_parser()
+        args = parser.parse_args()
+    else:
+        parser = uniprot_parser.build_parser(argv)
+        args = parser.parse_args()
+
+    if logger is None:
+        logger = logging.getLogger(__name__)
+        config_logger(args)
+    
+    # parse the configuration data (cache the uniprot data as .csv files)
+    connection, logger_name, cache_dir = cazy_scraper.connect_existing_db(args, time_stamp)
+
+    if args.cache_dir is not None:  # use user defined cache dir
+        cache_dir = args.cache_dir
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+    else:
+        cache_dir = cache_dir / "uniprot_data_retrieval"
+        file_io.make_output_directory(cache_dir, args.force, args.nodelete_cache)
+
+    (
+        config_dict,
+        class_filters,
+        family_filters,
+        kingdom_filters,
+        taxonomy_filter_dict,
+        ec_filters,
+    ) = get_expansion_configuration(args)
+
+    # add log to the local CAZyme database
+    logger.info("Adding log of scrape to the local CAZyme database")
+    with sql_orm.Session(bind=connection) as session:
+        retrieved_annotations = "UniProt accessions"
+        if len(config_dict['ec']) != 0:
+            retrieved_annotations = f"{retrieved_annotations}, EC numbers"
+        if len(config_dict['pdb']) != 0:
+            retrieved_annotations = f"{retrieved_annotations}, PDB accessions"
+        if len(config_dict['seq']) != 0:
+            retrieved_annotations = f"{retrieved_annotations}, Protein sequence"
+        if args.seq_update:
+            retrieved_annotations = f"{retrieved_annotations}, Updated UniProt protein sequences"
+        sql_interface.log_scrape_in_db(
+            time_stamp,
+            config_dict,
+            kingdom_filters,
+            taxonomy_filter_dict,
+            ec_filters,
+            'UniProt',
+            retrieved_annotations,
+            session,
+            args,
+        )
+
+    # retrieve dict of genbank accession and genbank accession ids from the local CAZyme db
+    gbk_dict = get_selected_gbks.get_genbank_accessions(
+        class_filters,
+        family_filters,
+        taxonomy_filter_dict,
+        kingdom_filters,
+        ec_filters,
+        connection,
+    )
+    genbank_accessions = list(gbk_dict.keys())
+
+    # retrieve the uniprot accessions for the genbank accessions
+    uniprot_gkb_dict = get_uniprot_accessions(genbank_accessions)  # {uniprot_acc: gbk_acc}
+    uniprot_dict, all_ecs = get_uniprot_data(uniprot_gkb_dict, cache_dir)
+
+    # add uniprot accessions (and sequences if seq retrieval is enabled)
+    add_uniprot_accessions(uniprot_dict, gbk_dict, connection, args)
+
+    # add ec numbers
+    if (args.ec) and (len(all_ecs) != 0):
+        add_ec_numbers(uniprot_dict, all_ecs, gbk_dict, connection, args)
+
+    # add pdb accessions
+    if args.pdb:
+        add_pdb_accessions(uniprot_dict, gbk_dict, connection, args)
+
+    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+    end_time = pd.to_datetime(end_time)
+    total_time = end_time - start_time
+
+    if args.verbose:
+        logger.info(
+            "Finished getting data from UniProt\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {cazy_scraper.VERSION_INFO}\n"
+            f"Citation: {cazy_scraper.CITATION_INFO}"
+        )
+    else:
+        print(
+            "=====================cazy_webscraper=====================\n"
+            "Finished getting data from UniProt\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {cazy_scraper.VERSION_INFO}\n"
+            f"Citation: {cazy_scraper.CITATION_INFO}"
+        )
+
+
+def get_uniprot_accessions(genbank_accessions, args):
+    """Retrieve UniProt accessions for the GenBank accessions from UniProt REST API.
+    
+    UniProt requests batch queries of no larger than 20,000, athough queries longer than 500
+    often raise HTTP 400 Error codes, especially in busy server times.
+
+    :param genbank_accessions: set, GenBank accessions to retrieve UniProt accessions for
+    :param args: cmd-line args parser
+
+    Return dict of {uniprot_accession: genbank_accession}
+    """
+    logger = logging.getLogger(__name__)
+    uniprot_url = 'https://www.uniprot.org/uploadlists/'
+
+    uniprot_rest_queries = get_chunks_list(genbank_accessions, args.uniprot_batch_size)
+
+    uniprot_gbk_dict = {}  # {uniprot_accession: gbk_accession}
+    failed_queries = {}  # {query: tries}
+
+    for query_chunk in tqdm(
+        uniprot_rest_queries,
+        desc='Batch retrieving UniProt accessions',
+    ):
+        if type(query) != str:
+            # convert the set of gbk accessions into str format
+            query = ' '.join(query_chunk)
+
+        params = {
+            'from': 'EMBL',
+            'to': 'ACC',
+            'format': 'tab',
+            'query': query
+        }
+
+        # submit query data
+        data = urllib.parse.urlencode(params)
+        data = data.encode('utf-8')
+        req = urllib.request.Request(uniprot_url, data)
+
+        # retrieve UniProt response
+        try:
+            with urllib.request.urlopen(req) as f:
+                response = f.read()
+        except HTTPError:
+            try:
+                failed_queries[query] += 1
+            except KeyError:
+                failed_queries[query] = 1
+            if failed_queries[query] > args.retries:
+                del failed_queries[query]
+            else:
+                uniprot_rest_queries.append(query)
+
+        uniprot_batch_response = response.decode('utf-8')
+
+        uniprot_batch_response = uniprot_batch_response.split('\n')
+
+        for line in uniprot_batch_response[1:]:  # the first line includes the titles, last line is an empty str
+            if line == '':  # add check incase last line is not an empty str 
+                continue
+            uniprot_gbk_dict[line.split('\t')[1]] = line.split('\t')[0]
+
+    logger.info(
+        f"Retrieved {len(genbank_accessions)} gbk accessions from the local db\n"
+        f"{len(list(uniprot_gbk_dict.keys()))} were assoicated with records in UniProt"
+    )
+
+    return uniprot_gbk_dict
+
+
+def get_uniprot_data(uniprot_gbk_dict, cache_dir, args):
+    """Batch query UniProt to retrieve protein data. Save data to cache directory.
+    
+    Bioservices requests batch queries no larger than 200.
+
+    :param uniprot_gbk_dict: dict, keyed by GenBank accession and valued by UniProt accession
+    :param cache_dir: path to directory to write out cache
+    :param args: cmd-line args parser
+    
+    Return
+    Dict of data retrieved from UniProt and to be added to the db 
+        {uniprot_acccession: {gbk_acccession: str, uniprot_name: str, pdb: set, ec: set}}
+    Set of all retrieved EC numbers
+    """
+    logger = logging.getLogger(__name__)
+
+    uniprot_dict = {}  # {uniprot: {genbank_accession: str, uniprot_name: str, pdb: set, ec: set}}
+    all_ecs = set()  # store all retrieved EC numbers as one tuple per unique EC number
+
+    # add PDB column to columns to be retrieved
+    UniProt()._valid_columns.append('database(PDB)')
+    
+    bioservices_queries = get_chunks_list(
+        list(uniprot_gbk_dict.keys()),
+        args.bioservices_batch_size,
+    )
+
+    for query in tqdm(bioservices_queries, "Batch retrieving protein data from UniProt"):
+        uniprot_df = UniProt().get_df(entries=query)
+
+        # cache UniProt response
+        _time_stamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
+        _path = cache_dir / f'uniprot_query_response_{_time_stamp}.csv'
+        uniprot_df.to_csv(_path)
+
+        index = 0
+        uniprot_df = uniprot_df[[
+            'Entry',
+            'Protein names',
+            'EC number',
+            'Sequence',
+            'Cross-reference (PDB)',
+        ]]
+
+        for index in tqdm(range(len(uniprot_df['Entry'])), desc="Parsing UniProt response"):
+            row = uniprot_df.iloc[index]
+            uniprot_acc = row['Entry']
+            uniprot_name = row['Protein names']
+
+            # checked if parsed before incase bioservices returned duplicate proteins
+            try:
+                uniprot_dict[uniprot_acc]
+                logger.warning(
+                    f'Multiple entries for UniProt:{uniprot_acc}, '
+                    f'GenBank:{uniprot_gbk_dict[uniprot_acc]} retrieved from UniProt,\n'
+                    'compiling data into a single record'
+                )
+            except KeyError:
+                try:
+                    uniprot_dict[uniprot_acc] = {
+                        "genbank_accession": uniprot_gbk_dict[uniprot_acc],
+                        "name": uniprot_name,
+                        }
+                except KeyError:
+                    logger.warning(
+                        f"Retrieved record with UniProt accession {uniprot_acc} but this "
+                        "accession was not\nretrieved from the UniProt REST API"
+                    )
+                    continue
+            
+            if args.ec:
+                # retrieve EC numbers
+                ec_numbers = row['EC number']
+                try:
+                    ec_numbers = ec_numbers.split('; ')
+                except AttributeError:
+                    # no EC numbers listed
+                    ec_numbers = set()
+                
+                try:
+                    uniprot_dict[uniprot_acc]["ec"]
+                except KeyError:
+                    uniprot_dict[uniprot_acc]["ec"] = set()
+
+                # add EC numbers to dict
+                for ec in ec_numbers:
+                    all_ecs.add( (ec,) )
+                    uniprot_dict[uniprot_acc]["ec"].add(ec)
+
+            if args.pdb:
+                # retrieve PDB accessions
+                pdb_accessions = row['Cross-reference (PDB)']
+                print('PDBS:', pdb_accessions)
+                try:
+                    pdb_accessions = pdb_accessions.split('; ')
+                except AttributeError:
+                    pdb_accessions = set()
+
+                try:
+                    uniprot_dict[uniprot_acc]["pdb"]
+                except KeyError:
+                    uniprot_dict[uniprot_acc]["pdb"] = set()
+
+                # add PDB accessions to dict
+                for pdb in pdb_accessions:
+                    uniprot_dict[uniprot_acc]["pdb"].add(pdb)
+            
+            if args.sequence:
+                sequence = row['Sequence']
+
+                try:
+                    uniprot_dict[uniprot_acc]["sequence"]
+                    existing_date = uniprot_dict[uniprot_acc]["seq_date"]
+                    new_date = row['Date of last sequence modification']
+                    
+                    # check which sequence is newer
+                    existing_date.split('-')
+                    existing_date = datetime(existing_date[0], existing_date[1], existing_date[2])
+                    new_date.split('-')
+                    new_date = datetime(existing_date[0], existing_date[1], existing_date[2])
+
+                    if new_date > existing_date:  # past < present is True
+                        uniprot_dict[uniprot_acc]["sequence"] = sequence
+                        uniprot_dict[uniprot_acc]["seq_date"] = row['Date of last sequence modification']
+                    # else keep the existing sequence
+                    logger.warning(
+                        f'Multiple sequences retrieved for {uniprot_acc}\n'
+                        'Using most recently updated sequence'
+                    )
+                    
+                except KeyError:
+                    uniprot_dict[uniprot_acc]["sequence"] = sequence
+                    uniprot_dict[uniprot_acc]["seq_date"] = row['Date of last sequence modification']
+
+    return uniprot_dict, all_ecs
+
+
+if __name__ == "__main__":
+    main()
diff --git a/scraper/expand/__init__.py b/cazy_webscraper/genomes/__init__.py
similarity index 93%
rename from scraper/expand/__init__.py
rename to cazy_webscraper/genomes/__init__.py
index f4c6ddcb..bc59be0b 100644
--- a/scraper/expand/__init__.py
+++ b/cazy_webscraper/genomes/__init__.py
@@ -36,4 +36,6 @@
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
-"""A module for expanding the the local CAZY database beyond what is provided within CAZy."""
+"""Retrieve genomic accessions for protein accessions in the local CAZyme database"""
+
+
diff --git a/cazy_webscraper/genomes/cazy_genbank_coverage.py b/cazy_webscraper/genomes/cazy_genbank_coverage.py
new file mode 100644
index 00000000..aee457e3
--- /dev/null
+++ b/cazy_webscraper/genomes/cazy_genbank_coverage.py
@@ -0,0 +1,723 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+
+# Contact
+# eemh1@st-andrews.ac.uk
+
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Explore the number of GenBank genomes annotated by CAZy."""
+
+
+import json
+import logging
+import os
+import time
+
+import pandas as pd
+
+from datetime import datetime
+from matplotlib import pyplot as plt
+from typing import List, Optional
+
+from Bio import Entrez
+from saintBioutils.genbank import entrez_retry
+from saintBioutils.utilities.file_io import make_output_directory
+from saintBioutils.utilities.logger import config_logger
+from tqdm import tqdm
+
+from cazy_webscraper import cazy_scraper
+from cazy_webscraper.expand import get_chunks_list
+from cazy_webscraper.genomes import entrez
+from cazy_webscraper.sql.sql_interface import get_table_dicts
+from cazy_webscraper.utilities.parsers import genbank_cov_parser
+
+
+KINGDOMS = ['Bacteria', 'Eukaryota', 'Archaea', 'Viruses', 'unclassified']
+
+
+def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = None):
+    time_stamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")  # used in naming files
+    start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
+    start_time = pd.to_datetime(start_time)
+
+    # Program preparation
+    if argv is None:
+        parser = genbank_cov_parser.build_parser()
+        args = parser.parse_args()
+    else:
+        parser = genbank_cov_parser.build_parser(argv)
+        args = parser.parse_args()
+
+    if logger is None:
+        logger = logging.getLogger(__name__)
+        config_logger(args)
+
+    Entrez.email = args.email
+
+    # check if need to build output dir
+    if os.getcwd() != args.output_dir:
+        make_output_directory(args.output_dir, args.force, args.nodelete)
+
+    # connect to the local CAZyme database
+    connection, logger_name, cache_dir = cazy_scraper.connect_existing_db(args, time_stamp)
+    # make cache_dir
+    make_output_directory(cache_dir, args.force_cache, args.nodelete_cache)
+
+    no_accession_logger = cache_dir / "logs"
+    # make logs dir
+    make_output_directory(no_accession_logger, args.force_cache, args.nodelete_cache)
+    no_accession_logger = no_accession_logger / f"no_genomic_accession_retrieved_{time_stamp}.log"
+
+    # load Genbank and Kingdom records from the db
+    logger.warning("Retrieving Genbanks, Taxs and Kingdoms records from the local CAZyme db")
+    genbank_kingdom_dict = get_table_dicts.get_gbk_kingdom_dict(connection)
+    logger.warning("Retrieved Genbanks, Taxs and Kingdoms records from the local CAZyme db")
+
+    nucleotide_accessions_dict = get_nucleotide_accessions(
+        genbank_kingdom_dict,
+        no_accession_logger,
+        args,
+    )
+
+    output_path = cache_dir / f"nucleotide_accessions_{time_stamp}.json"
+    with open(output_path, 'w') as fh:
+        json.dump(nucleotide_accessions_dict, fh)
+    
+    genomic_accession_dict = get_genomic_accessions(nucleotide_accessions_dict, no_accession_logger, args)
+    output_path = cache_dir / f"genomic_accession_numbers_{time_stamp}.json"
+    with open(output_path, 'w') as fh:
+        json.dump(genomic_accession_dict, fh)
+
+    write_out_genomic_accessions(genomic_accession_dict, time_stamp, args)
+
+    ncbi_genomes_totals = get_ncbi_counts(args)
+
+    write_out_genome_coverage(ncbi_genomes_totals, genomic_accession_dict, time_stamp, args)
+
+    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+    end_time = pd.to_datetime(end_time)
+    total_time = end_time - start_time
+
+    if args.verbose:
+        logger.info(
+            "Finished calculting the local CAZyme db's coverage of GenBank\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {cazy_scraper.VERSION_INFO}\n"
+            f"Citation: {cazy_scraper.CITATION_INFO}"
+        )
+    else:
+        print(
+            "=====================cazy_webscraper=====================\n"
+            "Finished calculting the local CAZyme db's coverage of GenBank\n"
+            f"Scrape initated at {start_time}\n"
+            f"Scrape finished at {end_time}\n"
+            f"Total run time: {total_time}"
+            f"Version: {cazy_scraper.VERSION_INFO}\n"
+            f"Citation: {cazy_scraper.CITATION_INFO}"
+        )
+
+
+def get_nucleotide_accessions(genbank_kingdom_dict, no_accession_logger, args):
+    """Retrieve the NCBI Nucleotide db records ID's containing the GenBank protein accessions.
+    
+    :param genbank_kingdom_dict: dict of Genbank and Kingdom records from db
+        {kingdom: {genus: {species: {protein_accessions}}}
+    :param no_accession_logger: Path, path to log file to save protein accessions for which no 
+        genomic accession was retrieved
+    :param args: cmd-line args parser
+    
+    Return dict {kingdom: {genus: {species: {nucleotide_accession: {protein_accessions},},},},}
+    """
+    logger = logging.getLogger(__name__)
+
+    nucleotide_accessions_dict = {}
+    # {kingdom: {genus: {species: {nucleotide record accessio: {protein_accessions},},},},}
+
+    retrieved_proteins = {}  # {protein_accession: nucleotide record accession}
+
+    # store all retrieved ncucleotide accessions to prevent retrieval of the same assembly record multiple times
+    
+    for kingdom in tqdm(genbank_kingdom_dict, desc="Retrieving nucleotide record accessions per kingdom"):
+        genera = genbank_kingdom_dict[kingdom]
+
+        gbk_organism_dict = {}  # protein_accession: {species: str, genus: str}
+
+        for genus in genera:
+            organisms = genera[genus]
+
+            for species in organisms:
+                for gbk_acc in organisms[species]:
+                    gbk_organism_dict[gbk_acc] = {'species': species, 'genus': genus}
+        
+        # retrieve all Gbk protein accessions for the given genera
+        gbk_accessions = list(gbk_organism_dict.keys())
+
+        # gbk accessions waiting for a nucleotide id to be retrieved
+        remaining_accessions = list(gbk_organism_dict.keys())
+
+        while len(remaining_accessions) != 0:
+            starting_loop_length = len(remaining_accessions)
+            logger.warning(f"{len(remaining_accessions)} Gbk accessions remaining to be parsed for {kingdom}")
+            
+            accessions_to_parse = remaining_accessions[:args.batch_size]
+
+            # eLink Protein to Nuccore db and retrieve IDs of linked nucleotide records
+            batch_query_ids = ",".join(accessions_to_parse)
+            with entrez_retry(
+                args.retries, Entrez.epost, "Protein", id=batch_query_ids,
+            ) as handle:
+                batch_post = Entrez.read(handle)
+
+            # {protein record ID: {nucleotide records IDs}}
+            nucleotide_ids = entrez.get_linked_nucleotide_record_ids(batch_post, args)
+
+            if nucleotide_ids is None: 
+                # issue with at least one accession in the batch
+                # e.g. it is not longer stored in NCBI
+                # pass individually to find/parse the bad accession(s)
+                nucleotide_ids, no_nucleotides = entrez.link_nucleotide_ids_individually(
+                    accessions_to_parse,
+                    no_accession_logger,
+                    args,
+                )
+
+                if len(no_nucleotides) != 0:
+                    logger.warning(f"{len(no_nucleotides)} proteins found with no linked Nucleotide records")
+                    for protein_accession in no_nucleotides:
+                        # already logged in link_nucleotide_ids_individually()
+                        try:
+                            remaining_accessions.remove(protein_accession)
+                        except ValueError:
+                            pass
+            
+            if len(list(nucleotide_ids.values())) == 0:
+                # no linked Nucleotide records retrieved for the current batch of protein accessions
+                for protein_accession in accessions_to_parse:
+                    logger.warning(
+                        f"Could not reitreve linked Nucleotide record for {protein_accession}"
+                    )
+                    # do not try to retrieve record again
+                    try:
+                        remaining_accessions.remove(protein_accession)
+                    except ValueError:
+                        pass
+                continue
+
+            # retrieves the nucleotide records IDs for protein records for whcih only one
+            # nuclotide ID was retrieved
+            single_nucleotide_ids = set()
+
+            # retrieve the protein_record_ids of protein records from which multiple nucletoide 
+            # record IDs were retrieved
+            protein_records_multi_nuc = set()
+
+            for protein_record_id in tqdm(nucleotide_ids, desc="Idenitfying proteins with multiple linked nucleotide records"):
+                if len(nucleotide_ids[protein_record_id]) == 1:
+                    single_nucleotide_ids.add(list(nucleotide_ids[protein_record_id])[0])
+                else:
+                    logger.warning(
+                        f"Found {len(nucleotide_ids[protein_record_id])} linked nucletoide records "
+                        f"for protein record {protein_record_id}"
+                        )
+                    protein_records_multi_nuc.add(protein_record_id)
+
+            # batch query to fetch nucletoide records for protein records
+            # from which only a sinlge nucleotide ID was retrieved
+            if len(single_nucleotide_ids) != 0:
+                retrieved_proteins, newly_retrieved_proteins, succcess = entrez.extract_protein_accessions(
+                    single_nucleotide_ids,
+                    retrieved_proteins,
+                    gbk_accessions,
+                    args,
+                )
+                if succcess is False:
+                    # issue with at least one accession in the batch
+                    # e.g. it is not longer stored in NCBI
+                    # pass individually to find/parse the bad accession(s)
+                    retrieved_proteins, newly_retrieved_proteins = entrez.extract_protein_accessions_individually(
+                        single_nucleotide_ids,
+                        retrieved_proteins,
+                        gbk_accessions,
+                        args,
+                    )
+
+                # add the nucleotide accessions to the nucleotide_accessions_dict
+                # {kingdom: {genus: {species: {nucleotide record accessio: {protein_accessions},},},},}
+                nucleotide_accessions_dict = add_nucleotide_accessions(
+                    nucleotide_accessions_dict,
+                    gbk_organism_dict,
+                    retrieved_proteins,
+                    newly_retrieved_proteins,
+                    kingdom,
+                    no_accession_logger,
+                )
+
+            # for Protein records for which multiple Nucleotide record IDs were retrieved
+            # Identify the longest Nucleotide record and retrieve protein accessions from it
+            # The longest record is most likely to be the most complete record
+            for protein_record_id in tqdm(protein_records_multi_nuc, "Parsing protein records with multiple linked Nucletide records"):
+                nucleotide_record_ids = nucleotide_ids[protein_record_id]
+
+                retrieved_proteins, newly_retrieved_proteins, success = entrez.parse_longest_record(
+                    nucleotide_record_ids,
+                    retrieved_proteins,
+                    gbk_accessions,
+                    args,
+                )
+
+                if success is False:
+                    # issue with at least one accession in the batch
+                    # e.g. it is not longer stored in NCBI
+                    # pass individually to find/parse the bad accession(s)
+                    retrieved_proteins, newly_retrieved_proteins, success = entrez.parse_longest_record_individually(
+                        nucleotide_record_ids,
+                        retrieved_proteins,
+                        gbk_accessions,
+                        args,
+                    )
+
+                nucleotide_accessions_dict = add_nucleotide_accessions(
+                    nucleotide_accessions_dict,
+                    gbk_organism_dict,
+                    retrieved_proteins,
+                    newly_retrieved_proteins,
+                    kingdom,
+                    no_accession_logger,
+                )
+
+            # remove protein accessions from remaining_accessions because the linked Nucleotide 
+            # record ID has already been retrieved
+            for protein_accession in retrieved_proteins:
+                try:
+                    remaining_accessions.remove(protein_accession)
+                except ValueError:
+                    pass
+
+            if starting_loop_length == len(remaining_accessions) and len(remaining_accessions) != 0:
+                # failing to retrieve data for protein accessions
+                nucleotide_ids, no_nucleotides = entrez.link_nucleotide_ids_individually(
+                    accessions_to_parse,
+                    no_accession_logger, 
+                    args,
+                )
+
+                if nucleotide_ids is None:
+                    for protein_accession in remaining_accessions:
+                        logger.warning(
+                                f"Could not retrieve  Nucletoide record for {protein_accession}"
+                            )
+                        try:
+                            remaining_accessions.remove(protein_accession)
+                        except ValueError:
+                            pass
+                    continue
+                
+            if len(list(nucleotide_ids.values())) == 0:
+                # no linked Nucleotide records retrieved for the current batch of protein accessions
+                for protein_accession in accessions_to_parse:
+                    logger.warning(
+                        f"Could not reitreve linked Nucleotide record for {protein_accession}"
+                    )
+                    # do not try to retrieve record again
+                    try:
+                        remaining_accessions.remove(protein_accession)
+                    except ValueError:
+                        pass
+        
+            if len(no_nucleotides) != 0:
+                for protein_accession in no_nucleotides:
+                    # already logged in link_nucleotide_ids_individually()
+                    try:
+                        remaining_accessions.remove(protein_accession)
+                    except ValueError:
+                        pass
+            
+    return nucleotide_accessions_dict
+
+
+def add_nucleotide_accessions(
+    nucleotide_accessions_dict,
+    gbk_organism_dict,
+    retrieved_proteins,
+    newly_retrieved_proteins,
+    kingdom,
+    no_accession_logger,
+):
+    """Add Nucleotide accession and protein acccession to the nucleotide_accessions_dict.
+    
+    :param nucleotide_accessions_dict: dict
+        {kingdom: {genus: {species: {nucleotide record accessio: {protein_accessions},},},},}
+    :param gbk_organism_dict: dict,
+        {protein_accession: {species: str, genus: str}}
+    :param retrieved_proteins: dict
+        {protein record id: {nucleotide records ids}}
+    :param newly_retrieved_proteins: list of protein's retrieved from Nucleotide records and 
+        not yet added to nucleotide_accessions_dict
+    :param kingdom: str, taxonomic kingdom of protein's source organism
+    :param no_accession_logger: Path, path to log file to save protein accessions for which no 
+            genomic accession was retrieved
+    
+    Return dict
+    """
+    logger = logging.getLogger(__name__)
+
+    for protein_accession in newly_retrieved_proteins:
+        nucleotide_accession = retrieved_proteins[protein_accession]
+
+        try:
+            species = gbk_organism_dict[protein_accession]['species']
+            genus = gbk_organism_dict[protein_accession]['genus']
+        except KeyError:
+            logger.error(
+                f"Retrieved protein accession from NCBI that is not in the local CAZyme db"
+            )
+            with open(no_accession_logger, 'a') as fh:
+                fh.write(
+                    f"{protein_accession}\t"
+                    "Retrieved protein accession from NCBI that is not in the local CAZyme db\t"
+                    f"{genus} {species}\n"
+                )
+            continue
+    try:
+        nucleotide_accessions_dict[kingdom]
+
+        try:
+            nucleotide_accessions_dict[kingdom][genus]
+
+            try:
+                nucleotide_accessions_dict[kingdom][genus][species]
+
+                try:
+                    nucleotide_accessions_dict[kingdom][genus][species][nucleotide_accession].add(protein_accession)
+                
+                except KeyError:
+                    nucleotide_accessions_dict[kingdom][genus][species][nucleotide_accession] = {protein_accession}
+
+            except KeyError:
+                nucleotide_accessions_dict[kingdom][genus][species] = {
+                    nucleotide_accession: {protein_accession},
+                }
+            
+        except KeyError:
+            nucleotide_accessions_dict[kingdom][genus] = {
+                species: {
+                    nucleotide_accession: {protein_accession},
+                },
+            }
+
+    except KeyError:
+        nucleotide_accessions_dict[kingdom] = {
+            genus: {
+                species: {
+                    nucleotide_accession: {protein_accession},
+                },
+            },
+        }
+    
+    return nucleotide_accessions_dict
+
+
+def get_genomic_accessions(nucleotide_accessions_dict, no_accession_logger, args):
+    """Retrieve genomic accessions for the genomic assemblies
+
+    :param nucleotide_accessions_dict: dict
+        {kingdom: {genus: {species: {nucleotide record accession: {protein_accessions},},},},}
+    :param no_accession_logger: Path, path to log file to write out assembly names for which no
+        genomic accession was retrieved
+    :param args: cmd-line args parser
+    
+    Return dict,
+    {kingdom: {genus: {species: {genomic_accession: {proteins: {protein_accessions}, count=int},},},},}
+    """
+    logger = logging.getLogger(__name__)
+
+    genomic_accession_dict = {}
+    # {kingdom: {genus: {species: {genomic_accession: {proteins: {protein_accessions}, count=int},},},},}
+
+    for kingdom in tqdm(nucleotide_accessions_dict, desc='Retrieving genomic accessions per kingdom'):
+        genera = nucleotide_accessions_dict[kingdom]
+        for genus in genera:
+            organisms = genera[genus]
+            for species in organisms:
+                # retrieve all genomic assembly names for the given species
+                assembly_names = list(organisms[species].keys()) 
+
+                # break up the list into a series of smaller lists that can be batched querried
+                batch_queries = get_chunks_list(args.batch_size, assembly_names)
+
+                for batch_query in tqdm(batch_queries, desc=f"Batch querying for {genus} {species}"):
+                    batch_query_ids = ",".join(batch_query)
+
+                # retrieve the records IDs for the assembly names
+                with entrez_retry(
+                    args.retries,
+                    Entrez.esearch,
+                    "Assembly",
+                    id=batch_query_ids,
+                ) as handle:
+                    batch_post = Entrez.read(handle)
+                
+                with entrez_retry(
+                    args.retries,
+                    Entrez.efetch,
+                    db="Assembly",
+                    query_key=batch_post['QueryKey'],
+                    WebEnv=batch_post['WebEnv'],
+                    retmode="xml",
+                ) as handle:
+                    batch_fetch = Entrez.read(handle)
+
+                genomic_accessions = {}
+
+                for genome_record in tqdm(batch_fetch, desc="Retrieving assembly IDs"):
+                    index = 0
+                    accessions = set()
+
+                    for index in range(len(genome_record['IdList'])):
+                        with entrez_retry(
+                            10, Entrez.efetch, db="Assembly", id=genome_record['IdList'][index], retmode="xml", rettype="docsum",
+                        ) as handle:
+                            result = Entrez.read(handle)
+                        genomic_accession = result['DocumentSummarySet']['DocumentSummary'][0]['AssemblyAccession']
+                        assembly_name = result['DocumentSummarySet']['DocumentSummary'][0]['AssemblyName']
+
+                        genomic_accessions[genomic_accession] = assembly_name
+
+                    accessions = list(genomic_accessions.keys)
+                    accessions.sort(reverse=True)
+                    latest_accession = accessions[0]
+                    latest_assembly_name = genomic_accessions[latest_accession]
+
+                    # replace assemlby name for genomic accession
+                    try:
+                        protein_accessions = nucleotide_accessions_dict[kingdom][genus][species][latest_assembly_name]
+
+                    except KeyError:
+                        logger.warning(f"Retrieved assembly name {latest_assembly_name}, but not retrieved previously")
+                        with open(no_accession_logger, 'a') as fh:
+                            fh.write(
+                                f"{latest_assembly_name}\tRetrieved assembly name, but not retrieved previously\t"
+                                f"{latest_accession}\t{genus} {species}\n"
+                            )
+                        continue
+
+                    try:
+                        genomic_accession_dict[kingdom]
+
+                        try:
+                            genomic_accession_dict[kingdom][genus]
+
+                            try:
+                                genomic_accession_dict[kingdom][genus][species]
+
+                            except KeyError:
+                                genomic_accession_dict[kingdom][genus][species] = {
+                                    genomic_accession: {
+                                        'proteins': protein_accessions,
+                                        'count': len(protein_accessions),
+                                    },
+                                }
+
+                        except KeyError:
+                            genomic_accession_dict[kingdom][genus] = {
+                                species: {
+                                    genomic_accession: {
+                                        'proteins': protein_accessions,
+                                        'count': len(protein_accessions),
+                                    },
+                                },
+                            }
+
+                    except KeyError:
+                        genomic_accession_dict[kingdom] = {
+                            genus: {
+                                species: {
+                                    genomic_accession: {
+                                        'proteins': protein_accessions,
+                                        'count': len(protein_accessions),
+                                    },
+                                },
+                            },
+                        }
+
+    return genomic_accession_dict
+
+
+def write_out_genomic_accessions(genomic_accession_dict, time_stamp, args):
+    """Compile and write output listing the genomic accessions.
+    
+    :param genomic_accession_dict:
+    :param time_stamp: str, date and time script was invoked
+    :param args: cmd-line args parser
+    
+    Return nothing
+    """
+    raw_json_path = args.output_dir / f"cazy_genomic_accessions_{time_stamp}.json"
+    with open(raw_json_path, 'w') as fh:
+        json.dump(genomic_accession_dict, fh)
+
+    genomic_df_column_names = ['Kingdom','Genus','Species','Genomic_accession','#ofProteins']
+    genomic_df = pd.DataFrame(columns=genomic_df_column_names)
+
+    protein_df_column_names = ['Kingdom', 'Genus', 'Species', 'Genomic_accession', 'Protein_accession']
+    protein_df = pd.DataFrame(columns=protein_df_column_names)
+
+    for kingdom in genomic_accession_dict:
+        genera = genomic_accession_dict[kingdom]
+
+        for genus in genera:
+            organisms = genera[genus]
+            
+            for organism in organisms:
+                genomes = organisms[organism]
+            
+                for genomic_accession in genomes:
+                    protein_accessions = organisms[genomic_accession]['Proteins']
+                    number_of_proteins = organisms[genomic_accession]['Count']
+
+                    g_new_row_data = [kingdom, genus, organism, genomic_accession, number_of_proteins]
+                    g_new_row = pd.DataFrame([g_new_row_data], columns=genomic_df_column_names)
+                    genomic_df = genomic_df.append(g_new_row)
+
+                    for protein in protein_accessions:
+                        p_new_row_data = [kingdom, genus, organism, genomic_accession, protein]
+                        p_new_row = pd.DataFrame([p_new_row_data], columns=protein_df_column_names)
+                        protein_df = protein_df.append(p_new_row)
+    
+    genomic_csv = args.output_dir / f"genomic_accessions_{time_stamp}.csv"
+    genomic_df.to_csv(genomic_csv)
+
+    protein_csv = args.output_dir / f"protein_genomic_accessions_{time_stamp}.csv"
+    protein_df.to_csv(protein_csv)
+
+    return
+
+
+def get_ncbi_counts(args):
+    """Retrieve the number of genomic assemblies per Kingdom from NCBI.
+    
+    :param args: cmd-line args parser
+    
+    Return dict {kingdom: count}
+    """
+    counts = {}
+    
+    for kingdom in KINGDOMS:
+        with entrez_retry(
+            args.retries,
+            Entrez.esearch,
+            db="Assembly",
+            term=kingdom,
+            retmode="xml",
+        ) as record_handle:
+            record = Entrez.read(record_handle, validate=False)
+    
+        number_of_assemblies = record['Count']
+        counts[kingdom] = number_of_assemblies
+
+    return counts
+
+
+def write_out_genome_coverage(ncbi_genomes_totals, genomic_accession_dict, time_stamp, args):
+    """Write out the genome coverage of NCBI GenBank database by the local CAZyme database
+    
+    :param ncbi_genomes_totals: dict {kingdom: number of NCBI GenBank genomes
+    :param genomic_accession_dict: dict
+        {kingdom: {genus: {species: {accession: {proteins: set(), counts: int}}}}
+    :param time_stamp: str, date and time script was invoked
+    :param args: cmd-line args parser
+    
+    Return nothing
+    """
+    column_names = ['Kingdom', 'NCBI_genomes', 'CAZy_genomes', 'Coverage_percent']
+    coverage_df = pd.DataFrame(columns=column_names)
+    graph_columns = ['Kingdom', 'NCBI', 'CAZy']
+    graph_df = pd.DateFrame(columns=graph_columns)
+
+    for kingdom in KINGDOMS:
+        ncbi = ncbi_genomes_totals[kingdom]
+
+        cazy = 0
+        genera = genomic_accession_dict[kingdom]
+        for genus in genera:
+            organisms = genera[genus]
+            for species in organisms:
+                species_genome_accessions = len(list(organisms[species].keys()))
+                cazy += species_genome_accessions
+
+        coverage = (cazy / ncbi) * 100
+
+        row_data = [kingdom, ncbi, cazy, coverage]
+        new_row = pd.DataFrame([row_data], columns=column_names)
+        coverage_df = coverage_df.append(new_row)
+
+        row_data = [kingdom, int(ncbi), int(cazy)]
+        new_row = pd.DataFrame([row_data], columns=graph_columns)
+        graph_df = graph_df.append(new_row)
+
+    output_path = args.output_dir / f"cazy_genbank_genome_coverage_{time_stamp}.csv"
+    coverage_df.to_csv(output_path)
+
+    fig, ax = plt.subplots()
+    # plot CAZy bars
+    ax.bar(
+        graph_df['Kingdom'],
+        graph_df['CAZy'],
+        label='CAZy',
+        color='orange',
+    )
+    # add NCBI bars (the higher bars)
+    ax.bar(
+        graph_df['Kingdom'],
+        graph_df['NCBI'],
+        bottom=graph_df['CAZy'],
+        label='NCBI',
+        color='dodgerblue',
+    )
+    ax.set_ylabel('Kingdom')
+    ax.set_xlabel('Number of genomes in the database')
+    ax.set_title('GenBank genomes included in CAZy')
+    
+    ax.legend()
+
+    output_path = args.output_dir / f"gbk_cazy_genomes_plot_{time_stamp}.png"
+    fig.savefig(output_path, bbox_inches='tight', dpi=360)
+
+    return
+
+
+if __name__ == "__main__":
+    main()
diff --git a/cazy_webscraper/genomes/entrez.py b/cazy_webscraper/genomes/entrez.py
new file mode 100644
index 00000000..651a362c
--- /dev/null
+++ b/cazy_webscraper/genomes/entrez.py
@@ -0,0 +1,459 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+
+# Contact
+# eemh1@st-andrews.ac.uk
+
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Functions that call to and parse data from NCBI.Entrez."""
+
+
+import logging
+
+from Bio import Entrez
+from saintBioutils.genbank import entrez_retry
+from tqdm import tqdm
+
+
+def get_linked_nucleotide_record_ids(batch_post, args):
+    """Use Entrez.elink to get the ID of the NCBI.Nucleotide db record linked to the NCBI.Protein
+    db record.
+    
+    In the resulting dict, the protein record ID is used to group nucleotide records that are 
+    linked to the same protein record, and from which it will need to be determined which is
+    the longest record, so that only one (and the longest) Nucleotide record is parsed for this protein
+    not all retrieved Nucleotide records.
+
+    :param batch_post: Entrez dict containing WebEnv and query_key from batch posting protein accessions
+    :param args: cmd-line args parser
+
+    Return dict {protein_accession: {nucleotide records IDs}}
+    Return None if cannot retrieve data from NCBI
+    """
+    logger = logging.getLogger(__name__)
+
+    nucleotide_ids = {}  # {protein_record_id: {nucleotide_ids}}
+    # used for identifying which nucleotide record to retrieve protein accessions from
+    with entrez_retry(
+        args.retries,
+        Entrez.elink,
+        dbfrom="Protein",
+        db="nuccore",
+        query_key=batch_post['QueryKey'],
+        WebEnv=batch_post['WebEnv'],
+        linkname='protein_nuccore',
+    ) as handle:
+        try:
+            batch_nuccore = Entrez.read(handle)
+        except Exception as err:
+            logger.warning(f"Failed Entrez connection: {err}\nNo nucletoide IDs retrieved for batch query")
+            return
+
+    for record in tqdm(batch_nuccore, desc="Retrieving Nucleotide db records IDs from nuccore"):
+        protein_record_id = record['IdList'][0]
+        record_nucleotide_ids = set()
+    
+        # Linked db records are contained in the 'LinkSetDb' field
+        # multiple linked records may be retrieved
+        
+        for link_dict in record['LinkSetDb']:
+            # record['LinkSetDb'] = [{'Link': [{'Id': '1127815138'}], 'DbTo': 'nuccore', 'LinkName': 'protein_nuccore'}]
+            # link_list = {'Link': [{'Id': '1127815138'}], 'DbTo': 'nuccore', 'LinkName': 'protein_nuccore'}
+            links = link_dict['Link']
+            for link in links:
+                # links = [{'Id': '1127815138'}]
+                # link = {'Id': '1127815138'}
+                nucleotide_id = link['Id']
+
+                record_nucleotide_ids.add(nucleotide_id)
+
+        try:
+            existing_ids = nucleotide_ids[protein_record_id]
+            all_ids = existing_ids.union(record_nucleotide_ids)
+            nucleotide_ids[protein_record_id] = all_ids
+        except KeyError:
+            nucleotide_ids[protein_record_id] = record_nucleotide_ids
+    
+    return nucleotide_ids
+
+
+def link_nucleotide_ids_individually(accessions_to_parse, no_accession_logger, args):
+    """Use Entrez.elink to get the ID of the NCBI.Nucleotide db record linked to the NCBI.Protein
+    db record.
+    
+    In the resulting dict, the protein record ID is used to group nucleotide records that are 
+    linked to the same protein record, and from which it will need to be determined which is
+    the longest record, so that only one (and the longest) Nucleotide record is parsed for this protein
+    not all retrieved Nucleotide records.
+
+    :param accessions_to_parse: list of protein accessions
+    :param no_accession_logger: path to log file for logging protein accessions for which no 
+        linked nucleotide record was retrieved
+    :param args: cmd-line args parser
+
+    Return dict {protein_accession: {nucleotide records IDs}}
+    no_nucleotides: set of protein accessions for which no nucleotide record could be retrieved
+    """
+    logger = logging.getLogger(__name__)
+
+    nucleotide_ids = {}  # {protein_record_id: {nucleotide_ids}}
+    no_nucleotides = set()  # protein accessions for which no nucleotide record could be retrieved
+
+    for protein_accession in tqdm(accessions_to_parse, "Retrieving Nucleotide IDs individually"):
+    # used for identifying which nucleotide record to retrieve protein accessions from
+        with entrez_retry(
+            args.retries,
+            Entrez.elink,
+            dbfrom="Protein",
+            db="nuccore",
+            id=protein_accession,
+        ) as handle:
+            try:
+                batch_nuccore = Entrez.read(handle)
+            except Exception as err:
+                logger.warning(
+                    f"Could not retrieve linked Nucletoide record for {protein_accession}"
+                )
+                with open(no_accession_logger, "a") as fh:
+                    fh.write(
+                        f"Could not retrieve linked Nucletoide record for {protein_accession}\t"
+                        f"Error: {err}"
+                    )
+                no_nucleotides.add(protein_accession)
+                continue
+
+        for record in tqdm(batch_nuccore, desc="Retrieving Nucleotide db records IDs from nuccore"):
+            protein_record_id = record['IdList'][0]
+            record_nucleotide_ids = set()
+        
+            # Linked db records are contained in the 'LinkSetDb' field
+            # multiple linked records may be retrieved
+
+            if len(record['LinkSetDb']) == 0:
+                logger.warning(
+                    f"Could not retrieve linked Nucletoide record for {protein_accession}"
+                )
+                err = "No data containiend in 'LinkSetDb' field from eLink output"
+                with open(no_accession_logger, "a") as fh:
+                    fh.write(
+                        f"Could not retrieve linked Nucletoide record for {protein_accession}\t"
+                        f"Error: {err}"
+                    )
+                no_nucleotides.add(protein_accession)
+                continue
+            
+            for link_dict in record['LinkSetDb']:
+                # record['LinkSetDb'] = [{'Link': [{'Id': '1127815138'}], 'DbTo': 'nuccore', 'LinkName': 'protein_nuccore'}]
+                # link_list = {'Link': [{'Id': '1127815138'}], 'DbTo': 'nuccore', 'LinkName': 'protein_nuccore'}
+                links = link_dict['Link']
+                for link in links:
+                    # links = [{'Id': '1127815138'}]
+                    # link = {'Id': '1127815138'}
+                    nucleotide_id = link['Id']
+
+                    record_nucleotide_ids.add(nucleotide_id)
+
+            try:
+                existing_ids = nucleotide_ids[protein_record_id]
+                all_ids = existing_ids.union(record_nucleotide_ids)
+                nucleotide_ids[protein_record_id] = all_ids
+            except KeyError:
+                nucleotide_ids[protein_record_id] = record_nucleotide_ids
+    
+    return nucleotide_ids, no_nucleotides
+
+
+def extract_protein_accessions(single_nucleotide_ids, retrieved_proteins, gbk_accessions, args):
+    """Retrieve and parse Nucleotide db records, for NCBI.Protein records from which only
+    one NCBI.Nucleotide db record ID was retrieved.
+    
+    :param retrieved_proteins: dict, {protein_accession: nucleotide record accession}
+    :param single_nucleotide_ids: list of nucloetide record IDs
+    :param gbk_accessions: list of protein GenBank accessions from the local CAZyme database
+    :param args: cmd-line args parser
+    
+    Return retrieved_proteins (dict)
+        newly_retrieved_proteins: set of CAZyme protein accessions retrieved from parsed records
+        bool: True if successful Entrez connection, False is connection fails
+    """
+    logger = logging.getLogger(__name__)
+    newly_retrieved_proteins = set()
+
+    batch_query_ids = ",".join(list(single_nucleotide_ids))
+    with entrez_retry(
+        args.retries, Entrez.epost, "Nucleotide", id=batch_query_ids,
+    ) as handle:
+        batch_post = Entrez.read(handle)
+
+    with entrez_retry(
+        args.retries,
+        Entrez.efetch,
+        db="Nucleotide",
+        query_key=batch_post['QueryKey'],
+        WebEnv=batch_post['WebEnv'],
+        retmode="xml",
+    ) as handle:
+        try:
+            batch_nucleotide = Entrez.read(handle)
+        except Exception as err:
+            logger.warning(
+                f"Failed Entrez connection for fetching Nucleotide records: {err}"
+            )
+            return retrieved_proteins, newly_retrieved_proteins, False
+    
+    for record in tqdm(batch_nucleotide, desc="Extracting data from Nucleotide records"):
+        nucleotide_accession = record['GBSeq_accession-version']
+
+        # retrieve protein accessions of proteins features in the nucletide record
+        for feature_dict in record['GBSeq_feature-table']:
+            # feature-table contains a list of features, one feature is one feature_dict
+
+            for feature_qual in feature_dict['GBFeature_quals']:
+                # feature_quals contains a list of dicts, one dict is feature_qual
+                # looking for dict containing protein accession (protein_id)
+                # e.g. {'GBQualifier_name': 'protein_id', 'GBQualifier_value': 'APS93952.1'}
+                if feature_qual['GBQualifier_name'] == 'protein_id':
+                    protein_accession = feature_qual['GBQualifier_value']
+
+                    if protein_accession in gbk_accessions:
+                        # protein is in the local CAZyme database
+                        try:
+                            retrieved_proteins[protein_accession].add(nucleotide_accession)
+                        except KeyError:
+                            retrieved_proteins[protein_accession] = {nucleotide_accession}
+                        
+                        newly_retrieved_proteins.add(protein_accession)
+
+    return retrieved_proteins, newly_retrieved_proteins, True
+
+
+def extract_protein_accessions_individually(single_nucleotide_ids, retrieved_proteins, gbk_accessions, args):
+    """Retrieve and parse Nucleotide db records, for NCBI.Protein records from which only
+    one NCBI.Nucleotide db record ID was retrieved.
+    
+    :param retrieved_proteins: dict, {protein_accession: nucleotide record accession}
+    :param single_nucleotide_ids: list of nucloetide record IDs
+    :param gbk_accessions: list of protein GenBank accessions from the local CAZyme database
+    :param args: cmd-line args parser
+    
+    Return retrieved_proteins (dict)
+        newly_retrieved_proteins: set of CAZyme protein accessions retrieved from parsed records
+    """
+    logger = logging.getLogger(__name__)
+
+    newly_retrieved_proteins = set()
+
+    for nucleotide_id in tqdm(single_nucleotide_ids, desc="Parsing nucelotide records individually"):
+        with entrez_retry(
+            args.retries,
+            Entrez.efetch,
+            db="Nucleotide",
+            id=nucleotide_id,
+            retmode="xml",
+        ) as handle:
+            try:
+                batch_nucleotide = Entrez.read(handle)
+            except Exception as err:
+                logger.warning(
+                    f"Failed Entrez connection for fetching Nucleotide records: {err}"
+                )
+                pass
+    
+        for record in tqdm(batch_nucleotide, desc="Extracting data from Nucleotide records"):
+            nucleotide_accession = record['GBSeq_accession-version']
+
+            # retrieve protein accessions of proteins features in the nucletide record
+            for feature_dict in record['GBSeq_feature-table']:
+                # feature-table contains a list of features, one feature is one feature_dict
+
+                for feature_qual in feature_dict['GBFeature_quals']:
+                    # feature_quals contains a list of dicts, one dict is feature_qual
+                    # looking for dict containing protein accession (protein_id)
+                    # e.g. {'GBQualifier_name': 'protein_id', 'GBQualifier_value': 'APS93952.1'}
+                    if feature_qual['GBQualifier_name'] == 'protein_id':
+                        protein_accession = feature_qual['GBQualifier_value']
+
+                        if protein_accession in gbk_accessions:
+                            # protein is in the local CAZyme database
+                            try:
+                                retrieved_proteins[protein_accession].add(nucleotide_accession)
+                            except KeyError:
+                                retrieved_proteins[protein_accession] = {nucleotide_accession}
+                            
+                            newly_retrieved_proteins.add(protein_accession)
+
+    return retrieved_proteins, newly_retrieved_proteins
+
+
+def parse_longest_record(nucleotide_record_ids, retrieved_proteins, gbk_accessions, args):
+    """Identify the longest NCBI.Nucleotide record, and extract Protein GenBank accessions
+    
+    :param nucleotide_record_ids: set, NCBI.Nucleotide records IDs retrieved for one Protein record
+    :param retrieved_proteins: dict, {protein_accession: nucleotide record accession}
+    :param gbk_accessions: list of protein GenBank accessions from the local CAZyme database
+    :param args: cmd-line args parser
+    
+    Return retrieved_proteins (dict)
+        newly_retrieved_proteins: set of CAZyme protein accessions retrieved from parsed records
+        bool: True if successful Entrez connection, False is connection fails
+    """
+    logger = logging.getLogger(__name__)
+    newly_retrieved_proteins = set()
+
+    batch_query_ids = ",".join(list(nucleotide_record_ids))
+    with entrez_retry(
+        args.retries, Entrez.epost, "Nucleotide", id=batch_query_ids,
+    ) as handle:
+        batch_post = Entrez.read(handle)
+    print("posted")
+    with entrez_retry(
+        args.retries,
+        Entrez.efetch,
+        db="Nucleotide",
+        query_key=batch_post['QueryKey'],
+        WebEnv=batch_post['WebEnv'],
+        retmode="xml",
+    ) as handle:
+        try:
+            batch_nucleotide = Entrez.read(handle)
+        except Exception as err:
+            logger.warning(
+                f"Failed Entrez connection for fetching Nucleotide records: {err}"
+            )
+            return retrieved_proteins, newly_retrieved_proteins, False
+    print("fetched")
+    record_lengths = {}  # {Nucleotide record accession: {len: Number of features (int), record: record}
+    # longest (most features) record interpretted as the most complete record
+    
+    for record in tqdm(batch_nucleotide, desc="Selecting longest Nucleotide record"):
+        nucleotide_accession = record['GBSeq_accession-version']
+
+        number_of_features = 0
+
+        for feature_dict in record['GBSeq_feature-table']:
+            for feature_qual in feature_dict['GBFeature_quals']:
+                if feature_qual['GBQualifier_name'] == 'protein_id':
+                    number_of_features += 1
+        
+        record_lengths[nucleotide_accession] = {'length': number_of_features, 'record': record}
+    
+    list_of_lengths = [acc['length'] for acc in list(record_lengths.keys())]
+    list_of_lengths.sort(reverse=True)
+    longest_length = list_of_lengths[0]
+
+    for nucleotide_accession in record_lengths:
+        if record_lengths[nucleotide_accession]['length'] == longest_length:
+            # found the longest record
+            # extract protein accessions for CAZymes in the local CAZyme database
+            # method explained in extract_protein_accessions()
+            for feature_qual in feature_dict['GBFeature_quals']:
+                if feature_qual['GBQualifier_name'] == 'protein_id':
+                    protein_accession = feature_qual['GBQualifier_value']
+                    if protein_accession in gbk_accessions:
+                        try:
+                            retrieved_proteins[protein_accession].add(nucleotide_accession)
+                        except KeyError:
+                            retrieved_proteins[protein_accession] = {nucleotide_accession}
+
+                        newly_retrieved_proteins.add(protein_accession)
+            break
+    
+    return retrieved_proteins, newly_retrieved_proteins, True
+
+
+def parse_longest_record_individually(nucleotide_record_ids, retrieved_proteins, gbk_accessions, args):
+    """Identify the longest NCBI.Nucleotide record, and extract Protein GenBank accessions
+    
+    :param nucleotide_record_ids: set, NCBI.Nucleotide records IDs retrieved for one Protein record
+    :param retrieved_proteins: dict, {protein_accession: nucleotide record accession}
+    :param gbk_accessions: list of protein GenBank accessions from the local CAZyme database
+    :param args: cmd-line args parser
+    
+    Return retrieved_proteins (dict)
+        newly_retrieved_proteins: set of CAZyme protein accessions retrieved from parsed records
+        bool: True if successful Entrez connection, False is connection fails
+    """
+    newly_retrieved_proteins = set()
+
+    record_lengths = {}  # {Nucleotide record accession: {len: Number of features (int), record: record}
+    # longest (most features) record interpretted as the most complete record
+
+    for nucleotide_id in tqdm(nucleotide_record_ids, desc="Parsing nucleotide records individually"):
+        with entrez_retry(
+            args.retries,
+            Entrez.efetch,
+            db="Nucleotide",
+            id=nucleotide_id,
+            retmode="xml",
+        ) as handle:
+            try:
+                nucleotide_records = Entrez.read(handle)
+            except Exception:
+                pass
+
+            for record in nucleotide_records:
+                nucleotide_accession = record['GBSeq_accession-version']
+
+                number_of_features = 0
+
+                for feature_dict in record['GBSeq_feature-table']:
+                    for feature_qual in feature_dict['GBFeature_quals']:
+                        if feature_qual['GBQualifier_name'] == 'protein_id':
+                            number_of_features += 1
+                
+                record_lengths[nucleotide_accession] = {'length': number_of_features, 'record': record}
+            
+    list_of_lengths = [acc['length'] for acc in list(record_lengths.keys())]
+    list_of_lengths.sort(reverse=True)
+    longest_length = list_of_lengths[0]
+
+    for nucleotide_accession in record_lengths:
+        if record_lengths[nucleotide_accession]['length'] == longest_length:
+            # found the longest record
+            # extract protein accessions for CAZymes in the local CAZyme database
+            # method explained in extract_protein_accessions()
+            for feature_qual in feature_dict['GBFeature_quals']:
+                if feature_qual['GBQualifier_name'] == 'protein_id':
+                    protein_accession = feature_qual['GBQualifier_value']
+                    if protein_accession in gbk_accessions:
+                        try:
+                            retrieved_proteins[protein_accession].add(nucleotide_accession)
+                        except KeyError:
+                            retrieved_proteins[protein_accession] = {nucleotide_accession}
+
+                        newly_retrieved_proteins.add(protein_accession)
+            break
+    
+    return retrieved_proteins, newly_retrieved_proteins, True
+
diff --git a/scraper/sql/__init__.py b/cazy_webscraper/sql/__init__.py
similarity index 100%
rename from scraper/sql/__init__.py
rename to cazy_webscraper/sql/__init__.py
diff --git a/scraper/sql/sql_interface/__init__.py b/cazy_webscraper/sql/sql_interface/__init__.py
similarity index 83%
rename from scraper/sql/sql_interface/__init__.py
rename to cazy_webscraper/sql/sql_interface/__init__.py
index 856747c8..e32bd1ee 100644
--- a/scraper/sql/sql_interface/__init__.py
+++ b/cazy_webscraper/sql/sql_interface/__init__.py
@@ -45,7 +45,7 @@
 
 from tqdm import tqdm
 
-from scraper.sql import sql_orm
+from cazy_webscraper.sql import sql_orm
 
 
 class SqlInterfaceException(Exception):
@@ -60,6 +60,9 @@ def log_scrape_in_db(
     config_dict,
     taxonomy_filters,
     kingdoms,
+    ec_filter,
+    db,
+    retrieved_annotations,
     session,
     args,
 ):
@@ -69,6 +72,9 @@ def log_scrape_in_db(
     :param config_dict: dict of CAZy classes and families to be scraped
     :param taxonomy_filters: dict of genera, species and strains to restrict the scrape to
     :param kingdoms: list of taxonomy Kingdoms to restrict scrape to
+    :param ec_filter: set of EC numbers to restrict retrieval of data to
+    :param db: str, name of the external database from which data is retrieved
+    :param retrieved_annotations: str, types of annotations retrieved (e.g. UniProt accessions)
     :param session: open SQL database session
     :param args: cmd arguments
 
@@ -78,7 +84,12 @@ def log_scrape_in_db(
     date = time_stamp[:time_stamp.find("--")]
     time = time_stamp[((time_stamp.find("--")) + 2):].replace("-", ":")
 
-    new_log = sql_orm.Log(date=date, time=time)
+    new_log = sql_orm.Log(
+        date=date,
+        time=time,
+        database=db,
+        retrieved_annotations=retrieved_annotations,
+    )
 
     if config_dict is not None:
         # get classes that user named to be scraped
@@ -146,6 +157,10 @@ def log_scrape_in_db(
         except TypeError:
             pass
 
+    if len(ec_filter) != 0:
+        cmd_line = cmd_line + "--ec_filter" + (args.ec) + "'"
+        new_log.ec_filter = ','.join(list(ec_filter))
+
     if len(cmd_line) != 0:
         cmd_line = cmd_line.strip()
         new_log.cmd_line = cmd_line
@@ -182,3 +197,20 @@ def insert_data(connection, table_name, column_names, insert_values):
             raise SqlInterfaceException(db_error)
 
     return
+
+
+def get_gbk_table_dict(connection):
+    """Compile a dict of the data in the Genbanks table
+    
+    :param connection: open connection to an SQLite3 database
+    
+    Return dict {gbk accession : gbk id}
+    """
+    with sql_orm.Session(bind=connection) as session:
+        all_genbank = session.query(sql_orm.Genbank).all()
+
+    db_gbk_dict = {}  # {genbank_accession: db genbank id number}
+    for gbk in all_genbank:
+        db_gbk_dict[f"{gbk.genbank_accession}"] = gbk.genbank_id
+    
+    return db_gbk_dict
diff --git a/cazy_webscraper/sql/sql_interface/add_cazyme_data.py b/cazy_webscraper/sql/sql_interface/add_cazyme_data.py
new file mode 100644
index 00000000..e02a46c9
--- /dev/null
+++ b/cazy_webscraper/sql/sql_interface/add_cazyme_data.py
@@ -0,0 +1,373 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Add CAZyme data to a local SQLite database"""
+
+
+import logging
+
+from sqlalchemy import delete, text
+from sqlalchemy.orm import relationship
+from tqdm import tqdm
+
+from cazy_webscraper.sql.sql_interface import insert_data, get_table_dicts
+from cazy_webscraper.sql.sql_orm import genbanks_families
+
+
+def add_kingdoms(cazy_taxa_dict, connection):
+    """Add new Kingdoms objects to database.
+    
+    Check existing kingdom objects in the db against kingdoms retrieved from the 
+    CAZy txt file, so as to only add new kingdoms.
+
+    :param cazy_taxa_dict: dict of kingdoms and organisms from the cazy_data dict
+        {kingdom: {organism}}
+    :param connection: open sqlalchemy connection to a local SQLite db engine
+    
+    Return nothing
+    """
+    kingdom_table_dict = get_table_dicts.get_kingdom_table_dict(connection)
+    # dict {kingdom: {organisms}}
+
+    # retrieve the Kingdoms retrieved from the CAZy txt file
+    cazy_kingdoms = list(cazy_taxa_dict.keys())
+    existing_kingdom_records = list(kingdom_table_dict.keys())
+
+    # create list of tuples for db insert
+    kingdoms_db_insert_values = [
+        (kngdm,) for kngdm in cazy_kingdoms if kngdm not in existing_kingdom_records
+    ]
+
+    if len(kingdoms_db_insert_values) != 0:
+        insert_data(connection, 'Kingdoms', ['kingdom'], kingdoms_db_insert_values)
+    
+    return
+
+
+def add_source_organisms(taxa_dict, connection):
+    """Add taxonomy (source organism) data to the local CAZyme database
+    
+    :param taxa_dict: dict of taxa data {kingdom: set(organism)} to be added to the db
+    :param connection: open sqlalchemy connection to SQLite engine
+    
+    Return nothing
+    """
+    logger = logging.getLogger(__name__)
+
+    # retrieve db kingdom objects for retrieving the kingdom_id for the Taxs table
+    kingdom_table_dict = get_table_dicts.get_kingdom_table_dict(connection)
+    # {kingdom: kingdom_id}
+    tax_table_dict = get_table_dicts.get_taxs_table_dict(connection)
+    # {genus species: {'tax_id': int(db_tax_id), 'kingdom_id': int(kingdom_id)}
+    
+    # compare taxa already in the db against taxa retrieved from the CAZy txt file
+    # to identify new taxa objects to be added to the db
+
+    taxonomy_db_insert_values = set()
+    records_to_update = set()  # used incase kingdom has changed for a species
+
+    for kingdom in tqdm(
+        taxa_dict,
+        total=len(list(taxa_dict.keys())),
+        desc='Create tax objects per Kingdom',
+    ):
+        kingdom_id = kingdom_table_dict[kingdom]
+        organisms = taxa_dict[kingdom]
+        
+        for organism in organisms:  # organisms from the CAZy txt file
+            
+            try:
+                existing_record_data = tax_table_dict[organism]
+                # check kingdom is correct
+                existing_record_kngdm_id = tax_table_dict[organism]['kingdom_id']
+                if existing_record_kngdm_id != kingdom_id:
+                    records_to_update.add( (genus, species, kingdom_id,) )
+                
+            except KeyError:  # organism not in the db, build new record
+                genus = organism.split(" ")[0]
+                species = ' '.join(organism.split(" ")[1:])
+                new_record = (genus, species, kingdom_id,)
+                taxonomy_db_insert_values.add( new_record )
+    
+    if len(taxonomy_db_insert_values) != 0:
+        logger.info(
+            f"Adding {len(taxonomy_db_insert_values)} new tax records to the db"
+        )
+        insert_data(
+            connection,
+            'Taxs',
+            ['genus', 'species', 'kingdom_id'],
+            list(taxonomy_db_insert_values),
+        )
+    else:
+        logger.info(
+            "No new tax records to add to the db"
+        )
+    if len(records_to_update) != 0:
+        logger.info(
+            f"Updating the parent Kingdom for {len(records_to_update)} tax records in the db"
+        )
+        for record in records_to_update:
+            connection.execute(
+                text(
+                    "UPDATE Taxs "
+                    f"SET kingdom_id = {record[2]} "
+                    f"WHERE genus = '{record[0]}' AND species = '{record[1]}'"
+                )
+            )
+
+    return
+
+
+def add_cazy_families(cazy_data, connection):
+    """Add CAZy families and subfamilies to local CAZyme database
+    
+    :param cazy_data: dict of data extracted from the txt file
+        {gbk_accession: {kingdom:str, organism:str, families{fam:subfam}}}
+    :param connection: open sqlalchemy connection to an SQLite db engine
+    
+    Return nothing"""
+    logger = logging.getLogger(__name__)
+
+    # get list of CAZy (sub)families already present in the db
+    fam_table_dict = get_table_dicts.get_fams_table_dict(connection)  # {family subfamily: db_family_id}
+
+    existing_fam_records = list(fam_table_dict.keys()) 
+
+    families_db_insert_values = set()  # new fam records to add to db
+
+    for genbank_accession in tqdm(cazy_data, desc='Extracting CAZy fams from CAZy data'):
+        for cazy_fam in cazy_data[genbank_accession]["families"]:
+            subfamilies = cazy_data[genbank_accession]["families"][cazy_fam]
+            
+            for subfam in subfamilies:
+                if subfam is None:
+                    cazy_subfam = "_"
+                else:
+                    cazy_subfam = subfam
+                fam_key = f"{cazy_fam} {cazy_subfam}"
+                
+                if fam_key not in existing_fam_records:
+                    families_db_insert_values.add( (cazy_fam, subfam) )  # add None for subfam if is None
+            
+    if len(families_db_insert_values) != 0:
+        logger.info(
+            f"Inserting {len(families_db_insert_values)} "
+            "new family records into the CazyFamilies table"
+        )
+        insert_data(connection, 'CazyFamilies', ['family', 'subfamily'], list(families_db_insert_values))
+    else:
+        logger.info("Found no new CAZy family records to add the CazyFamilies table")
+
+    return
+
+
+def add_genbanks(cazy_data, connection):
+    """Add GenBank accessions with tax data to the db
+    
+    :param cazy_data: dict of CAZy data
+        {gbk_acc: {kingdom: {str}, organism: {str}, families: {(fam, subfam, )}}}
+    :param connection: open sqlalchemy connection to an SQLite db
+    
+    Return Nothing
+    """
+    logger = logging.getLogger(__name__)
+
+    # retrieve existing records from the db
+    gbk_table_dict = get_table_dicts.get_gbk_table_dict(connection)
+    existing_gbk_records = list(gbk_table_dict.keys())
+
+    taxa_table_dict = get_table_dicts.get_taxs_table_dict(connection)
+    # {genus species: {'tax_id': db_tax_id, 'kingdom_id': kingdom_id}
+
+    gbk_record_updates = set()  # {gbk_accession: 'taxa_id': (new taxa_id) int, 'gbk_id': int}
+    gbk_db_insert_values = set()
+
+    for gbk_accession in tqdm(cazy_data, desc="Compiling Genbank records for insertion"):
+        if gbk_accession not in existing_gbk_records:
+            organism = list(cazy_data[gbk_accession]['organism'])[0]
+            taxa_id = taxa_table_dict[organism]['tax_id']
+            gbk_db_insert_values.add( (gbk_accession, taxa_id,) )
+        
+        else:  # check if need to update taxa_id
+            # get the taxa_id for the existing record
+            existing_record_id = gbk_table_dict[gbk_accession]['taxa_id']
+            
+            # get the taxa_id for the organism listed in the CAZy txt file
+            organism = list(cazy_data[gbk_accession]['organism'])[0]
+            cazy_data_taxa_id = taxa_table_dict[organism]['tax_id']
+            
+            if cazy_data_taxa_id != existing_record_id:
+                # need to update the record
+                gbk_record_updates.add( (cazy_data_taxa_id, gbk_table_dict[gbk_accession]['gbk_id']) )
+
+    if len(gbk_db_insert_values) != 0:
+        logger.info(
+            f"Inserting {len(gbk_db_insert_values)} GenBank accessions into the db"
+        )
+        insert_data(connection, 'Genbanks', ['genbank_accession', 'taxonomy_id'], list(gbk_db_insert_values))
+    
+    if len(gbk_record_updates) != 0:
+        logger.info(
+            f"Updating {len(gbk_record_updates)} Genbank table records with new taxonomy IDs"
+        )
+        for record in gbk_record_updates:
+            connection.execute(
+                text(
+                    "UPDATE Taxs "
+                    f"SET taxonomy_id = {record[1]} "
+                    f"WHERE genbank_id = '{record[0]}'"
+                )
+            )
+    
+    return
+
+
+def add_genbank_fam_relationships(cazy_data, connection, args):
+    """Add GenBank accession - CAZy family relationships to db
+    
+    :param cazy_data: dict of data extracted from the txt file
+        {gbk_accession: {kingdom:str, organism:str, families{fam:subfam}}}
+    :param connection: open sqlalchemy connection to an SQLite db engine
+    :param args: cmd-line args parser
+
+    Return nothing
+    """
+    logger = logging.getLogger(__name__)
+
+    gbk_fam_db_insert_values = set()  # new records to add
+    gbk_fam_records_to_del = set()  # records/relationships to delete
+
+    # get dict of GenBank and CazyFamilies tables, used for getting gbk_ids  and fam_ids of accessions and
+    # families without entries in the CazyFamilies_Genbanks table
+    
+    gbk_table_dict = get_table_dicts.get_gbk_table_dict(connection) 
+    # {genbank_accession: 'taxa_id': int, 'gbk_id': int}
+
+
+    fam_table_dict = get_table_dicts.get_fams_table_dict(connection) 
+    # {'fam subfam': fam_id}
+
+    # load current relationships in the db
+    gbk_fam_table_dict, existing_rel_tuples = get_table_dicts.get_gbk_fam_table_dict(connection)
+    # {genbank_accession: {'families': {str(fam subfam): int(fam_id)}, 'gbk_id': int(gbk_db_id)} }
+
+    for genbank_accession in tqdm(cazy_data, desc="Extracting Genbank-Family relationships from CAZy data"):
+        
+        gbk_id = gbk_table_dict[genbank_accession]['gbk_id']
+        
+        cazy_fam_dict = cazy_data[genbank_accession]['families']
+        # cazy_data = { gbk_acc: {'families': {fam: {subfamilies}} } }
+        # cazy_fam_dict = {fam : {subfamilies}}
+        
+        try:
+            existing_relation_dict = gbk_fam_table_dict[genbank_accession]['families']
+            # existing_relation_dict = { str(fam subfam) : fam_id }
+            
+            for fam in cazy_fam_dict:
+                subfamilies = cazy_fam_dict[fam] # set of subfams for parent fam from CAZy data
+                
+                for subfam in subfamilies:  # add each fam-subfam pair
+                    if subfam is None:
+                        family_key = f"{fam} _"
+                    else:
+                        family_key = f"{fam} {subfam}"
+                    
+                    try:
+                        existing_relation_dict[family_key]
+                        # already in db, don't add relationship again
+                    
+                    except KeyError:
+                        fam_id = fam_table_dict[family_key]
+                        
+                        new_row = (gbk_id, fam_id,)
+                        if new_row not in existing_rel_tuples:
+                            gbk_fam_db_insert_values.add( (gbk_id, fam_id,) )
+        
+        except KeyError:  # GenBank not present in the Genbanks_CazyFamilies table, create new record
+            
+            for fam in cazy_fam_dict:
+                subfamilies = cazy_fam_dict[fam] # set of subfams for parent fam from CAZy data
+                
+                for subfam in subfamilies:  # add each fam-subfam pair
+                    if subfam is None:
+                        family_key = f"{fam} _"
+                    else:
+                        family_key = f"{fam} {subfam}"
+
+                    fam_id = fam_table_dict[family_key]
+                
+                    new_row = (gbk_id, fam_id,)
+                    if new_row not in existing_rel_tuples:
+                        gbk_fam_db_insert_values.add( (gbk_id, fam_id,) )
+
+    if len(gbk_fam_db_insert_values) != 0:
+
+        logger.info(
+            f"Adding {len(gbk_fam_db_insert_values)} new GenBank accession - "
+            "CAZy (sub)family relationships to the db"
+        )
+        insert_data(
+            connection,
+            'Genbanks_CazyFamilies',
+            ['genbank_id', 'family_id'],
+            list(gbk_fam_db_insert_values),
+        )
+    else:
+        logger.info(
+            "No new Genbank accession-CAZy (sub)family relationships to add to the db"
+        )
+    
+    if (len(gbk_fam_records_to_del) != 0):
+        logger.info(
+            "Deleting {(len(gbk_fam_records_to_del)} GenBank accession - "
+            "CAZy (sub)family relationships\n"
+            "that are the db but are no longer in CAZy"
+        )
+        for record in gbk_fam_records_to_del:
+            # record = (genbank_id, fam_id,)
+            stmt = (
+                delete(genbanks_families).\
+                where(genbanks_families.c.genbank_id == record[0]).\
+                where(genbanks_families.c.family_id == record[1])
+            )
+            connection.execute(stmt)
+
+    return
diff --git a/cazy_webscraper/sql/sql_interface/add_genbank_data.py b/cazy_webscraper/sql/sql_interface/add_genbank_data.py
new file mode 100644
index 00000000..69d18223
--- /dev/null
+++ b/cazy_webscraper/sql/sql_interface/add_genbank_data.py
@@ -0,0 +1,84 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Add protein sequences retrieved from GenBank to a local SQLite database"""
+
+
+from sqlalchemy import text
+from tqdm import tqdm
+
+from cazy_webscraper.sql.sql_interface import get_table_dicts
+
+
+def add_gbk_seqs_to_db(seq_dict, retrieval_date, connection, args):
+    """Add sequences retrieved from GenBank to the local CAZyme db
+    
+    :param seq_dict: dict of retrieved seq {gbk_acc: seq}
+    :param retrieval_date: str, date seqs were retrieved in ISO format
+    :param connection: open sqlalchemy connection to a SQLite db
+    :param args: cmd-line args parser
+    
+    Return nothing
+    """
+    # load the current Genbanks table into a dict
+    gbk_table_dict = get_table_dicts.get_gbk_table_seq_dict(connection)
+
+    records = set()  # records to update, contains tuples (gbk_acc, seq, )
+
+    for gbk_accession in seq_dict:
+        existing_record = gbk_table_dict[gbk_accession]
+
+        if existing_record['sequence'] is None:
+            records.add( (gbk_accession, seq_dict[gbk_accession]) )
+
+        else:
+            if (args.sequence_update) and (retrieval_date < existing_record['seq_date']):
+                records.add( (gbk_accession, seq_dict[gbk_accession]) )
+
+    for record in tqdm(records, desc="Adding seqs to db"):
+            connection.execute(
+                text(
+                    "UPDATE Genbanks "
+                    f"SET sequence = {record[1]}, seq_update_date = {retrieval_date} "
+                    f"WHERE genbank_accession = '{record[0]}'"
+                )
+            )
+    
+    return
diff --git a/cazy_webscraper/sql/sql_interface/add_uniprot_data.py b/cazy_webscraper/sql/sql_interface/add_uniprot_data.py
new file mode 100644
index 00000000..4d42d2c3
--- /dev/null
+++ b/cazy_webscraper/sql/sql_interface/add_uniprot_data.py
@@ -0,0 +1,313 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Add data retrieved from UniProt to a local SQLite database"""
+
+
+from sqlalchemy import delete, text
+from tqdm import tqdm
+
+from cazy_webscraper.sql.sql_interface import insert_data, get_table_dicts
+from cazy_webscraper.sql.sql_orm import genbanks_ecs
+
+
+def add_uniprot_accessions(uniprot_dict, gbk_dict, connection, args):
+    """Add UniProt data to the local CAZyme database
+    
+    :param uniprot_dict: dict containing data retrieved from UniProt
+    :param gbk_dict: dict representing data from the Genbanks table
+    :param connection: open sqlalchemy conenction to an SQLite db engine
+    :param args: cmd-line args parser
+
+    Return nothing.
+    """
+    # load the current Uniprot records in the local CAZyme db
+    # {acc: {name: str, gbk_id: int, seq: str, seq_date:str } }
+    uniprot_table_dict = get_table_dicts.get_uniprot_table_dict(connection)
+
+    uniprot_insert_values = set()
+
+    # the following variables containing objects in the local db that are to be updated
+    update_record_gbk_id = set()  # ((uniprot_accession, gbk_id),)
+    update_record_name = set()  # ((uniprot_accession, retrieved_name), )
+    update_record_seq = set()  # ((uniprot_accession, retrieved_seq), )
+
+    for uniprot_acc in tqdm(uniprot_dict, desc="Separating new and existing records"):
+        # check if the UniProt accession is already in the local CAZyme db
+        try:
+            uniprot_table_dict[uniprot_acc]
+            # Uniprot accession is already in the local CAZyme db
+
+            # check if the GenBank id is the same
+            existing_gbk_id = uniprot_table_dict[uniprot_acc]['genbank_id']
+            retrieved_gbk_acc = uniprot_dict[uniprot_acc]["genbank_accession"]
+            retrieved_gbk_id = gbk_dict[retrieved_gbk_acc]
+            if existing_gbk_id != retrieved_gbk_id:
+                update_record_gbk_id.add( (uniprot_acc, retrieved_gbk_id) )
+                
+            if args.update_name:
+                # check if the name has changed
+                existing_name = uniprot_table_dict[uniprot_acc]['name']
+                retrieved_name = uniprot_dict[uniprot_acc]['name']
+                
+                if existing_name != retrieved_name:
+                    update_record_name.add( (uniprot_acc, retrieved_name) )
+            
+            if args.sequence_new:  # only add seq if sequence is not there
+                if uniprot_table_dict[uniprot_acc]['seq'] is None:
+                    update_record_seq.add( (uniprot_acc, uniprot_dict[uniprot_acc]["sequence"]) )
+                
+            if args.sequence_update:  # update seq if a newer version is available or no seq present
+                if uniprot_table_dict[uniprot_acc]['seq'] is None:
+                        update_record_seq.add( (
+                            uniprot_acc,
+                            uniprot_dict[uniprot_acc]["sequence"],
+                            uniprot_dict[uniprot_acc]["seq_date"],
+                        ) )
+                else:
+                    existing_date = uniprot_table_dict[uniprot_acc]['seq_date']
+                    retrieved_date = uniprot_dict[uniprot_acc]['seq_date']
+                    
+                    if existing_date < retrieved_date:  # existing date is older, update seq
+                        update_record_seq.add( (
+                            uniprot_acc,
+                            uniprot_dict[uniprot_acc]["sequence"],
+                            uniprot_dict[uniprot_acc]["seq_date"],
+                        ) )
+        
+        except KeyError:  # new record to add to local CAZyme db
+            
+            if args.sequence_new or args.sequence_update:
+                genbank_acc = uniprot_dict[uniprot_acc]["genbank_accession"]
+                gbk_id = gbk_dict[genbank_acc]
+                uniprot_name = uniprot_dict[uniprot_acc]["name"]
+                seq = uniprot_dict[uniprot_acc]["sequence"]
+                date = uniprot_dict[uniprot_acc]["seq_date"]
+
+                uniprot_insert_values.add( (gbk_id, uniprot_acc, uniprot_name, seq, date) )
+
+            else:  # not retrieving protein sequences
+                genbank_acc = uniprot_dict[uniprot_acc]["genbank_accession"]
+                gbk_id = gbk_dict[genbank_acc]
+                uniprot_name = uniprot_dict[uniprot_acc]["name"]
+                
+                uniprot_insert_values.add( (gbk_id, uniprot_acc, uniprot_name) )
+    
+    if len(uniprot_insert_values) != 0:
+        if args.sequence_new or args.sequence_update:
+            columns = ['genbank_id', 'uniprot_accession', 'uniprot_name', 'sequence', 'seq_update_date']
+        else:
+            columns = ['genbakn_id', 'uniprot_accession', 'uniprot_name']
+    
+        insert_data(connection, "Uniprots", columns, list(uniprot_insert_values))
+
+    if len(update_record_gbk_id) != 0:
+        for record in tqdm(update_record_gbk_id, desc="Updating UniProt-Gbk relationships"):
+            connection.execute(
+                text(
+                    "UPDATE Uniprots "
+                    f"SET genbank_id = {record[1]} "
+                    f"WHERE uniprot_accession = '{record[0]}'"
+                )
+            )
+        
+    if len(update_record_name) != 0:
+        for record in tqdm(update_record_name, desc="Updating UniProt protein names"):
+            connection.execute(
+                text(
+                    "UPDATE Uniprots "
+                    f"SET uniprot_name = {record[1]} "
+                    f"WHERE uniprot_accession = '{record[0]}'"
+                )
+            )
+
+    if len(update_record_seq) != 0:
+        for record in tqdm(update_record_seq, desc="Updating UniProt protein seqs"):
+            connection.execute(
+                text(
+                    "UPDATE Uniprots "
+                    f"SET sequence = {record[1]}, seq_update_date = {record[2]} "
+                    f"WHERE uniprot_accession = '{record[0]}'"
+                )
+            )
+
+    return
+
+
+def add_ec_numbers(uniprot_dict, all_ecs, gbk_dict, connection, args):
+    """Add EC numbers to the local CAZyme database
+
+    :param uniprot_dict: dict containing data retrieved from UniProt
+    :param all_ecs: set of all EC numbers retrieved from UniProt
+    :param gbk_dict: dict representing data from the Genbanks table
+    :param connection: open sqlalchemy conenction to an SQLite db engine
+    :param args: cmd-line args parser
+
+    Return nothing.
+    """
+    # load EC records in the local CAZyme db
+    ec_table_dict = get_table_dicts.get_ec_table_dict(connection)
+
+    # identify new EC numbers to add to the EC table
+    existing_ecs = list(ec_table_dict.keys())
+    ec_insert_values = set()
+
+    for ec in all_ecs:
+        if ec not in existing_ecs:
+            ec_insert_values.add( (ec,) )
+
+    if len(ec_insert_values) != 0:
+        insert_data(connection, "Ecs", ["ec_number"], list(ec_insert_values))
+
+        # load in the newly updated EC table from the local CAZyme db
+        ec_table_dict = get_table_dicts.get_ec_table_dict(connection)
+    
+    # load in gbk_ec table, contains the gbk-ec number relationships
+    ec_gbk_table_dict = get_table_dicts.get_ec_gbk_table_dict(connection)  # {ec_id: {gbk ids}}
+
+    if args.delete_old_ec:
+        gbk_ec_table_dict = get_table_dicts.get_gbk_ec_table_dict(connection)  # {gbk_id: {ec ids}}
+        ecs_rel_to_delete = set()  # set of tuples (gbk_id, ec_id)
+
+    # compile list of tuples to insert into the Genbanks_Ecs table
+    gbk_ec_insert_values = set()
+
+    for uniprot_acc in tqdm(uniprot_dict, desc="Updating EC numbers"):
+        genbank_acc = uniprot_dict[uniprot_acc]["genbank_accession"]
+        gbk_id = gbk_dict[genbank_acc]
+        
+        retrieved_ec_numbers = uniprot_dict[uniprot_acc]["ec"]  # EC#s retrieved from UniProt
+        for ec in retrieved_ec_numbers:
+            ec_id = ec_table_dict[ec]
+            
+            existing_gbk_relationships = ec_gbk_table_dict[ec_id]
+            if gbk_id not in existing_gbk_relationships:
+                gbk_ec_insert_values.add( (gbk_id, ec_id) )
+        
+        if args.delete_old_ec:
+            existing_ec_relationships = gbk_ec_table_dict[gbk_id]
+            for ec in retrieved_ec_numbers:
+                ec_id = ec_table_dict[ec]
+                if ec_id not in existing_ec_relationships:
+                    ecs_rel_to_delete.add( (gbk_id, ec_id) )
+
+    if len(gbk_ec_insert_values) != 0:
+        insert_data(connection, "Genbanks_Ecs", ["genbank_id", "ec_id"], list(gbk_ec_insert_values))
+
+    if args.delete_old_ec and len(ecs_rel_to_delete) != 0:
+        for record in tqdm(ecs_rel_to_delete, desc="Deleteing old GenBank-EC relationships"):
+            # record = (genbank_id, ec_id,)
+            stmt = (
+                delete(genbanks_ecs).\
+                where(genbanks_ecs.c.genbank_id == record[0]).\
+                where(genbanks_ecs.c.ec_id == record[1])
+            )
+            connection.execute(stmt)
+
+    return
+
+
+def add_pdb_accessions(uniprot_dict, gbk_dict, connection, args):
+    """Add PDB accessions to the local CAZyme database
+
+    :param uniprot_dict: dict containing data retrieved from UniProt
+    :param gbk_dict: dict representing data from the Genbanks table
+    :param connection: open sqlalchemy conenction to an SQLite db engine
+    :param args: cmd-line args parser
+
+    Return nothing.
+    """
+    # load in PDB objects in the local CAZyme db
+    pdb_table_dict = get_table_dicts.get_pdb_table_dict(connection)
+
+    pdb_insert_values = set()  # new records to add to db
+    update_pdbs = set()  # records to update
+    existing_pdbs = set(pdb_table_dict.values())  # records already in the db
+    if args.delete_old_pdbs:
+        delete_pdbs = set()  # records to delete, PDB acc no longer listed in UniProt for the given protein
+
+    for uniprot_acc in tqdm(uniprot_dict, desc="Identifying new and updated PDB records"):
+        genbank_acc = uniprot_dict[uniprot_acc]["genbank_accession"]
+        gbk_id = gbk_dict[genbank_acc]
+
+        retrieved_pdbs = uniprot_dict[uniprot_acc]["pdb"]
+        if len(retrieved_pdbs) == 0:
+            continue
+
+        for pdb in retrieved_pdbs:
+            if pdb_acc in existing_pdbs:
+                # check if the gbk_id is the same
+                current_pdbs = pdb_table_dict[gbk_id]
+                if pdb_acc not in current_pdbs:
+                    update_pdbs.add( (pdb_acc, gbk_id) )
+            
+            else:
+                pdb_insert_values.add( (pdb_acc, gbk_id) )
+        
+        if args.delete_old_pdbs:
+            existing_pdb_records = pdb_table_dict[gbk_id]
+            for pdb_acc in existing_pdb_records:
+                if pdb_acc not in retrieved_pdbs:
+                    delete_pdbs.add( (pdb_acc,) )
+
+    if len(pdb_insert_values) != 0:
+        insert_data(connection, "Pdbs", ["pdb_accession", "genbank_id"], list(pdb_insert_values))
+
+    if len(update_pdbs) != 0:
+        for record in tqdm(update_pdbs, desc="Updating PDB records"):
+            connection.execute(
+                text(
+                    "UPDATE Pdbs "
+                    f"SET genbank_id = {record[1]} "
+                    f"WHERE pdb_accession = '{record[0]}'"
+                )
+            )
+
+    if args.delete_old_pdbs and len(delete_pdbs) != 0:
+        for record in tqdm(delete_pdbs, desc="Deleteing old PDB accessions"):
+            # record = (pdb_acc,)
+            connection.execute(
+                text(
+                    "DELETE Pdbs "
+                    f"WHERE pdb_accession = '{record[0]}'"
+                )
+            )
+            
+    return
diff --git a/cazy_webscraper/sql/sql_interface/get_api_data.py b/cazy_webscraper/sql/sql_interface/get_api_data.py
new file mode 100644
index 00000000..490ec331
--- /dev/null
+++ b/cazy_webscraper/sql/sql_interface/get_api_data.py
@@ -0,0 +1,441 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2021
+# (c) University of Strathclyde 2021
+# (c) James Hutton Institute 20201
+# Author:
+# Emma E. M. Hobbs
+
+# Contact
+# eemh1@st-andrews.ac.uk
+
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Retrieve protein data for provided GenBank accessions"""
+
+
+import logging
+import re
+
+from tqdm import tqdm
+
+from cazy_webscraper.sql.sql_orm import (
+    CazyFamily,
+    Ec,
+    Genbank,
+    Kingdom,
+    Pdb,
+    Taxonomy,
+    Session,
+    Uniprot,
+)
+
+
+def get_class_fam_annotations(gbk_dict, query_data, connection, args):
+    """Retrieve CAZy class and/or family annotations for the provided Gbks.
+    
+    :param gbk_dict: dict of selected GenBank accessions {acc: id}
+    :param query_data: dict containing all data retrieved from the db
+    :param connection: open sqlaclchemy connection for an SQLite db
+    :param args: cmd-line args parser
+
+    Return query_data: dict containing all data retrieved from the db
+    """
+    logger = logging.getLogger(__name__)
+    
+    gbk_accessions = list(gbk_dict.keys())
+
+    # retrieve the data from the CAZy Family table
+    with Session(bind=connection) as session:
+        fam_table_query = session.query(Genbank.genbank_id).\
+            join(CazyFamily, Genbank.families).\
+            filter(Genbank.genbank_accession.in_(gbk_accessions)).\
+            all()
+
+    if len(fam_table_query) == 0:
+        logger.warning(
+            "No CAZy class/family annotations retrieved for any of the selected "
+            "GenBank accessions."
+        )
+        return query_data
+
+    for record in tqdm(fam_table_query, desc="Getting CAZy class/family annotations"):
+        gbk_acc = record[0].genbank_accession
+
+        if args.cazy_class:
+            fam = record[1].family
+            cazy_class = re.match(r"\D{2,3}\d", fam).group()[:-1]
+
+            try:
+                query_data[gbk_acc]
+                
+                try:
+                    query_data[gbk_acc]['class'].add(cazy_class)
+                except KeyError:
+                    query_data[gbk_acc]['class'] = {cazy_class}
+
+            except KeyError:
+                query_data[gbk_acc] = {'class': {cazy_class}}
+        
+        if args.cazy_family:
+            fam = record[1].family
+
+            try:
+                query_data[gbk_acc]
+                
+                try:
+                    query_data[gbk_acc]['family'].add(fam)
+                except KeyError:
+                    query_data[gbk_acc]['family'] = {fam}
+
+            except KeyError:
+                query_data[gbk_acc] = {'family': {fam}}
+
+        if args.cazy_subfamily:
+            subfam = record[1].subfamily
+
+            try:
+                query_data[gbk_acc]
+                
+                try:
+                    query_data[gbk_acc]['subfamily'].add(subfam)
+                except KeyError:
+                    query_data[gbk_acc]['subfamily'] = {subfam}
+
+            except KeyError:
+                query_data[gbk_acc] = {'subfamily': {subfam}}
+
+    return query_data
+
+
+def get_tax_annotations(gbk_dict, query_data, connection, args):
+    """Retrieve kingdom, genus and/or scientific name of the source organism for the provided Gbks.
+    
+    :param gbk_dict: dict of selected GenBank accessions {acc: id}
+    :param query_data: dict containing all data retrieved from the db
+    :param connection: open sqlaclchemy connection for an SQLite db
+    :param args: cmd-line args parser
+
+    Return query_data: dict containing all data retrieved from the db
+    """
+    logger = logging.getLogger(__name__)
+    
+    gbk_accessions = list(gbk_dict.keys())
+
+    # retrieve the data from the Taxonomy and Kingdom tables
+    with Session(bind=connection) as session:
+        tax_query = session.query(Genbank, Taxonomy, Kingdom).\
+            join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
+            join(Genbank, (Genbank.taxonomy_id == Taxonomy.taxonomy_id)).\
+            filter(Genbank.genbank_accession.in_(gbk_accessions)).\
+            all()
+
+    if len(tax_query) == 0:
+        logger.warning("No taxonomy data retrieved for any of the selected GenBank accessions.")
+        return query_data
+
+    for record in tqdm(tax_query, desc="Getting taxonomy data"):
+        gbk_acc = record[0].genbank_accession
+
+        if args.kingdom:
+            kingdom = record[2].kingdom
+
+            try:
+                query_data[gbk_acc]
+                
+                try:
+                    query_data[gbk_acc]['kingdom']
+                    logger.warning(
+                        f"Multiple taxa found for {gbk_acc}\n"
+                        "Retreiving only one record."
+                    )
+                    query_data[gbk_acc]['kingdom'] = kingdom
+                except KeyError:
+                    query_data[gbk_acc]['kingdom'] = kingdom
+
+            except KeyError:
+                query_data[gbk_acc] = {'kingdom': kingdom}
+        
+        if args.genus:
+            genus = record[1].genus
+
+            try:
+                query_data[gbk_acc]
+                
+                try:
+                    query_data[gbk_acc]['genus']
+                    logger.warning(
+                        f"Multiple taxa found for {gbk_acc}\n"
+                        "Retreiving only one record."
+                    )
+                    query_data[gbk_acc]['genus'] = genus
+                except KeyError:
+                    query_data[gbk_acc]['genus'] = genus
+
+            except KeyError:
+                query_data[gbk_acc] = {'genus': genus}
+
+        if args.organism:
+            genus = record[1].genus
+            species = record[1].species
+            organism = f"{genus} {species}"
+
+            try:
+                query_data[gbk_acc]
+                
+                try:
+                    query_data[gbk_acc]['organism']
+                    logger.warning(
+                        f"Multiple taxa found for {gbk_acc}\n"
+                        "Retreiving only one record."
+                    )
+                    query_data[gbk_acc]['organism'] = organism
+                except KeyError:
+                    query_data[gbk_acc]['organism'] = organism
+
+            except KeyError:
+                query_data[gbk_acc] = {'organism': organism}
+
+    return query_data
+
+
+def get_ec_annotations(gbk_dict, query_data, connection):
+    """Retrieve EC number annotations for the provided Gbks.
+    
+    :param gbk_dict: dict of selected GenBank accessions {acc: id}
+    :param query_data: dict containing all data retrieved from the db
+    :param connection: open sqlaclchemy connection for an SQLite db
+
+    Return query_data: dict containing all data retrieved from the db
+    """
+    logger = logging.getLogger(__name__)
+    
+    gbk_accessions = list(gbk_dict.keys())
+
+    # retrieve the data from the Taxonomy and Kingdom tables
+    with Session(bind=connection) as session:
+        ec_query = session.query(Genbank, Ec).\
+            join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
+            join(Ec, Genbank.ecs).\
+            filter(Genbank.genbank_accession.in_(gbk_accessions)).\
+            all()
+
+    if len(ec_query) == 0:
+        logger.warning("No EC annotations retrieved for any of the selected GenBank accessions.")
+        return query_data
+
+    for record in tqdm(ec_query, desc="Getting EC number annotations"):
+        gbk_acc = record[0].genbank_accession
+
+        ec_number = record[1].ec_number
+
+        try:
+            query_data[gbk_acc]
+            
+            try:
+                query_data[gbk_acc]['ec_numbers'].add(ec_number)
+            except KeyError:
+                query_data[gbk_acc]['ec_numbers'] = {ec_number}
+
+        except KeyError:
+            query_data[gbk_acc] = {'ec_numbers': {ec_number}}
+
+    return query_data
+
+
+def get_pdb_accessions(gbk_dict, query_data, connection):
+    """Retrieve PDB accessions for the provided Gbks.
+    
+    :param gbk_dict: dict of selected GenBank accessions {acc: id}
+    :param query_data: dict containing all data retrieved from the db
+    :param connection: open sqlaclchemy connection for an SQLite db
+
+    Return query_data: dict containing all data retrieved from the db
+    """
+    logger = logging.getLogger(__name__)
+    
+    gbk_accessions = list(gbk_dict.keys())
+
+    # retrieve the data from the Taxonomy and Kingdom tables
+    with Session(bind=connection) as session:
+        pdb_query = session.query(Genbank, Pdb).\
+            join(Pdb, (Pdb.genbank_id == Genbank.genbank_id)).\
+            filter(Genbank.genbank_accession.in_(gbk_accessions)).\
+            all()
+
+    if len(pdb_query) == 0:
+        logger.warning("No PDB accessions retrieved for any of the selected GenBank accessions.")
+        return query_data
+
+    for record in tqdm(pdb_query, desc="Getting PDB accessions"):
+        gbk_acc = record[0].genbank_accession
+
+        pdb_accession = record[1].pdb_accession
+
+        try:
+            query_data[gbk_acc]
+            
+            try:
+                query_data[gbk_acc]['pdb_accessions'].add(pdb_accession)
+            except KeyError:
+                query_data[gbk_acc]['pdb_accessions'] = {pdb_accession}
+
+        except KeyError:
+            query_data[gbk_acc] = {'pdb_accessions': {pdb_accession}}
+
+    return query_data
+
+
+def get_uniprot_data(gbk_dict, query_data, connection, args):
+    """Retrieve UniProt data for the provided Gbks.
+    
+    :param gbk_dict: dict of selected GenBank accessions {acc: id}
+    :param query_data: dict containing all data retrieved from the db
+    :param connection: open sqlaclchemy connection for an SQLite db
+    :param args: cmd-line args parser
+
+    Return query_data: dict containing all data retrieved from the db
+    """
+    logger = logging.getLogger(__name__)
+    
+    gbk_accessions = list(gbk_dict.keys())
+
+    # retrieve the data from the Taxonomy and Kingdom tables
+    with Session(bind=connection) as session:
+        uniprot_query = session.query(Genbank, Uniprot).\
+            join(Uniprot, (Uniprot.genbank_id == Genbank.genbank_id)).\
+            filter(Genbank.genbank_accession.in_(gbk_accessions)).\
+            all()
+
+    if len(uniprot_query) == 0:
+        logger.warning("No UniProt records retrieved for any of the selected GenBank accessions.")
+        return query_data
+
+    for record in tqdm(uniprot_query, desc="Getting UniProt data"):
+        gbk_acc = record[0].genbank_accession
+
+        if args.uniprot:
+            uniprot_accession = record[1].uniprot_accession
+            uniprot_name = record[1].uniprot_name
+
+            try:
+                query_data[gbk_acc]
+                
+                try:
+                    query_data[gbk_acc]['uniprot_accession']
+                    query_data[gbk_acc]['uniprot_name']
+                    logger.warning(
+                        f"Multiple UniProt records found for GBK acc {gbk_acc}\n"
+                        "Retreiving only one."
+                    )
+                    query_data[gbk_acc]['uniprot_accession'] = uniprot_accession
+                    query_data[gbk_acc]['uniprot_name'] = uniprot_name
+                except KeyError:
+
+                    query_data[gbk_acc]['uniprot_accession'] = uniprot_accession
+                    query_data[gbk_acc]['uniprot_name'] = uniprot_name
+
+            except KeyError:
+                query_data[gbk_acc] = {
+                    'uniprot_accession': uniprot_accession,
+                    'uniprot_name': uniprot_name,
+                }
+        
+        if args.seq_uniprot:
+            seq = record[1].sequence
+            seq_date = record[1].seq_update_date
+
+            try:
+                query_data[gbk_acc]
+                
+                try:
+                    query_data[gbk_acc]['uniprot_sequence']
+                    logger.warning(
+                        f"Multiple UniProt records found for GBK acc {gbk_acc}\n"
+                        "Retreiving only one record."
+                    )
+                    query_data[gbk_acc]['uniprot_sequence'] = seq
+                    query_data[gbk_acc]['uniprot_sequence_date'] = seq_date
+                except KeyError:
+                    query_data[gbk_acc]['uniprot_sequence'] = seq
+                    query_data[gbk_acc]['uniprot_sequence_date'] = seq_date
+
+            except KeyError:
+                query_data[gbk_acc] = {'sequence': seq, 'sequence_date': seq_date}
+
+    return query_data
+
+
+def get_gbk_seq(gbk_dict, query_data, connection):
+    """Retrieve GenBank protein sequences for the provided Gbks.
+    
+    :param gbk_dict: dict of selected GenBank accessions {acc: id}
+    :param query_data: dict containing all data retrieved from the db
+    :param connection: open sqlaclchemy connection for an SQLite db
+
+    Return query_data: dict containing all data retrieved from the db
+    """
+    logger = logging.getLogger(__name__)
+    
+    gbk_accessions = list(gbk_dict.keys())
+
+    # retrieve the data from the Taxonomy and Kingdom tables
+    with Session(bind=connection) as session:
+        gbk_query = session.query(Genbank).\
+            filter(Genbank.genbank_accession.in_(gbk_accessions)).\
+            all()
+
+    if len(gbk_query) == 0:
+        logger.warning("No GenBank records retrieved for any of the selected GenBank accessions.")
+        return query_data
+
+    for record in tqdm(gbk_query, desc="Getting GenBank protein sequences"):
+        gbk_acc = record[0].genbank_accession
+        seq = record[0].sequence
+        seq_date = record[0].seq_update_date
+
+        try:
+            query_data[gbk_acc]
+            
+            try:
+                logger.warning(
+                    f"Multiple GBK records found for GBK acc {gbk_acc}\n"
+                    "Retreiving only one gbk sequence."
+                )
+                query_data[gbk_acc]['gbk_sequence'] = seq
+                query_data[gbk_acc]['gbk_sequence_date'] = seq_date
+            except KeyError:
+                query_data[gbk_acc]['gbk_sequence'] = seq
+                query_data[gbk_acc]['gbk_sequence_date'] = seq_date
+
+        except KeyError:
+            query_data[gbk_acc] = {'gbk_sequence': seq, 'gbk_sequence_date': seq_date}
+
+    return query_data
+
diff --git a/cazy_webscraper/sql/sql_interface/get_selected_gbks.py b/cazy_webscraper/sql/sql_interface/get_selected_gbks.py
new file mode 100644
index 00000000..4de4b2cd
--- /dev/null
+++ b/cazy_webscraper/sql/sql_interface/get_selected_gbks.py
@@ -0,0 +1,329 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Funcs for retrieving GenBank accessions of interest, matching user filter criteria"""
+
+
+import logging
+import sys
+
+from tqdm import tqdm
+
+from cazy_webscraper.sql.sql_orm import (
+    CazyFamily,
+    Ec,
+    Genbank,
+    Kingdom,
+    Taxonomy,
+    Session,
+)
+
+
+def get_genbank_accessions(
+    class_filters,
+    family_filters,
+    taxonomy_filters,
+    kingdom_filters,
+    ec_filters,
+    connection,
+):
+    """Retrieve the GenBank accessions of proteins of interest (to retrieve data for).
+    
+    :param class_filters: set of CAZy classes to retrieve data for
+    :param family_filters: set of CAZy families to retrieve data for
+    :param taxonomy_filters: dict of taxonom filters to limit the retrieval of data to
+    :param kingdom_filters: set of tax kingdoms to limit the retrieval of data to
+    :param ec_filters: set of EC numbers to limit the retrieval of data to
+    :param connection: open sqlaclchemy connection for an SQLite db
+    
+    Return dict {gbk_acc: gbk_id}
+    """
+    logger = logging.getLogger(__name__)
+    
+    # retrieve GenBank accessions of proteins in user selected CAZy classes and (sub)families
+    initially_selected_gbk = get_class_fam_genbank_accessions(
+        class_filters,
+        family_filters,
+        connection,
+    )
+    
+    if len(initially_selected_gbk) == 0:
+        logger.error(
+            "Retrieved NO proteins for the user selected CAZy classes and (sub)families\n"
+            "Ensure proteins belonging to these classes and (sub)families are catalouged into the local CAZyme db\n"
+            "Terminating program"
+        )
+        sys.exit(1)
+    
+    logger.info(
+        f"Retrieved {len(initially_selected_gbk)} from user selected CAZy class and (sub)families"
+    )
+    
+    # Retrieve the db ID numbers of taxonomy entries matching the users taxonomy/kingdom filters
+    filtered_gbk_accessions = apply_tax_filters(
+        initially_selected_gbk,
+        taxonomy_filters,
+        kingdom_filters,
+        connection
+    )
+    
+    if len(filtered_gbk_accessions) == 0:
+        logger.error(
+            "Retrieved NO proteins for the user selected taxonomy and kingdom filters\n"
+            "Ensure proteins belonging to these taxa are catalouged into the local CAZyme db\n"
+            "Terminating program"
+        )
+        sys.exit(1)
+    
+    # Apply EC number filter if provided
+    if len(ec_filters) != 0:
+        filtered_gbk_accessions = apply_ec_filters(
+            ec_filters,
+            connection,
+        )
+    
+    # extract the accession numbers from the db Genbank objects and their db genbank_id
+    gbk_dict = {}
+    for obj in filtered_gbk_accessions:
+        gbk_dict[obj.genbank_accession] = obj.genbank_id
+    
+    if len(list(gbk_dict.keys())) == 0:
+        logger.error(
+            "No proteins in the local CAZyme db matched the provided critiera.\n"
+            "Check the critieria matches data in the local CAZyme db.\n"
+            "Terminating program"
+        )
+        sys.exit(1)
+    
+    return gbk_dict
+
+    
+def get_class_fam_genbank_accessions(
+    class_filters,
+    family_filters,
+    connection,
+):
+    """Retrieve the GenBank accessions of proteins from user selected CAZy classes and (sub)families
+
+    :param class_filters: set of CAZy classes to retrieve data for
+    :param family_filters: set of CAZy families to retrieve data for
+    :param connection: open sqlaclchemy connection for an SQLite db
+    
+    Return list of db objects containing a Genbank obj, Taxonomy obj and Kingdom obj.
+    """
+    initially_selected_gbk = []
+
+    if len(class_filters) == 0 and len(family_filters) == 0:
+        # could retrieve all GenBank accessions
+        with Session(bind=connection) as session:
+            gbk_query = session.query(Genbank, Taxonomy, Kingdom).\
+                join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
+                join(Genbank, (Genbank.taxonomy_id == Taxonomy.taxonomy_id)).\
+                join(CazyFamily, Genbank.families).\
+                all()
+
+            initially_selected_gbk = gbk_query
+
+    else:
+        for cazy_class in tqdm(class_filters, desc="Retrieving GenBank accessions for selected CAZy classes"):
+            with Session(bind=connection) as session:
+                class_subquery = session.query(Genbank.genbank_id).\
+                join(CazyFamily, Genbank.families).\
+                filter(CazyFamily.family.like(f'{cazy_class}%')).\
+                subquery()
+
+            with Session(bind=connection) as session:
+                gbk_query = session.query(Genbank, Taxonomy, Kingdom).\
+                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
+                    join(Genbank, (Genbank.taxonomy_id == Taxonomy.taxonomy_id)).\
+                    join(CazyFamily, Genbank.families).\
+                    filter(Genbank.genbank_id.in_(class_subquery)).\
+                    all()
+
+            initially_selected_gbk += gbk_query
+    
+        for cazy_fam in tqdm(family_filters, desc="Retrieving GenBank accessions for selected CAZy families"):
+            if cazy_fam.find('_') != -1:  # subfamily
+                with Session(bind=connection) as session:
+                    fam_subquery = session.query(Genbank.genbank_id).\
+                    join(CazyFamily, Genbank.families).\
+                    filter(CazyFamily.subfamily == cazy_fam).\
+                    subquery()
+
+            else:  # family
+                with Session(bind=connection) as session:
+                    fam_subquery = session.query(Genbank.genbank_id).\
+                    join(CazyFamily, Genbank.families).\
+                    filter(CazyFamily.family == cazy_fam).\
+                    subquery()
+
+            with Session(bind=connection) as session:
+                gbk_query = session.query(Genbank, Taxonomy, Kingdom).\
+                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
+                    join(Genbank, (Genbank.taxonomy_id == Taxonomy.taxonomy_id)).\
+                    join(CazyFamily, Genbank.families).\
+                    filter(Genbank.genbank_id.in_(fam_subquery)).\
+                    all()
+
+            initially_selected_gbk += gbk_query
+        
+    return initially_selected_gbk
+
+
+def apply_tax_filters(
+    initially_selected_gbk,
+    taxonomy_filters,
+    kingdom_filters,
+    connection,
+):
+    """Filter retrieved GenBank accessions by taxonomy filters.
+    
+    :param initally_selected_gbk: list of db Genbank objs retrieved from the db
+    :param taxonomy_filters: dict of taxonom filters to limit the retrieval of data to
+    :param kingdom_filters: set of tax kingdoms to limit the retrieval of data to
+    :param connection: open sqlaclchemy connection for an SQLite db
+    
+    Return set of db Genbank objs
+    """
+    logger = logging.getLogger(__name__)
+    
+    if len(taxonomy_filters) == 0 and len(kingdom_filters) == 0:
+        gbks = [obj[0] for obj in initially_selected_gbk]
+        return set(gbks)
+    
+    tax_ids = set()
+    
+    for kingdom in tqdm(kingdom_filters, desc="Retrieving IDs of species from selected kingdoms"):
+        with Session(bind=connection) as session:
+            kingdom_query = session.query(Taxonomy.taxonomy_id).\
+                join(Kingdom, (Kingdom.kingdom_id == Taxonomy.kingdom_id)).\
+                filter(Kingdom.kingdom == kingdom).\
+                all()
+            for taxa in kingdom_query:
+                tax_ids.add(taxa[0])
+
+    genera = taxonomy_filters['genus']
+    species = taxonomy_filters['species']
+    strains = taxonomy_filters['strains']
+
+    for genus in tqdm(genera, desc="Retrieving IDs of species from selected genera"):
+        with Session(bind=connection) as session:
+            tax_query = session.query(Taxonomy.taxonomy_id).\
+                filter(Taxonomy.genus == genus).\
+                all()
+            for taxa in tax_query:
+                tax_ids.add(taxa[0])
+
+    for species in tqdm(genera, desc="Retrieving IDs of species from selected species"):
+        with Session(bind=connection) as session:
+            tax_query = session.query(Taxonomy.taxonomy_id).\
+                filter(Taxonomy.species.like(f'{species}%')).\
+                all()
+            for taxa in tax_query:
+                tax_ids.add(taxa[0])
+
+    for strain in tqdm(strains, desc="Retrieving IDs of species from selected strains"):
+        with Session(bind=connection) as session:
+            tax_query = session.query(Taxonomy.taxonomy_id).\
+                filter(Taxonomy.species == strain).\
+                all()
+            for taxa in tax_query:
+                tax_ids.add(taxa[0])
+                
+    if len(tax_ids) == 0:
+        logger.error(
+            "Retrieve NO taxonomy objects matching the provided kingdom and tax filters\n"
+            "Therefore, retrieved NO proteins matching the provided criteria.\n"
+            "Check the database contains the selected kingdoms, genera, species and strains\n"
+            "Terminating program"
+        )
+        sys.exit(1)
+    
+    filtered_gbk = set()
+    for gbk in initially_selected_gbk:
+        if gbk[1].taxonomy_id in tax_ids:
+            filtered_gbk.add(gbk[0])
+            
+    return filtered_gbk
+
+
+def apply_ec_filters(
+    current_gbk_objs,
+    ec_filters,
+    connection,
+):
+    """Apply EC number filter to the retrieved Genbank records.
+    
+    :param current_gbk_objs: list of db Genbank objs retrieved from the db
+    :param ec_filters: set of EC numbers to limit the retrieval of data to
+    :param connection: open sqlaclchemy connection for an SQLite db
+    
+    Return set of db Genbank objects.
+    """
+    logger = logging.getLogger(__name__)
+    
+    # retrieve the Genbank db IDs for all GenBank accessions associated with each user selected EC num in the db
+    gbk_ids = set()
+
+    for ec in tqdm(ec_filters, desc="Retrieving selected EC numbers from db"):
+        with Session(bind=connection) as session:
+            gbk_query = session.query(Genbank.genbank_id).\
+                join(Ec, Genbank.ecs).\
+                filter(Ec.ec_number == ec).\
+                all()
+        for gbk_id in gbk_query:
+            gbk_ids.add(gbk_id)
+
+    if len(gbk_ids) == 0:
+        logger.error(
+        "Retrieved NO proteins matching the provided EC numbers\n"
+        "Therefore, not retrievign UniProt data for any proteins.\n"
+        "Check the local CAZyme db contains the EC numbers provided\n"
+        "Terminating program"
+    )
+        sys.exit(1)
+    
+    filtered_gbks = set()
+    for gbk in current_gbk_objs:
+        if gbk.genbank_id in gbk_ids:
+            filtered_gbks.add(gbk)
+    
+    return filtered_gbks
+
diff --git a/cazy_webscraper/sql/sql_interface/get_selected_pdbs.py b/cazy_webscraper/sql/sql_interface/get_selected_pdbs.py
new file mode 100644
index 00000000..daba0423
--- /dev/null
+++ b/cazy_webscraper/sql/sql_interface/get_selected_pdbs.py
@@ -0,0 +1,100 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+# Bio.PDB reference:
+# Hamelryck, T., Manderick, B. (2003) PDB parser and structure class 
+# implemented in Python. Bioinformatics 19: 2308–2310
+"""Retrieve PDB accessions matching user criteria from the local CAZyme db"""
+
+
+from cazy_webscraper.sql.sql_interface import get_selected_gbks, get_table_dicts
+
+
+def get_pdb_accessions(
+    class_filters,
+    family_filters,
+    taxonomy_filters,
+    kingdom_filters,
+    ec_filters,
+    connection,
+):
+    """Retrieve PDB accessions matching user criteria from the local CAZyme db
+    
+    :param class_filters: set of CAZy classes to retrieve data for
+    :param family_filters: set of CAZy families to retrieve data for
+    :param taxonomy_filters: dict of taxonom filters to limit the retrieval of data to
+    :param kingdom_filters: set of tax kingdoms to limit the retrieval of data to
+    :param ec_filters: set of EC numbers to limit the retrieval of data to
+    :param connection: open sqlalchemy connection to an SQLite db engine
+    
+    Return list of PDB accessions
+    """
+
+    # retrieve the GenBank accessions of Gbk records matching the user criteria
+    selected_gbks = get_selected_gbks.get_genbank_accessions(
+        class_filters,
+        family_filters,
+        taxonomy_filters,
+        kingdom_filters,
+        ec_filters,
+        connection,
+    )
+    
+    # retrieve all PDB accessions for each GenBank accession retrieved for the local CAZyme db
+
+    gbk_table_dict = get_table_dicts.get_gbk_table_dict(connection)  # used to convert acc to id
+    pdb_table_dict = get_table_dicts.get_pdb_table_dict(connection)  # used to retrieve PDB accs
+
+    pdb_accessions = set()
+
+    for gbk_accession in selected_gbks:
+        gbk_id = gbk_table_dict[gbk_accession]['gbk_id']
+
+        try:
+            pdbs = pdb_table_dict[gbk_id]
+
+            for pdb in pdbs:
+                # remove chain annotation if present
+                parent_pdb = pdb.split("[")[0]
+                pdb_accessions.add(parent_pdb)
+        
+        except KeyError:
+            continue
+
+    return list(pdb_accessions)
diff --git a/cazy_webscraper/sql/sql_interface/get_table_dicts.py b/cazy_webscraper/sql/sql_interface/get_table_dicts.py
new file mode 100644
index 00000000..97aa2a9f
--- /dev/null
+++ b/cazy_webscraper/sql/sql_interface/get_table_dicts.py
@@ -0,0 +1,386 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Retrieve all objects from a db table and parse the data to build a dict, repr the current table state."""
+
+
+from tqdm import tqdm
+
+from cazy_webscraper.sql.sql_orm import (
+    CazyFamily,
+    Ec,
+    Genbank,
+    Kingdom,
+    Pdb,
+    Session,
+    Taxonomy,
+    Uniprot,
+)
+
+
+def get_ec_table_dict(connection):
+    """Create dict of objects present in the CazyFamilies table.
+    
+    :param connection: open sqlalchemy db engine connection
+    
+    Return dict {ec_number: ec_id}
+    """
+    with Session(bind=connection) as session:
+        db_ec_records = session.query(Ec).all()
+    
+    ec_table_dict = {}  # {ec_number: ec_id}
+    for record in tqdm(db_ec_records, desc="Retrieving existing EC# records"):
+        ec_table_dict[record.ec_number] = record.ec_id
+
+
+def get_ec_gbk_table_dict(connection):
+    """Load the Genbanks_Ecs table into memory and compile a dict.
+
+    The the result dict is keyed by EC IDS.
+    
+    The table contains the current Genbank and EC number relationships in 
+    the local CAZyme db.
+    
+    :param connection: open sqlalchemy connection to an SQLite db
+    
+    Return dict {ec_id: {gbk ids}}
+    """
+    with Session(bind=connection) as session:
+        all_gbk_ec_records = session.query(Genbank, Ec).\
+            join(Ec, Genbank.ecs).\
+            all()
+        
+    ec_gbk_table_dict = {}
+    
+    for record in all_gbk_ec_records:
+        genbank_id = record[0].genbank_id
+        ec_id = record[1].ec_id
+        
+        try:
+            ec_gbk_table_dict[ec_id].add(genbank_id)
+        except KeyError:
+            ec_gbk_table_dict[ec_id] = {genbank_id}
+    
+    return ec_gbk_table_dict
+
+
+def get_gbk_ec_table_dict(connection):
+    """Load the Genbanks_Ecs table into memory and compile a dict.
+
+    The the result dict is keyed by GENBANK IDS.
+    
+    The table contains the current Genbank and EC number relationships in 
+    the local CAZyme db.
+    
+    :param connection: open sqlalchemy connection to an SQLite db
+    
+    Return dict {gbk_id: {ec ids}}
+    """
+    with Session(bind=connection) as session:
+        all_gbk_ec_records = session.query(Genbank, Ec).\
+            join(Ec, Genbank.ecs).\
+            all()
+        
+    gbk_ec_table_dict = {}
+    
+    for record in all_gbk_ec_records:
+        genbank_id = record[0].genbank_id
+        ec_id = record[1].ec_id
+        
+        try:
+            gbk_ec_table_dict[genbank_id].add(ec_id)
+        except KeyError:
+            gbk_ec_table_dict[genbank_id] = {ec_id}
+    
+    return gbk_ec_table_dict
+
+
+def get_fams_table_dict(connection):
+    """Create dict of objects present in the CazyFamilies table.
+    
+    :param connection: open sqlalchemy db engine connection
+    
+    Return dict {family subfamily: db_family_id}
+    """
+    with Session(bind=connection) as session:
+        all_families = session.query(CazyFamily).all()
+        
+    db_fam_dict = {}
+
+    for fam in all_families:
+        if fam.subfamily is None:
+            subfam = '_'
+        else:
+            subfam = fam.subfamily
+            
+        db_fam_dict[f"{fam.family} {subfam}"] = fam.family_id
+    
+    return db_fam_dict
+
+
+def get_gbk_table_dict(connection):
+    """Compile a dict of the data in the Genbanks table
+    
+    :param connection: open connection to an SQLite3 database
+    
+    Return dict {genbank_accession: 'taxa_id': int, 'gbk_id': int}
+    """
+    with Session(bind=connection) as session:
+        all_genbank = session.query(Genbank).all()
+
+    db_gbk_dict = {}  # {genbank_accession: 'taxa_id': str, 'id': int}
+    
+    for gbk in all_genbank:
+        db_gbk_dict[f"{gbk.genbank_accession}"] = {
+            'taxa_id': gbk.taxonomy_id,
+            'gbk_id': gbk.genbank_id
+        }
+    
+    return db_gbk_dict
+
+
+def get_gbk_table_seq_dict(connection):
+    """Compile a dict of the data in the Genbanks table
+    
+    :param connection: open connection to an SQLite3 database
+    
+    Return dict {genbank_accession: 'taxa_id': int, 'gbk_id': int}
+    """
+    with Session(bind=connection) as session:
+        all_genbank = session.query(Genbank).all()
+
+    db_gbk_dict = {}  # {genbank_accession: 'taxa_id': str, 'id': int}
+    
+    for gbk in all_genbank:
+        db_gbk_dict[f"{gbk.genbank_accession}"] = {
+            'sequence': gbk.sequence,
+            'seq_date': gbk.seq_update_date
+        }
+    
+    return db_gbk_dict
+
+
+def get_gbk_fam_table_dict(connection):
+    """Build dict representing the records present in the Genbanks_CazyFamilies table
+
+    If a GenBank accession is in the db but not has not CazyFamilies instances related to it,
+    the GenBank accession is not returned when quering the db.
+    
+    :param connection: open sqlalchemy connection to an SQLite3 db engine
+    
+    Return 
+    - dict: {gbk_acc: {'families': {'fam subfam': fam_id}}, 'gbk_id': gbk_id }
+    - set of tuples: (gbk_id, fam_id), each representing one row in the table
+    """
+    with Session(bind=connection) as session:
+        all_gbk_fam_records = session.query(Genbank, CazyFamily).\
+        join(CazyFamily, Genbank.families).\
+        all()
+
+    existing_rel_tuples = set()  # set of tuples (gbk_id, fam_id)
+
+    gbk_fam_table_dict = {}
+    # {gbk_acc: {'families': {'fam subfam': fam_id}}, 'gbk_id': gbk_id }
+
+    for record in tqdm(all_gbk_fam_records, ' Retreving existing gbk-fam relationships from db'):
+        gbk_accession = record[0].genbank_accession
+        gbk_id = record[0].genbank_id
+
+        family = record[1].family
+        if record[1].subfamily is None:
+            subfamily = '_'
+        else:
+            subfamily = record[1].subfamily
+        fam_id = record[1].family_id
+
+        existing_rel_tuples.add( (gbk_id, fam_id) )
+
+        try:
+            gbk_fam_table_dict[gbk_accession]
+
+            try:
+                gbk_fam_table_dict[gbk_accession][f'{family} {subfamily}']
+
+            except KeyError:
+                gbk_fam_table_dict[gbk_accession][f'{family} {subfamily}'] = fam_id
+
+        except KeyError:
+            gbk_fam_table_dict[gbk_accession] = {
+                'families': {f'{family} {subfamily}': fam_id},
+                'gbk_id': gbk_id,
+            }
+
+    return gbk_fam_table_dict, existing_rel_tuples
+
+
+def get_kingdom_table_dict(connection):
+    """Load and parse the Kingdoms table from the db and compile a dict {kgnd: id}
+    
+    :param connection:
+    
+    Return dict {kingdom: kindom_db_id}
+    """
+    with Session(bind=connection) as session:
+        kingdom_table = session.query(Kingdom).all()
+    
+    kingdom_dict = {}  # {kingdom: kindom_db_id}
+    
+    for kingdom_obj in kingdom_table:
+        kingdom_dict[kingdom_obj.kingdom] = kingdom_obj.kingdom_id
+        
+    return kingdom_dict
+
+
+def get_pdb_table_dict(connection):
+    """Create dict of objects present in the Pdbs table.
+    
+    :param connection: open sqlalchemy db engine connection
+    
+    Return dict {gbk_id: {pdb accessions}}
+    """
+    with Session(bind=connection) as session:
+        db_pdb_records = session.query(Pdb).all()
+
+    pdb_table_dict = {}  # {gbk_id: {pdb accessions}}
+
+    for record in tqdm(db_pdb_records, desc="Loading existing PDB db records"):
+        gbk_id = record.genbank_id
+        try:
+            pdb_table_dict[gbk_id].add(record.pdb_accession)
+            
+        except KeyError:
+            pdb_table_dict[gbk_id] = {record.pdb_accession}
+
+    pdb_table_dict 
+
+
+def get_taxs_table_dict(connection):
+    """Create dict of objects present in the Taxs table.
+    
+    :param connection: open sqlalchemy db engine connection
+    
+    Return dict {genus species: {'tax_id': db_tax_id, 'kingdom_id': kingdom_id}
+    """
+    with Session(bind=connection) as session:
+        all_taxa = session.query(Taxonomy).all()
+        
+    db_tax_dict = {}
+    for taxa in all_taxa:
+        if len(taxa.species) == 0:
+            db_tax_dict[f"{taxa.genus}"] = {
+                'tax_id': taxa.taxonomy_id,
+                'kingdom_id': taxa.kingdom_id,
+            }
+        else:
+            db_tax_dict[f"{taxa.genus} {taxa.species}"] = {
+                'tax_id': taxa.taxonomy_id,
+                'kingdom_id': taxa.kingdom_id,
+            }
+    
+    return db_tax_dict
+
+
+def get_uniprot_table_dict(connection):
+    """Create dict of objects present in the Uniprots table.
+    
+    :param connection: open sqlalchemy db engine connection
+    
+    Return dict {acc: {name: str, gbk_id: int, seq: str, seq_date:str } }
+    """
+    with Session(bind=connection) as session:
+        db_uniprot_records = session.query(Uniprot).all()
+
+    uniprot_table_dict = {}  # {acc: {name: str, gbk_id: int, seq: str, seq_date:str } }
+
+    for record in tqdm(db_uniprot_records, desc="Retrieving existing UniProt records from db"):
+        uniprot_table_dict[record.uniprot_accession] = {
+            "name": record.uniprot_name,
+            "genbank_id": record.genbank_id,
+            "seq": record.sequence,
+            "seq_date": record.seq_update_date,
+        }
+    
+    return uniprot_table_dict
+
+
+def get_gbk_kingdom_dict(connection):
+    """Compile dict of Genbank, Taxonomy and Kingdom records
+    
+    :param connection: open sqlalchemy db connection
+    
+    Return dict {kingdom: {genus: {species: {protein_accessions}}}
+    """
+    with Session(bind=connection) as session:
+        query_results = session.query(Genbank, Taxonomy, Kingdom).\
+            join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
+            join(Genbank, (Genbank.taxonomy_id == Taxonomy.taxonomy_id)).\
+            all()
+
+    genbank_kingdom_dict = {}  # kingdom: {genus: {species: {protein_accessions}}}
+
+    for result in tqdm(query_results, desc="Retreving GenBank accessions and taxonomy"):
+        genbank_accession = result[0].genbank_accession
+        genus = result[1].genus
+        species = result[1].species
+        kingdom = result[2].kingdom
+
+        try:
+            genbank_kingdom_dict[kingdom]
+
+            try:
+                genbank_kingdom_dict[kingdom][genus]
+
+                try:
+                    genbank_kingdom_dict[kingdom][genus][species].add(genbank_accession)
+                
+                except KeyError:
+                    genbank_kingdom_dict[kingdom][genus][species] = {genbank_accession}
+
+            except KeyError:
+                genbank_kingdom_dict[kingdom][genus] = {species: {genbank_accession}}
+
+        except KeyError:
+            genbank_kingdom_dict[kingdom] = {
+                genus: {
+                    species: {genbank_accession},
+                },
+            }
+
+    return genbank_kingdom_dict
diff --git a/scraper/sql/sql_orm.py b/cazy_webscraper/sql/sql_orm.py
similarity index 76%
rename from scraper/sql/sql_orm.py
rename to cazy_webscraper/sql/sql_orm.py
index 33174b1d..baefbbc8 100644
--- a/scraper/sql/sql_orm.py
+++ b/cazy_webscraper/sql/sql_orm.py
@@ -41,14 +41,11 @@
 """Submodule to build a local SQL database"""
 
 import logging
-import os
 import re
-import sys
 
 import sqlite3
 
 from sqlalchemy import (
-    Boolean,
     Column,
     ForeignKey,
     Index,
@@ -160,6 +157,15 @@ def sqlite_engine_connect(dbapi_connection, connection_record):
     PrimaryKeyConstraint("genbank_id", "family_id"),
 )
 
+genbanks_ecs = Table(
+    "Genbanks_Ecs",
+    Base.metadata,
+    Column("genbank_id", Integer, ForeignKey("Genbanks.genbank_id")),
+    Column("ec_id", Integer, ForeignKey("Ecs.ec_id")),
+    PrimaryKeyConstraint("genbank_id", "ec_id"),
+)
+
+
 
 # Define class tables
 class Genbank(Base):
@@ -175,10 +181,15 @@ class Genbank(Base):
     
     genbank_id = Column(Integer, primary_key=True)
     genbank_accession = Column(String, index=True)
-    sequence = Column(String)
-    seq_update_date = Column(String)
+    sequence = Column(ReString)
+    seq_update_date = Column(ReString)
     taxonomy_id = Column(Integer, ForeignKey("Taxs.taxonomy_id"))
     
+    organism = relationship(
+        "Taxonomy",
+        back_populates="genbanks",
+    )
+    
     families = relationship(
         "CazyFamily",
         secondary=genbanks_families,
@@ -186,6 +197,23 @@ class Genbank(Base):
         lazy="dynamic",
     )
     
+    ecs = relationship(
+        "Ec",
+        secondary=genbanks_ecs,
+        back_populates="genbanks",
+        lazy="dynamic",
+    )
+    
+    pdbs = relationship(
+        "Pdb",
+        back_populates="genbank",
+    )
+    
+    uniprot = relationship(
+        "Uniprot",
+        back_populates="genbank",
+        # uselist=False,
+    ) # 1-1 relationship
     
     def __str__(self):
         return f"-Genbank accession={self.genbank_accession}-"
@@ -208,6 +236,7 @@ class Taxonomy(Base):
     species = Column(String)
     kingdom_id = Column(Integer, ForeignKey("Kingdoms.kingdom_id"))
     
+    genbanks = relationship("Genbank", back_populates="organism")
     tax_kingdom = relationship("Kingdom", back_populates="taxonomy")
     
     def __str__(self):
@@ -270,23 +299,103 @@ def __str__(self):
     def __repr__(self):
         """Return string representation of source organism."""
         return(
-            f"<Class Family, family={self.family}, subfamily={self.subfamily}, id={self.family_id}"
+            f"<Class Family, family={self.family}, subfamily={self.subfamily}, id={self.family_id}>"
+        )
+    
+    
+class Uniprot(Base):
+    """Table containing UniProt accessions and protein sequences retrieved from UniProtKB"""
+    __tablename__ = "Uniprots"
+    
+    __table_args__ = (
+        UniqueConstraint("uniprot_accession",),
+        Index("uniprot_option", "uniprot_id", "uniprot_accession")
+    )
+    
+    genbank_id = Column(Integer, ForeignKey('Genbanks.genbank_id'))
+    uniprot_id = Column(Integer, primary_key=True)
+    uniprot_accession = Column(String)
+    uniprot_name = Column(ReString)
+    sequence = Column(ReString)
+    seq_update_date = Column(ReString)
+    
+    genbank = relationship("Genbank", back_populates="uniprot")
+    
+    def __str__(self):
+        return f"-Uniprot, accession={self.uniprot_accession}, name={self.uniprot_name}, id={self.uniprot_id}-"
+
+    def __repr__(self):
+        """Return string representation of source organism."""
+        return(
+            f"<Uniprot, accession={self.uniprot_accession}, name={self.uniprot_name}, id={self.uniprot_id}>"
         )
 
 
+class Ec(Base):
+    """Describe EC numbers."""
+    __tablename__ = "Ecs"
+    __table_args__ = (
+        UniqueConstraint("ec_number"),
+    )
+
+    ec_id = Column(Integer, primary_key=True)
+    ec_number = Column(String, index=True)
+
+    genbanks = relationship(
+        "Genbank",
+        secondary=genbanks_ecs,
+        back_populates="ecs",
+        lazy="dynamic",
+    )
+    
+    def __str__(self):
+        return f"-EC{self.ec_number}-ec_id={self.ec_number}-"
+
+    def __repr__(self):
+        return f"<Class EC, EC{self.ec_number}, ec_id={self.ec_number}>"
+    
+
+class Pdb(Base):
+    """Describe a PDB accession number of protein structure."""
+    __tablename__ = "Pdbs"
+    __table_args__ = (
+        UniqueConstraint("pdb_accession"),
+    )
+
+    pdb_id = Column(Integer, primary_key=True)
+    pdb_accession = Column(String)
+    genbank_id = Column(Integer, ForeignKey('Genbanks.genbank_id'))
+    
+    Index('pdb_idx', pdb_accession)
+
+    genbank = relationship(
+        "Genbank",
+        back_populates="pdbs",
+    )
+    
+    def __str__(self):
+        return f"-PDB accession={self.pdb_accession}, id={self.pdb_id}-"
+
+    def __repr__(self):
+        return f"<Class Pdb accession={self.pdb_accession}, id={self.pdb_id}>"
+
+
 class Log(Base):
     """Record what data was added to the database and when."""
-    __tablename__ = "logs"
+    __tablename__ = "Logs"
 
     log_id = Column(Integer, primary_key=True)
     date = Column(String)  # date CAZy scrape was initiated
     time = Column(String)  # time scrape was initated
+    database = Column(String)
+    retrieved_annotations = Column(String)
     classes = Column(String)  # CAZy classes scraped
     families = Column(String)  # CAZy families scraped
     kingdoms = Column(String)  # Taxonomy Kingdoms to retrieve CAZymes from
     genera_filter = Column(String)
     species_filter = Column(String)
     strains_filter = Column(String)
+    ec_filter = Column(String)
     cmd_line = Column(String)  # command line arguments
 
     def __str__(self):
diff --git a/cazy_webscraper/taxonomy/__init__.py b/cazy_webscraper/taxonomy/__init__.py
new file mode 100644
index 00000000..f036945e
--- /dev/null
+++ b/cazy_webscraper/taxonomy/__init__.py
@@ -0,0 +1,320 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Parse taxonomy data from NCBI and CAZy to replace when multiple taxa are retrieved from CAZy"""
+
+
+import logging
+
+from saintBioutils.genbank import entrez_retry
+from tqdm import tqdm
+
+from Bio import Entrez
+
+
+def identify_multiple_taxa(cazy_data, multiple_taxa_logger):
+    """Identify GenBank accessions in the CAZy data
+    
+    :param cazy_data: dict of CAZy data, keyed by GenBank accessions, valued by dict with the keys
+        kingdom, organism, families
+    :param multiple_taxa_logger: logger, used for logging GenBank accessions with multiple taxa in CAZy
+        
+    Return list of GenBank accessions
+    """
+    multiple_taxa_gbk = []
+    
+    for genbank_accession in tqdm(cazy_data, total=len(list(cazy_data.keys())), desc='Searching for multiple taxa annotations'):
+        
+        gbk_organisms = cazy_data[genbank_accession]["organism"]
+        
+        if len(gbk_organisms) > 1:
+            multiple_taxa_gbk.append(genbank_accession)
+
+            kingdoms = ",".join(list(cazy_data[genbank_accession]["kingdom"]))
+            gbk_organisms = ",".join(list(gbk_organisms))
+
+            multiple_taxa_logger.warning(
+                f"{genbank_accession}\t{kingdoms}\t{gbk_organisms}"
+            )
+            
+    return multiple_taxa_gbk
+
+
+def replace_multiple_tax(cazy_data, genbank_accessions, replaced_taxa_logger, args, invalid_ids):
+    """Identify GenBank accessions which have multiple source organisms listedi in CAZy. Replace with
+    the latest source organism from NCBI.
+
+    :param cazy_data: dict of CAZy data
+    :param genbank_accessions: list of genbank accessions with multiple taxa in CAZy
+    :param replaced_taxa_logger: logger, used for logging GenBank accessions with multiple taxa in CAZy,
+        and the data this replaced and what is replaced by
+    :param args: cmd-line args parser
+    :param invalid_ids: boolean, potential presence of invalid GenBank accessions
+        Set as true when func is called by replace_multiple_tax_with_invalid_ids()
+
+    Return dict updated cazy_data dict and boolean, if multiple taxa were replaced by single taxa
+    """
+    logger = logging.getLogger(__name__)
+    
+    id_post_list = str(",".join(genbank_accessions))
+
+    success = False
+
+    try:
+        epost_results = Entrez.read(
+            entrez_retry(
+                args.retries,
+                Entrez.epost,
+                "Protein",
+                id=id_post_list,
+            )
+        )
+        success = True
+
+    except (TypeError, AttributeError):  # if no record is returned from call to Entrez
+        logger.error(
+            f"Entrez failed to post assembly IDs.\n"
+            "Not retrieving taxonomy classification from NCBI.\n"
+            "Selecting the first organism retrieved from CAZy as the source organism"
+        )
+        cazy_data = select_first_organism(cazy_data, genbank_accessions)
+        success = True
+
+    except RuntimeError:
+        logger.warning("Found GenBank accessions in CAZy data that are no longer in NCBI")
+
+        if invalid_ids:
+            # replace_multiple_tax was called by replace_multiple_tax_with_invalid_ids
+            # return results, don't use recursive programming
+            return cazy_data, success 
+
+        else:
+            # first time replace_multiple_tax was called
+            cazy_data, success = replace_multiple_tax_with_invalid_ids(cazy_data, args)
+
+    if success is False:
+        logger.error(
+            "Could not retrieve taxonomy data from NCBI,\n"
+             "Using the first source organism retrieved from CAZy for each GenBank accession"
+        )
+
+        cazy_data = select_first_organism(cazy_data, genbank_accessions, replaced_taxa_logger)
+        success = True
+    
+    else:
+        logger.info("Parsing data retrieved from NCBI")
+        cazy_data = get_ncbi_tax(epost_results, cazy_data, replaced_taxa_logger, args)
+        
+    return cazy_data, success
+    
+
+
+def get_ncbi_tax(epost_results, cazy_data, replaced_taxa_logger, args):
+    """Parse the ePost output from Entrez and add the NCBI tax classifications to the CAZy data
+    
+    :param epost_results: Entrez ePost output
+    :param cazy_data: dict, data retrieved from CAZy
+    :param args: cmd-line args parser
+    :param replaced_taxa_logger: logger, used for logging GenBank accessions with multiple taxa in CAZy
+    
+    Return cazy_data (dict)
+    """
+    logger = logging.getLogger(__name__)
+    
+    # Retrieve web environment and query key from Entrez epost
+    epost_webenv = epost_results["WebEnv"]
+    epost_query_key = epost_results["QueryKey"]
+
+    try:
+        with entrez_retry(
+            args.retries,
+            Entrez.efetch,
+            db="Protein",
+            query_key=epost_query_key,
+            WebEnv=epost_webenv,
+            retmode="xml",
+        ) as record_handle:
+            protein_records = Entrez.read(record_handle, validate=False)
+
+    # if no record is returned from call to Entrez
+    except (TypeError, AttributeError) as error:
+        logger.error(
+            f"Entrez failed to retireve accession numbers."
+            "Exiting retrieval of accession numbers, and returning null value 'NA'"
+        )
+    
+    for protein in tqdm(protein_records, desc="Retrieving organism from NCBI"):
+        # retrieve NCBI taxonomy data
+        accession = protein['GBSeq_accession-version']
+        organism = protein['GBSeq_organism']
+        kingdom = protein['GBSeq_taxonomy'].split(';')[0]
+
+        # retrieve CAZy taxonomy data
+        cazy_kingdom = cazy_data[accession]["kingdom"]
+        cazy_organisms = cazy_data[accession]["organism"]
+    
+        cazy_kingdom_str = ",".join(cazy_kingdom)
+        cazy_organism_str = ','.join(cazy_organisms)
+        
+        try:
+            cazy_data[accession]['kingdom'] = {kingdom}
+            cazy_data[accession]['organism'] = {organism}
+
+            # log the difference
+            replaced_taxa_logger.warning(
+               f"{accession}\t{cazy_kingdom_str}: {cazy_organism_str}\t{kingdom}: {organism}"
+            )
+
+        except KeyError:
+            err = f'GenBank accession {accession} retrieved from NCBI, but it is not present in CAZy'
+            logger.error(err)
+            
+            replaced_taxa_logger.warning(
+               f"{accession}\t{cazy_kingdom_str}: {cazy_organism_str}\t{err}"
+            )
+
+    return cazy_data
+
+
+def replace_multiple_tax_with_invalid_ids(cazy_data, gbk_accessions, replaced_taxa_logger, args):
+    """Retrieve the latest taxonomy classification for proteins with multiple source organisms.
+    
+    If a GenBank accession is not longer present in NCBI, select the first organism retrieved from CAZy.
+    
+    :param cazy_data: dict of CAZy data
+    :param genbank_accessions: list of GenBank accessions
+    :param replaced_taxa_logger: logger, log tax data that is replaced and what it is replaced with
+    :param args: cmd-line args parser
+
+    Return dict updated cazy_data dict
+    """
+    # retrieve the first half of the list
+    mid_point = int((len(gbk_accessions)/2))
+    
+    half_gbk_accs = gbk_accessions[:mid_point]
+    
+    cazy_data, success = replace_multiple_tax(
+        cazy_data,
+        half_gbk_accs,
+        replaced_taxa_logger, 
+        args,
+        invalid_ids=True,
+    )
+      
+    if success:
+        # invalid IDs are stored in the second half of the accessions list
+        half_gbk_accs = gbk_accessions[mid_point:]
+        
+        for accession in tqdm(half_gbk_accs, desc='Retrieving taxonomies individually'):
+            cazy_data, success = replace_multiple_tax(
+                cazy_data,
+                [accession],
+                replaced_taxa_logger,
+                args,
+                invalid_ids=True,
+            )
+            
+            if success is False:
+                cazy_data = select_first_organism(cazy_data, [accession], replaced_taxa_logger)
+    
+    else:
+        # invalid gbk ID present in the first half of the accessions list
+        for accession in tqdm(half_gbk_accs, desc='Retrieving taxonomies individually'):
+            cazy_data, success = replace_multiple_tax(
+                cazy_data,
+                [accession],
+                replaced_taxa_logger,
+                args,
+                invalid_ids=True,
+            )
+            
+            if success is False:
+                cazy_data = select_first_organism(cazy_data, [accession], replaced_taxa_logger)
+                
+        # parse the second half of the accessions list
+        half_gbk_accs = gbk_accessions[mid_point:]
+        
+        cazy_data, success = replace_multiple_tax(cazy_data, half_gbk_accs, True)
+        
+        if success is False:
+            # invalid gbk ID present in the second half of the accessions list
+            for accession in tqdm(half_gbk_accs, desc='Retrieving taxonomies'):
+                cazy_data, success = replace_multiple_tax(
+                    cazy_data,
+                    [accession],
+                    replaced_taxa_logger,
+                    args,
+                    invalid_ids=True,
+                )
+                
+                if success is False:
+                    cazy_data = select_first_organism(
+                        cazy_data,
+                        [accession],
+                        replaced_taxa_logger,
+                    )
+                
+    return cazy_data, True
+
+
+def select_first_organism(cazy_data, gbk_accessions, replaced_taxa_logger):
+    """Select the first organism listed for each GenBank accession
+    
+    :param cazy_data: dict of data retrieved from CAZy
+    :param gbk_accessions: list of GenBank accessions
+    :param replaced_taxa_logger: logger, used for logging GenBank accessions with multiple taxa in CAZy
+    
+    Return cazy_data (dict)
+    """
+    for accession in tqdm(gbk_accessions, desc='Selecting the first retrieved organism'):
+        cazy_kingdoms_str = ",".join(list(cazy_data[accession]["kingdom"]))
+        cazy_organisms_str = ",".join(list(cazy_data[accession]["organism"]))
+
+        cazy_kingdom = list(cazy_data[accession]["kingdom"])[0]
+        cazy_organism = list(cazy_data[accession]["organism"])[0]
+
+        cazy_data[accession]["kingdom"] = {cazy_kingdom}
+        cazy_data[accession]["organism"] = {cazy_organism}
+
+        # log the data that was replaced and the data it was replaced with
+        replaced_taxa_logger.warning(
+            f"{accession}\t{cazy_kingdoms_str}: {cazy_organisms_str}\t{cazy_kingdom}: {cazy_organism}"
+        )
+    
+    return cazy_data
diff --git a/scraper/utilities/__init__.py b/cazy_webscraper/utilities/__init__.py
similarity index 100%
rename from scraper/utilities/__init__.py
rename to cazy_webscraper/utilities/__init__.py
diff --git a/scraper/utilities/file_io/__init__.py b/cazy_webscraper/utilities/file_io/__init__.py
similarity index 100%
rename from scraper/utilities/file_io/__init__.py
rename to cazy_webscraper/utilities/file_io/__init__.py
diff --git a/scraper/utilities/parse_configuration/__init__.py b/cazy_webscraper/utilities/parse_configuration/__init__.py
similarity index 82%
rename from scraper/utilities/parse_configuration/__init__.py
rename to cazy_webscraper/utilities/parse_configuration/__init__.py
index c9859c2e..780cbb0e 100644
--- a/scraper/utilities/parse_configuration/__init__.py
+++ b/cazy_webscraper/utilities/parse_configuration/__init__.py
@@ -47,9 +47,7 @@
 import sys
 import yaml
 
-from pathlib import Path
-
-from scraper.utilities.parse_configuration.cazy_class_synonym_dict import cazy_synonym_dict
+from cazy_webscraper.utilities.parse_configuration.cazy_class_synonym_dict import cazy_synonym_dict
 
 
 def parse_configuration(args):
@@ -98,15 +96,7 @@ def parse_configuration(args):
             args,
         )
 
-        # add configuration from the cmd-line
-        config_dict, taxonomy_filter_dict, kingdom_filters = get_cmd_scrape_config(
-            config_dict,
-            cazy_class_synonym_dict,
-            taxonomy_filter_dict, 
-            kingdom_filters,
-            args,
-        )
-
+    # add configuration from the cmd-line
     config_dict, taxonomy_filter_dict, kingdom_filters = get_cmd_scrape_config(
         config_dict,
         cazy_class_synonym_dict,
@@ -116,17 +106,13 @@ def parse_configuration(args):
     )
 
     if len(kingdom_filters) != 0:
-        kingdoms = {}
+        kingdoms = set()
         correct_kingdoms = {"Archaea", "Bacteria", "Eukaryota", "Viruses", "Unclassified"}
         for kingdom in kingdom_filters:
             user_kingdom = f"{kingdom[0].upper()}{kingdom[1:].lower()}"
             if user_kingdom in correct_kingdoms:
                 kingdoms.add(f"{kingdom[0].upper()}{kingdom[1:].lower()}")
         kingdom_filters = kingdoms
-    
-    # When no classes or families are defined scrape every class and family
-    if args.classes is None and args.families is None and args.config is None:
-        config_dict["classes"] = set(cazy_class_synonym_dict.keys())
 
     # get list of CAZy classes not to scrape
     excluded_classes = get_excluded_classes(config_dict, cazy_class_synonym_dict)
@@ -134,21 +120,15 @@ def parse_configuration(args):
     # convert dict into a single set of filters
     taxonomy_filter_set = get_filter_set(taxonomy_filter_dict)
 
-    user_class_filters = set(config_dict['classes'])
-    class_filters = set()
-    for class_filter in user_class_filters:
-        new_filter = class_filter[class_filter.find('(')+1:class_filter.find(')')-1]
-        class_filters.add(new_filter)
-    
+    # retrieve class abbreivations
+    class_filters = set([cazy_class.split(" (")[0] for cazy_class in set(config_dict['classes'])])
+
     family_filters = set()
     for key in config_dict:
         if key != 'classes':
             for fam in config_dict[key]:
                 family_filters.add(fam)
 
-    # convert empty sets to None type objects
-    config_dict = convert_empty_sets_to_none(config_dict)
-
     return (
         excluded_classes,
         config_dict,
@@ -225,12 +205,6 @@ def get_yaml_configuration(
         )
         sys.exit(1)
 
-    # retrieve CAZy classes defined in the YAML configuration file
-    config_dict["classes"] = get_yaml_cazy_classes(
-        yaml_config_dict,
-        cazy_class_synonym_dict,
-    )
-
     for key in yaml_config_dict:
         if key == "classes":
             yaml_cazy_classes = get_yaml_cazy_classes(yaml_config_dict, cazy_class_synonym_dict)
@@ -255,7 +229,11 @@ def get_yaml_configuration(
         elif key == "kingdoms":
             if yaml_config_dict["kingdoms"] is not None:
                 for kingdom in yaml_config_dict["kingdoms"]:
-                    kingdom_filters["kingdoms"].add(kingdom)
+                    if kingdom.lower() == 'unclassified':
+                        kingdom_filters["kingdoms"].add(kingdom.lower())
+                    else:
+                        new_kingdom = kingdom[0].upper() + kingdom[1:].lower()
+                        kingdom_filters["kingdoms"].add(new_kingdom)
 
         else:  # key is a CAZy class and underneath are CAZy families to scrape
             if yaml_config_dict[key] is not None:  # CAZy families were listed
@@ -380,7 +358,11 @@ def get_cmd_scrape_config(
     if args.kingdoms is not None:
         cmd_kingdoms = (args.kingdoms).split(",")
         for kingdom in cmd_kingdoms:
-            kingdom_filters.add(kingdom)
+            if kingdom.lower() == 'unclassified':
+                kingdom_filters.add(kingdom.lower())
+            else:
+                new_kingdom = kingdom[0].upper() + kingdom[1:].lower()
+                kingdom_filters.add(new_kingdom)
     
     return config_dict, taxonomy_filter, kingdom_filters
 
@@ -448,13 +430,15 @@ def get_excluded_classes(config_dict, cazy_class_synonym_dict):
 
     Return list of CAZy classes not to be scraped.
     """
-    cazy_classes_to_scrape = config_dict["classes"]
 
+    cazy_classes_to_scrape = set()
+    
     # retrieve the names of classes for which specific families to be scraped HAVE BEEN named
     for key in config_dict:
         if (key != "classes") and (len(config_dict[key]) != 0):
             # add the class of families to be scraped to the list of CAZy classes to be scraped
             cazy_classes_to_scrape.add(key)
+    cazy_classes_to_scrape.union(config_dict["classes"])
 
     # create list of CAZy classes not to be scraped
     excluded_classes = list(cazy_class_synonym_dict.keys())
@@ -493,7 +477,7 @@ def get_filter_set(taxonomy_filters_dict):
 
     Return a set.
     """
-    taxonomy_filters = {}
+    taxonomy_filters = set()
 
     for key in taxonomy_filters_dict:
         try:
@@ -505,39 +489,95 @@ def get_filter_set(taxonomy_filters_dict):
     return taxonomy_filters
 
 
-def get_configuration(args):
+def get_expansion_configuration(args):
     """Get configuration for the Expand module.
 
-    :param file_io_path: Path to file_io module
     :param args: cmd-line argument parser
 
     Return configuration dictionary, set of taxonomy filters and set of Taxonomy Kingdoms.
     """
-    # retrieve inital parsing of configuration data
-    (
-        excluded_classes, config_dict, cazy_class_synonym_dict, taxonomy_filters_dict, kingdoms, ec_filter,
-    ) = parse_configuration(args)
-    # excluded_classes and cazy_class_synonym_dict are used in the crawler module but are not needed for the
-    # the expand module
+    # Get dictionary of accepted CAZy class synonyms
+    cazy_class_synonym_dict = get_cazy_class_synonym_dict(args)
 
-    taxonomy_filters = []
+    # create dictionary to store families and classes to be scraped
+    config_dict = {
+        'classes': set(),
+        'Glycoside Hydrolases (GHs)': set(),
+        'GlycosylTransferases (GTs)': set(),
+        'Polysaccharide Lyases (PLs)': set(),
+        'Carbohydrate Esterases (CEs)': set(),
+        'Auxiliary Activities (AAs)': set(),
+        'Carbohydrate-Binding Modules (CBMs)': set(),
+    }
 
-    for key in taxonomy_filters_dict:
-        try:
-            if len(taxonomy_filters_dict[key]) != 0:
-                taxonomy_filters += taxonomy_filters_dict[key]
-        except (TypeError, KeyError):
-            pass
+    taxonomy_filter_dict = {"genera": set(), "species": set(), "strains": set()}
+    kingdom_filters = set()
+    ec_filters = set()
 
-    if len(taxonomy_filters) == 0:
-        taxonomy_filters = None
+    # retrieve user configuration, defining for which records data should be retrieved
+    if args.config is not None:
+        config_dict, taxonomy_filter_dict, kingdom_filters = get_yaml_configuration(
+            config_dict,
+            cazy_class_synonym_dict,
+            taxonomy_filter_dict,
+            kingdom_filters,
+            args,
+        )
 
-    else:
-        taxonomy_filters = set(taxonomy_filters)
+    # add configuration from the cmd-line
+    config_dict, taxonomy_filter_dict, kingdom_filters = get_cmd_scrape_config(
+        config_dict,
+        cazy_class_synonym_dict,
+        taxonomy_filter_dict, 
+        kingdom_filters,
+        args,
+    )
+
+    ec_filters = get_ec_config(ec_filters, args)
+    
+    class_filters = set(config_dict['classes'])
+
+    family_filters = set()
+    for key in config_dict:
+        if key != 'classes':
+            for fam in config_dict[key]:
+                family_filters.add(fam)
+
+    return (
+        config_dict,
+        class_filters,
+        family_filters,
+        kingdom_filters,
+        taxonomy_filter_dict,
+        ec_filters,
+    )
+
+
+def get_ec_config(ec_filters, args):
+    logger = logging.getLogger(__name__)
+
+    if args.config is not None:
+        # open configuration file
+        try:
+            with open(args.config) as fh:
+                yaml_config_dict = yaml.full_load(fh)
+        except FileNotFoundError:
+            logger.error(
+                "Could not find configuration file when option was enabled.\n"
+                "Make sure path to the configuration file is correct\n"
+                "Scrapping will not be performed becuase configuration is wrong.\n"
+                "Had looked for the configuration file at:\n"
+                f"{args.config}"
+                "Terminating program."
+            )
+            sys.exit(1)
+
+        ec_filters = set(yaml_config_dict['ec'])
 
-    if kingdoms == "all":
-        kingdoms = ['Archaea', 'Bacteria', 'Eukaryota', 'Viruses', 'Unclassified']
+    ec_filters = ec_filters.union(set((args.ec_filter).split(",")))
 
-    kingdoms = set(kingdoms)
+    ec_filters = [ec.replace("EC","") for ec in ec_filters]
+    ec_filters = [ec.replace("ec","") for ec in ec_filters]
+    ec_filters = [ec.replace("*","-") for ec in ec_filters]
 
-    return config_dict, taxonomy_filters, kingdoms
+    return set(ec_filters)
diff --git a/scraper/utilities/parse_configuration/cazy_class_synonym_dict.py b/cazy_webscraper/utilities/parse_configuration/cazy_class_synonym_dict.py
similarity index 100%
rename from scraper/utilities/parse_configuration/cazy_class_synonym_dict.py
rename to cazy_webscraper/utilities/parse_configuration/cazy_class_synonym_dict.py
diff --git a/scraper/utilities/parse_configuration/cazy_dictionary.json b/cazy_webscraper/utilities/parse_configuration/cazy_dictionary.json
similarity index 100%
rename from scraper/utilities/parse_configuration/cazy_dictionary.json
rename to cazy_webscraper/utilities/parse_configuration/cazy_dictionary.json
diff --git a/scraper/utilities/parsers/__init__.py b/cazy_webscraper/utilities/parsers/__init__.py
similarity index 100%
rename from scraper/utilities/parsers/__init__.py
rename to cazy_webscraper/utilities/parsers/__init__.py
diff --git a/cazy_webscraper/utilities/parsers/api_parser.py b/cazy_webscraper/utilities/parsers/api_parser.py
new file mode 100644
index 00000000..0b068497
--- /dev/null
+++ b/cazy_webscraper/utilities/parsers/api_parser.py
@@ -0,0 +1,301 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Submodule for building cmd-line parsers"""
+
+
+import argparse
+
+from pathlib import Path
+from typing import List, Optional
+
+
+def build_parser(argv: Optional[List] = None):
+    """Return ArgumentParser parser for script."""
+    # Create parser object
+    parser = argparse.ArgumentParser(
+        prog="cw_query_database",
+        description="Interrogate a local CAZyme database",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+
+    parser.add_argument(
+        "database",
+        type=Path,
+        help="Path to local CAZyme database"
+    )
+
+    parser.add_argument(
+        "file_types",
+        dest="file_types",
+        action="store",
+        nargs="+",
+        choices=["csv", "json"],
+        help="File types to write the query output in [csv,json]"
+    )
+
+    parser.add_argument(
+        "--cache_dir",
+        type=Path,
+        default=None,
+        help="Target path for cache dir to be used instead of default path",
+    )
+
+    # Add option to use own CAZy class synoymn dict
+    parser.add_argument(
+        "--cazy_synonyms",
+        type=Path,
+        default=None,
+        help="Path to JSON file containing CAZy class synoymn names",
+    )
+
+    parser.add_argument(
+        "--cazy_class",
+        dest="class",
+        action="store_true",
+        default=False,
+        help="Include CAZy class annotations in the query output"
+    )
+
+    # Add option to define complete classes to scrape
+    parser.add_argument(
+        "--classes",
+        type=str,
+        default=None,
+        help="CAZy classes to retrieve UniProt data for. Separate classes by ','"
+    )
+
+    # Add option to specify path to configuration file
+    parser.add_argument(
+        "-c",
+        "--config",
+        type=Path,
+        metavar="config file",
+        default=None,
+        help="Path to configuration file. Default: None, scrapes entire database",
+    )
+
+    parser.add_argument(
+        "--delete_old_ec",
+        dest="delete_old_ec",
+        action="store_true",
+        default=False,
+        help=(
+            "Delete EC number-GenBank relationships/annotations\n"
+            "that are not longer listed in UniProt for a given protein"
+        ),
+    )
+
+    parser.add_argument(
+        "--ec",
+        dest="ec",
+        action="store_true",
+        default=False,
+        help="Include EC number annotations in the query output",
+    )
+
+    parser.add_argument(
+        "--ec_filter",
+        type=str,
+        default=None,
+        help="Limit retrieval to proteins annotated with the provided EC numbers. Separate EC numbers with single commas"
+    )
+
+    parser.add_argument(
+        "--cazy_family",
+        dest="cazy_family",
+        action="store_true",
+        default=False,
+        help="Include CAZy family annotations in the query output. This is only CAZy families NOT subfamilies"
+    )
+
+    parser.add_argument(
+        "--cazy_subfamily",
+        dest="cazy_subfamily",
+        action="store_true",
+        default=False,
+        help="Include CAZy subfamily annotations in the query output."
+    )
+
+    # Add option to specify families to scrape
+    parser.add_argument(
+        "--families",
+        type=str,
+        default=None,
+        help="CAZy families to UniProt data for. Separate families by commas 'GH1,GH2' (case sensitive)"
+    )
+
+    parser.add_argument(
+        "--genus",
+        dest="genus",
+        action="store_true",
+        default=False,
+        help="Include Genus annotations in the query output"
+    )
+
+    # Add option to restrict scrape to specific genera
+    parser.add_argument(
+        "--genera",
+        type=str,
+        default=None,
+        help="Genera to UniProt data for"
+    )
+
+    parser.add_argument(
+        "--kingdom",
+        dest="kingdom",
+        action="store_true",
+        default=False,
+        help="Include taxonomy Kingdom annotations in the query output"
+    )
+
+    parser.add_argument(
+        "--kingdoms",
+        type=str,
+        default=None,
+        help="Tax Kingdoms to UniProt data for"
+    )
+
+    # Add log file name option
+    # If not given, no log file will be written out
+    parser.add_argument(
+        "-l",
+        "--log",
+        type=Path,
+        metavar="log file name",
+        default=None,
+        help="Defines log file name and/or path",
+    )
+
+    # Add option to not delete content in the existing cache dir
+    parser.add_argument(
+        "--nodelete_cache",
+        dest="nodelete_cache",
+        action="store_true",
+        default=False,
+        help="When called, content in the existing cache dir is NOT deleted",
+    )
+
+    parser.add_argument(
+        "--organism",
+        dest="organism",
+        action="store_true",
+        default=False,
+        help="Include scientific name of the source organism in the query output"
+    )
+
+    parser.add_argument(
+        "-p",
+        "--pdb",
+        dest="pdb",
+        action="store_true",
+        default=False,
+        help="Include PDB accessions in the query output",
+    )
+
+    parser.add_argument(
+        "--seq_gebank",
+        dest="seq_genbank",
+        action="store_true",
+        default=False,
+        help="Include the protein sequences retrieve from GenBank in the query output"
+    )
+
+    parser.add_argument(
+        "--seq_uniprot",
+        dest="seq_uniprot",
+        action="store_true",
+        default=False,
+        help="Include the protein sequences retrieved from UniProt in the query output"
+    )
+
+    # Add option to force file over writting
+    parser.add_argument(
+        "--sql_echo",
+        dest="sql_echo",
+        action="store_true",
+        default=False,
+        help="Set SQLite engine echo to True (SQLite will print its log messages)",
+    )
+
+    # Add option to UniProt data for specific species. This will scrape CAZymes from
+    # all strains belonging to each listed species
+    parser.add_argument(
+        "--species",
+        type=str,
+        default=None,
+        help="Species (written as Genus Species) to UniProt data for"
+    )
+
+    # Add option to restrict scraping to specific strains of organisms
+    parser.add_argument(
+        "--strains",
+        type=str,
+        default=None,
+        help=(
+            "Specific strains of organisms to UniProt data for "
+            "(written as Genus Species Strain)"
+        ),
+    )
+
+    parser.add_argument(
+        "--uniprot",
+        dest="uniprot",
+        action="store_true",
+        default=False,
+        help="Include UniProt ID and protein name in the query output"
+    )
+
+    # Add option for more detail (verbose) logging
+    parser.add_argument(
+        "-v",
+        "--verbose",
+        dest="verbose",
+        action="store_true",
+        default=False,
+        help="Set logger level to 'INFO'",
+    ) 
+
+    if argv is None:
+        # parse command-line
+        return parser
+    else:
+        # return namespace
+        return parser.parse_args(argv)
diff --git a/scraper/utilities/parsers/cazy_webscraper_parser.py b/cazy_webscraper/utilities/parsers/cazy_webscraper_parser.py
similarity index 89%
rename from scraper/utilities/parsers/cazy_webscraper_parser.py
rename to cazy_webscraper/utilities/parsers/cazy_webscraper_parser.py
index e1273cb9..009fbd50 100644
--- a/scraper/utilities/parsers/cazy_webscraper_parser.py
+++ b/cazy_webscraper/utilities/parsers/cazy_webscraper_parser.py
@@ -66,12 +66,26 @@ def build_parser(argv: Optional[List] = None):
 
     # Add option to specify path to configuration file
     parser.add_argument(
-        "-c",
-        "--config",
+        "--cache_dir",
         type=Path,
-        metavar="config file",
         default=None,
-        help="Path to configuration file. Default: None, scrapes entire database",
+        help="Target path for cache dir to be used instead of default path",
+    )
+
+    # Add option to use a pre-downloaded CAZy txt file
+    parser.add_argument(
+        "--cazy_data",
+        type=Path,
+        default=None,
+        help="Path predownloaded CAZy txt file",
+    )
+
+    # Add option to use own CAZy class synoymn dict
+    parser.add_argument(
+        "--cazy_synonyms",
+        type=Path,
+        default=None,
+        help="Path to JSON file containing CAZy class synoymn names",
     )
 
     # Add option to define complete classes to scrape
@@ -82,12 +96,13 @@ def build_parser(argv: Optional[List] = None):
         help="Classes from which all families are to be scraped. Separate classes by ','"
     )
 
-    # Add option to use own CAZy class synoymn dict
     parser.add_argument(
-        "--cazy_synonyms",
+        "-c",
+        "--config",
         type=Path,
+        metavar="config file",
         default=None,
-        help="Path to JSON file containing CAZy class synoymn names",
+        help="Path to configuration file. Default: None, scrapes entire database",
     )
 
     # Add option to display citation
@@ -101,14 +116,7 @@ def build_parser(argv: Optional[List] = None):
     )
 
     parser.add_argument(
-        "--cache_dir",
-        type=Path,
-        default=None,
-        help="Target path for cache dir to be used instead of default path",
-    )
-
-    parser.add_argument(
-        "-d",
+        "-o",
         "--db_output",
         type=Path,
         default=None,
@@ -116,13 +124,25 @@ def build_parser(argv: Optional[List] = None):
     )
 
     parser.add_argument(
-        "-D",
+        "-d",
         "--database",
         type=Path,
         default=None,
         help="Path to an existing local CAZy SQL database",
     )
 
+    parser.add_argument(
+        "--delete_old_relationships",
+        dest="delete_old_relationships",
+        action="store_true",
+        default=False,
+        help=(
+            "Delete old GenBank accession - CAZy family relationships (annotations)\n"
+            "that are in the local db but are not in CAZy, e.g. when CAZy has moved a\n"
+            "protein from one fam to another, delete the old family annotation."
+        ),
+    )
+
     # Add option to force file over writting
     parser.add_argument(
         "-f",
@@ -167,6 +187,15 @@ def build_parser(argv: Optional[List] = None):
         help="Defines log file name and/or path",
     )
 
+    parser.add_argument(
+        "-n",
+        "--nodelete",
+        dest="nodelete",
+        action="store_true",
+        default=False,
+        help="When called, content in the existing out dir is NOT deleted",
+    )
+
     # Add option to not delete content in the existing cache dir
     parser.add_argument(
         "--nodelete_cache",
diff --git a/cazy_webscraper/utilities/parsers/extract_seq_parser.py b/cazy_webscraper/utilities/parsers/extract_seq_parser.py
new file mode 100644
index 00000000..4125e026
--- /dev/null
+++ b/cazy_webscraper/utilities/parsers/extract_seq_parser.py
@@ -0,0 +1,241 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Submodule for building cmd-line parsers"""
+
+
+import argparse
+
+from pathlib import Path
+from typing import List, Optional
+
+
+def build_parser(argv: Optional[List] = None):
+    """Return ArgumentParser parser for the script 'expand.genbank_sequences.py'."""
+    # Create parser object
+    parser = argparse.ArgumentParser(
+        prog="genbank_sequences.py",
+        description="Populates local CAZy database with protein sequences from GenBank",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+
+    # Add positional/required arguments
+    parser.add_argument(
+        "database",
+        type=Path,
+        metavar="local CAZy database",
+        help="Path to local CAZy database",
+    )
+
+    # Add optional arguments to parser
+
+    # Add option for building a BLAST database of retrieved protein sequences
+    parser.add_argument(
+        "-b",
+        "--blastdb",
+        type=Path,
+        default=None,
+        help=(
+            "Create BLAST database of extracted protein sequences.\n"
+            "Provide the path to the directory to store the database"
+        ),
+    )
+
+    # Add option to specify path to configuration file
+    parser.add_argument(
+        "-c",
+        "--config",
+        type=Path,
+        metavar="config file",
+        default=None,
+        help="Path to configuration file. Default: None, scrapes entire database",
+    )
+
+    parser.add_argument(
+        "--cache_dir",
+        type=Path,
+        default=None,
+        help="Target path for cache dir to be used instead of default path",
+    )
+
+    # Add option to use own CAZy class synoymn dict
+    parser.add_argument(
+        "--cazy_synonyms",
+        type=Path,
+        default=None,
+        help="Path to JSON file containing CAZy class synoymn names",
+    )
+
+    # Add option to define classes to retrieve protein sequences for
+    parser.add_argument(
+        "--classes",
+        type=str,
+        default=None,
+        help="Classes from which all families are to be scraped. Separate classes by ','"
+    )
+
+    parser.add_argument(
+        "--ec_filter",
+        type=str,
+        default=None,
+        help="Limit retrieval to proteins annotated with the provided EC numbers. Separate EC numbers with single commas"
+    )
+
+    # Add option to force file over writting
+    parser.add_argument(
+        "-f",
+        "--force",
+        dest="force",
+        action="store_true",
+        default=False,
+        help="Force file over writting",
+    )
+
+    # Add option to specify families to retrieve protein sequences for
+    parser.add_argument(
+        "--families",
+        type=str,
+        default=None,
+        help="Families to scrape. Separate families by commas 'GH1,GH2'",
+    )
+
+    # Add option to enable writing sequences to FASTA file or files, or not at all
+    parser.add_argument(
+        "--fasta_dir",
+        type=Path,
+        default=None,
+        help=(
+            "Write out each extracted sequence to a separate FASTA file in the provided dir\n"
+            "Provide a path to a dir to write out FASTA files"
+        ),
+    )
+
+    parser.add_argument(
+        "--fasta_file",
+        type=Path,
+        default=None,
+        help=(
+            "Write out all extracted sequences to a single FASTA file.\n"
+            "Provide a path to write out the FASTA to"
+        ),
+    )
+
+    # Add option to restrict the scrape to specific kingdoms
+    parser.add_argument(
+        "--kingdoms",
+        type=str,
+        default=None,
+        help=(
+            "Kingdoms to scrape. Separate by a single comma.\n"
+            "Options= archaea, bacteria, eukaryota, viruses, unclassified (not case sensitive)"
+        ),
+    )
+
+    # Add option to restrict scrape to specific genera
+    parser.add_argument(
+        "--genera",
+        type=str,
+        default=None,
+        help="Genera to restrict the scrape to"
+    )
+
+    # Add log file name option
+    # If not given, no log file will be written out
+    parser.add_argument(
+        "-l",
+        "--log",
+        type=Path,
+        metavar="log file name",
+        default=None,
+        help="Defines log file name and/or path",
+    )
+
+    parser.add_argument(
+        "--nodelete",
+        dest="nodelete",
+        action="store_true",
+        default=False,
+        help="Do not delete content in existing output dir",
+    )
+
+    parser.add_argument(
+        "--nodelete_cache",
+        dest="nodelete_cache",
+        action="store_true",
+        default=False,
+        help="Do not delete content in existing cache dir",
+    )
+
+    # Add option to restrict the scrape to specific species. This will scrape CAZymes from
+    # all strains belonging to each listed species
+    parser.add_argument(
+        "--species",
+        type=str,
+        default=None,
+        help="Species (written as Genus Species) to restrict the scrape to"
+    )
+
+    # Add option to restrict scraping to specific strains of organisms
+    parser.add_argument(
+        "--strains",
+        type=str,
+        default=None,
+        help=(
+            "Specific strains of organisms to restrict the scrape to "
+            "(written as Genus Species Strain)"
+        ),
+    )
+
+    # Add option for more detail (verbose) logging
+    parser.add_argument(
+        "-v",
+        "--verbose",
+        dest="verbose",
+        action="store_true",
+        default=False,
+        help="Set logger level to 'INFO'",
+    )
+
+    if argv is None:
+        # parse command-line
+        return parser
+    else:
+        # return namespace
+        return parser.parse_args(argv)
diff --git a/scraper/utilities/parsers/gbk_seq_parser.py b/cazy_webscraper/utilities/parsers/gbk_seq_parser.py
similarity index 84%
rename from scraper/utilities/parsers/gbk_seq_parser.py
rename to cazy_webscraper/utilities/parsers/gbk_seq_parser.py
index 6d0a9bcf..1a407261 100644
--- a/scraper/utilities/parsers/gbk_seq_parser.py
+++ b/cazy_webscraper/utilities/parsers/gbk_seq_parser.py
@@ -42,13 +42,12 @@
 
 
 import argparse
-import sys
 
 from pathlib import Path
 from typing import List, Optional
 
 
-def build_genbank_sequences_parser(argv: Optional[List] = None):
+def build_parser(argv: Optional[List] = None):
     """Return ArgumentParser parser for the script 'expand.genbank_sequences.py'."""
     # Create parser object
     parser = argparse.ArgumentParser(
@@ -74,18 +73,6 @@ def build_genbank_sequences_parser(argv: Optional[List] = None):
 
     # Add optional arguments to parser
 
-    # Add option for building a BLAST database of retrieved protein sequences
-    parser.add_argument(
-        "-b",
-        "--blastdb",
-        type=Path,
-        default=None,
-        help=(
-            "Create BLAST database of retrieved GenBank protein sequences.\n"
-            "Give the path to the directory to store the database"
-        ),
-    )
-
     # Add option to specify path to configuration file
     parser.add_argument(
         "-c",
@@ -96,6 +83,14 @@ def build_genbank_sequences_parser(argv: Optional[List] = None):
         help="Path to configuration file. Default: None, scrapes entire database",
     )
 
+    # Add option to use own CAZy class synoymn dict
+    parser.add_argument(
+        "--cazy_synonyms",
+        type=Path,
+        default=None,
+        help="Path to JSON file containing CAZy class synoymn names",
+    )
+
     # Add option to define classes to retrieve protein sequences for
     parser.add_argument(
         "--classes",
@@ -104,10 +99,16 @@ def build_genbank_sequences_parser(argv: Optional[List] = None):
         help="Classes from which all families are to be scraped. Separate classes by ','"
     )
 
+    parser.add_argument(
+        "--ec_filter",
+        type=str,
+        default=None,
+        help="Limit retrieval to proteins annotated with the provided EC numbers. Separate EC numbers with single commas"
+    )
+
     # specify the number of accessions posted in single ePost to NCBI
     parser.add_argument(
-        "-e",
-        "--epost",
+        "--entrez_batch_size",
         type=int,
         default=150,
         help="Number of accessions posted to NCBI per epost, advice to be max 200. Default=150"
@@ -131,19 +132,6 @@ def build_genbank_sequences_parser(argv: Optional[List] = None):
         help="Families to scrape. Separate families by commas 'GH1,GH2'",
     )
 
-    # Add option to enable writing sequences to FASTA file or files, or not at all
-    parser.add_argument(
-        "--fasta",
-        type=str,
-        default=None,
-        help=(
-            "Enable writing out retrieved sequences to FASTA file(s).\n"
-            "Writing 'separate' produces a single FASTA file per retrieved protein sequence,\n"
-            "else, write the path to the FASTA to add retrieved protein sequences to\n"
-            "(this can be a pre-existing or non-existing FASTA file."
-        ),
-    )
-
     # Add option to restrict the scrape to specific kingdoms
     parser.add_argument(
         "--kingdoms",
@@ -174,16 +162,35 @@ def build_genbank_sequences_parser(argv: Optional[List] = None):
         help="Defines log file name and/or path",
     )
 
-    # enable retrieving protein sequences for only primary GenBank accessions
     parser.add_argument(
-        "-p",
-        "--primary",
-        dest="primary",
+        "--nodelete_cache",
+        dest="nodelete_cache",
         action="store_true",
         default=False,
-        help="Enable retrieveing protein sequences for only primary GenBank accessions",
+        help="Do not delete content in existing cache dir",
     )
 
+    # Add option to update sequences if the retrieved sequence is different
+    # If not enabled then sequences will only be retrieved and added for proteins that do not
+    # already have a protein sequence
+    parser.add_argument(
+        "--seq_update",
+        dest="seq_update",
+        action="store_true",
+        default=False,
+        help="Enable overwriting sequences in the database if the retrieved sequence is different",
+    )
+
+    # Add option to force file over writting
+    parser.add_argument(
+        "--sql_echo",
+        dest="sql_echo",
+        action="store_true",
+        default=False,
+        help="Set SQLite engine echo to True (SQLite will print its log messages)",
+    )
+
+
     # Add option to restrict the scrape to specific species. This will scrape CAZymes from
     # all strains belonging to each listed species
     parser.add_argument(
@@ -204,18 +211,6 @@ def build_genbank_sequences_parser(argv: Optional[List] = None):
         ),
     )
 
-    # Add option to update sequences if the retrieved sequence is different
-    # If not enabled then sequences will only be retrieved and added for proteins that do not
-    # already have a protein sequence
-    parser.add_argument(
-        "-u",
-        "--update",
-        dest="update",
-        action="store_true",
-        default=False,
-        help="Enable overwriting sequences in the database if the retrieved sequence is different",
-    )
-
     # Add option for more detail (verbose) logging
     parser.add_argument(
         "-v",
diff --git a/cazy_webscraper/utilities/parsers/genbank_cov_parser.py b/cazy_webscraper/utilities/parsers/genbank_cov_parser.py
new file mode 100644
index 00000000..3939515c
--- /dev/null
+++ b/cazy_webscraper/utilities/parsers/genbank_cov_parser.py
@@ -0,0 +1,185 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Submodule for building cmd-line parsers"""
+
+
+import argparse
+import os
+
+from pathlib import Path
+from typing import List, Optional
+
+
+def build_parser(argv: Optional[List] = None):
+    """Return ArgumentParser parser for the script 'expand.genbank_sequences.py'."""
+    # Create parser object
+    parser = argparse.ArgumentParser(
+        prog="cazy_genbank_coverage.py",
+        description="Retrieves genomic accessions from NCBI for proteins in the local CAZyme database",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+
+    # Add positional/required arguments
+    parser.add_argument(
+        "database",
+        type=Path,
+        metavar="local CAZy database",
+        help="Path to local CAZy database",
+    )
+
+    parser.add_argument(
+        "email",
+        type=str,
+        metavar="user email address",
+        help="User email address, requirement of NCBI-Entrez",
+    )
+
+    # Add optional arguments to parser
+
+    # specify the number of accessions posted in single ePost to NCBI
+    parser.add_argument(
+        "--batch_size",
+        type=int,
+        default=150,
+        help="Number of accessions posted to NCBI per epost, advice to be max 200. Default=150"
+    )
+
+    # Add option to force file over writting
+    parser.add_argument(
+        "-f",
+        "--force",
+        dest="force",
+        action="store_true",
+        default=False,
+        help="Force writing in existing output directory",
+    )
+    parser.add_argument(
+        "--force_cache",
+        dest="force_cache",
+        action="store_true",
+        default=False,
+        help="Force writing in existing cache dir",
+    )
+
+    # Add option to restrict scrape to specific genera
+    parser.add_argument(
+        "--genera",
+        type=str,
+        default=None,
+        help="Genera to restrict the scrape to"
+    )
+
+    # Add log file name option
+    # If not given, no log file will be written out
+    parser.add_argument(
+        "-l",
+        "--log",
+        type=Path,
+        metavar="log file name",
+        default=None,
+        help="Defines log file name and/or path",
+    )
+
+    parser.add_argument(
+        "--nodelete",
+        dest="nodelete",
+        action="store_true",
+        default=False,
+        help="Do not delete content in existing output dir",
+    )
+    parser.add_argument(
+        "--nodelete_cache",
+        dest="nodelete_cache",
+        action="store_true",
+        default=False,
+        help="Do not delete content in existing cache dir",
+    )
+
+    parser.add_argument(
+        "-o",
+        "--output_dir",
+        type=Path,
+        metavar="log file name",
+        default=os.getcwd(),
+        help="Define path to output directory. Default: cwd",
+    )
+
+    # Add option to force file over writting
+    parser.add_argument(
+        "--sql_echo",
+        dest="sql_echo",
+        action="store_true",
+        default=False,
+        help="Set SQLite engine echo to True (SQLite will print its log messages)",
+    )
+
+    parser.add_argument(
+        "--retries",
+        type=int,
+        default=10,
+        help="Number of times to retry connection to NCBI"
+    )
+
+    # Add option to restrict the scrape to specific species. This will scrape CAZymes from
+    # all strains belonging to each listed species
+    parser.add_argument(
+        "--species",
+        type=str,
+        default=None,
+        help="Species (written as Genus Species) to restrict the scrape to"
+    )
+
+    # Add option for more detail (verbose) logging
+    parser.add_argument(
+        "-v",
+        "--verbose",
+        dest="verbose",
+        action="store_true",
+        default=False,
+        help="Set logger level to 'INFO'",
+    )
+
+    if argv is None:
+        # parse command-line
+        return parser
+    else:
+        # return namespace
+        return parser.parse_args(argv)
diff --git a/scraper/utilities/parsers/pdb_strctre_parser.py b/cazy_webscraper/utilities/parsers/pdb_strctre_parser.py
similarity index 89%
rename from scraper/utilities/parsers/pdb_strctre_parser.py
rename to cazy_webscraper/utilities/parsers/pdb_strctre_parser.py
index 349d39af..a6d93dac 100644
--- a/scraper/utilities/parsers/pdb_strctre_parser.py
+++ b/cazy_webscraper/utilities/parsers/pdb_strctre_parser.py
@@ -49,7 +49,7 @@
 
 
 
-def build_pdb_structures_parser(argv: Optional[List] = None):
+def build_parser(argv: Optional[List] = None):
     """Return ArgumentParser parser for the script 'expand.genbank_sequences.py'."""
     # Create parser object
     parser = argparse.ArgumentParser(
@@ -85,6 +85,21 @@ def build_pdb_structures_parser(argv: Optional[List] = None):
         help="Path to configuration file. Default: None, scrapes entire database",
     )
 
+    parser.add_argument(
+        "--cache_dir",
+        type=Path,
+        default=None,
+        help="Target path for cache dir to be used instead of default path",
+    )
+
+    # Add option to use own CAZy class synoymn dict
+    parser.add_argument(
+        "--cazy_synonyms",
+        type=Path,
+        default=None,
+        help="Path to JSON file containing CAZy class synoymn names",
+    )
+
     # Add option to define classes to retrieve protein sequences for
     parser.add_argument(
         "--classes",
@@ -152,6 +167,14 @@ def build_pdb_structures_parser(argv: Optional[List] = None):
         help="enable/disable deletion of exisiting files",
     )
 
+    parser.add_argument(
+        "--nodelete_cache",
+        dest="nodelete_cache",
+        action="store_true",
+        default=False,
+        help="Do not delete content in existing cache dir",
+    )
+
     # enable specifying an output directory
     parser.add_argument(
         "-o",
diff --git a/cazy_webscraper/utilities/parsers/uniprot_parser.py b/cazy_webscraper/utilities/parsers/uniprot_parser.py
new file mode 100644
index 00000000..e22741c6
--- /dev/null
+++ b/cazy_webscraper/utilities/parsers/uniprot_parser.py
@@ -0,0 +1,296 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# (c) University of St Andrews 2020-2021
+# (c) University of Strathclyde 2020-2021
+# (c) James Hutton Institute 2020-2021
+#
+# Author:
+# Emma E. M. Hobbs
+#
+# Contact
+# eemh1@st-andrews.ac.uk
+#
+# Emma E. M. Hobbs,
+# Biomolecular Sciences Building,
+# University of St Andrews,
+# North Haugh Campus,
+# St Andrews,
+# KY16 9ST
+# Scotland,
+# UK
+#
+# The MIT License
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Submodule for building cmd-line parsers"""
+
+
+import argparse
+
+from pathlib import Path
+from typing import List, Optional
+
+
+def build_parser(argv: Optional[List] = None):
+    """Return ArgumentParser parser for script."""
+    # Create parser object
+    parser = argparse.ArgumentParser(
+        prog="cazy_webscraper.py",
+        description="Scrapes the CAZy database",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+
+    parser.add_argument(
+        "database",
+        type=Path,
+        help="Path to local CAZyme database"
+    )
+
+    # Add optional arguments to parser
+    parser.add_argument(
+        "--bioservices_batch_size",
+        type=int,
+        default=150,
+        help="Batch size for queries parsed by bioservices to UniProt to retrieve protein data"
+    )
+
+    parser.add_argument(
+        "--cache_dir",
+        type=Path,
+        default=None,
+        help="Target path for cache dir to be used instead of default path",
+    )
+
+    # Add option to use own CAZy class synoymn dict
+    parser.add_argument(
+        "--cazy_synonyms",
+        type=Path,
+        default=None,
+        help="Path to JSON file containing CAZy class synoymn names",
+    )
+
+    # Add option to define complete classes to scrape
+    parser.add_argument(
+        "--classes",
+        type=str,
+        default=None,
+        help="CAZy classes to retrieve UniProt data for. Separate classes by ','"
+    )
+
+    # Add option to specify path to configuration file
+    parser.add_argument(
+        "-c",
+        "--config",
+        type=Path,
+        metavar="config file",
+        default=None,
+        help="Path to configuration file. Default: None, scrapes entire database",
+    )
+
+    parser.add_argument(
+        "--delete_old_ec",
+        dest="delete_old_ec",
+        action="store_true",
+        default=False,
+        help=(
+            "Delete EC number-GenBank relationships/annotations\n"
+            "that are not longer listed in UniProt for a given protein"
+        ),
+    )
+
+    parser.add_argument(
+        "--delete_old_pdbs",
+        dest="delete_old_pdbs",
+        action="store_true",
+        default=False,
+        help="Delete PDB accessions that are not longer listed in UniProt for a given protein",
+    )
+
+    parser.add_argument(
+        "-e",
+        "--ec",
+        dest="ec",
+        action="store_true",
+        default=False,
+        help="Retrieve EC numbers from UniProt",
+    )
+
+    parser.add_argument(
+        "--ec_filter",
+        type=str,
+        default=None,
+        help="Limit retrieval to proteins annotated with the provided EC numbers. Separate EC numbers with single commas"
+    )
+
+    # Add option to specify families to scrape
+    parser.add_argument(
+        "--families",
+        type=str,
+        default=None,
+        help="CAZy families to UniProt data for. Separate families by commas 'GH1,GH2' (case sensitive)"
+    )
+
+    # Add option to restrict scrape to specific genera
+    parser.add_argument(
+        "--genera",
+        type=str,
+        default=None,
+        help="Genera to UniProt data for"
+    )
+
+    parser.add_argument(
+        "--kingdoms",
+        type=str,
+        default=None,
+        help="Tax Kingdoms to UniProt data for"
+    )
+
+    # Add log file name option
+    # If not given, no log file will be written out
+    parser.add_argument(
+        "-l",
+        "--log",
+        type=Path,
+        metavar="log file name",
+        default=None,
+        help="Defines log file name and/or path",
+    )
+
+    parser.add_argument(
+        "--name_update",
+        dest="name_update",
+        action="store_true",
+        default=False,
+        help=(
+            "Overwrite the existing protein name from UniProt if a different\n"
+            "protein name is retrieve for the same UniProt accession"
+        ),
+    )
+
+    # Add option to not delete content in the existing cache dir
+    parser.add_argument(
+        "--nodelete_cache",
+        dest="nodelete_cache",
+        action="store_true",
+        default=False,
+        help="When called, content in the existing cache dir is NOT deleted",
+    )
+
+    parser.add_argument(
+        "-p",
+        "--pdb",
+        dest="pdb",
+        action="store_true",
+        default=False,
+        help="Retrieve PDB accessions from UniProt",
+    )
+
+    # Add option to enable number of times to retry scraping
+    parser.add_argument(
+        "-r",
+        "--retries",
+        type=int,
+        default=10,
+        help="Number of times to retry scraping a family or class page if error encountered",
+    )
+
+    parser.add_argument(
+        "-s",
+        "--sequence",
+        dest="sequence",
+        action="store_true",
+        default=False,
+        help=(
+            "Retrieve protein Aa sequences from UniProt for records that do NOT\n"
+            "have a sequence in the local CAZyme database"
+        ),
+    )
+
+    parser.add_argument(
+        "--seq_update",
+        dest="seq_update",
+        action="store_true",
+        default=False,
+        help=(
+            "Retrieve protein Aa sequences from UniProt for records and overwrite the existing\n"
+            "sequence in the local CAZyme database if a newer sequence is retireved from UniProt"
+        ),
+    )
+
+    # Add option to force file over writting
+    parser.add_argument(
+        "--sql_echo",
+        dest="sql_echo",
+        action="store_true",
+        default=False,
+        help="Set SQLite engine echo to True (SQLite will print its log messages)",
+    )
+
+    # Add option to UniProt data for specific species. This will scrape CAZymes from
+    # all strains belonging to each listed species
+    parser.add_argument(
+        "--species",
+        type=str,
+        default=None,
+        help="Species (written as Genus Species) to UniProt data for"
+    )
+
+    # Add option to restrict scraping to specific strains of organisms
+    parser.add_argument(
+        "--strains",
+        type=str,
+        default=None,
+        help=(
+            "Specific strains of organisms to UniProt data for "
+            "(written as Genus Species Strain)"
+        ),
+    )
+
+    # Add option to define time out limit for trying to connect to CAZy
+    parser.add_argument(
+        "-t",
+        "--timeout",
+        type=int,
+        default=45,
+        help="Connection timeout limit (seconds)"
+    )
+
+    parser.add_argument(
+        "--uniprot_batch_size",
+        type=int,
+        default=150,
+        help="Batch size for queries sent to the UniProt REST API to retrieved UniProt accessions"
+    )
+
+    # Add option for more detail (verbose) logging
+    parser.add_argument(
+        "-v",
+        "--verbose",
+        dest="verbose",
+        action="store_true",
+        default=False,
+        help="Set logger level to 'INFO'",
+    ) 
+
+    if argv is None:
+        # parse command-line
+        return parser
+    else:
+        # return namespace
+        return parser.parse_args(argv)
diff --git a/configuration_files/README.md b/configuration_files/README.md
new file mode 100644
index 00000000..d4bc5ef4
--- /dev/null
+++ b/configuration_files/README.md
@@ -0,0 +1,9 @@
+# Configuration files
+
+To faciliate the reproducbility of compiling a local CAZyme database, `cazy_webscraper` can be configured using a YAML file.
+
+This directory contains:
+- [`template_cazy_config.yaml`]() - an empty template cofig file for configuring scraping CAZy
+- [`template_get)data_config.yaml`]() - an empty template config file for configuring the retrieval of data from UniProt, GenBank and PDB
+- [`example_cazy_config.yaml`]() - An example configuration file for scraping CAZy, to scrape all proteins from Bacteria and Viruses in GH and CE, PL families PL1 and PL2
+- [`example_get_data_config.yaml`]() - An example configuration file for retrieving data from UniProt, GenBank or PDB for proteins from genera _Trichoderma_ and _Aspergillus_, and the species _Pythium ultimum_, and belonging for GT families GT1, GT5 and GT6
diff --git a/configuration_files/example_cazy_config.yaml b/configuration_files/example_cazy_config.yaml
new file mode 100644
index 00000000..26314bd8
--- /dev/null
+++ b/configuration_files/example_cazy_config.yaml
@@ -0,0 +1,24 @@
+# Under 'classes' list class from which all proteins will retrieved
+# Under each families respective name, list the specific families/subfamilies to be scraped
+# Write the FULL family name, e.g. 'GH1', NOT only its number, e.g. '1'
+# To list multiple families, each familiy must be on a new line starting indented once
+# relative to the parent class name, and the name written within quotation marks.
+# For more information on writing lists in Yaml please see:
+# https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html 
+classes:  # classes from which all proteins will be retrieved
+  - "GH"
+  - "CE"
+Glycoside Hydrolases (GHs):
+GlycosylTransferases (GTs):
+Polysaccharide Lyases (PLs):
+  - "PL`"
+  - "PL2"
+Carbohydrate Esterases (CEs):
+Auxiliary Activities (AAs):
+Carbohydrate-Binding Modules (CBMs):
+genera:  # list genera to be scraped
+species:  # list species, this will scrape all strains under the species
+strains:  # list specific strains to be scraped
+kingdoms:  # Archaea, Bacteria, Eukaryota, Viruses, Unclassified
+ - "Bacteria"
+ - "Viruses"
diff --git a/configuration_files/example_get_data_config.yaml b/configuration_files/example_get_data_config.yaml
new file mode 100644
index 00000000..f92fd455
--- /dev/null
+++ b/configuration_files/example_get_data_config.yaml
@@ -0,0 +1,28 @@
+# Under 'classes' list class from which all proteins will retrieved
+# Under each families respective name, list the specific families/subfamilies to be scraped
+# Write the FULL family name, e.g. 'GH1', NOT only its number, e.g. '1'
+# To list multiple families, each familiy must be on a new line starting indented once
+# relative to the parent class name, and the name written within quotation marks.
+# For more information on writing lists in Yaml please see:
+# https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html 
+classes:  # classes from which all proteins will be retrieved
+  - "GH"
+  - "CE"
+Glycoside Hydrolases (GHs):
+GlycosylTransferases (GTs):
+Polysaccharide Lyases (PLs):
+  - "GT1"
+  - "GT5"
+  - "GT6"
+Carbohydrate Esterases (CEs):
+Auxiliary Activities (AAs):
+Carbohydrate-Binding Modules (CBMs):
+genera:  # list genera to be scraped
+ - "Trichoderma"
+ - "Aspergillus"
+species:  # list species, this will scrape all strains under the species
+- "Pythium ultimum"
+strains:  # list specific strains to be scraped
+kingdoms:  # Archaea, Bacteria, Eukaryota, Viruses, Unclassified
+ec:
+- "EC1.2.3.4"
\ No newline at end of file
diff --git a/scraper/scraper_config.yaml b/configuration_files/template_cazy_config.yaml
similarity index 95%
rename from scraper/scraper_config.yaml
rename to configuration_files/template_cazy_config.yaml
index 577a57b6..10765e2c 100644
--- a/scraper/scraper_config.yaml
+++ b/configuration_files/template_cazy_config.yaml
@@ -9,13 +9,10 @@ classes:  # classes from which all proteins will be retrieved
 Glycoside Hydrolases (GHs):
 GlycosylTransferases (GTs):
 Polysaccharide Lyases (PLs):
-  - "PL28"
 Carbohydrate Esterases (CEs):
 Auxiliary Activities (AAs):
 Carbohydrate-Binding Modules (CBMs):
 genera:  # list genera to be scraped
- - "Trichoderma"
 species:  # list species, this will scrape all strains under the species
 strains:  # list specific strains to be scraped
 kingdoms:  # Archaea, Bacteria, Eukaryota, Viruses, Unclassified
- - "Bacteria"
diff --git a/configuration_files/template_get_data_config.yaml b/configuration_files/template_get_data_config.yaml
new file mode 100644
index 00000000..6b6b5326
--- /dev/null
+++ b/configuration_files/template_get_data_config.yaml
@@ -0,0 +1,19 @@
+# Under 'classes' list class from which all proteins will retrieved
+# Under each families respective name, list the specific families/subfamilies to be scraped
+# Write the FULL family name, e.g. 'GH1', NOT only its number, e.g. '1'
+# To list multiple families, each familiy must be on a new line starting indented once
+# relative to the parent class name, and the name written within quotation marks.
+# For more information on writing lists in Yaml please see:
+# https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html 
+classes:  # classes from which all proteins will be retrieved
+Glycoside Hydrolases (GHs):
+GlycosylTransferases (GTs):
+Polysaccharide Lyases (PLs):
+Carbohydrate Esterases (CEs):
+Auxiliary Activities (AAs):
+Carbohydrate-Binding Modules (CBMs):
+genera:  # list genera to be scraped
+species:  # list species, this will scrape all strains under the species
+strains:  # list specific strains to be scraped
+kingdoms:  # Archaea, Bacteria, Eukaryota, Viruses, Unclassified
+ec:  # list of EC numbers proteins must be annotated with in the local db to retrieve data for
diff --git a/docs/source/cache.rst b/docs/source/cache.rst
new file mode 100644
index 00000000..2888daba
--- /dev/null
+++ b/docs/source/cache.rst
@@ -0,0 +1,89 @@
+=========================
+``cazy_webscraper`` cache
+=========================
+
+To facilitate reproducibility of scrapping CAZy, ``cazy_webscraper`` logs each scrape within the compiled 
+CAZyme datbase and writes out cache files.
+
+This pages walks through the data logged in the CAZyme database and the different cache files written by 
+``cazy_webscraper``, as well as how to define a different cache directory, instead of the default.
+
+--------------
+The Logs table
+--------------
+
+The database built by ``cazy_webscraper`` contains a table called 'Logs'. This table logs every 
+scrape of CAZy, UniProt and GenBank which added data to the database.
+
+The table contains the following columns and data:
+
+* **log_id:** Autoincrement ID number
+* **date:** Date scrape was initated (in ISO format)
+* **time:** Time scrape was initated (in ISO format)
+* **database:** Name of the external database from which data was retrieved (i.e. 'CAZy', 'UniProt' or 'GenBank')
+* **retrieved_annotations:** List of annotation types retrieved (e.g. 'EC number, PDB accession, Sequence')
+* **classes:** CAZy classes for which data was retrieved
+* **families:** CAZy families for which data was retrievedclasses
+* **kingdoms:** taxonomy Kingdoms filteres applied
+* **genera_filter:** taxonomy Kingdoms filteres applied
+* **species_filter:** taxonomy Kingdoms filteres applied
+* **strains_filter:** taxonomy Kingdoms filteres applied
+* **ec_filter:** EC fliters applied to retrieve data for CAZymes annotated with the specified EC numbers (only applies to retrieval of data from UniProt, GenBank and PDB)
+* **cmd_line:** Reproduction of the command line arguments passed to ``cazy_webscraper``.
+
+The 'Logs' table allows any one who uses the database to see how the dataset was compiled.
+
+-----------
+Cache files
+-----------
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Cache files when retrieving data from CAZy
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+``cazy_webscraper`` writes out 2 cache files. These are:
+
+* ``cazy_db_<date-time>.zip`` which is the txt file downloaded from CAZy containing all CAZy Data
+* ``<db_name>_<date-time>_connection_failures.log`` which contains :
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Cache files when retrieving data from UniProt
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When retreiving data from UniProt several cache files are produced, and the number depends on the size of the 
+dataset retrieval:
+
+
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Cache files when retrieving data from GenBank
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When retrieving protein sequences from GenBank two cache file are produced:
+
+* ``genbank_sequence_retrieved.txt`` contains the GenBank protein accessions for which a protein sequence **was** successfully retrieved
+* ``genbank_no_sequence.txt`` contains the GenBank protein accessions for which a protein sequence was not retrieved, as well as the possible reason
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Cache files when retrieving data from PDB
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+
+
+
+---------------
+Cache directory
+---------------
+
+By default ``cazy_webscraper`` creates the cache directory in the same directory that the datbase is created, and 
+with the name ``.cazy_webscraper``.
+
+To use a different cache directory instead add the ``--cache_dir`` flag, followed by the path to the cache directory.
+
+.. NOTE::
+    The cache directory does not need to already exist, ``cazy_webscraper`` will build the cache directory 
+    and all it's parent directories.
+
+If the target cache directory already exists, by default ``cazy_webscraper`` will delete the content already 
+present in the already existing cache directory. To not delete the exsiting directory content add the 
+``nodelete_cache`` flag.
diff --git a/docs/source/conf.py b/docs/source/conf.py
index 4de55caa..ce2b19ff 100644
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -29,7 +29,7 @@
 author = 'Emma E. M. Hobbs'
 
 # The full version, including alpha/beta/rc tags
-release = '0.1.2'
+release = '2.0.0'
 
 
 # -- General configuration ---------------------------------------------------
diff --git a/docs/source/genbank.rst b/docs/source/genbank.rst
index 7c451f74..ef979b0c 100644
--- a/docs/source/genbank.rst
+++ b/docs/source/genbank.rst
@@ -1,74 +1,115 @@
-
+=========================================
 Retrieving Protein Sequences from GenBank
-==========================================
+=========================================
+
+``cazy_webscraper`` can be used to retrieve protein amino acid sequences from NCBI GenBank for user-specified data sets of CAZymes 
+in the local CAZymes database. 
 
-The CAZy webscraper supports the automated retrieval of protein sequences from GenBank for the s
-craped CAZymes. This is performed by using the **BioPython** ``Bio.entrez`` module.
+The retrieval of data from NCBI is performed by using the **BioPython** `Bio.entrez <https://biopython.org/docs/1.75/api/Bio.Entrez.html>_` module [Cock *et al.*, 2009].
+
+    Cock, P. J. A, Antao, T., Chang, J. T., Chapman, B. A., Cox, C. J., Dalke, A. _et al._ (2009) 'Biopython: freely available Python tools for computaitonal molecular biology and bioinformatics', _Bioinformatics_, 25(11), pp. 1422-3.
 
 .. note::
-    For specific information of the ``bio.entrez`` module please see the 
+    For specific information of the ``Bio.entrez`` module please see the 
     `entrez documentation <https://biopython.org/docs/1.75/api/Bio.Entrez.html>`_.
 
 
-Expected practise
+-----------
+Quick Start
+-----------
+
+To download protein sequences for all CAZymes in the local CAZyme database, and write them to the local CAZyme database, 
+use the following command structure:
+
+.. code-block:: bash
+    cw_get_genbank_seqs <path to local CAZyme db> <user email address>
+
+For example:
+
+.. code-block:: bash
+    cw_get_genbank_seqs cazy/cazyme.db myemail@domain.com
+
+.. NOTE::
+   The ``cw`` prefix is an abbreviation of ``cazy_webscraper``.
+
+
+--------------------
+Command line options
 --------------------
 
-Before using Entrez to access the NCBI’s online resources (via any modules), please read the 
-`NCBI’s Entrez User Requirements <https://www.ncbi.nlm.nih.gov/books/NBK25497/>`_.  
-**If you are found to be abusing NCBI's systems, they can and will ban your access!** 
+``database`` - **REQUIRED** Path to a local CAZyme database to add UniProt data to.
 
-In summary:
+``email`` - **REQUIRED** User email address, required by NCBI Entrez.
 
-For scrapes that would perform more than 100 requests to NCBI, perform the scrape at the 
-**weekend** or **outside USA peak times**.
+``--cache_dir`` - Path to cache dir to be used instead of default cache dir path.
 
-Do not perform more than 10 queries to NCBI per second, this is already handled by the 
-webscraper.
+``--cazy_data`` - Path to a txt file downloaded from CAZy containing a CAZy database dump
 
-Provide a user email so that NCBI can contact you if there is a problem.
+``--cazy_synonyms`` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
 
+``--config``, ``-c`` - Path to a configuration YAML file. Default: None.
 
-.. note::
-    For large requests NCBI suggests using the session history feature of Entrez. At the moment the 
-    scraper performs a single request per GenBank accession retrieved from CAZy. Overtime this will 
-    may be replaced with using the session history feature to better meet expected practise of the NCBI.
+``--classes`` - list of classes to retrieve UniProt data for.
 
+``--ec_filter`` - List of EC numbers to limit the retrieval of protein data for proteins annotated with at least one of the given EC numbers **in the local CAZyme database**.
 
-Enabling sequence retrieval
------------------------------
+``--entrez_batch_size`` - Change the query batch size submitted via [`Entrez`]() to retrieve protein sequences from GenBank data. Default is 150. `Entrez <https://www.ncbi.nlm.nih.gov/books/NBK179288/>_` recommands queries not larger than XXX objects in length.
 
-To enable retrieval of protein sequences from GenBank use the ``-g`` or ``--genbank`` option 
-followed by an email address. The provision of the email address is a requirement of Entrez, the 
-search system of NCBI.
+``--families`` - List of CAZy (sub)families to retrieve UniProt protein data for.
 
-For example:  
-``cazy_webscraper -g dummy_email@domain.com``
+``--genera`` - List of genera to restrict the retrieval of protein to data from UniProt to proteins belonging to one of the given genera.
 
+``--log``, ``-l`` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
 
-Output
-------
+``--nodelete_cache`` - When called, content in the existing cache dir will **not** be deleted. Default: False (existing content is deleted).
 
-Phe protein sequence of the scraped CAZymes are retrieved from GenBank are retrieved in 
-the FASTA file format and can be written to STDOUT to facilitate piping to a subsequent program or 
-written out to disk, within a specified directory.
+``--nodelete_log`` - When called, content in the existing log dir will **not** be deleted. Default: False (existing content is deleted).
 
-To write out the FASTA files to a specified directory use the option ``-genbank_output`` followed 
-by the path to specified directory.
+``--retries``, ``-r`` - Define the number of times to retry making a connection to CAZy if the connection should fail. Default: 10.
 
+``--seq_update`` - If a newer version of the protein sequence is available, overwrite the existing sequence for the protein in the database. Default is false, the protein sequence is **not** overwritten and updated.
 
-GenBank synonyms
-----------------------------
+``--sql_echo`` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
 
-Often multiple GenBank accession numbers are listed for a given CAZyme within CAZy. However, only the 
-first listed accession number is hyperlinked to the GenBank database. Examination of the other listed 
-synonyms (referred to as **genbank synonyms** in the webscraper) shows that these GenBank synonyms are 
-the result of submission of identical protein sequences, splice site and protein isoforms. It has been 
-interpreted that it is the first GenBank accession that is listed and hyperlinked to GenBank is the accession 
-number of protein sequence which was used by CAZy to catalogue the CAZyme and the GenBank synonyms were 
-identified and listed by having extremely high sequence identity to the catalogued CAZyme. Therefore, the 
-webscraper only retrieves the FASTA file of first listed GenBank accession of each CAZyme. 
+``--species`` - List of species (organsim scientific names) to restrict the retrieval of protein to data from UniProt to proteins belonging to one of the given species.
 
-.. note::
-    If you wish to see a feature where the FASTA files of the GenBank synonyms are also retrieved, please 
-    raise an issue in the ``cazy_webscraper`` GitHub repository to let us know that this is a feature 
-    you would like added.
+``--strains`` - List of species strains to restrict the retrieval of protein to data from UniProt to proteins belonging to one of the given strains.
+
+``--timeout``, ``-t`` - Connection timout limit (seconds). Default: 45.
+
+``--verbose``, ``-v`` - Enable verbose logging. This does **not** set the SQLite engine ``echo`` parameter to True. Default: False.
+
+-----------
+Basic Usage
+-----------
+
+The command-line options listed above can be used in combination to customise the scraping of CAZy. Some options (e.g. ``--families`` and ``--classes``) define the broad group of data that will be scraped, others (e.g. ``--species``) are used to filter and fine-tune the data that is scraped.
+
+The ``--classes``, ``--families``, ``--kingdoms``, ``--genera``, ``--species``, and ``--strains`` filteres are applied 
+in the exactly same for retrieving data from CAZy as retrieving data from UniProt. Examples of using these flags 
+can be found in the ``cazy_webscraper`` tutorial in this documentation.
+
+The ``--seq_update`` flag is used in the same way for retrieving protein sequences from UniProt and GenBank.
+
+.. NOTE::
+    To retrieve data for members of specific CAZy subfamilies, list the subfamilies after the ``--families`` 
+    flag.
+
+------------------------
+Updating local sequences
+------------------------
+
+When using ``--sequence`` flag, ``cazy_webscraper`` will only add *new* protein sequences to the database, i.e.
+it will only add protein sequences to records that do not have a sequence. Therefore, if a protein
+already has a sequence in the local database, this sequence is **not** overwritten.
+
+You may wish to update the protein sequences in your local CAZyme database. To do this use the ``--sequence``/``-s`` 
+flag to tell ``cazy_webscraper`` to retrieve protein sequences, **and** use the ``--seq_update`` flag.
+
+.. code-block:: bash
+
+    cw_get_genbank_seqs cazy_db.db -s --seq_update
+
+This instructs ``cazy_webscraper`` to overwriting existing protein sequences in the local database *if* a newer version 
+of the sequence is retrieved from UniProt. This is checked by comparing the 'last modified date' of the 
+protein sequence in the local database against the sequence retrieved from UniProt.
diff --git a/scraper/expand/genbank_sequences/write_fasta_from_db.py b/docs/source/genbanktutorial.rst
similarity index 100%
rename from scraper/expand/genbank_sequences/write_fasta_from_db.py
rename to docs/source/genbanktutorial.rst
diff --git a/docs/source/index.rst b/docs/source/index.rst
index a4120bdc..6b7a6b36 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -1,7 +1,5 @@
 .. cazy_webscraper documentation master file, created by
    sphinx-quickstart on Fri Nov 20 15:33:10 2020.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root ``toctree`` directive.
 
 ===========================================
 Welcome to cazy_webscraper's documentation!
@@ -28,10 +26,6 @@ Build Information
    :target: https://circleci.com/gh/HobnobMancer/cazy_webscraper
 .. image:: https://codecov.io/gh/HobnobMancer/cazy_webscraper/branch/master/graph/badge.svg
    :target: https://codecov.io/gh/HobnobMancer/cazy_webscraper
-.. image:: https://img.shields.io/badge/Python-v3.8.---orange
-   :target: https://www.python.org/about/
-.. image:: https://img.shields.io/badge/Bioinformatics-EASTBio-ff69b4
-   :target: http://www.eastscotbiodtp.ac.uk/eastbio-student-cohort-2019
 
 
 --------
@@ -60,22 +54,47 @@ Build Information
 ``cazy_webscraper``
 -------------------
 
-``cazy_webscraper`` is a Python3 package for the automated retrieval of carbohydrate-active enzyme (CAZyme) data from the `CAZy <http://wwww.cazy.org/>`_ database. This program is free to use under the MIT license, and we kindly request that, if you use this program or Python package, you cite it as indicated below.
+``cazy_webscraper`` is a Python3 package for the automated retrieval of Carbohydrate-Active enZyme (CAZyme) data from the `CAZy <http://wwww.cazy.org/>`_ database. This program is free to use under the MIT license, and we kindly request that, if you use this program or Python package, you cite it as indicated below.
 
    Hobbs, Emma E. M.; Pritchard, Leighton; Chapman, Sean; Gloster, Tracey M. (2021): cazy_webscraper Microbiology Society Annual Conference 2021 poster. figshare. Poster. https://doi.org/10.6084/m9.figshare.14370860.v7 
 
-``cazy_webscraper`` retrieves data from CAZy, writing it to a local SQLite3 file. ``cazy_webscraper`` can retrieve the protein sequences from NCBI for CAZymes in the local database, and also write out those sequences in FASTA format. Additionally, ``cazy_webscraper`` can retrieve protein structures from the Research Collaboratory for Structural Bioinformatics (RCSB) Protein Data Bank, `PDB <https://www.rcsb.org/>`_, for CAZymes in the local database.
+``cazy_webscraper`` retrieves data from CAZy, writing it to a local SQLite3 file. 
 
-``cazy_webscraper`` can be configured to scrape the entire CAZy database, to recover only CAZymes filtered by user-supplied criteria, such as CAZy classes, CAZy (sub)family, or taxonomy. 
+``cazy_webscraper`` can also retrieve the protein data from `UniProt <https://www.uniprot.org/>`_ for CAZymes in the local database. 
+This data includes:  
+- UniProt accession
+- Protien name
+- Protein amino acid sequence
+- EC numbers
+- PDB accessions
+
+``cazy_webscraper`` can retrieve the protein sequences from NCBI for CAZymes in the local database.
+
+``cazy_webscraper`` can write out protein sequences retrieved from UniPtot and NCBI in FASTA format, 
+and build a local BLAST database.
+
+Additionally, ``cazy_webscraper`` can retrieve protein structures from the Research Collaboratory 
+for Structural Bioinformatics (RCSB) Protein Data Bank, `PDB <https://www.rcsb.org/>`_, 
+for CAZymes in the local database.
+
+``cazy_webscraper`` can be configured to scrape the entire CAZy database, to recover only CAZymes 
+filtered by user-supplied criteria, such as CAZy classes, CAZy (sub)family, or taxonomy. 
 
 ----------
 Quickstart
 ----------
 
-.. toctree::
-   :maxdepth: 4
-   
-   quickstart
+We have produced a "Getting Started With ``cazy_webscraper``" `poster <https://hobnobmancer.github.io/cazy_webscraper/getting_started_poster.pdf>`_.
+
+To download the entire CAZy dataset, and save the data set to the current working directory with the final name 
+``cazy_webscraper_<date>_<time>.db``, use the following command structure:  
+
+.. code-block:: bash
+   cazy_webscraper <user_email>
+
+.. NOTE::
+   The user email address is a requirement of NCBI. NCBI is queried to identify the currect source organism 
+   for a given protein, when multiple source organisms are retrieved from CAZy for a single protein.
 
 -------------
 Best practice
@@ -86,7 +105,9 @@ When performing a series of many automated, repeated calls to a server it is pol
 The webscraper can appear to run slowly but this may be due to the bandwidth at the CAZy server, or server speed. ``cazy_webscraper`` provides a progress bar to reassure the user that the webscraper is working. 
 
 .. WARNING::
-   **Please do not perform a complete scrape of the CAZy database unless you specifically require to reproduce the entire CAZy dataset. A complete scrape will take several hours and may unintentionally deny the service to others.**
+   **Please do not perform a retrieval of UniProt, NCBI and/or PDB data for the entire CAZy dataset, unless 
+   absolutely unavoidable. Retrieving the data from any of these exteranl databases for the entire CAZy 
+   dataset will take several hours and may unintentionally deny the service to others.**
 
 -------------
 Documentation
@@ -101,8 +122,13 @@ For details and updates on development, please consult the `GitHub repository <h
    usage
    tutorial
    database
+   uniprot
+   uniprottutorial
    genbank
+   genbanktutorial
    pdb
+   pdbtutorial
+   cache
    license
 
 
@@ -121,4 +147,4 @@ Development and issues
 
 If there are additional features you wish to be added, you have problems with the scraper, or would like to contribute please raise an issue at the `GitHub repository <https://github.com/HobnobMancer/cazy_webscraper>`_.
 
-* Issues Page: `https://github.com/HobnobMancer/cazy_webscraper/issues <https://github.com/HobnobMancer/cazy_webscraper/issues>`_
\ No newline at end of file
+* Issues Page: `https://github.com/HobnobMancer/cazy_webscraper/issues <https://github.com/HobnobMancer/cazy_webscraper/issues>`_
diff --git a/docs/source/pdb.rst b/docs/source/pdb.rst
index cf0de3c5..b0390db8 100644
--- a/docs/source/pdb.rst
+++ b/docs/source/pdb.rst
@@ -1,27 +1,36 @@
 ================================
-Retrieving Structures from PDB
+Retrieving structure files from PDB
 ================================
 
-The CAZy webscraper supports the automated retrieval of 3D structures from PDB for the scraped CAZymes. 
-This is performed by using the ``BioPython`` module ``Bio.PDB``. All protein structures that are listed 
-for each CAZyme are retrieved from the `RSCB PDB database <https://www.rcsb.org/>`_.
+``cazy_webscraper`` can be used to retrieve protein structure files for PDB accessions in a local CAZyme database from `RSCB PDB database <https://www.rcsb.org/>`_. The downloading of the structure files is handled by the ``BioPython`` module ``Bio.PDB``. 
 
 For specific information of the ``Bio.PDB`` module please see the 
 `BioPython documentation <https://biopython.org/wiki/The_Biopython_Structural_Bioinformatics_FAQ>`_.
 
-
 .. warning::
-    If many requests are going to be made in a series to PDB (for example a series of 100 
-    requests), then it is expected practise to perform the scraper at the **weekend** or 
-    **outside USA peak times**.
+        If many PDB structure files are going to retrieved PDB (for example more than 100), it is expected practise to perform the
+        opretional output side peak times.
+
+.. note::
+    PDB structure files are retrieved for the PDB accessions *in* a local CAZyme database created using ``cazy_webscraper``.
+
+-----------
+Quick Start
+-----------
 
+To download the protein structure file for all PDB accessions in a local CAZyme database, use the following command structure:
 
-Enabling structure retrieval
------------------------------
+.. code-block:: bash
+   cw_get_pdb_structures <path to local CAZyme db> <desired file formats>
 
-To enable retrieval of protein structures from PDB then use the option ``-p`` or ``--pdb_output`` 
-followed by the desired file format. The options of file format are (as specified in the BioPython 
-`documentation <https://biopython.org/docs/1.75/api/Bio.PDB.PDBList.html>`_):
+.. NOTE::
+   The ``cw`` prefix on command is an abbreviation of ``cazy_webscraper``.
+   
+----------------------
+Structure file formats
+----------------------
+
+``cw_get_pdb_structures`` can retrieve protein structure files in a series of file formats. The options of file format are (as specified in the BioPython `documentation <https://biopython.org/docs/1.75/api/Bio.PDB.PDBList.html>`_):
 
 * mmCif (default, PDBx/mmCif file),
 * pdb (format PDB),
@@ -29,16 +38,79 @@ followed by the desired file format. The options of file format are (as specifie
 * mmtf (highly compressed),
 * bundle (PDB formatted archive for large structure}
 
+Any combination of file formats can be provided to ``cw_get_pdb_structures`` to download every file type for each PDB accession in the local CAZyme database. To list multiple file formats, separate each file format with a single comma. For example, to download the mmCif and xml files for every PDB accession in a local CAZyme database (located at ``cazy/cazyme_db.db``), use the following command:
+
+.. code-block:: bash
+    cw_get_pdb_structures cazy/cazyme_db.db mmCif,xml
+
+.. WARNING::
+    The file formats are case sensitive. For example, make sure to use 'mmCif' not 'mmcif'.
+
+--------------------
+Command line options
+--------------------
+
+``database`` - **REQUIRED** Path to a local CAZyme database to add UniProt data to.
+
+``pdb`` - **REQUIRED** List of file formats to retrieve from PDB for each PDB accession.
+
+``--batch_size`` - Size of an individual batch query of PDB accessions submitted to PDB. Default is 150.
+
+``--cache_dir`` - Path to cache dir to be used instead of default cache dir path.
+
+``--cazy_synonyms`` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
+
+``--config``, ``-c`` - Path to a configuration YAML file. Default: None.
+
+``--classes`` - list of classes to retrieve UniProt data for.
+
+``--ec_filter`` - List of EC numbers to limit the retrieval of structure files to proteins with at least one of the given EC numbers *in the local CAZyme database*.
+
+``--families`` - List of CAZy (sub)families to retrieve UniProt protein data for.
+
+``--genera`` - List of genera to restrict the retrieval of protein to data from UniProt to proteins belonging to one of the given genera.
+
+``--log``, ``-l`` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
+
+``--nodelete``, ``-n`` - When called, content in the existing output  will **not** be deleted. Default: False (existing content is deleted).
+
+``--nodelete_cache`` - When called, content in the existing cache dir will **not** be deleted. Default: False (existing content is deleted).
+
+``--outdir``, ``-o`` - Define output directory to write out structure files. Default, write structure files to current working directory.
+
+``--retries``, ``-r`` - Define the number of times to retry making a connection to CAZy if the connection should fail. Default: 10.
+
+``--sql_echo`` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
+
+``--species`` - List of species (organsim scientific names) to restrict the retrieval of protein to data from UniProt to proteins belonging to one of the given species.
+
+``--strains`` - List of species strains to restrict the retrieval of protein to data from UniProt to proteins belonging to one of the given strains.
+
+``--timeout``, ``-t`` - Connection timout limit (seconds). Default: 45.
+
+``--update_seq`` - If a newer version of the protein sequence is available, overwrite the existing sequence for the protein in the database. Default is false, the protein sequence is **not** overwritten and updated.
+
+``--verbose``, ``-v`` - Enable verbose logging. This does **not** set the SQLite engine ``echo`` parameter to True. Default: False.
+
+
+
+-----------
+Basic Usage
+-----------
+
+The command-line options listed above can be used in combination to customise the retrieval of protein structure files from PDB for proteins of interest. Some options (e.g. ``--families`` and ``--classes``) define the broad group of proteins for which structure files are retrieved, others (e.g. ``--species``) are used to filter and fine-tune the protein dataset for which structure files are retrieved.
+
+The ``--classes``, ``--families``, ``--kingdoms``, ``--genera``, ``--species``, and ``--strains`` filteres are applied 
+in the exactly same for retrieving data from CAZy and UniProt, as retrieving data from PDB. Examples of using these flags 
+can be found in the ``cazy_webscraper`` and ``cw_get_uniprot_data`` tutorial in this documentation.
+
+.. NOTE::
+    To retrieve protein structures for members of specific CAZy subfamilies, list the subfamilies after the ``--families`` 
+    flag.
 
-Output
-------
 
-The retrieved protein structures can be written to a specified directory by using the ``-pdb_output`` 
-option followed by the path to desired output directory. This directory does not already need to be 
-existing, the webscraper can make the directory.
+---------------------------------
+Structure file retrieval from PDB
+---------------------------------
 
-The ``BioPython`` ``Bio.PDB`` module does not support writing out the protein structures to standard 
-out. Therefore, if ``--pdb_output`` is not given the webscraper will write out the protein structures 
-to the output directory specified by the ``-o``/``--output`` option. If the ``-o``/``--output`` option 
-is set as standard out (STDOUT), for example if the ``-o``/``--output`` option is not called, then 
-the protein structure files will be written to the current working directory.
+The command for using ``cazy_webscraper`` for retrieval of PDB structure files from PDB is ``cw_get_pdb_structures``.
diff --git a/docs/source/pdbtutorial.rst b/docs/source/pdbtutorial.rst
new file mode 100644
index 00000000..b76c52ea
--- /dev/null
+++ b/docs/source/pdbtutorial.rst
@@ -0,0 +1,256 @@
+==========================================================================
+Tutorials on configuring ``cazy_webscraper`` to retrieve data from PDB
+==========================================================================
+
+``cazy_webscraper`` can be configured to retrieve user specified data sets from UniProt for specific date 
+sets of CAZymes in a local CAZyme database. Many of the configuration options 
+apply to the retrieval of protein data from CAZy, UniProt, GenBank and PDB.
+
+.. NOTE::
+   ``cazy_webscraper`` retrieves protein data from UniProt for CAZymes in a local CAZyme database. It 
+   will not add new proteins to the database.
+
+``cazy_webscraper`` can be configured via the **command line** and/or via a **YAML configuration file**.
+
+This page runs through examples of how to combine the various 'filters' that can be applied, to fully customised 
+the retrieval of data from UniProt. These tutorials are designed for those with less experience using command-line tools.
+
+.. NOTE::
+  If you installed ``cazy_webscraper`` using ``bioconda`` or ``pip`` to invoke ``cazy_webscraper`` to retrieve UniProt data call it using ``cw_get_uniprot_data`` - this is the method used in this tutorial.  
+  If you installed ``cazy_webscraper`` from source then you will need to invoke ``cazy_webscraper`` from the root of the repo using the command ``python3 cazy_webscraper/expand/uniprot/get_uniprot_data.py``.
+
+From this point on, we will be discussed the ``cw_get_uniprot_data``, which is the entry point for 
+retrieving data from UniProt. We also presume you are comfortable configuring ``cazy_webscraper`` for the 
+scraping of data from CAZy.
+
+All data retrieved from UniProt by ``cw_get_uniprot_data`` is added to the local CAZyme database.
+
+----------------------------------
+Configuration via the command line
+----------------------------------
+
+``cw_get_uniprot_data`` has only one required argument: the path to the local CAZyme database created 
+using ``cazy_webscraper``. Therefore, ``cw_get_uniprot_data`` can be enabled using a simple command structure:
+
+.. code-block:: bash
+
+  cazy_webscraper <path to the local CAZyme db>
+
+For example, if our database was stored in ``cazy/cazyme.db``, we would used:
+
+.. code-block:: bash
+   
+  cazy_webscraper cazy/cazyme.db
+
+When no optional arguments are provided, the default behaviour is invoked. The default behaviour is to:
+
+* Retrieve protein data for **all** CAZymes in the local CAZyme db
+* Retrieve UniProt protein accessions and protein names
+
+-----------------------------------------
+Options configurable at the command line 
+-----------------------------------------
+
+The following behaviours of the ``cw_get_uniprot_data`` can be configured at the command-line in the terminal:  
+
+* Limit the retrieve of UniProt protein data to CAZymes in the local databaes from specific CAZy classes, CAZy families, kingdoms, genuera, species, strains and/or EC numbers
+* Enable retrieving EC number annotations
+* Enable retrieving PDB accessions
+* Enable retrieving protein amino acid sequences
+* Enable updating protein sequences in the local CAZyme database if newer versions are retrieved from UniProt
+* Enable verbose logging during the operation of the webscraper
+
+`Here <https://cazy-webscraper.readthedocs.io/en/latest/configuration_scraper.html>`_ you can find a full list of the command-line flags and options.
+
+
+---------------------------------------------------------------
+Retrieving protein data for CAZy classes and families to scrape
+---------------------------------------------------------------
+
+The ``--classes`` and ``--families`` flags from scraping data from CAZy are applied in the extact same way 
+for retrieve data from UniProt.
+
+For instance, if instead of retrieving protein data for all CAZymes in your local CAZyme database, you want to 
+retrieve protein data for CAZymes in specific CAZy classes then add the 
+``--classes`` flag followed by the classes you want to retrieve protein data for.
+
+.. TIP::
+   To list multiple classes, separate the classes with a single comma. 
+
+For example, if you want to retrieve protein data for all CAZymes from Glycoside Hydrolase and Carbohydrate Esterases then use the command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --classes GH,CE
+
+OR
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --classes Glycoside Hydrolases,Carbohydrate Esterases
+
+Retrieving protein data for proteins from specific specific CAZy families is achieved using the ``--families`` flag. For 
+example, to retrieve protein data for all proteins in PL1, PL2 and PL3 in the local CAZyme database use the 
+following command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --families PL1,PL2,PL3
+
+.. WARNING::
+   ``cw_get_uniprot_data`` only accpets families written in the proper CAZy family syntax.
+   GH1 is accepted.
+   gh1 and GlycosideHydrolases1 are not accepted.
+
+As with scraping data from CAZy, the ``--classes`` and ``--families`` flags can be combined. To retrieve 
+protein data for all CAZymes in PL1, PL2, PL3 and *all* of GH and CE both:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --families PL1,PL2,PL3 --classes GH,CE
+
+**AND**
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --classes GH,CE --families PL1,PL2,PL3
+
+are accepted.
+
+
+------------------
+Applying taxonomic
+------------------
+
+The ``--kingdoms``, ``--genera``, ``--species`` and ``--strains`` flags can be used to refine the dataset 
+of proteins to retrieve protein data by taxonomy. These flags are applied in the exact same way as they 
+are used for the scraping of data from CAZy. Only proteins in the local CAZyme database and matching at least on of the provided taxonomy 
+criteria will have protein data retrieved from UniProt and added to the local CAZyme datbase.
+
+For example, if you want to retrieve protein data for all CAZymes in a local CAZyme database from bacterial and eukaryotic species then use the command 
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --kingdoms bacteria,eukaryota
+
+.. warning::
+   The kingdoms must be spelt the same way CAZy spells them, for example use 'eukaryot**a**' instead of 'eukaryot**e**'.
+   
+.. NOTE:: 
+   The kingdoms are **not** case sensitive, therefore, both ``bacteria`` *and* ``Bacteria`` are accepted. 
+
+.. NOTE::
+   You can list the kingdoms in *any* order. Thus, both ``bacteria,eukaryota`` *and* ``eukaryota,bacteria`` are accepted.
+
+You can combine any combination of the optional flags, including combining the taxonomic filters. For example,
+you may wish to retrieve protein data for all CAZymes in a local CAZyme database that are derived from all viral species, Aspergillus species, Layia carnosa, Layia chrysanthemoides, Trichoderma reesei QM6a and 
+Trichoderma reesei QM9414. To do this we would combine the respective flags for a single ``cw_get_uniprot_data`` command. The command 
+we would use would be:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --kingdoms viruses --genera Aspergillus --species Layia carnosa,Layia chrysanthemoides --strains Trichoderma reesei QM6a,Trichoderma reesei QM9414
+
+.. note::
+   The order that the flags are used and the order taxa  are listed does **not** matter, and separate multiple taxa names with a single comma 
+   with **no** spaces.
+
+.. warning::
+   Use the standard scientific name formating. Captialise the first letter of *genus* and write a lower 
+   case letter for the first letter of the species.
+
+   Aspergillus niger is **correct**
+
+   asepergillus niger is **incorrect**
+
+   ASPERGILLUS NIGER is **incorrect**
+
+.. warning::
+   When you specify a species ``cw_get_uniprot_data`` will retrieval CAZymes from *all* strains of the species.
+
+
+-------------------------
+Applying EC number filter
+-------------------------
+
+The retrieval of protein data from UniProt can also be limited to proteins in a local CAZyme database that are
+annotated with specific EC numbers.
+
+Having previously retrieved EC number annotations and added them to the local CAZyme database, you  may 
+wish to retrieve protein data for CAZymes annotated with specific EC numbers. To do this add the 
+``--ec_filter`` flag to the command, follwed by a list of EC numbers.
+
+.. code-block:: bash
+   
+   cw_get_uniprot_data cazy/cazyme.db --ec_filter "EC1.2.3.4,EC2.3.4.5"
+
+
+.. NOTE::
+    Provide complete EC numbers. 
+    Both dashes ('-') and asterixes ('*') are accepted for missing digits in EC numbers.
+
+    EC1.2.3.- and EC1.2.3.* are accepted.
+    EC1.2.3. and EC 1.2.3 are **not** accepted.
+
+.. NOTE::
+   The 'EC' prefix is not necessary.
+   EC1.2.3.4 and 1.2.3.4 are accepted.
+
+.. WARNING::
+    If using dashes to represent missing digits in EC numbers, it is recommended to bookend the entire 
+    EC number list in single or double quotation marks. Some terminals may misinterpret EC1.2.-.- as trying to invoke the options '.'
+
+.. NOTE::
+    ``cazy_webscraper`` will retrieve the specified UniProt data for all proteins in the local CAZyme 
+    database that are annotated with **at least one** of the given EC numbers. Therefore, if multiple 
+    EC numbers are given this **does not mean** only CAZymes will all provided EC numbers will have data retrieved
+    from UniProt for them.
+
+``--ec_filter`` is based upon EC number annotations stored within the local CAZyme database. For 
+example, if protein A is annotated with the EC1.2.3.4, but this annotation is not stored in the 
+local CAZyme database, using ``--ec_filter EC1.2.3.4`` will **not** cause ``cazy_webscraper`` to retrieve
+data for protein A. This is because ``cazy_webscraper`` does not know protein A is annotated with 
+EC1.2.3.4, because this annotation is not within its database.
+
+.. WARNING::
+    If ``--ec_filter`` is used along side ``--ec``, ``cazy_webscraper`` will retrieve **all** EC number 
+    annotations from UniProt for all proteins in the local CAZyme database that are associated with 
+    at least one of the EC numbers provided via ``--ec_filter`` within the CAZyme database.
+
+
+---------------------
+Combining all filters
+---------------------
+
+The ``--classes``, ``--families``, ``--ec_filter``, ``--kingdoms``, ``--genera``, ``--species`` and ``--strains`` flags can 
+be used in any combination to define a specific subset of proteins in the local CAZyme database for whom
+protein data from UniProt will be retrieved. These flags can be used with any combination of 
+``--ec``, ``--pdb``, ``--sequence``, ``--update_seq`` to customise what data is retrieved from UniProt and 
+added to the local CAZyme database.
+
+Below we run through 3 example commands of combining these flags, and the resulting behaviour.
+
+**Example 1:**
+To retrieve PDB accessions for all CAZymes in GH, GT, CE1, CE5 and CE8, and which are derived from baceterial species we use the command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --pdb --classes GH,CE --families CE1,CE5,CE8 --kingdoms bacteria
+
+
+**Example 2:**
+To retrieve EC numbers and PDB accessions for all CAZymes in GH and which are derived from *Aspegillus* and *Trichoderma* species we use the command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --pdb --classes GH --genera Aspegillus,Trichoderma
+
+
+**Example 3:**
+To retrieve EC numbers and sequences for all CAZymes in GH,CE and CBM which are derived from baceterial species and are annotated with at least one of 
+EC3.2.1.23, EC3.2.1.37 and EC3.2.1.85, we use the command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --pdb --sequences --classes GH,CE,CBM --kingdoms bacteria --ec_filter "3.2.1.23,3.2.1.37,3.2.1.85"
+
diff --git a/docs/source/tutorial.rst b/docs/source/tutorial.rst
index f0173fb6..13f0bb80 100644
--- a/docs/source/tutorial.rst
+++ b/docs/source/tutorial.rst
@@ -1,52 +1,54 @@
 ================================================================
-Tutorials on configuring ``cazy_webscraper``
+Tutorials on configuring ``cazy_webscraper`` to scrape CAZy
 ================================================================
 
 ``cazy_webscraper`` can be configured to retrieve user specified data sets from CAZy. The configuration 
-applies to the retrieval of protein sequences from GenBank and protein structure files from PDB.
+applies to the retrieval of protein data from UniProt, protein sequences from GenBank and protein structure files from PDB.
 
 ``cazy_webscraper`` can be configured via the **command line** and/or via a **YAML configuration file**.
 
 This page runs through examples of how to combine the various 'filters' that can be applied, to fully customised 
 the scraping of CAZy. These tutorials are designed for those with less experience using command-line tools.
 
-
-.. note::
+.. NOTE::
   If you installed ``cazy_webscraper`` using ``bioconda`` or ``pip`` to invoke ``cazy_webscraper`` call it using ``cazy_webscraper`` - this is the method used in this tutorial.  
-  If you installed ``cazy_webscraper`` from source then you will need to invoke ``cazy_webscraper`` using the command ``python3 <path to cazy_webscraper.py>``. For example, if you were located in the root of the repository, you would use: ``python3 scraper/cazy_webscraper.py``.
+  If you installed ``cazy_webscraper`` from source then you will need to invoke ``cazy_webscraper`` using the command ``python3 <path to cazy_webscraper.py>``. For example, if you were located in the root of the repository, you would use: ``python3 cazy_webscraper/cazy_scraper.py``.
 
 
+----------------------------------
 Configuration via the command line
-###########################################
+----------------------------------
 
-There are no required arguments for ``cazy_webscraper``, therefore the scraper can be enabled 
-by simply calling the scraper at the command line in the terminal: 
+``cazy_webscraper`` has only one required argument: the user email address. Therefore, 
+ the scraper can be enabled by simply calling the scraper at the command line in the terminal: 
 
 .. code-block:: bash
-  cazy_webscraper
+  cazy_webscraper myemail@domain.com
 
-When NO optional arguments are provided the default behaviour of the scraper will be performed. 
+When no optional arguments are provided the default behaviour of the scraper will be performed. 
 The default behaviour is to:
 
 * Scrape the entire CAZy databases
-* Write the resulting database to standard out (STDOUT)
+* Write the resulting database to the current working directory
 * Not to retrieve subfamilies (members of subfamilies will be retrieved but only their parent family will be listed)
 
 .. note::
    **For those new to using command-line tools: arguments**  
    Arguments are additional pieces of information we add onto the end of the command. They are used to configure the specific behaviour 
-   performed by computer when we tell it to perform a specific command. In the examples above the command is ``cazy_webscraper.py``, 
-   where we have told to computer to run the Python program ``cazy_webscraper``, and because no additional information was provided, the computer 
+   performed by computer when we tell it to perform a specific command. In the examples above the command is ``cazy_webscraper myemail@domain.com``, 
+   where we have told to computer to run the Python program ``cazy_webscraper`` and submit the user email 
+   address to NCBI for the retrieval of source orgnaism data. No additional information was provided, the computer 
    will invoke ``cazy_webscraper`` using its default behaviour. If you do not want the default behaviour of ``cazy_webscraper`` then we need to 
    pass additionally arguments to the computer when telling it to run ``cazy_webscraper``, which we cover in the section below.
 
 
-Options configurable at the command line
-############################################
+-----------------------------------------
+Options configurable at the command line 
+-----------------------------------------
 
 The following behaviours of the ``cazy_webscraper`` can be configured at the command-line in the terminal:  
 
-* Limit the scraping of CAZy to specific CAZy classes, CAZy families, kingdoms, genuera, species, strains and/or EC numbers.
+* Limit the scraping of CAZy to specific CAZy classes, CAZy families, kingdoms, genuera, species, and/or strains
 * Force writing out the database to a a new or existing directory
 * Write out a log file of the packages operation
 * Not delete content already present in the output directory
@@ -56,14 +58,15 @@ The following behaviours of the ``cazy_webscraper`` can be configured at the com
 `Here <https://cazy-webscraper.readthedocs.io/en/latest/configuration_scraper.html>`_ you can find a full list of the command-line flags and options.
 
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 How to use the command-line options
-#############################################
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The command-line options listed above can be used in any combination to customise the scraping of CAZy. The options that apply a 'filter' 
-to restrict which CAZymes are scraped from CAZy are applied in combination. For example, if the ``--families`` option and ``--ec`` option are called then 
-only CAZymes from the specified families **and** annotated with the listed EC numbers will be retrieved.
+to restrict which CAZymes are scraped from CAZy are applied in combination. For example, if the ``--families`` option and ``--genera`` option are called then 
+only CAZymes from the specified families **and** belonging to source organisms within the defined genera will be retrieved.
 
-We will now walk through some examples of how to use ``cazy_webscraper``. All example code provided in this section will presume that the terminal is pointed at the ``scraper`` directory, which contains the ``cazy_webscraper.py`` file.
+We will now walk through some examples of how to use ``cazy_webscraper``. All example code provided in this section will presume that the terminal is pointed at the ``cazy_webscraper`` directory, which contains the ``cazy_webscraper.py`` file.
 
 .. note::
    **For those new to using command-line tools: flags**
@@ -77,21 +80,30 @@ We will now walk through some examples of how to use ``cazy_webscraper``. All ex
    ``cazy_webscraper --help``.
 
 
+-------------------------------------
 Configuring were the output is saved
-*************************************************
+-------------------------------------
+
+Instead of writing out database to the current working directory using the default database name 
+(``cazy_webscraper_<date>_<time>.db``), we can name the database and directory that the database 
+created by ``cazy_webscraper`` is written by calling the ``--output`` flag. 
 
-We can name the directory that the database created by ``cazy_webscraper`` is written to by calling the ``--output`` flag. 
-We add the flag to the command that invokes ``cazy_webscraper``. For example, to write the output to the directory 'cazyme_database' we can use:
+We add the flag to the command that invokes ``cazy_webscraper``. For example, to write the output to the directory 'cazyme_database' with the file 
+name 'cazyme_database.db' we can use:
 
 .. code-block:: bash
 
-   cazy_webscraper --output cazyme_database
+   cazy_webscraper --output cazyme_database/cazyme_database.db
 
 OR we can use the short hand version of the ``--output`` flag, ``-o``:
 
 .. code-block:: bash
 
-   cazy_webscraper -o cazyme_database
+   cazy_webscraper -o cazyme_database/cazyme_database.db
+
+.. NOTE::
+   The final element of the path provided after the ``--output``/``-o`` flag is the name of database compiled by 
+   ``cazy_webscraper``.
 
 The output directory does not have to exist when ``cazy_webscraper`` is invoked. ``cazy_webscraper`` can make 
 the output directory, including all necessary parent directories. 
@@ -100,33 +112,20 @@ The ``--output`` flag can take an infinetly long path. For example, we could use
 
 .. code-block:: bash
 
-   cazy_webscraper -o data/cazyme_research/cazyme_database
+   cazy_webscraper -o data/cazyme_research/cazyme_database/cazyme_database.db
 
 If the directories ``cazymes_research`` and ``cazyme_database`` did not exist then ``cazy_webscraper`` will build 
 these for you.
 
-In the Bash terminal paths are **relative**, meaning that the terminal starts in the directory it is currently 
-looking at and follows the path from there. The installation section of this tutorial covers this when 
-discussing how to change directory.
-
-.. warning::
-   When requesting ``cazy_webscraper`` make an output directory, the parent of the directory we wish to make 
-   **must already exist**. For examlple, if we asked ``cazy_webscraper`` to write its output to the directory 
-   'data/cazyme_research/cazyme_database' and the directory 'cazyme_database' did not exist, *if* the directory 
-   'cazyme_research' did exist ``cazy_webscraper`` would build the directory 'cazyme_database' within 'cazyme_research'. 
-   However, if 'cazyme_research' *and* 'cazyme_database' did not exist, then ``cazy_webscraper`` would raise an error saying 
-   the path 'data/cazyme_research' does not exist.
-
-
 **Writing the output to an existing database**
 If you want to write the output CAZyme database to a directory that already exists, you will need to add the force (``--force`` *or* ``-f``) flag 
 anywhere to the ``cazy_webscraper`` command. For example:
 
 .. code-block:: bash
 
-   cazy_webscraper -o data/cazyme_research/cazyme_database -f
+   cazy_webscraper -o data/cazyme_research/cazyme_database/cazyme_database.db -f
 
-By default ``cazy_webscraper`` will delete or content in an already existing output directory. Therefore, in the above example, 
+By default ``cazy_webscraper`` will delete all content in an already existing output directory. Therefore, in the above example, 
 if the directory ``cazyme_database`` already existed, ``cazy_webscraper`` would delete all content in the directory then proceed. 
 
 You may wish to retain the data already in that directory. To do this add the 'no delete' (``--nodelete`` *or* ``-n``) flag anywhere 
@@ -134,35 +133,40 @@ to the ``cazy_webscraper`` command. For example:
 
 .. code-block:: bash
 
-   cazy_webscraper -o data/cazyme_research/cazyme_database -f -n
+   cazy_webscraper -o data/cazyme_research/cazyme_database/cazyme_database.db -f -n
 
 The order you invoke *any* of the optional flags does not matter, for example the following three examples perform the 
 exact same operation as the code given above:
 
 .. code-block:: bash
 
-   cazy_webscraper --force -o data/cazyme_research/cazyme_database -f
+   cazy_webscraper --force -o data/cazyme_research/cazyme_database/cazyme_database.db -f
 
 .. code-block:: bash
 
-   cazy_webscraper -n -o data/cazyme_research/cazyme_database -f
+   cazy_webscraper -n -o data/cazyme_research/cazyme_database/cazyme_database.db -f
 
 .. code-block:: bash
 
-   cazy_webscraper --nodelete --force --output data/cazyme_research/cazyme_database
+   cazy_webscraper --nodelete --force --output data/cazyme_research/cazyme_database/cazyme_database.db
 
 The above examples also highlight that it does not matter if you use the long or short versions of each of the flags.
 
+.. NOTE::
+   If you elect to write the database to a file in the current working directory, you do not need to worry 
+   about ``cazy_webscraper`` deleting content in the current working directory. This only applies if you chose to
+   write the database to a directory over than the current working directory.
 
-
+----------------------------------------------
 Specifying CAZy classes and families to scrape
-**************************************************
+----------------------------------------------
 
+^^^^^^^^^^^^^^^^^^^^^^^^^
 Scraping specific classes
-==============================
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 If instead of scraping all of CAZy, you want to only scrape CAZymes from specific CAZy classes then add the 
-``--classes`` flag followed by the classes you want to scrape. If you want to list multiple families, separate the families 
+``--classes`` flag followed by the classes you want to scrape. If you want to list multiple classes, separate the classes 
 with a single comma. When you specify a CAZy class to scrape, *all* CAZy families within that class will be scraped.
 
 For example, if you want to scrape all CAZymes from Glycoside Hydrolase and Carbohydrate Esterases then use the command:
@@ -185,17 +189,18 @@ For example, if you want to scrape all CAZymes from Glycoside Hydrolase and Carb
 
 * **Carbohydrate-Binding Modules (CBMs):** Carbohydrate-Binding-Modules, Carbohydrate_Binding_Modules, Carbohydrate_Binding Modules, CarbohydrateBindingModules, CARBOHYDRATE-BINDING-MODULES, CARBOHYDRATE_BINDING_MODULES, CARBOHYDRATE_BINDING MODULES, CARBOHYDRATEBINDINGMODULES, carbohydrate-binding-modules, carbohydrate_binding_modules, carbohydrate_binding modules, carbohydratebindingmodules, CBMs, CBM, cbms, cbm
 
-.. note::
+.. TIP::
    These synonyms are stored in a JSON found at ``scraper/utilities/parse_configuration/cazy_dictionary.json``. 
    Storing these synonyms allows you to modify this file if you wish to add your own synonoms for each CAZy class.
 
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 Scraping specific families
-===============================
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To specify specific CAZy families to scrape, add the ``--families`` flag followed by the families you want 
-to scrape. If you want to scrape multiple families, add the ``--families`` flag *once* followed by a list of *all* 
-the CAZy families you want to scrape, separated by a single comma.
+to scrape. If you want to scrape multiple families, list all the families you wish to scrape, with each family 
+separated with a single comma.
 
 For example, if you want to scrape all CAZymes from GH2, PL5, CE1, CE2 and AA10 use:
 
@@ -203,17 +208,12 @@ For example, if you want to scrape all CAZymes from GH2, PL5, CE1, CE2 and AA10
 
    cazy_webscraper --families GH2,PL5,CE1,CE2,AA10
 
-Make sure to use the accepted CAZy nomenclature; 'GH2' is accepted but 'gh2' is not.
-
-.. note::
-   When ``--families`` is invoked any CAZy classes that do **not** include an of the CAZy families specified will 
-   **not** be scraped. Therefore, using the example above, CAZymes from the families GH2, PL5, CE1, CE2 and AA10 
-   **will** be retrieved; however, CAZymes from any other families from those classes **will not** be retrieved, and CAZymes 
-   from the Carbohydrate Binding Modules (CBM) and GlycoslyTransferases classes will **not** be retrieved.
-
+.. WARNING::
+   Make sure to use the accepted CAZy nomenclature; 'GH2' is accepted but 'gh2' is not.
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Scraping specific classes AND families
-========================================
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 If you want to specify specific CAZy classes *and* families to scrape then add *both* the ``--classess`` *and* ``-families`` 
 flags, because you can combine, mix-and-match, any combination of optional flags when invoking ``cazy_webscraper``.
@@ -225,7 +225,7 @@ For example, if we wanted to scrape all CAZymes from GH1, PL9 and *all* of CE we
    cazy_webscraper --families GH1,PL9 --classes CE
 
 It does **not** matter what order you add the optional flags to your command. Therefore, if we wanted to 
-scrape all CAZymes from PL1, PL2, PL3 and *all* of GH and CE we both:
+scrape all CAZymes from PL1, PL2, PL3 and *all* of GH and CE, both:
 
 .. code-block:: bash
 
@@ -240,16 +240,18 @@ scrape all CAZymes from PL1, PL2, PL3 and *all* of GH and CE we both:
 are accepted.
 
 .. note::
-   In the example ``cazy_webscraper.py --classes GH,CE --families PL1,PL2,PL3`` all CAZymes from PL1, 
+   In the example ``cazy_webscraper --classes GH,CE --families PL1,PL2,PL3`` all CAZymes from PL1, 
    PL2 and PL3 would be retrieved, but no CAZymes from the other PL families, in addition all CAZymes from all GH and CE 
    families would be retrieved, but no CAZymes from AA, GT or CBM families would be retrieved.
 
 
-Applying taxonomic and EC number filters
-********************************************
+------------------
+Applying taxonomic
+------------------
 
+^^^^^^^^^^^^^^^^^^^
 Specifying kingdoms
-================================
+^^^^^^^^^^^^^^^^^^^
 
 You may only be interest in CAZymes that are derived from species from a specific taxonomic kingdom. 
 CAZy classifies source organisms under one of 5 kingdoms:
@@ -260,10 +262,10 @@ CAZy classifies source organisms under one of 5 kingdoms:
 * Viruses
 * Unclassified
 
-To restrict the scraping of CAZy to retrieve CAZymes only derived from species from specific taxonomic kingdoms 
-then add the ``--kingdoms`` flag to the ``cazy_webscraper`` command followed by the kingdoms to limit the retrieval 
+To restrict the scraping of CAZy to retrieve CAZymes only derived from species from specific taxonomic kingdoms
+ add the ``--kingdoms`` flag to the ``cazy_webscraper`` command followed by the kingdoms to limit the retrieval 
 of CAZymes to. To list multiple kingdoms you need only add the ``--kingdoms`` flag *once*, then list all the kingdoms 
-you want to restrict the restrival of CAZymes to, separated by a single comma.
+you are interested in, separated by a single comma.
 
 For example, if you want to retrieve CAZymes only from bacterial and eukaryotic species then use the command 
 
@@ -273,19 +275,25 @@ For example, if you want to retrieve CAZymes only from bacterial and eukaryotic
 
 
 .. warning::
-   The kingomds must be spelt the same way CAZy spells them, for example use 'eukaryot**a**' instead of 'eukaryot**e**'. The kingdoms 
-   are **not** case sensitive, therefore, both ``bacteria`` *and* ``Bacteria`` are accepted. You can also list the kingdoms in 
-   *any* order. Thus, both ``bacteria,eukaryota`` *and* ``eukaryota,bacteria`` are accepted.
+   The kingdoms must be spelt the same way CAZy spells them, for example use 'eukaryot**a**' instead of 'eukaryot**e**'.
+   
+.. NOTE:: 
+   The kingdoms are **not** case sensitive, therefore, both ``bacteria`` *and* ``Bacteria`` are accepted. 
 
+.. NOTE::
+   You can list the kingdoms in *any* order. Thus, both ``bacteria,eukaryota`` *and* ``eukaryota,bacteria`` are accepted.
 
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 Speciying Genera to scrape
-=======================================
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 You can customise the scraping of CAZy to retrieve only CAZymes from *all* species from specific 
-genera. To do this add the ``--genera`` flag to the ``cazy_webscraper`` command followed by all 
-the genera you want to retrieve CAZymes from. CAZymes from any genera that you do not list will 
-**not** be retrieved. To list multiple genera, you need to only add the ``--genera`` flag once followed 
-by a list of all genera, with each genera separated with a single comma and *no* spaces.
+genera. To do this add the ``--genera`` flag to the ``cazy_webscraper`` command followed by your
+genera of interes.
+
+To list multiple genera, you need to only add the ``--genera`` flag *once* followed 
+by a list of all your genera, with each genera separated with a single comma and *no* spaces.
 
 For example, if we wanted to retrieve all CAZymes from *all* Aspergillus, Trichoderma and Streptomyces species 
 we would use the command:
@@ -310,15 +318,15 @@ we would use the command:
    ASPERGILLUS is **incorrect**
 
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Specifying species of organisms to scrape
-=============================================
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-
-You can specify to retrieve CAZymes only derived from specific species. To do this add the ``--species`` 
+You can specify to retrieve only CAZymes derived from specific species. To do this add the ``--species`` 
 flag to the ``cazy_webscraper`` command, followed by a list of all species you wish to retrist the retrieval of 
 CAZymes to. Separate each species with a single comma. Also for each species use the full scientific name for the species.
 
-For example, if we wanted to retrieve all CAZymes from Aspergillus niger and Aspergillus fumigatus we would use the command:  
+For example, if we wanted to retrieve all CAZymes from *Aspergillus niger* and *Aspergillus fumigatus* we would use the command:  
 
 .. code-block:: bash
 
@@ -344,13 +352,16 @@ For example, if we wanted to retrieve all CAZymes from Aspergillus niger and Asp
    When you specify a species ``cazy_webscraper`` will retrieval CAZymes from *all* strains of the species.
 
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Specify specific strains of species to scrape
-====================================================
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+You may only be interested in specific strains of a species. Instead of scraping CAZymes for all strains 
+of a given speices, add the ``--strains`` flag followed by the specific species strains you wish to restrict 
+the retrieval of CAZymes to.
 
-You may only be interested in specific strains of a species. Therefore, ``cazy_webscraper`` allows you to 
-restrict the retrieval of CAZymes to only those derived from specific strains of species. To do this 
-add the ``--strains`` flag to the ``cazy_webscraper`` command, followed by a list of all the strains 
-of interest. Separate each strain with a single command and no spaces.
+List the full scientific name followed by the strain name. To specify multiple strains, list all 
+strains of interest and separate with a single comma with **no** space.
 
 For example, if we wanted to retrieve all CAZymes from Aspergillus niger ATCC 1015 and Aspergillus uvarum CBS 121591  we would use the command:
 
@@ -358,31 +369,25 @@ For example, if we wanted to retrieve all CAZymes from Aspergillus niger ATCC 10
 
    cazy_webscraper --strains Aspergillus niger ATCC 1015,Aspergillus uvarum CBS 121591
 
-.. note::
-   The order that the strains are listed does **not** matter, and separate multiple species names with a single comma 
-   with **no** spaces.
-
-.. note::
-   Sometimes in CAZy only the species name is given and no specific strain identifer. To retrieve CAZymes from these 
-   species then you can list the species name and it will only retrieve CAZymes that are listed with the exact species 
-   and with no strain identifers. For example, listing 'Aspergillus niger' will only retrieve CAZymes with their source 
-   organism specifically listed as 'Aspergillus niger' and will not retrieve CAZymes from ''.
+he order that the strains are listed does **not** matter.
 
-.. warning::
+.. NOTE::
    If you use the ``--species``, ``--genera`` and ``--strains`` flags in any combination and a source organism matches 
-   multiple of the taxonomy critera, the CAZymes derived from that species will only be retrieved **once**. For example, 
-   using the command ``cazy_webscraper --genera Aspergillus --species Aspergillus niger --strains Aspergillus niger ATCC 1015`` 
-   will retrieve all CAZymes from *all* Aspergillus species *once*. The higher taxonomy levels take president, and the command 
-   will not retrieve all CAZymes from all Aspergillus species once AND all CAZymes from Aspergillus niger strains as well, and then 
-   retrieve another copy of all CAZymes from Aspergillus niger ATCC 1015.
+   multiple of the taxonomy critera, the CAZymes derived from that species will only be retrieved **once**.
+   
+   For example, using the command ``cazy_webscraper --genera Aspergillus --species Aspergillus niger --strains Aspergillus niger ATCC 1015`` 
+   will retrieve all CAZymes from *all* Aspergillus species *once*.
+   
+The higher taxonomy levels take president, and the command will not retrieve all CAZymes from all Aspergillus species once AND all CAZymes from Aspergillus niger strains as well, and then retrieve another copy of all CAZymes from Aspergillus niger ATCC 1015.
 
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Combining taxonomic filters
-=====================================
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-You can combine any combination of ``cazy_webscraper`` optional flags, including combining the taxonomic filters. For example,
-you may wish to retrieve all CAZyme derived from all viral and Aspergillus species, Layia carnosa, Layia chrysanthemoides, Trichoderma reesei QM6a and 
-Trichoderma reesei QM9414, we would combine the respective flags for a single ``cazy_webscraper`` command. The command 
+You can combine any combination of ``cazy_webscraper`` optional flags, including combining the taxonomic filtersFor example,
+you may wish to retrieve all CAZyme derived from all viral species, Aspergillus species, Layia carnosa, Layia chrysanthemoides, Trichoderma reesei QM6a and 
+Trichoderma reesei QM9414. To do this we would combine the respective flags for a single ``cazy_webscraper`` command. The command 
 we would use would be:
 
 .. code-block:: bash
@@ -391,65 +396,49 @@ we would use would be:
 
 .. note::
    This is a single command written on a single line. When typing the command into the terminal do not fit enter until you have finished the command. 
-   Visually the command may spread over multiple lines but it is a *single* command.
 
 .. warning::
    If you use the ``--species``, ``--genera`` and ``--strains`` flags in any combination and a source organism matches 
    multiple of the taxonomy critera, the CAZymes derived from that species will only be retrieved **once**. For example, 
    using the command ``cazy_webscraper --genera Aspergillus --species Aspergillus niger --strains Aspergillus niger ATCC 1015`` 
-   will retrieve all CAZymes from *all* Aspergillus species *once*. The higher taxonomy levels take president, and the command 
-   will not retrieve all CAZymes from all Aspergillus species once AND all CAZymes from Aspergillus niger strains as well, and then 
-   retrieve another copy of all CAZymes from Aspergillus niger ATCC 1015.
-
-
-EC numbers
-==============
-
-If you are interested in CAZymes with specific activities you can limit the retrieval of CAZymes from CAZy to only those 
-annotated with *at least one* EC number from a set of EC numbers you specify. To specify a set of EC numbers 
-add the ``--ec`` flag to the ``cazy_webscraper`` command, followed by a list of EC numbers. Separate each EC number with a single 
-comma and *no* spaces. Do **not** forget to include the 'EC' prefix from your EC numbers. 
-
-.. note::
-   Use the international accepted '-' (dash) to indicate missing identifiers (numbers) in the EC number.
-   EC1.2.3.- is accepted but EC1.2.3. and EC1.2.3.* are not.
-
-To limit the scraping of CAZy to only retrieve CAZymes that are annotated with *at least one* of the EC numbers 
-EC4.2.2.-, EC1.3.2.- and EC5.4.-.-, use the command:
+   will retrieve all CAZymes from *all* Aspergillus species *once*.
 
+When combining taxonomy filters, the higher taxonomy levels take president. For example, the :command:
+   
 .. code-block:: bash
+   cazy_webscraper --genera Aspergillus --species Aspergillus niger --strains Aspergillus niger ATCC 1015
 
-   cazy_webscraper --ec "EC4.2.2.-,EC1.3.2.-,EC5.4.-.-"
+will not retrieve all CAZymes from all Aspergillus species once AND all CAZymes from Aspergillus niger strains as well. 
+``cazy_webscraper`` will retrieval all CAZymes for all strains of *Aspergillus niger*.
 
-.. warning::
-   Some terminals may misinterpret ``EC1.2.-.-`` as trying to invoke the options ``.``, therefore, it is 
-   recommend practise to encase the entire EC number list in single or double quotation marks if any of the EC numbers 
-   include missing identifiers. ``"EC4.2.2.-,EC1.3.2.-,EC5.4.-.-"`` or ``'EC4.2.2.-,EC1.3.2.-,EC5.4.-.-'`` are recommended, 
-   ``EC4.2.2.-,EC1.3.2.-,EC5.4.-.-`` is not recommended, and ``"EC4.2.2.-,EC1.3.2.-,EC5.4.-.-'`` (mismatching double and single 
-   quotation marks) will raise errors.
+-----------------------------------------
+Enabling retrieving subfamily annotations
+-----------------------------------------
 
+By default ``cazy_webscraper`` only retrieves the CAZy family annotation for each protein, it does not 
+retrieve the CAZy subfamily annotation. For example, a CAZyme within the CAZy subfamily GH3_1, will be 
+stored in the local CAZyme database as only a GH3 CAZyme.
 
-Taxonomy and EC numbers
-=============================
-
-You can use any combination of ``cazy_webscraper`` optional flags to fully customise the scraping of CAZy. 
-For example, you may which to retrieve all CAZymes annotated with the EC number EC4.2.2.- which are only from bacterial 
-species. To do that you would add the ``--kingdoms`` and ``--ec`` flags:
+To retrieve the CAZy family **and** CAZy subfamily annotations, add the ``-subfamilies``/``-s`` flag, anywhere in the 
+``cazy_webscraper`` command. For example:
 
 .. code-block:: bash
+   cazy_webscraper --families GH3 --subfamilies
 
-   cazy_webscraper --ec "EC4.2.2.-" --kingdoms bacteria
-
-The order you add the optional flags **does not** matter, and you can specify multiple EC numbers, kingdoms, strains etc.
+This command will retrieve all CAZymes from GH3, and will retrieve the CAZy family **and** CAZy subfamily 
+annotations. For example, a CAZyme in CAZy subfamily GH3_1 will be stored in the local database under the 
+CAZy family GH3 and the CAZy subfamily GH3_1.
 
+------------------------------------------------------
+Combining CAZy class, CAZy family and taxonomy filters
+------------------------------------------------------
 
-Combining Taxonomy, EC numbers, CAZy classes and CAZy families filters
-*************************************************************************
+You can use any combination of the CAZy class, CAZy family and taxonomy filters to fully customise the scrape of 
+CAZy.
 
-The optional flags for ``cazy_webscraper`` can be used in any combination and any order. For example, 
-you can combine the EC number, taxonomy, CAZy class and CAZy family configurations. Below are some examples:
+Below are some examples:
 
-**Example 1**  
+**Example 1**
 To retrieve all CAZymes from all CBM families, GH1, GH2 and PL9, and that are derived from any Aspergillus species:
 
 .. code-block:: bash
@@ -457,18 +446,11 @@ To retrieve all CAZymes from all CBM families, GH1, GH2 and PL9, and that are de
    cazy_webscraper --classes CBM --families GH1,GH2,PL9 --genera Aspergillus
 
 **Example 2**  
-To retrieve all CAZymes from GH1, and GH2, if they are annotated with EC1.2.-.-, and are derived from any bacterial species:
+To retrieve all CAZymes from GH1, and GH2 that are derived from any bacterial species:
 
 .. code-block:: bash
 
-   cazy_webscraper --families GH1,GH2 --ec "EC1.2.-.-" --kingdoms bacteria 
-
-.. warning::
-   Some terminals may misinterpret ``EC1.2.-.-`` as trying to invoke the options ``.``, therefore, it is 
-   recommend practise to encase the entire EC number list in single or double quotation marks if any of the EC numbers 
-   include missing identifiers. ``"EC4.2.2.-,EC1.3.2.-,EC5.4.-.-"`` or ``'EC4.2.2.-,EC1.3.2.-,EC5.4.-.-'`` are recommended, 
-   ``EC4.2.2.-,EC1.3.2.-,EC5.4.-.-`` is not recommended, and ``"EC4.2.2.-,EC1.3.2.-,EC5.4.-.-'`` (mismatching double and single 
-   quotation marks) will raise errors.
+   cazy_webscraper --families GH1,GH2 --kingdoms bacteria 
 
 **Example 3**  
 To retrieve CAZymes from all viral species, and all Aspergillus niger strains which are catalogued within GH3_1 and GH3_2
@@ -477,23 +459,26 @@ To retrieve CAZymes from all viral species, and all Aspergillus niger strains wh
 
    cazy_webscraper --families GH3_1,GH3_2 --subfamilies --species Aspergillus niger --kingdoms Bacteria
 
-
+------------------
 Configuration file
-################################
+------------------
 
 Whenever ``cazy_webscraper`` is invoked and adds data to a database, the configuration of ``cazy_webscraper`` 
-(this is the kingdoms, genera, species, strains, EC numbers, CAZy classes and CAZy family filters which were applied) 
+(this is the kingdoms, genera, species, strains, CAZy classes and CAZy family filters which were applied) 
 and the data and time the scrape was initiated is logged in the database. However, for optimal reproduction of 
 how ``cazy_webscraper`` was used in your research, you can create shareable documentation that others can use to 
-invoke ``cazy_webscraper`` and apply the exact sample filters as yourself. This is achieved by creating a configuration file 
+reproduce your CAZy dateset. This is achieved by creating a configuration file 
 rather than configuring the performance of ``cazy_webscraper`` at the command line.
 
-
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Creating a configuration file
-****************************************
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+``cazy_webscraper`` uses the YAML file type for its configuraiton file; 
+if you are new to YAML files please find more detailed information on YAML files [here](https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html).
 
-An example and template configuration file is included in ``cazy_webscraper``, it can be found at ``scraper/scraper_config.yaml``. 
-This is a YAML file; if you are new to YAML files please find more detailed information on YAML files [here](https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html).
+A template and example configuration file for scrapping CAZy using ``cazy_webscraper`` can be found in 
+the repo, in the ``configuration_files`` directory.
 
 The configuration YAML **must** contain the following tags/headings (identical to how they are presented below):
 
@@ -507,34 +492,17 @@ The configuration YAML **must** contain the following tags/headings (identical t
 * genera
 * species
 * strains
-* kingoms
-* ECs
-
-
-Specifying CAZy classes to scrape
-====================================
-
-Under the **classes** heading list any classes to be scrapped. For classes listed under 'classes', 
-all proteins catalogued under that class will be retrieved, **unless** specific families have been 
-listed under the respective classes heading in the configuration file. Then scraping only the 
-specific families takes precident and the entire class is not scraped. _If you believe this should 
-be changed please raise an issue. It is invisioned that very few users would want to simultanious 
-scrape an entire class and also scrape only specific families from that same class._
-
-A ``cazy_dictionary.json`` has been created and packaged within the ``cazy_webscraper`` 
-(the specific location is ``./scraper/file_io/cazy_dictionary.json``, where '.' is the directory 
-where the webscraper is installed). This allows users to use a variety of synonoms for the CAZy 
-classes, for example both "GH" and "Glycoside-Hydrolases" are accepted as synonoms for 
-"Glycoside Hydrolases (GHs)". Additionally, the retrieval of CAZy classes from the configuration 
-file is **not** case sensitive, therefore, both "gh" and "GH" are excepted. The excepted class 
-synonoms have beeen written out in a json file to enale easy editing of this file if additional 
-accepted synonoms are to be added, of it a new CAZy class is defined then this class only needs 
-to be added to the json file, without needing to modify the entire webscraper. 
-
-If you having issues with the scraper retrieving the list of CAZy classes that are written under 
-'classes' in the configuration file, please check the dictionary first to see the full list of 
-accepted synonoms. If you are comfortable modifying json files then feel free to add your own 
-synonoms to the dictionary.
+* kingdoms
+
+.. NOTE::
+   The order of the tags/headings does not matter.
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Scraping specific CAZy classes
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Under the **classes** heading list any classes to be scrapped. For each CAZy class listed under 'classes', CAZymes 
+will be retrieved for every CAZy family within the CAZy class.
 
 Each class must be listed on a separate line, indented by 4 spaces, and the class name encapsulated 
 with single or double quotation marks. For example:
@@ -543,17 +511,22 @@ with single or double quotation marks. For example:
 
     classes:
         - "GH"
-        - "pl"
+        - "PL"
 
-Specifying CAZy families to scrape
-=========================================
+The same CAZy class name synonyms used for the command line are accepted for the configuration file.
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Scraping specific CAZy families
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Under the each of the class names listed in the configuration file, list the names of specific 
 **families** to be scraped from that class. The respective classes of the specificed families do 
 **not** need to be added to the 'classes' list.
 
 Write the true name of the family not only it's number, for example **GH1** is excepted by **1** is 
-not. Name families using the standard CAZy nomenclature, such as **"GT2"** and 
+not.
+
+Name families using the standard CAZy nomenclature, such as **"GT2"** and 
 **NOT "GlycosylTransferases_2"**. Additionally, use the standard CAZy notation for subfamilies 
 (**GH3_1**).
 
@@ -569,12 +542,16 @@ marks. For example:
     Glycoside Hydrolases (GHs):
         - "GH1"
         - "GH2"
+        - "GH3_1"
 
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 Example configuration file
-*************************************
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Below is an example of the content you may wish to put in a configuration file.
+Below is an example of the content you may wish to put in a configuration file. Using this file 
+will retrieve all CAZymes in CAZy class AA, CAZy families GH1, GH3 and PL9 that are either derived from 
+a bacterial or *Trichoderma* species.
 
 .. code-block:: yaml
 
@@ -595,9 +572,6 @@ Below is an example of the content you may wish to put in a configuration file.
    strains:
    kingdoms:
       - "Bacteria"
-   ECs:
-      - EC4.2.2.-
-      - EC5.4.-.-
 
 
 .. note::
@@ -628,15 +602,13 @@ allow you to add notes to your configuration file. For example:
    ECs:  # only CAZymes with at least one of these EC numbers will be scrapped
 
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 Using a configuration file
-************************************
+^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Once you have created a configuration file (we recommend modifying the template one provided with ``cazy_webscraper`` 
 you then need to invoke ``cazy_webscraper`` and tell it you are using a configuration file. To do this we add the 
-``--config`` flag to the ``cazy_webscraper`` command, followed by the path to the configuration file.
-
-.. note::
-   You can use the long form of the configuration file flag (``--config``) *or* the short hand (``-c``).
+``--config``/``-c`` flag to the ``cazy_webscraper`` command, followed by the path to the configuration file.
 
 The path we pass to ``cazy_webscraper`` is a *relative* path. This means ``cazy_webscraper`` will start in the directory 
 the terminal is currently pointed out, and follow the path from there. For example, if we used the command:
@@ -645,8 +617,8 @@ the terminal is currently pointed out, and follow the path from there. For examp
 
    cazy_webscraper -c scraper/scraper_config.yaml
 
-Then the computer will look for a directory called ``scraper`` in the directory the terminal is looking at, then within the 
-``scraper`` directory it will look for a yaml file called ``scraper_config.yaml``.
+Then the computer will look for a directory called ``scraper`` in the directory the terminal is looking at, then look within the 
+``scraper`` directory for a yaml file called ``scraper_config.yaml``.
 
 .. note::
    To check which directory ``cazy_webscraper`` is pointed at type ``pwd`` into the terminal and hit enter. This is the 
@@ -655,13 +627,14 @@ Then the computer will look for a directory called ``scraper`` in the directory
 .. warning::
    Your path must point directly to the YAML file. Don't forget the '.yaml' file extension!
 
-
+------------------------------------------
 Using a configuration and the command-line
-###############################################
+------------------------------------------
+
+You can configure ``cazy_webscraper`` using a combination of command line arguments and a configuration file. 
 
-If you so wished, you can use a configuration file *and* the command line to configure ``cazy_webscraper``. If you do this 
-``cazy_webscraper`` will **not** retrieve duplicates of the data. If a CAZyme matches at least one of the configuration data then 
-one copy of the CAZyme record will be added to the SQL database, and only one copy will be added to the database no matter how many of the 
+If a CAZyme matches at least one of the configuration data (whether if be from the terminal of the configuration file),  
+one copy of the CAZyme record will be added to the SQL database, and only **one copy**, no matter how many of the 
 configuration data the CAZyme meets.
 
 To use a configuration file and a the command-line to configure ``cazy_webscraper``, use the configuration file 
@@ -671,31 +644,26 @@ To use a configuration file and a the command-line to configure ``cazy_webscrape
    The order you invoke the optional flags **does not** matter.
 
 
+-------------------------------------------------------------------
 Additional operations to fine tune how ``cazy_webscraper`` operates
-#############################################################################
-
-
-Retrieving CAZy family and CAZy subfamily annotations
-********************************************************
-
-The default behaviour of ``cazy_webscraper`` retrieves only the CAZy family annotations of CAZymes, and 
-does **not** catalogue the child CAZy subfamily annotations as well. If you want to retrieve the CAZy subfamily 
-annotations then add the ``--subfamilies`` flag anywhere to the ``cazy_webscraper`` command. For example:
-
-.. code-block:: bash
-
-   cazy_webscraper --subfamilies
+-------------------------------------------------------------------
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Add the scraped data to an existing CAZyme database
-*********************************************************
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-You may wish to scrape CAZy in multiple stages, maybe your internet dropped out while scraping CAZy 
-and you don't want to start again, or maybe you scraped CAZy but forget missed out a species of interest. No matter 
+You may wish to scrape CAZy in multiple stages; maybe your internet dropped out while scraping CAZy 
+and you don't want to start again, or maybe you scraped CAZy but missed out a species of interest. No matter 
 the reason ``cazy_webscraper`` allows you to add more CAZyme data to an existing database previously created by 
 ``cazy_webscraper``.
 
 To do this add the database (``--database`` or ``-d``) flag to the ``cazy_webscraper`` command, followed by the path 
-to the SQL database you want to add your scraped CAZy data to.
+to the CAZyme database you want to add your scraped CAZy data to. For example, to add data to an existing 
+database in ``cazy/cazyme_db.db`` use the command:
+
+.. code-block:: bash
+
+   cazy_webscraper -- database cazy/cazyme_db.db
 
 .. note::
    Don't forget the .db file extension at the end of the path!
@@ -711,11 +679,34 @@ Then the computer will look for a directory called ``my_cazyme_databases`` in th
 ``my_cazyme_databases`` directory the computer will look for the file ``my_cazyme_database.db``.
 
 
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Scraping data from a previously downloaded CAZy txt file
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+CAZy provides access to data within its database via text files. ``cazy_webscraper`` downloads the CAZy 
+text file containing all data within the CAZy database, providing a database dump. This file is then written to the cache directory 
+(by default, called ``.cazy_webscraper_<date>_<time>``).
+
+For consistency in the dataset, you may wish to perform multiple scrapes of CAZyme data from the same CAZy text file. 
+This could be a CAZy text file you have downloaded from CAZy or a text file downloaded by ``cazy_webscrapper``.
+
+To direct ``cazy_webscraper`` to retrieve CAZyme data from a previously downloaded CAZy text file, using the 
+``--cazy_data`` flag, followed by the path to the text file. For example:
+
+.. code-block:: bash
+   cazy_webscraper --cazy_data cazy_db/cazy_data.txt
+
+.. WARNING::
+   ``--cazy_data`` must be pointed directly at the text file, **not** a zipped file containing the CAZy 
+   data text file.
+
+
+^^^^^^^^^^^^^^^^^^^^^^
 Writing out a log file
-******************************
+^^^^^^^^^^^^^^^^^^^^^^
 
 If you want to have a log file of all terminal output produced by ``cazy_webscraper`` then add the log 
-``--log`` flag (or the shorthand version ``-l``) anywhere to the ``cazy_webscraper`` command, followed by a 
+``--log``/``-l`` anywhere to the ``cazy_webscraper`` command, followed by a 
 path to write the log file to. This path is a *relative* path and must include target a log file specifically. 
 For example:
 
@@ -727,13 +718,13 @@ For example:
    The log file does not already have to exist for ``cazy_webscraper`` to write to it; however, all 
    directories included in the path must already exist.
 
-
+^^^^^^^^^^^^^^^
 Verbose logging
-*********************************
+^^^^^^^^^^^^^^^
 
-For more detailed logging (logging more detail and not only when warnings and errors are raised by 
-``cazy_webscraper``), add the verbose logging flag (``--verbose`` or ``-v``) anywhere to the ``cazy_webscraper`` 
-command. You need only add the verbose flag and nothing else, for example:
+For more detailed logging (which includes not only error and warning messages (the default) but also 
+configuration setup, number of proteins retrieved etc.), add the verbose logging flag (``--verbose`` or ``-v``) anywhere to the ``cazy_webscraper`` 
+command. For example:
 
 .. code-block:: bash
 
@@ -741,14 +732,16 @@ command. You need only add the verbose flag and nothing else, for example:
 
 The verbose flag can be used in combination with the log flag to write all terminal output to a log file.
 
-
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Changing the connection timeout limit
-*****************************************
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Sometimes the connection to the CAZy server times out. By default if a connection is attempted to made to CAZy 
 and no response is recieved within 45 seconds, then ``cazy_webscraper`` interprets this as the connection 
-timing out, waits 10 seconds and retries the connection.  You can change how long the computer waits for a 
-response from the CAZy server before classifying the connection as timed out by adding the timeout flag to the 
+timing out. ``cazy_webscraper`` then waits 10 seconds and retries the connection.
+
+You can change how long the computer waits for a 
+response from the CAZy server before classifying the connection as timed out by adding the ``--timeout`` flag to the 
 ``cazy_webscraper`` command, followed by the number of seconds you want the computer to wait for a response from CAZy 
 before classifying the connection as timing out.
 
@@ -763,32 +756,3 @@ The timeout flag can be used in combination with other flags, for example:
 .. code-block:: bash
 
    cazy_webscraper --subfamilies --genera Aspergillus -v --timeout 30
-
-You can use the long version ``--timeout`` or short version ``-t`` of the timeout flag.
-
-.. code-block:: bash
-
-   cazy_webscraper --subfamilies --genera Aspergillus -v -t 60
-
-
-Configuration when scraping subfamilies
-################################################
-
-The default behaviour of ``cazy_webscraper`` retrieves only the CAZy family annotations of CAZymes, and 
-does **not** catalogue the child CAZy subfamily annotations as well. If you want to retrieve the CAZy subfamily 
-annotations then add the ``--subfamilies`` flag anywhere to the ``cazy_webscraper`` command. For example:
-
-.. code-block:: bash
-
-   cazy_webscraper --subfamilies
-
-This will retrieve both the parent CAZy family annotations and the child CAZy subfamily annotations for all applicable CAZymes. 
-If a CAZyme is not part of a subfamily only its CAZy family annotations will be catagloued.
-
-If any subfamilies are listed within the configuration file, the retrieval of subfamilies **must** 
-be enabled at the command line uisng ``--subfamilies``.
-
-If the parent family, e.g GH3, is listed in the configuration file and ``--subfamilies`` is enabled, 
-all proteins catalogued under GH3 and its subfamilies will be retrieved. This is to save time 
-having to write out all the subfamilies for a given CAZy family. The scraper will remove any 
-duplicate proteins automatically.
diff --git a/docs/source/uniprot.rst b/docs/source/uniprot.rst
new file mode 100644
index 00000000..a3a9cdb1
--- /dev/null
+++ b/docs/source/uniprot.rst
@@ -0,0 +1,296 @@
+============================
+Retrieving data from UniProt
+============================
+
+``cazy_webscraper`` can be used to retrieve user-specified data sets from the UniProt database, for a given subset
+of proteins in a local CAZyme database created using ``cazy_webscraper``. The ``cazy_webscraper`` application can be invoked *via* the command line
+
+-----------
+Quick Start
+-----------
+
+To download UniProt protein accessions and names from UniProt for all protein in the local CAZyme database, and save the data to
+the local CAZyme database, use the following command structure:  
+
+.. code-block:: bash
+   cw_get_uniprot_data <path to local CAZyme db>
+
+.. NOTE::
+   The ``cw`` prefix on command is an abbreviation of ``cazy_webscraper``.
+
+
+--------------------
+Command line options
+--------------------
+
+``database`` - **REQUIRED** Path to a local CAZyme database to add UniProt data to.
+
+``bioservices_batch_size`` - Change the size of the batch query size submitted via `bioservices <https://bioservices.readthedocs.io/en/master/>`_ to UniProt
+to retrieve protein data. Default 150. ``bioservices`` recommends submitting    ueries no larger than 200 objects.
+
+``--cache_dir`` - Path to cache dir to be used instead of default cache dir path.
+
+``--cazy_synonyms`` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
+
+``--config``, ``-c`` - Path to a configuration YAML file. Default: None.
+
+``--classes`` - list of classes to retrieve UniProt data for.
+
+``--ec``, ``-e`` - Enable retrieval of EC number annotations from UniProt. Default, EC number annotations are **not** retrieved.
+
+``--ec_filter`` - List of EC numbers to limit the retrieval of protein data for proteins annotated with at least one of the given EC numbers **in the local CAZyme database**.
+
+``--families`` - List of CAZy (sub)families to retrieve UniProt protein data for.
+
+``--genera`` - List of genera to restrict the retrieval of protein to data from UniProt to proteins belonging to one of the given genera.
+
+``--log``, ``-l`` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
+
+``--nodelete_cache`` - When called, content in the existing cache dir will **not** be deleted. Default: False (existing content is deleted).
+
+``--nodelete_log`` - When called, content in the existing log dir will **not** be deleted. Default: False (existing content is deleted).
+
+``--pdb``, ``-p`` - Enable retrieval of PDB accessions. Default, PDB accessions not retrieved.
+
+``--retries``, ``-r`` - Define the number of times to retry making a connection to CAZy if the connection should fail. Default: 10.
+
+``--sequence``, ``-s`` - Enable retrieving protein amino acid sequences. Default, sequences are **not** retrieved.
+
+``--seq_update`` - If a newer version of the protein sequence is available, overwrite the existing sequence for the protein in the database. Default is false, the protein sequence is **not** overwritten and updated.
+
+``--sql_echo`` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
+
+``--species`` - List of species (organsim scientific names) to restrict the retrieval of protein to data from UniProt to proteins belonging to one of the given species.
+
+``--strains`` - List of species strains to restrict the retrieval of protein to data from UniProt to proteins belonging to one of the given strains.
+
+``--timeout``, ``-t`` - Connection timout limit (seconds). Default: 45.
+
+``--uniprot_batch_size`` - Size of an individual batch query submitted to the `UniProt REST API <https://www.uniprot.org/help/programmatic_access>_` to retrieve the UniProt accessions of proteins identified by the GenBank accession. Default is 150. The UniProt API documentation recommands batch sizes of less than 20,000 but batch sizes of 1,000 often result in HTTP 400 errors. It is recommend to keep batch sizes less than 1,000, and ideally less than 200.
+
+``--verbose``, ``-v`` - Enable verbose logging. This does **not** set the SQLite engine ``echo`` parameter to True. Default: False.
+
+-----------
+Basic Usage
+-----------
+
+The command-line options listed above can be used in combination to customise the retrieval of protein data from UniProt. Some options (e.g. ``--families`` and ``--classes``) define the broad group of proteins for which data will be retrieved from UniProt, others (e.g. ``--species``) are used to filter and fine-tune the protein dataset for which protein data will be retrieved.
+
+The ``--classes``, ``--families``, ``--kingdoms``, ``--genera``, ``--species``, and ``--strains`` filteres are applied 
+in the exactly same for retrieving data from CAZy as retrieving data from UniProt. Examples of using these flags 
+can be found in the ``cazy_webscraper`` tutorial in this documentation.
+
+Here we discuss using the new flags ``--ec``, ``--pdb``, ``--sequence``, ``--seq_update``, and ``--ec_filter``.
+
+.. NOTE::
+    To retrieve data for members of specific CAZy subfamilies, list the subfamilies after the ``--families`` 
+    flag.
+
+.. NOTE::
+    The command for retrieving protein data from UniProt for proteins in a local CAZyme database is ``cw_get_uniprot_data``.
+
+-----------------------------
+Data retrievable from UniProt
+-----------------------------
+
+By default ``cw_get_uniprot_data`` retrieves the UniProt protein accession and protein name from UniProt, for proteins in a 
+local CAZyme database. ``cw_get_uniprot_data`` can also retrieve from UniProt:
+
+* EC number annotations
+* PDB accessions
+* Protein amino acid sequences
+
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Retrieving EC number annotations
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To retrieve EC number annotations from UniProt add the ``--ec``/``-e`` flag to the command:
+
+.. code-block:: bash
+
+    cw_get_uniprot_data cazy_db.db --ec
+
+OR
+
+.. code-block:: bash
+
+    cw_get_uniprot_data cazy_db.db -e
+
+.. NOTE::
+    **All** EC number annotations are retrieved for all CAZymes matching the given filter criteria. In the example 
+    command above, no filters were provided therefore, all EC number annotations will be retrieved for all
+    CAZymes in the local CAZyme database (in this case called ``cazy_db.db``).
+
+
+^^^^^^^^^^^^^^^^^^^^^^^^^
+Retrieving PDB accessions
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To retrieve all PDB accessions for all CAZymes in the local CAZyme database matching the given filter criteria,
+add the ``--pdb``/``-p`` flag to the command:
+
+.. code-block:: bash
+
+    cw_get_uniprot_data cazy_db.db --pdb
+
+OR
+
+.. code-block:: bash
+
+    cw_get_uniprot_data cazy_db.db -p
+
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Retrieving protein sequences
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To retrieve all protein amino acid sequences for all CAZymes in the local CAZyme database matching the given filter criteria,
+add the ``--sequence``/``-s`` flag to the command:
+
+.. code-block:: bash
+
+    cw_get_uniprot_data cazy_db.db --sequence
+
+OR
+
+.. code-block:: bash
+
+    cw_get_uniprot_data cazy_db.db -s
+
+``cw_get_uniprot_data`` stores the protein amino acids sequence within the local CAZyme database, as well 
+as the 'last modified date' retrieved from UniProt.
+
+
+^^^^^^^^^^^^^^^^^^^^^^^^
+Updating local sequences
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+When using ``--sequence`` flag, ``cw_get_uniprot_data`` will only add *new* protein sequences to the database, i.e.
+it will only add protein sequences to records that do not have a sequence. Therefore, if a protein
+already has a sequence in the local database, this sequence is **not** overwritten.
+
+You may wish to update the protein sequences in your local CAZyme database. To do this use the ``--sequence``/``-s`` 
+flag to tell ``cw_get_uniprot_data`` to retrieve protein sequences, **and** use the ``--seq_update`` flag.
+
+.. code-block:: bash
+
+    cw_get_uniprot_data cazy_db.db -s --seq_update
+
+This instructs ``cw_get_uniprot_data`` to overwriting existing protein sequences in the local database *if* a newer version 
+of the sequence is retrieved from UniProt. This is checked by comparing the 'last modified date' of the 
+protein sequence in the local database against the sequence retrieved from UniProt.
+
+
+--------------------------
+Using the EC number filter
+--------------------------
+
+Having previously retrieved EC number annotations and added them to the local CAZyme database, you  may 
+wish to retrieve protein data for CAZymes annotated with specific EC numbers. To do this add the 
+``--ec_filter`` flag to the command, follwed by a list of EC numbers.
+
+.. NOTE::
+    Provide complete EC numbers. 
+    Both dashes ('-') and asterixes ('*') are accepted for missing digits in EC numbers.
+
+    EC1.2.3.- and EC1.2.3.* are accepted.
+    EC1.2.3. and EC 1.2.3 are **not** accepted.
+
+.. WARNING::
+    If using dashes to represent missing digits in EC numbers, it is recommended to bookend the entire 
+    EC number list in single or double quotation marks. Some terminals may misinterpret EC1.2.-.- as trying to invoke the options '.'
+
+.. NOTE::
+    ``cw_get_uniprot_data`` will retrieve the specified UniProt data for all proteins in the local CAZyme 
+    database that are annotated with **at least one** of the given EC numbers. Therefore, if multiple 
+    EC numbers are given this **does not mean** only CAZymes will all provided EC numbers will have data retrieved
+    from UniProt for them.
+
+``--ec_filter`` is based upon EC number annotations stored within the local CAZyme database. For 
+example, if protein A is annotated with the EC1.2.3.4, but this annotation is not stored in the 
+local CAZyme database, using ``--ec_filter EC1.2.3.4`` will **not** cause ``cw_get_uniprot_data`` to retrieve
+data for protein A. This is because ``cw_get_uniprot_data`` does not know protein A is annotated with 
+EC1.2.3.4, because this annotation is not within its database.
+
+.. WARNING::
+    If ``--ec_filter`` is used along side ``--ec``, ``cw_get_uniprot_data`` will retrieve **all** EC number 
+    annotations from UniProt for all proteins in the local CAZyme database that are associated with 
+    at least one of the EC numbers provided via ``--ec_filter`` within the CAZyme database.
+
+-------------------------------
+Configuration using a YAML file
+-------------------------------
+
+As with scraping CAZy, a YAML file can be provided to define the filters for retrieving data from UniProt. 
+The same YAML file can be used both for scraping CAZy and UniProt. However, the configuration file for
+retrieving data from UniProt can contain the additionl ``ec`` tag.
+
+Using a config file supports reproducible documentation of ``cazy_webscraper`` usage.
+
+An template YAML file is provided in the ``cazy_webscraper`` repository (``configuration_files/template-get_data_config.yaml``):
+
+.. code-block:: yaml
+
+    # Under 'classes' list class from which all proteins will retrieved
+    # Under each families respective name, list the specific families/subfamilies to be scraped
+    # Write the FULL family name, e.g. 'GH1', NOT only its number, e.g. '1'
+    # To list multiple families, each familiy must be on a new line starting indented once
+    # relative to the parent class name, and the name written within quotation marks.
+    # For more information on writing lists in Yaml please see:
+    # https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html 
+    classes:  # classes from which all proteins will be retrieved
+    - "GH"
+    - "CE"
+    Glycoside Hydrolases (GHs):
+    GlycosylTransferases (GTs):
+    Polysaccharide Lyases (PLs):
+    - "GT1"
+    - "GT5"
+    - "GT6"
+    Carbohydrate Esterases (CEs):
+    Auxiliary Activities (AAs):
+    Carbohydrate-Binding Modules (CBMs):
+    genera:  # list genera to be scraped
+    - "Trichoderma"
+    - "Aspergillus"
+    species:  # list species, this will scrape all strains under the species
+    - "Pythium ultimum"
+    strains:  # list specific strains to be scraped
+    kingdoms:  # Archaea, Bacteria, Eukaryota, Viruses, Unclassified
+    ec:
+    - "EC1.2.3.4"
+
+.. ATTENTION::
+  The YAML configuration file must contain all tags/headings indicated in the example configuration file found in the repository:
+
+  * classes
+  * Glycoside Hydrolases (GHs)
+  * GlycosylTransferases (GTs)
+  * Polysaccharide Lyases (PLs)
+  * Carbohydrate Esterases (CEs)
+  * Auxiliary Activities (AAs)
+  * Carbohydrate-Binding Modules (CBMs)
+  * genera
+  * species
+  * strains
+  * kingdoms
+  * ec
+
+Each value in the YAML mappings for these arguments must be listed on a separate line, indented by 4 spaces, and the class name encapsulated with single or double quotation marks. For example:
+
+.. code-block:: yaml
+
+    classes:
+        - "GT"
+        - "pl"
+    Glycoside Hydrolases (GHs):
+        - "GH1"
+        - "GH2"
+
+
+^^^^^^^^^^^^^^^^^^^^^^^^^
+Synonyms for CAZy classes
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A number of synonyms may be provided for CAZy classes, e.g. both "GH" and "Glycoside-Hydrolases" are accepted as synonyms for "Glycoside Hydrolases (GHs)" (the name recorded at CAZy). These alternatives are defined in the ``cazy_webscraper`` repository, in the file ``scraper/utilities/parse_configuration/cazy_dictionary.json``.
diff --git a/docs/source/uniprottutorial.rst b/docs/source/uniprottutorial.rst
new file mode 100644
index 00000000..16dd4f9d
--- /dev/null
+++ b/docs/source/uniprottutorial.rst
@@ -0,0 +1,256 @@
+==========================================================================
+Tutorials on configuring ``cazy_webscraper`` to retrieve data from UniProt
+==========================================================================
+
+``cazy_webscraper`` can be configured to retrieve user specified data sets from UniProt for specific date 
+sets of CAZymes in a local CAZyme database. Many of the configuration options 
+apply to the retrieval of protein data from CAZy, UniProt, GenBank and PDB.
+
+.. NOTE::
+   ``cazy_webscraper`` retrieves protein data from UniProt for CAZymes in a local CAZyme database. It 
+   will not add new proteins to the database.
+
+``cazy_webscraper`` can be configured via the **command line** and/or via a **YAML configuration file**.
+
+This page runs through examples of how to combine the various 'filters' that can be applied, to fully customised 
+the retrieval of data from UniProt. These tutorials are designed for those with less experience using command-line tools.
+
+.. NOTE::
+  If you installed ``cazy_webscraper`` using ``bioconda`` or ``pip`` to invoke ``cazy_webscraper`` to retrieve UniProt data call it using ``cw_get_uniprot_data`` - this is the method used in this tutorial.  
+  If you installed ``cazy_webscraper`` from source then you will need to invoke ``cazy_webscraper`` from the root of the repo using the command ``python3 cazy_webscraper/expand/uniprot/get_uniprot_data.py``.
+
+From this point on, we will be discussed the ``cw_get_uniprot_data``, which is the entry point for 
+retrieving data from UniProt. We also presume you are comfortable configuring ``cazy_webscraper`` for the 
+scraping of data from CAZy.
+
+All data retrieved from UniProt by ``cw_get_uniprot_data`` is added to the local CAZyme database.
+
+----------------------------------
+Configuration via the command line
+----------------------------------
+
+``cw_get_uniprot_data`` has only one required argument: the path to the local CAZyme database created 
+using ``cazy_webscraper``. Therefore, ``cw_get_uniprot_data`` can be enabled using a simple command structure:
+
+.. code-block:: bash
+
+  cazy_webscraper <path to the local CAZyme db>
+
+For example, if our database was stored in ``cazy/cazyme.db``, we would used:
+
+.. code-block:: bash
+   
+  cazy_webscraper cazy/cazyme.db
+
+When no optional arguments are provided, the default behaviour is invoked. The default behaviour is to:
+
+* Retrieve protein data for **all** CAZymes in the local CAZyme db
+* Retrieve UniProt protein accessions and protein names
+
+-----------------------------------------
+Options configurable at the command line 
+-----------------------------------------
+
+The following behaviours of the ``cw_get_uniprot_data`` can be configured at the command-line in the terminal:  
+
+* Limit the retrieve of UniProt protein data to CAZymes in the local databaes from specific CAZy classes, CAZy families, kingdoms, genuera, species, strains and/or EC numbers
+* Enable retrieving EC number annotations
+* Enable retrieving PDB accessions
+* Enable retrieving protein amino acid sequences
+* Enable updating protein sequences in the local CAZyme database if newer versions are retrieved from UniProt
+* Enable verbose logging during the operation of the webscraper
+
+`Here <https://cazy-webscraper.readthedocs.io/en/latest/configuration_scraper.html>`_ you can find a full list of the command-line flags and options.
+
+
+---------------------------------------------------------------
+Retrieving protein data for CAZy classes and families to scrape
+---------------------------------------------------------------
+
+The ``--classes`` and ``--families`` flags from scraping data from CAZy are applied in the extact same way 
+for retrieve data from UniProt.
+
+For instance, if instead of retrieving protein data for all CAZymes in your local CAZyme database, you want to 
+retrieve protein data for CAZymes in specific CAZy classes then add the 
+``--classes`` flag followed by the classes you want to retrieve protein data for.
+
+.. TIP::
+   To list multiple classes, separate the classes with a single comma. 
+
+For example, if you want to retrieve protein data for all CAZymes from Glycoside Hydrolase and Carbohydrate Esterases then use the command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --classes GH,CE
+
+OR
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --classes Glycoside Hydrolases,Carbohydrate Esterases
+
+Retrieving protein data for proteins from specific specific CAZy families is achieved using the ``--families`` flag. For 
+example, to retrieve protein data for all proteins in PL1, PL2 and PL3 in the local CAZyme database use the 
+following command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --families PL1,PL2,PL3
+
+.. WARNING::
+   ``cw_get_uniprot_data`` only accpets families written in the proper CAZy family syntax.
+   GH1 is accepted.
+   gh1 and GlycosideHydrolases1 are not accepted.
+
+As with scraping data from CAZy, the ``--classes`` and ``--families`` flags can be combined. To retrieve 
+protein data for all CAZymes in PL1, PL2, PL3 and *all* of GH and CE both:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --families PL1,PL2,PL3 --classes GH,CE
+
+**AND**
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --classes GH,CE --families PL1,PL2,PL3
+
+are accepted.
+
+
+------------------
+Applying taxonomic
+------------------
+
+The ``--kingdoms``, ``--genera``, ``--species`` and ``--strains`` flags can be used to refine the dataset 
+of proteins to retrieve protein data by taxonomy. These flags are applied in the exact same way as they 
+are used for the scraping of data from CAZy. Only proteins in the local CAZyme database and matching at least on of the provided taxonomy 
+criteria will have protein data retrieved from UniProt and added to the local CAZyme datbase.
+
+For example, if you want to retrieve protein data for all CAZymes in a local CAZyme database from bacterial and eukaryotic species then use the command 
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --kingdoms bacteria,eukaryota
+
+.. warning::
+   The kingdoms must be spelt the same way CAZy spells them, for example use 'eukaryot**a**' instead of 'eukaryot**e**'.
+   
+.. NOTE:: 
+   The kingdoms are **not** case sensitive, therefore, both ``bacteria`` *and* ``Bacteria`` are accepted. 
+
+.. NOTE::
+   You can list the kingdoms in *any* order. Thus, both ``bacteria,eukaryota`` *and* ``eukaryota,bacteria`` are accepted.
+
+You can combine any combination of the optional flags, including combining the taxonomic filters. For example,
+you may wish to retrieve protein data for all CAZymes in a local CAZyme database that are derived from all viral species, Aspergillus species, Layia carnosa, Layia chrysanthemoides, Trichoderma reesei QM6a and 
+Trichoderma reesei QM9414. To do this we would combine the respective flags for a single ``cw_get_uniprot_data`` command. The command 
+we would use would be:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --kingdoms viruses --genera Aspergillus --species Layia carnosa,Layia chrysanthemoides --strains Trichoderma reesei QM6a,Trichoderma reesei QM9414
+
+.. note::
+   The order that the flags are used and the order taxa  are listed does **not** matter, and separate multiple taxa names with a single comma 
+   with **no** spaces.
+
+.. warning::
+   Use the standard scientific name formating. Captialise the first letter of *genus* and write a lower 
+   case letter for the first letter of the species.
+
+   Aspergillus niger is **correct**
+
+   asepergillus niger is **incorrect**
+
+   ASPERGILLUS NIGER is **incorrect**
+
+.. warning::
+   When you specify a species ``cw_get_uniprot_data`` will retrieval CAZymes from *all* strains of the species.
+
+
+-------------------------
+Applying EC number filter
+-------------------------
+
+The retrieval of protein data from UniProt can also be limited to proteins in a local CAZyme database that are
+annotated with specific EC numbers.
+
+Having previously retrieved EC number annotations and added them to the local CAZyme database, you  may 
+wish to retrieve protein data for CAZymes annotated with specific EC numbers. To do this add the 
+``--ec_filter`` flag to the command, follwed by a list of EC numbers.
+
+.. code-block:: bash
+   
+   cw_get_uniprot_data cazy/cazyme.db --ec_filter "EC1.2.3.4,EC2.3.4.5"
+
+
+.. NOTE::
+    Provide complete EC numbers. 
+    Both dashes ('-') and asterixes ('*') are accepted for missing digits in EC numbers.
+
+    EC1.2.3.- and EC1.2.3.* are accepted.
+    EC1.2.3. and EC 1.2.3 are **not** accepted.
+
+.. NOTE::
+   The 'EC' prefix is not necessary.
+   EC1.2.3.4 and 1.2.3.4 are accepted.
+
+.. WARNING::
+    If using dashes to represent missing digits in EC numbers, it is recommended to bookend the entire 
+    EC number list in single or double quotation marks. Some terminals may misinterpret EC1.2.-.- as trying to invoke the options '.'
+
+.. NOTE::
+    ``cazy_webscraper`` will retrieve the specified UniProt data for all proteins in the local CAZyme 
+    database that are annotated with **at least one** of the given EC numbers. Therefore, if multiple 
+    EC numbers are given this **does not mean** only CAZymes will all provided EC numbers will have data retrieved
+    from UniProt for them.
+
+``--ec_filter`` is based upon EC number annotations stored within the local CAZyme database. For 
+example, if protein A is annotated with the EC1.2.3.4, but this annotation is not stored in the 
+local CAZyme database, using ``--ec_filter EC1.2.3.4`` will **not** cause ``cazy_webscraper`` to retrieve
+data for protein A. This is because ``cazy_webscraper`` does not know protein A is annotated with 
+EC1.2.3.4, because this annotation is not within its database.
+
+.. WARNING::
+    If ``--ec_filter`` is used along side ``--ec``, ``cazy_webscraper`` will retrieve **all** EC number 
+    annotations from UniProt for all proteins in the local CAZyme database that are associated with 
+    at least one of the EC numbers provided via ``--ec_filter`` within the CAZyme database.
+
+
+---------------------
+Combining all filters
+---------------------
+
+The ``--classes``, ``--families``, ``--ec_filter``, ``--kingdoms``, ``--genera``, ``--species`` and ``--strains`` flags can 
+be used in any combination to define a specific subset of proteins in the local CAZyme database for whom
+protein data from UniProt will be retrieved. These flags can be used with any combination of 
+``--ec``, ``--pdb``, ``--sequence``, ``--update_seq`` to customise what data is retrieved from UniProt and 
+added to the local CAZyme database.
+
+Below we run through 3 example commands of combining these flags, and the resulting behaviour.
+
+**Example 1:**
+To retrieve PDB accessions for all CAZymes in GH, GT, CE1, CE5 and CE8, and which are derived from baceterial species we use the command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --pdb --classes GH,CE --families CE1,CE5,CE8 --kingdoms bacteria
+
+
+**Example 2:**
+To retrieve EC numbers and PDB accessions for all CAZymes in GH and which are derived from *Aspegillus* and *Trichoderma* species we use the command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --pdb --classes GH --genera Aspegillus,Trichoderma
+
+
+**Example 3:**
+To retrieve EC numbers and sequences for all CAZymes in GH,CE and CBM which are derived from baceterial species and are annotated with at least one of 
+EC3.2.1.23, EC3.2.1.37 and EC3.2.1.85, we use the command:
+
+.. code-block:: bash
+
+   cw_get_uniprot_data cazy/cazyme.db --pdb --sequences --classes GH,CE,CBM --kingdoms bacteria --ec_filter "3.2.1.23,3.2.1.37,3.2.1.85"
+
diff --git a/docs/source/usage.rst b/docs/source/usage.rst
index 325105e3..817eb3c5 100644
--- a/docs/source/usage.rst
+++ b/docs/source/usage.rst
@@ -4,121 +4,87 @@ Using ``cazy_webscraper``
 
 ``cazy_webscraper`` can be used to retrieve user-specified data sets from the CAZy database. The ``cazy_webscraper`` application can be invoked *via* the command line
 
--------------
-Example Usage
--------------
+----------------------
+Quick Start
+----------------------
 
-To download the single CAZy family GH169, use the command:
+To download the entire CAZy dataset, and save the data set to the current working directory with the final name 
+``cazy_webscraper_<date>_<time>.db``, use the following command structure:  
 
 .. code-block:: bash
+   cazy_webscraper <user_email>
 
-  cazy_webscraper --families GH169 -o GH169
-
-This will create a new directory ``GH169`` in the current working directory, and will download all CAZy entries in the GH169 family to a new SQLite3 database in that directory.
-
-This page provides a brief summary of command-line options for ``cazy_webscraper`` that control the retrieval of data sets from the CAZy database, including:
-
-* Retrieve only specified CAZy classes and families or subfamilies
-* Retrieve only CAZymes from specified taxonomic kingdoms, genera, species, or strains
-* Recover only CAZymes with specified EC numbers
-* Local SQLite3 database path
-* Verbosity level and logging options
+.. NOTE::
+   The user email address is a requirement of NCBI. NCBI is queried to identify the currect source organism 
+   for a given protein, when multiple source organisms are retrieved from CAZy for a single protein.
 
 --------------------
 Command line options
 --------------------
 
-.. list-table:: Command line options
-   :header-rows: 1
-
-   * - Short option
-     - Long option
-     - Action
-     - Default
-   * - ``-c``
-     - ``--config``
-     - path to a YAML configuration file
-     - do not use YAML configuration file
-   * -
-     - ``--classes``
-     - define CAZy classes to be retrieved (comma-separated list for multiple classes)
-     - no specified value
-   * - ``-d``
-     - ``--database``
-     - path to SQLite3 database
-     - create a new database with default name
-   * - 
-     - ``--ec``
-     - define EC numbers to filter CAZyme data (comma-separated list for multiple values)
-     - no specified value
-   * - ``-f``
-     - ``--force``
-     - force overwriting of existing output
-     - do not force overwrite
-   * -
-     - ``--families``
-     - define CAZy families to be retrieved (comma-separated list for multiple families)
-     - no specified value
-   * -
-     - ``--genera``
-     - filter CAZyme data on taxonomic genus (comma-separated list for multiple values)
-     - do not filter on genus
-   * - 
-     - ``--get_pages``
-     - retrieve HTML from CAZy for specified CAZy families and write to disk
-     - do not retrieve HTML to disk
-   * - ``-h``
-     - ``--help``
-     - display command line options
-     -  
-   * - 
-     - ``--kingdoms``
-     - filter CAZyme data on taxonomic kingdom (comma-separated list for multiple values)
-     - do not filter on kingdom
-   * - ``-l``
-     - ``--log``
-     - path to log file
-     - do not write log file
-   * - ``-n``
-     - ``--nodelete``
-     - do not delete ("clobber") existing output when overwriting
-     - delete existing output when overwriting
-   * - ``-o``
-     - ``--output`` 
-     - path to output database
-     - write to STDOUT
-   * - ``-r``
-     - ``--retries``
-     - number of times to retry CAZy web requests.
-     - 10
-   * -
-     - ``--scrape_files``
-     - path to local CAZy HTML files; data will be scraped from these files instead of CAZy website
-     - do not use local CAZy HTML files
-   * - ``-s``
-     - ``--subfamilies``
-     - define CAZy subfamilies to be retrieved (comma-separated list for multiple families)
-     - no specified value
-   * - 
-     - ``--species``
-     - filter CAZyme data on taxonomic species (comma-separated list for multiple values)
-     - do not filter on species
-   * - 
-     - ``--strains``
-     - filter CAZyme data on strain (comma-separated list for multiple values)
-     - do not filter on strain
-   * - 
-     - ``--streamline``
-     - override CAZy metadata for each recovered record
-     - do not override CAZy metadata
-   * - ``-t``
-     - ``--timeout``
-     - wait time before CAZy web connection is considered timed out
-     - 45
-   * - ``-v``
-     - ``--verbose``
-     - enable verbose logging and output
-     - standard logging/output level
+
+``email`` - **REQUIRED** User email address. This is required by NCBI Entrez for querying the Entrez server.
+
+``--cache_dir`` - Path to cache dir to be used instead of default cache dir path.
+
+``--cazy_data`` - Path to a txt file downloaded from CAZy containing a CAZy database dump
+
+``--cazy_synonyms`` - Path to a JSON file containing accepted CAZy class synonsyms if the default are not sufficient.
+
+``--config``, ``-c`` - Path to a configuration YAML file. Default: None.
+
+``--citation``, ``-C`` - Print the `cazy_webscraper` citation. When called, the program terminates after printng the citation and CAZy is **not** scraped.
+
+``--classes`` - list of classes from which all families are to be scrape.
+
+``--database``, ``-d`` - Path to an **existings** local CAZyme database to add newly scraped too. Default: None.
+
+``--db_output``, ``-o`` - Path to write out a **new** local CAZyme database to. Include the name of the new database, including the `.db` extension. Default: None.
+
+.. WARNING::
+  **Do not use ``--db_output`` and ``--database`` at the same time.**
+
+.. NOTE::
+  If ``--db_output`` **and** ``--database`` are **not** called,
+  ``cazy_webscraper`` will write out a local CAZyme database to the cwd with the standardised name ``cazy_webscraper_<date>_<time>.db``
+
+``--delete_old_relationships`` - Detele old CAZy family annotations of GenBank accessions. These are CAZy family annotations of a given GenBank accession are in the local database but the accession is not longer associated with those CAZy families, so delete old accession-family relationships.
+
+``--families`` - List of CAZy (sub)families to scrape.
+
+``--force``, ``-f`` - force overwriting existing output file. Default: False.
+
+.. WARNING::
+  If a specified output directory already exists, if ``--force`` is not called, ``cazy_webscraper`` 
+  will not overwrite the output and terminate.
+
+``--genera`` - List of genera to restrict the scrape to. Default: None, filter not applied to scrape.
+
+``--log``, ``-l`` - Target path to write out a log file. If not called, no log file is written. Default: None (no log file is written out).
+
+``--nodelete``, ``-n`` - When called, content in the existing output dir will **not** be deleted. Default: False (existing content is deleted).
+
+``--nodelete_cache`` - When called, content in the existing cache dir will **not** be deleted. Default: False (existing content is deleted).
+
+``--nodelete_log`` - When called, content in the existing log dir will **not** be deleted. Default: False (existing content is deleted).
+
+``--retries``, ``-r`` - Define the number of times to retry making a connection to CAZy if the connection should fail. Default: 10.
+
+``--sql_echo`` - Set SQLite engine echo parameter to True, causing SQLite to print log messages. Default: False.
+
+``--subfamilies``, ``-s`` - Enable retrival of CAZy subfamilies, otherwise **only** CAZy family annotations will be retrieved. Default: False.
+
+``--species`` - List of species written as Genus Species) to restrict the scraping of CAZymes to. CAZymes will be retrieved for **all** strains of each given species.
+
+``--strains`` - List of specific species strains to restrict the scraping of CAZymes to.
+
+``--timeout``, ``-t`` - Connection timout limit (seconds). Default: 45.
+
+``--validate``, - Retrieve CAZy family population sizes from the CAZy website and check against the number of family members added to the local CAZyme database, as a method for validating the complete retrieval of CAZy data.
+
+``--verbose``, ``-v`` - Enable verbose logging. This does **not** set the SQLite engine ``echo`` parameter to True. Default: False.
+
+``--version``, ``-V`` - Print ``cazy_webscraper`` version number. When called and the version number is printed, ``cazy_webscraper`` is immediately terminated.
 
 -----------
 Basic Usage
@@ -134,68 +100,98 @@ The 'definition' arguments (e.g. ``--classes`` and ``--families``) indicate whic
 
 .. code-block:: bash
 
-  cazy_webscraper --families GH169 -o GH169
-  cazy_webscraper --classes AA -o AA
+  cazy_webscraper --families GH169 -o GH169.db
+  cazy_webscraper --classes AA -o AA.db
 
 will download all CAZymes from the GH169 family, and the AA class, respectively. More than one class or family can be specified, e.g.
 
 .. code-block:: bash
 
-  cazy_webscraper --families GH169,GH1,GH2,GH3 -o GH_families
-  cazy_webscraper --classes AA,CBM -o other_classes
+  cazy_webscraper --families GH169,GH1,GH2,GH3 -o GH_families.db
+  cazy_webscraper --classes AA,CBM -o other_classes.db
 
 and members of distinct families and classes can be selected simultaneously, e.g.
 
 .. code-block:: bash
 
-  cazy_webscraper --families GH169,GH1,GH2,GH3 --classes AA,CBM -o complex_query
+  cazy_webscraper --families GH169,GH1,GH2,GH3 --classes AA,CBM -o complex_query.db
 
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Filtering CAZy families and classes
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. NOTE::
+  CAZy families should be named using the standard CAZy syntax.
+  GH1 is **accepted**.
+  gh1 and Glycoside hydrolase 1 are **note** accepted.
+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Specifying output data location
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Options that apply a *filter* to restrict which CAZymes from a class or familiy are scraped from CAZy (e.g. ``--species`` and ``--ec``) may be applied in combination. For example:
+By default ``cazy_webscraper`` writes out a SQL database file to the current working directory, with a 
+filename with the following structure ``cazy_webscraper_<date>_<time>.db``, where the date and time mark 
+the time ``cazy_webscraper`` was called.
+
+To specify the location of the output database the ``--db_output``/``-o`` option can be used:
 
 .. code-block:: bash
 
-  cazy_webscraper --families GH169 \
-      --ec 1.1.1.1 --species "Escherichia coli" \
-      -o GH169_ec1.1.1.1_speciesEscherichia_coli
+  cazy_webscraper --families GH169 -o GH169_output.db
 
-will download only the CAZymes in the GH169 family that have EC number 1.1.1.1 *and* are from the species *Escherichia coli*. The command:
+will write an SQL database file to ``GH169_output.db``.
+
+If the target output file already exists, ``cazy_webscraper`` by default will not overwrite the existing file and will terminate. To 
+overwrite an existing file use the ``--force``/``-f`` options:
 
 .. code-block:: bash
 
-  cazy_webscraper --families PL14 \
-      --ec 1.2.3.4 --kingdoms bacteria \
-      -o PL14_ec1.2.3.4_kingdomBacteria
+  cazy_webscraper --families GH169 -o GH169_output.db -f
 
-will download only CAZymes in the PL14 familiy that have EC number 1.2.3.4 *and* are from the kingdom *Bacteria*.
+A multi-layered path can be provided to ``cazy_webscraper``. If any of the parent directories for the target 
+output path do not exist, ``cazy_webscraper`` will build the necessary output direcotires. In the following command if 
+the ``cazy`` and ``families`` directories do not exist, ``cazy_webscraper`` will build these directories:
 
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Specifying output data location
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. code-block:: bash
+
+  cazy_webscraper --families GH169 -o cazy/families/GH169_output.db 
+
+If any of the output directories exist, by default, ``cazy_webscraper`` will terminate. To write to an existing output 
+directory use the ``--force``/``-f`` options:
+
+.. code-block:: bash
+
+  cazy_webscraper --families GH169 -o GH169_output.db -f
+
+By default ``cazy_webscraper`` will delete the existing content in the existing output files. To not delete the content 
+in the existing output directories use the ``--nodelete``/``-n``:
 
-To specify the location of the output database and log files, the ``--output``/``-o`` option can be used:
+.. code-block:: bash
+
+  cazy_webscraper --families GH169 -o GH169_output.db -f -n
+
+If you already have an existing CAZy database, then specifying this database with the ``-d``/``--database`` option will cause the scraper to use the existing database rather than creating a new one:
 
 .. code-block:: bash
 
-  cazy_webscraper --families GH169 -o GH169_output
+  cazy_webscraper --families GH169 -d GH169/GH169_output.db
 
-will write output to the directory ``GH169_output``, and create a new CAZy database in that directory.
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Filtering CAZy families and classes
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-If you already have an existing CAZy output directory with a database, then specifying this database with the ``-d``/``--database`` option will cause the scraper to use the existing database rather than creating a new one:
+Options that apply a *filter* to restrict which CAZymes from a class or familiy are scraped from CAZy (e.g.  ``--families`` and ``--species``) may be applied in combination. For example:
 
 .. code-block:: bash
 
-  cazy_webscraper --families GH169 -d GH169_output/cazy.db
+  cazy_webscraper --families GH169 \
+      --species "Escherichia coli" \
+      -o GH169_speciesEscherichia_coli.db
 
-To write output to an existing directory without deleting the content already present, use the ``--force``/``-f`` and ``--nodelete``/``-n`` options:
+will download only the CAZymes in the GH169 family that are from the species *Escherichia coli*. The command:
 
 .. code-block:: bash
 
-  cazy_webscraper --families GH169 -d GH169_output -f -n
+  cazy_webscraper --families PL14,PL15,PL16 \
+      -o PL14_ec1.2.3.4_kingdomBacteria
 
+will download only CAZymes in the PL14, PL15 and PL16 families that are from the kingdom *Bacteria*.
 
 .. NOTE::
   ``cazy_webscraper`` input options can also be specified in a **YAML configuration file**, to enable transparency and reproducibility.
diff --git a/requirements.txt b/requirements.txt
index 1613ff8a..8abb2223 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6,5 +6,6 @@ mechanicalsoup
 numpy
 pandas
 requests
+saintBioutils
 sqlalchemy>=1.4.20
 tqdm
diff --git a/scraper/expand/get_genbank_sequences.py b/scraper/expand/get_genbank_sequences.py
deleted file mode 100644
index 344e9043..00000000
--- a/scraper/expand/get_genbank_sequences.py
+++ /dev/null
@@ -1,874 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-# (c) University of St Andrews 2020-2021
-# (c) University of Strathclyde 2020-2021
-# Author:
-# Emma E. M. Hobbs
-
-# Contact
-# eemh1@st-andrews.ac.uk
-
-# Emma E. M. Hobbs,
-# Biomolecular Sciences Building,
-# University of St Andrews,
-# North Haugh Campus,
-# St Andrews,
-# KY16 9ST
-# Scotland,
-# UK
-
-# The MIT License
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-
-# The above copyright notice and this permission notice shall be included in all
-# copies or substantial portions of the Software.
-
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-"""Retrieve proteins sequences from GenBank and populate the local database and write to FASTA"""
-
-
-import logging
-import os
-import re
-import sys
-import time
-
-import pandas as pd
-
-from datetime import datetime
-from typing import List, Optional
-
-from Bio import Entrez, SeqIO
-from tqdm import tqdm
-
-from scraper.sql.sql_orm import (
-    Cazyme,
-    CazyFamily,
-    Cazymes_Genbanks,
-    Genbank,
-    Kingdom,
-    Taxonomy,
-    get_db_session,
-)
-from scraper.utilities import config_logger, file_io, parse_configuration
-from scraper.utilities.parsers import build_genbank_sequences_parser
-
-
-def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = None):
-    """Set up programme and initate run."""
-    start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
-    start_time = pd.to_datetime(start_time)
-    date_today = datetime.now().strftime("%Y/%m/%d")  # used as seq_update_date in the db
-
-    # parse cmd-line arguments
-    if argv is None:
-        parser = build_genbank_sequences_parser()
-        args = parser.parse_args()
-    else:
-        args = build_genbank_sequences_parser(argv).parse_args()
-
-    if logger is None:
-        logger = logging.getLogger(__name__)
-        config_logger(args)
-
-    # check database was passed
-    if os.path.isfile(args.database) is False:
-        logger.error(
-            "Could not find local CAZy database. Check path is correct. Terminating programme."
-        )
-        sys.exit(1)
-
-    Entrez.email = args.email
-
-    # create session to local database
-    session = get_db_session(args)
-
-    # retrieve configuration data
-    file_io_path = file_io.__file__
-    config_dict, taxonomy_filters, kingdoms = parse_configuration.get_configuration(
-        args,
-    )
-
-    if config_dict is None:
-        if args.update:
-            # get sequence for everything without a sequence and those with newer remote sequence
-            add_and_update_all_sequences(date_today, taxonomy_filters, kingdoms, session, args)
-
-        else:
-            # get sequences for everything without a sequence
-            get_missing_sequences_for_everything(
-                date_today,
-                taxonomy_filters,
-                kingdoms,
-                session,
-                args,
-            )
-
-    else:
-        # get sequences for only specified classes/families
-        if args.update:
-            update_sequences_for_specific_records(
-                date_today,
-                config_dict,
-                taxonomy_filters,
-                kingdoms,
-                session,
-                args,
-            )
-        else:
-            get_missing_sequences_for_specific_records(
-                date_today,
-                config_dict,
-                taxonomy_filters,
-                kingdoms,
-                session,
-                args,
-            )
-
-    if args.blastdb is not None:
-        file_io.build_blast_db(args)
-
-    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
-    end_time = pd.to_datetime(start_time)
-    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
-    end_time = pd.to_datetime(end_time)
-    total_time = end_time - start_time
-
-    logger.info(
-        "Finished populating local CAZy database with GenBank protein sequences. "
-        "Terminating program.\n"
-        f"Scrape initated at {start_time}\n"
-        f"Scrape finished at {end_time}\n"
-        f"Total run time: {total_time}"
-    )
-
-    print(
-        "=====================cazy_webscraper-expand-genank_sequences=====================\n"
-        "Finished populating local CAZy database with GenBank protein sequences. "
-        "Terminating program.\n"
-        f"Scrape initated at {start_time}\n"
-        f"Scrape finished at {end_time}\n"
-        f"Total run time: {total_time}\n"
-    )
-
-
-# The folowing functions are for querying the local database to get GenBank accessions
-
-
-def get_missing_sequences_for_everything(date_today, taxonomy_filters, kingdoms, session, args):
-    """Retrieve protein sequences for all CAZymes in the local CAZy database that don't have seq.
-
-    :param date_today: str, today's date, used for logging the date the seq is retrieved in the db
-    :param taxonomy_filters: set of genera, species and strains to restrict sequence retrieval
-    :param kingdoms: set of taxonomy Kingdoms to retrieve sequences for
-    :param session: open SQLite db session
-    :param args: cmd-line argument parser
-
-    Return nothing.
-    """
-    logger = logging.getLogger(__name__)
-
-    # retrieve only sequences for primary GenBank accessions, and those without sequences
-    if args.primary is True:
-        logger.warning(
-            "Retrieving sequences for all primary GenBank accessions that do not have sequences"
-        )
-        genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-            join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-            join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-            join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-            join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-            filter(Cazymes_Genbanks.primary == True).\
-            filter(Genbank.sequence == None).\
-            all()
-
-    # retrieve sequences for all GenBank accessions without sequences
-    else:
-        logger.warning(
-            "Retrieving sequences for all GenBank accessions that do not have sequences"
-        )
-        genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-            join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-            join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-            join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-            join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-            filter(Genbank.sequence == None).\
-            all()
-
-    # retrieve the genbank_accessions
-    accessions = extract_accessions(genbank_query, taxonomy_filters)
-
-    if len(accessions) == 0:
-        logger.warning(
-            "Did not retrieve any GenBank accessions from the local database\n"
-            "that have sequences missing. Not adding sequences to the local database."
-        )
-        return
-
-    # separate accesions in to separate lists of length args.epost, epost doesn't like more than 200
-    accessions = get_accession_chunks(accessions, args.epost)  # args.epost = number per chunk
-    for lst in accessions:
-        get_sequences_add_to_db(lst, date_today, session, args)
-    return
-
-
-def add_and_update_all_sequences(date_today, taxonomy_filters, kingdoms, session, args):
-    """Retrieve sequences for all proteins in the database.
-
-    For records with no sequences, add the retrieved sequence.
-    For records with a sequence, check if the remove sequence is more recent than the existing
-    sequence. It it is, update the local sequence.
-
-    :param date_today: str, today's date, used for logging the date the seq is retrieved in the db
-    :param taxonomy_filters: set of genera, species and strains to retrieve sequences for
-    :param kingdoms: set of taxonomy Kingdoms to retrieve sequences for
-    :param session: open SQLite db session
-    :param args: cmd-line argument parser
-
-    Return nothing.
-    """
-    logger = logging.getLogger(__name__)
-
-    # retrieve only sequences for primary GenBank accessions, and those without sequences
-    if args.primary is True:
-        logger.warning(
-            "Retrieving sequences for all primary GenBank accessions that do not have sequences\n"
-            "and those whose sequences have been updated in NCBI "
-            "since they were retrieved previously"
-        )
-        genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-            join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-            join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-            join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-            join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-            filter(Cazymes_Genbanks.primary == True).\
-            all()
-
-    # retrieve sequences for all GenBank accessions
-    else:
-        logger.warning(
-            "Retrieving sequences for all GenBank accessions that do not have sequences\n"
-            "and those whose sequences have been updated in NCBI "
-            "since they were retrieved previously"
-        )
-        genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-            join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-            join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-            join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-            join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-            all()
-
-    # create dictionary of {genbank_accession: 'sequence update date' (str)}
-    accessions = extract_accessions_and_dates(genbank_query, taxonomy_filters)
-
-    if len(accessions.keys()) == 0:
-        logger.warning(
-            "Did not retrieve any GenBank accessions from the local database.\n"
-            "Not adding sequences to the local database."
-        )
-        return
-
-    accessions = get_accessions_for_new_sequences(accessions)  # list of genkbank_accession
-
-    if len(accessions) == 0:
-        logger.warning(
-            "Did not retrieve any GenBank accessions whose sequences need updating.\n"
-            "Not adding sequences to the local database."
-        )
-        return
-
-    # separate accesions in to separate lists of length args.epost, epost doesn't like more than 200
-    accessions = get_accession_chunks(accessions, args.epost)  # args.epost = number per chunk
-    for lst in accessions:
-        get_sequences_add_to_db(lst, date_today, session, args)
-    return
-
-
-def get_missing_sequences_for_specific_records(
-    date_today,
-    config_dict,
-    taxonomy_filters,
-    kingdoms,
-    session,
-    args,
-):
-    """Coordinate getting the sequences for specific CAZymes, not with seqs in the db.
-
-    :param date_today: str, today's date, used for logging the date the seq is retrieved in the db
-    :param config_dict: dict, defines CAZy classes and families to get sequences for
-    :param taxonomy_filters: set of genera, species and strains to restrict sequence retrieval
-    :param kingdoms: set of taxonomy Kingdoms to retrieve sequences for
-    :param session: open SQL database session
-    :param args: cmd-line args parser
-
-    Return nothing.
-    """
-    logger = logging.getLogger(__name__)
-    logger.warning(
-        "Retrieving sequences for GenBank accessions that do not have a sequence in the database"
-    )
-
-    # start with the classes
-    if len(config_dict["classes"]) != 0:
-        # retrieve list of CAZy classes to get sequences for
-        cazy_classes = config_dict["classes"]
-
-        for cazy_class in tqdm(cazy_classes, desc="Parsing CAZy classes"):
-            # retrieve class name abbreviation
-            cazy_class = cazy_class[((cazy_class.find("(")) + 1):((cazy_class.find(")")) - 1)]
-
-            # get the CAZymes within the CAZy class
-            class_subquery = session.query(Cazyme.cazyme_id).\
-                join(CazyFamily, Cazyme.families).\
-                filter(CazyFamily.family.regexp(rf"{cazy_class}\d+")).\
-                subquery()
-
-            # retrieve the GenBank accessions of the CAZymes in the CAZy class without seqs
-            if args.primary:
-                genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                    join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                    join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-                    join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-                    filter(Cazyme.cazyme_id.in_(class_subquery)).\
-                    filter(Cazymes_Genbanks.primary == True).\
-                    filter(Genbank.sequence == None).\
-                    all()
-            else:
-                genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                    join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                    join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-                    join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-                    filter(Cazyme.cazyme_id.in_(class_subquery)).\
-                    filter(Genbank.sequence == None).\
-                    all()
-
-            # retrieve the genbank_accessions from the sql collection object returned by the query
-            accessions = extract_accessions(genbank_query, taxonomy_filters)
-
-            if len(accessions) == 0:
-                logger.warning(
-                    f"Did not retrieve any GenBank accessions for the CAZy class {cazy_class}\n"
-                    "that have missing sequences. Not adding sequences to the local database."
-                )
-                continue
-
-            # separate accesions in to separate lists of length args.epost
-            # epost doesn't like posting more than 200 at once
-            accessions = get_accession_chunks(accessions, args.epost)  # args.epost = number/chunk
-            for lst in accessions:
-                get_sequences_add_to_db(lst, date_today, session, args)
-            continue
-
-    # Retrieve protein sequences for specified families
-    for key in config_dict:
-        if key == "classes":
-            continue
-        if config_dict[key] is None:
-            continue  # no families to parse
-
-        for family in tqdm(config_dict[key], desc=f"Parsing families in {key}"):
-            if family.find("_") != -1:  # subfamily
-                # Retrieve GenBank accessions catalogued under the subfamily
-                family_subquery = session.query(Cazyme.cazyme_id).\
-                    join(CazyFamily, Cazyme.families).\
-                    filter(CazyFamily.subfamily == family).\
-                    subquery()
-
-            else:  # family
-                # Retrieve GenBank accessions catalogued under the family
-                family_subquery = session.query(Cazyme.cazyme_id).\
-                    join(CazyFamily, Cazyme.families).\
-                    filter(CazyFamily.family == family).\
-                    subquery()
-
-            # get the GenBank accessions of thes CAZymes, without sequences
-            if args.primary:
-                genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                    join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                    join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-                    join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-                    filter(Cazyme.cazyme_id.in_(family_subquery)).\
-                    filter(Cazymes_Genbanks.primary == True).\
-                    filter(Genbank.sequence == None).\
-                    all()
-            else:
-                genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                    join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                    join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-                    join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-                    filter(Cazyme.cazyme_id.in_(family_subquery)).\
-                    filter(Genbank.sequence == None).\
-                    all()
-
-            # retrieve a list of GenBank accessions from the sql collection returned from the query
-            accessions = extract_accessions(genbank_query, taxonomy_filters)
-
-            if len(accessions) == 0:
-                logger.warning(
-                    f"Did not retrieve any GenBank accessions for the CAZy class {family}\n"
-                    "that have missing sequences. Not adding sequences to the local database."
-                )
-                continue
-            # separate accesions in to separate lists of length args.epost
-            # epost doesn't like posting more than 200 at once
-            accessions = get_accession_chunks(accessions, args.epost)  # args.epost = acc/chunk
-            for lst in accessions:
-                get_sequences_add_to_db(lst, date_today, session, args)
-
-    return
-
-
-def update_sequences_for_specific_records(
-    date_today,
-    config_dict,
-    taxonomy_filters,
-    kingdoms,
-    session,
-    args,
-):
-    """Coordinate getting the sequences for specific CAZymes, not with seqs in the db nad those
-    whose seq in NCBI has been updated since the last retrieval.
-
-    For records with no sequences, add the retrieved sequence.
-    For records with a sequence, check if the remove sequence is more recent than the existing
-    sequence. It it is, update the local sequence.
-
-    :param date_today: str, today's date, used for logging the date the seq is retrieved in the db
-    :param config_dict: dict, defines CAZy classes and families to get sequences for
-    :param taxonomy_filters: set of genera, species and strains to restrict sequence retrieval
-    :param kingdoms: set of taxonomy Kingdoms to retrieve sequences for
-    :param session: open SQL database session
-    :param args: cmd-line args parser
-
-    Return nothing.
-    """
-    logger = logging.getLogger(__name__)
-    logger.warning(
-        "Retrieving sequences for GenBank accessions that do not have a sequence in the database,\n"
-        "and those whose sequence in NCBI has been updated since they were previously retrieved."
-    )
-
-    # start with the classes
-    if len(config_dict["classes"]) != 0:
-        # retrieve list of CAZy classes to get sequences for
-        cazy_classes = config_dict["classes"]
-
-        for cazy_class in tqdm(cazy_classes, desc="Parsing CAZy classes"):
-            # retrieve class name abbreviation
-            cazy_class = cazy_class[((cazy_class.find("(")) + 1):((cazy_class.find(")")) - 1)]
-
-            # get the CAZymes within the CAZy class
-            class_subquery = session.query(Cazyme.cazyme_id).\
-                join(CazyFamily, Cazyme.families).\
-                filter(CazyFamily.family.regexp(rf"{cazy_class}\d+")).\
-                subquery()
-
-            # retrieve the GenBank accessions of the CAZymes in the CAZy class without seqs
-            if args.primary:
-                genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                    join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                    join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-                    join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-                    filter(Cazyme.cazyme_id.in_(class_subquery)).\
-                    filter(Cazymes_Genbanks.primary == True).\
-                    all()
-            else:
-                genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                    join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                    join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-                    join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-                    filter(Cazyme.cazyme_id.in_(class_subquery)).\
-                    all()
-
-            # create dictionary of genbank_accession: 'sequence update date' (str)
-            accessions = extract_accessions_and_dates(genbank_query, taxonomy_filters)
-
-            if len(accessions.keys()) == 0:
-                logger.warning(
-                    f"Did not retrieve any GenBank accessions for the CAZy class {cazy_class}.\n"
-                    "Not adding sequences to the local database."
-                )
-                continue
-
-            accessions = get_accessions_for_new_sequences(accessions)  # list of genkbank_accession
-
-            if len(accessions) == 0:
-                logger.warning(
-                    "Did not retrieve any GenBank accessions whose sequences need updating for "
-                    f"the CAZy class {cazy_class}.\n"
-                    "Not adding sequences to the local database."
-                )
-                continue
-            # separate accesions in to separate lists of length args.epost
-            # epost doesn't like posting more than 200 at once
-            accessions = get_accession_chunks(accessions, args.epost)  # args.epost = acc/chunk
-            for lst in accessions:
-                get_sequences_add_to_db(lst, date_today, session, args)
-
-    # Retrieve protein sequences for specified families
-    for key in config_dict:
-        if key == "classes":
-            continue
-        if config_dict[key] is None:
-            continue  # no families to parse
-
-        for family in tqdm(config_dict[key], desc=f"Parsing families in {key}"):
-            if family.find("_") != -1:  # subfamily
-                # Retrieve GenBank accessions catalogued under the subfamily
-                family_subquery = session.query(Cazyme.cazyme_id).\
-                    join(CazyFamily, Cazyme.families).\
-                    filter(CazyFamily.subfamily == family).\
-                    subquery()
-
-            else:  # family
-                # Retrieve GenBank accessions catalogued under the family
-                family_subquery = session.query(Cazyme.cazyme_id).\
-                    join(CazyFamily, Cazyme.families).\
-                    filter(CazyFamily.family == family).\
-                    subquery()
-
-            # get the GenBank accessions of thes CAZymes, without sequences
-            if args.primary:
-                genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                    join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                    join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-                    join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-                    filter(Cazyme.cazyme_id.in_(family_subquery)).\
-                    filter(Cazymes_Genbanks.primary == True).\
-                    filter(Genbank.sequence == None).\
-                    all()
-            else:
-                genbank_query = session.query(Genbank, Cazymes_Genbanks, Cazyme, Taxonomy, Kingdom).\
-                    join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                    join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                    join(Cazymes_Genbanks, (Cazymes_Genbanks.cazyme_id == Cazyme.cazyme_id)).\
-                    join(Genbank, (Genbank.genbank_id == Cazymes_Genbanks.genbank_id)).\
-                    filter(Cazyme.cazyme_id.in_(family_subquery)).\
-                    filter(Genbank.sequence == None).\
-                    all()
-
-            # create dictionary of {genbank_accession: 'sequence update date' (str)}
-            accessions = extract_accessions_and_dates(genbank_query, taxonomy_filters)
-
-            if len(accessions.keys()) == 0:
-                logger.warning(
-                    f"Did not retrieve any GenBank accessions for the CAZy class {family}.\n"
-                    "Not adding sequences to the local database."
-                )
-                continue
-
-            accessions = get_accessions_for_new_sequences(accessions)  # list of genkbank_accession
-
-            if len(accessions) == 0:
-                logger.warning(
-                    "Did not retrieve any GenBank accessions whose sequences need updating for "
-                    f"the CAZy class {family}.\n"
-                    "Not adding sequences to the local database."
-                )
-                continue
-            # separate accesions in to separate lists of length args.epost
-            # epost doesn't like posting more than 200 at once
-            accessions = get_accession_chunks(accessions, args.epost)  # args.epost = acc/chunk
-            for lst in accessions:
-                get_sequences_add_to_db(lst, date_today, session, args)
-
-    return
-
-
-# The following functions are retrieving the list of Genbank accessions to retrieve sequences for #
-
-
-def extract_accessions(genbank_query, taxonomy_filters):
-    """The query contains GenBank accessions and Cazymes_Genbanks records, retrieve the accessions.
-
-    :param genbank_query: sql collection
-    :param taxonomy_filters: set of genera, species and strains to restrict retrieval of sequences
-
-    Return a list of GenBank accessions. Each element is a string of a unique accession.
-    """
-    if taxonomy_filters is None:
-        accessions = [item[0] for item in genbank_query]
-        return [x for x in accessions if "NA" != x]
-
-    else:
-        accessions = []
-        for item in genbank_query:
-            if item[0] != "NA":  # if GenBank accession not stored as 'NA'
-                source_organism = item[-1].genus + item[-1].species
-                if any(filter in source_organism for filter in taxonomy_filters):
-                    accessions.append(item[0])
-    return accessions
-
-
-def extract_accessions_and_dates(genbank_query, taxonomy_filters):
-    """Retrieve the GenBank accessions and retrieval dates of existing sequences from the db query.
-
-    :param genbank_query: sql collection
-    :param taxonomy_filters: set of genera, species and strains to restrict retrieval of sequences
-
-    Return a dict {GenBank_accession: retrieval_date}
-    """
-    accessions = {}
-
-    if taxonomy_filters is None:
-        for item in genbank_query:
-            if item[0].genbank_accession == "NA":  # no GenBank accession stored in CAZy
-                continue
-            accessions[item[0].genbank_accession] = item[0].seq_update_date
-
-    else:
-        for item in genbank_query:
-            if item[0].genbank_accession == "NA":  # no GenBank accession stored in CAZy
-                continue
-            source_organism = item[-1].genus + item[-1].species
-            if any(filter in source_organism for filter in taxonomy_filters):
-                accessions[item[0].genbank_accession] = item[0].seq_update_date
-
-    return accessions
-
-
-def get_accessions_for_new_sequences(accessions):
-    """Get the GenBank accessions of sequences to be added to the local database.
-
-    For records currently with no protein sequence, the retrieved protein sequence will be added
-    to the record. For records with a sequence, the 'UpdateDate' for the sequence from NCBI will
-    be compared against the  'seq_update_date' in the local database. The 'seq_update_date' is the
-    'UpdateDate' previosuly retrieved from NCBI. If the NCBI sequence is newer,
-    the local database will be updated with the new sequence.
-
-    :param accessions: dict, {GenBank accessions (str):sequence retrieval data (str)}
-    :param session: open SQL database session
-
-    Return nothing.
-    """
-    logger = logging.getLogger(__name__)
-
-    accessions_list = list(accessions.keys())
-    accessions_string = ",".join(accessions_list)
-    # perform batch query of Entrez
-    epost_result = Entrez.read(
-        entrez_retry(
-            Entrez.epost, "Protein", id=accessions_string, retmode="text",
-        )
-    )
-    # retrieve the web environment and query key from the Entrez post
-    epost_webenv = epost_result["WebEnv"]
-    epost_query_key = epost_result["QueryKey"]
-
-    # retrieve summary docs to check the sequence 'UpdateDates' in NCBI
-    with entrez_retry(
-        Entrez.efetch,
-        db="Protein",
-        query_key=epost_query_key,
-        WebEnv=epost_webenv,
-        rettype="docsum",
-        retmode="xml",
-    ) as handle:
-        summary_docs = Entrez.read(handle)
-
-    for doc in summary_docs:
-        try:
-            temp_accession = doc["AccessionVersion"]  # accession of the current working protein
-        except KeyError:
-            logger.warning(
-                f"Retrieved protein with accession {temp_accession} but this accession is not in "
-                "the local database.\n"
-                "Not retrieving a sequence for this accession."
-            )
-            continue
-        previous_data = accessions[temp_accession]
-        if previous_data is not None:
-            # sequence retrieved previosuly, thus check if the NCBI seq has been updated since
-            previous_data = previous_data.split("/")  # Y=[0], M=[1], D=[]
-            update_date = doc["UpdateDate"]
-            update_date = update_date.split("/")  # Y=[0], M=[1], D=[]
-            if datetime.date(
-                previous_data[0], previous_data[1], previous_data[2],
-            ) < datetime.data(
-                update_date[0], update_date[1], update_date[2],
-            ) is False:
-                # the sequence at NCBI has not been updated since the seq was retrieved
-                # thus no need to retrieve it again
-                accessions_list.remove(temp_accession)
-
-    return accessions_list
-
-
-def get_accession_chunks(lst, chunk_length):
-    """Separate the long list into separate chunks.
-
-    :param lst: list to be separated into smaller lists (or chunks)
-    :param chunk_length: int, the length of the lists the longer list is to be split up into
-
-    Return a generator object containing lists.
-    """
-    for i in range(0, len(lst), chunk_length):
-        yield lst[i:i + chunk_length]
-
-
-# The following functions are for retrieving sequences, adding to the db and writing fasta files
-
-
-def get_sequences_add_to_db(accessions, date_today, session, args):
-    """Retrieve protein sequences from Entrez and add to the local database.
-
-    :param accessions: list, GenBank accessions
-    :param date_today: str, YYYY/MM/DD
-    :param session: open SQL database session
-    :param args: cmb-line args parser
-
-    Return nothing.
-    """
-    logger = logging.getLogger(__name__)
-    # perform batch query of Entrez
-    accessions_string = ",".join(accessions)
-    epost_result = Entrez.read(
-        entrez_retry(
-            Entrez.epost, "Protein", id=accessions_string,
-        )
-    )
-    # retrieve the web environment and query key from the Entrez post
-    epost_webenv = epost_result["WebEnv"]
-    epost_query_key = epost_result["QueryKey"]
-
-    # retrieve the protein sequences
-    with entrez_retry(
-        Entrez.efetch,
-        db="Protein",
-        query_key=epost_query_key,
-        WebEnv=epost_webenv,
-        rettype="fasta",
-        retmode="text",
-    ) as seq_handle:
-        for record in SeqIO.parse(seq_handle, "fasta"):
-            # retrieve the accession of the record
-            temp_accession = record.id  # accession of the current working protein record
-
-            if temp_accession.find("|") != -1:  # sometimes multiple items are listed
-                success = False   # will be true if finds protein accession
-                temp_accession = temp_accession.split("|")
-                for item in temp_accession:
-                    # check if a accession number
-                    try:
-                        re.match(
-                            (
-                                r"(\D{3}\d{5,7}\.\d+)|(\D\d(\D|\d){3}\d)|"
-                                r"(\D\d(\D|\d){3}\d\D(\D|\d){2}\d)"
-                            ),
-                            item,
-                        ).group()
-                        temp_accession = item
-                        success = True
-                        break
-                    except AttributeError:  # raised if not an accession
-                        continue
-
-            else:
-                success = True  # have protein accession number
-
-            if success is False:
-                logger.error(
-                    f"Could not retrieve accession from {record.id}, therefore, "
-                    "protein sequence not added to the database,\n"
-                    "because cannot retrieve the necessary CAZyme record"
-                )
-                continue
-
-            # check the retrieve protein accession is in the list of retrieved accession
-            if temp_accession not in accessions:
-                logger.warning(
-                    f"Retrieved the accession {temp_accession} from the record id={record.id}, "
-                    "but this accession is not in the database.\n"
-                    "Therefore, not adding this protein seqence to the local database"
-                )
-                continue
-
-            # retrieve the GenBank record from the local data base to add the seq to
-            genbank_record = session.query(Genbank).\
-                filter(Genbank.genbank_accession == temp_accession).first()
-
-            retrieved_sequence = str(record.seq)  # convert to a string becuase SQL expects a string
-            genbank_record.sequence = retrieved_sequence
-            genbank_record.seq_update_date = date_today
-            session.commit()
-
-            if args.fasta is not None:
-                file_io.write_out_fasta(record, temp_accession, args)
-
-            if args.blastdb is not None:
-                file_io.write_fasta_for_db(record, temp_accession, args)
-
-            # remove the accession from the list
-            accessions.remove(temp_accession)
-
-    if len(accessions) != 0:
-        logger.warning(
-            "Protein sequences were not retrieved for the following CAZyme in the local database"
-        )
-        for acc in accessions:
-            logger.warning(f"GenBank accession: {acc}")
-
-    return
-
-
-def entrez_retry(entrez_func, *func_args, **func_kwargs):
-    """Call to NCBI using Entrez.
-
-    Maximum number of retries is 10, retry initated when network error encountered.
-
-    :param logger: logger object
-    :param retries: parser argument, maximum number of retries excepted if network error encountered
-    :param entrez_func: function, call method to NCBI
-
-    :param *func_args: tuple, arguments passed to Entrez function
-    :param ** func_kwargs: dictionary, keyword arguments passed to Entrez function
-
-    Returns record.
-    """
-    logger = logging.getLogger(__name__)
-    record, retries, tries = None, 10, 0
-
-    while record is None and tries < retries:
-        try:
-            record = entrez_func(*func_args, **func_kwargs)
-
-        except IOError:
-            # log retry attempt
-            if tries < retries:
-                logger.warning(
-                    f"Network error encountered during try no.{tries}.\nRetrying in 10s",
-                    exc_info=1,
-                )
-                time.sleep(10)
-            tries += 1
-
-    if record is None:
-        logger.error(
-            "Network error encountered too many times. Exiting attempt to call to NCBI"
-        )
-        return
-
-    return record
-
-
-if __name__ == "__main__":
-    main()
diff --git a/scraper/expand/get_genbank_sequences/from_sql_db/__init__.py b/scraper/expand/get_genbank_sequences/from_sql_db/__init__.py
deleted file mode 100644
index 3e519425..00000000
--- a/scraper/expand/get_genbank_sequences/from_sql_db/__init__.py
+++ /dev/null
@@ -1,334 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-# (c) University of St Andrews 2020-2021
-# (c) University of Strathclyde 2020-2021
-# Author:
-# Emma E. M. Hobbs
-#
-# Contact
-# eemh1@st-andrews.ac.uk
-#
-# Emma E. M. Hobbs,
-# Biomolecular Sciences Building,
-# University of St Andrews,
-# North Haugh Campus,
-# St Andrews,
-# KY16 9ST
-# Scotland,
-# UK
-#
-# The MIT License
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in all
-# copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-"""Retrieve proteins sequences from GenBank and populate the local database and write to FASTA"""
-
-
-import logging
-import math
-import sys
-
-from tqdm import tqdm
-
-from scraper.expand import get_genbank_sequences
-from scraper.expand.get_genbank_sequences.from_sql_db import query_sql_db
-from scraper.expand.get_genbank_sequences.ncbi import query_entrez
-from scraper.sql.sql_orm import get_db_session
-from scraper.utilities import file_io, parse_configuration
-
-
-def sequences_for_proteins_from_db(date_today, args):
-    """Coordinate retrievel of protein sequences for proteins in a SQL database.
-    
-    :param date_today: str, date script was invoked, used for naming files
-    :param args: cmd-line args parser
-    
-    Return nothing.
-    """
-    logger = logging.getLogger(__name__)
-
-    # get database session
-    try:
-        session = get_db_session(args)
-    except Exception as err:
-        logger.error(
-            "Could not connect to local CAZyme database.\nThe following error was raised:\n"
-            f"{err}\nTerminating program\n"
-        )
-        sys.exit(1)
-
-    file_io.make_output_directory(args.fasta, args.force, args.nodelete)
-
-    if args.blastdb is not None:  # build directory to store FASTA file for BLAST db
-        file_io.make_output_directory(args.blastdb, args.force, args.nodelete)
-
-    # retrieve configuration data, as a dict of CAZy classes and families to retrieve seqs for
-    parse_configuration_path = parse_configuration.__file__
-    (
-        config_dict, taxonomy_filters, kingdoms, ec_filters,
-    ) = parse_configuration.parse_configuration_for_cazy_database(
-        parse_configuration_path,
-        args,
-    )
-
-    genbank_accessions = get_genbank_accessions(
-        args,
-        session,
-        config_dict,
-        taxonomy_filters,
-        kingdoms,
-        ec_filters,
-    )
-
-    # break up protein_list into multiple, smaller lists for batch querying Entrez
-    # batches of greater than 200 can be rejected by Entrez during busy periods
-    # args.epost=size of chunks
-
-    accessions_lists_for_individual_queries = []
-
-    for accession_list in tqdm(
-        get_genbank_sequences.get_accession_chunks(genbank_accessions, args.epost),
-        desc="Batch retrieving sequences from NCBI",
-        total=(math.ceil(len(genbank_accessions) / args.epost)),
-    ):
-        try:
-            accession_list.remove("NA")
-        except ValueError:
-            pass
-
-        try:
-            query_entrez.get_sequences_add_to_db(accession_list, date_today, args)
-        except RuntimeError as err:  # typically Some IDs have invalid value and were omitted.
-            logger.warning(
-                "RuntimeError raised for accession list. Will query accessions individualy after.\n"
-                f"The following error was raised:\n{err}"
-            )
-            with open("legihton_error.txt", "a") as fh:
-                fh.write(f"{err}\n{str(accession_list)}")
-            accessions_lists_for_individual_queries.append(accession_list)
-
-    if len(accessions_lists_for_individual_queries) != 0:
-        for accession_list in tqdm(
-            accessions_lists_for_individual_queries,
-            desc="Performing individual queries to parse GenBank accessions without records",
-        ):
-            for accession in tqdm(accession_list, desc="Retrieving individual sequences"):
-                try:
-                    query_entrez.get_sequences_for_dict([accession], date_today, args)
-                except RuntimeError as err:
-                    logger.warning(
-                        f"Querying NCBI for {accession} raised the following RuntimeError:\n"
-                        f"{err}"
-                    )
-    return
-
-    return
-
-
-def get_genbank_accessions(args, session, config_dict, taxonomy_filters, kingdoms, ec_filters):
-    """Retrieve the GenBank accessions for all proteins for which a sequence will be retrieved.
-
-    :param args: cmd-line argument parser
-    :param session: open SQLite db session
-    :param config_dict: dict, defines CAZy classes and families to get sequences for
-    :param taxonomy_filters: set of genera, species and strains to retrieve sequences for
-    :param kingdoms: set of taxonomy Kingdoms to retrieve sequences for
-    :param ec_filters: set of EC numbers annotations CAZymes must have at least one to retrieve
-        a sequence
-
-    Return a list of GenBank accessions, containing no duplicate GenBank accessions
-    """
-    logger = logging.getLogger(__name__)
-
-    if config_dict:  # there are specific CAZy classes/families to retrieve sequences for
-        genbank_query_class, genbank_query_family = get_cazy_class_fam_genbank_records(
-            session,
-            config_dict,
-        )
-
-        class_genbank_accessions = parse_genbank_query(
-            genbank_query_class,
-            taxonomy_filters,
-            kingdoms,
-            ec_filters,
-            session,
-        )
-
-        family_genbank_accessions = parse_genbank_query(
-            genbank_query_family,
-            taxonomy_filters,
-            kingdoms,
-            ec_filters,
-            session,
-        )
-
-        genbank_accessions = class_genbank_accessions + family_genbank_accessions
-
-    else:
-        if args.update:  # retrieve all GenBank accessions
-
-            if args.primary:
-                logger.warning(
-                    "Retrieving sequences for all PRIMARY GenBank accessions that:\n"
-                    "Do not have a sequence in the db OR the sequence has been updated in NCBI"
-                )
-                genbank_query = query_sql_db.get_all_prim_genbank_acc(session)
-
-            else:
-                logger.warning(
-                    "Retrieving sequences for all PRIMARY GenBank accessions that\n"
-                    "do not have a sequence in the db"
-                )
-                genbank_query = query_sql_db.get_all_genbank_acc(session)
-
-        else:  # retrieve GenBank accesions of records that don't have a sequence
-            if args.primary:
-                logger.warning(
-                    "Retrieving sequences for all PRIMARY GenBank accessions that\n"
-                    "do not have a sequence in the db"
-                )
-                genbank_query = query_sql_db.get_prim_genbank_accessions_with_no_seq(session)
-
-            else:
-                logger.warning(
-                    "Retrieving sequences for ALL GenBank accessions that\n"
-                    "do not have a sequence in the db"
-                )
-                genbank_query = query_sql_db.get_genbank_accessions_with_no_seq(session)
-
-        genbank_accessions = parse_genbank_query(
-            genbank_query,
-            taxonomy_filters,
-            kingdoms,
-            ec_filters,
-            session,
-        )
-
-    return list(set(genbank_accessions))  # prevent quering the same accession multiple times
-
-
-def get_cazy_class_fam_genbank_records(args, session, config_dict):
-    """GenBank acc query results from the local CAZyme database for CAZyme from specific classes/fams
-
-    :param args: cmd-line argument parser
-    :param session: open SQLite db session
-    :param config_dict: dict, defines CAZy classes and families to get sequences for
-
-    Return CAZy class and CAZy family GenBank accession query results
-    """
-    logger = logging.getLogger(__name__)
-    if args.update:  # retrieve all GenBank accessions
-        if args.primary:
-            logger.warning(
-                "Retrieving sequences for PRIMARY GenBank accessions that:\n"
-                "belong to specific CAZy classes/families AND\n"
-                "do not have a sequence in the db OR the sequence has been updated in NCBI"
-            )
-            (
-                genbank_query_class,
-                genbank_query_family,
-            ) = query_sql_db.get_prim_gnbk_acc_from_clss_fams(
-                session,
-                config_dict,
-            )
-
-        else:
-            logger.warning(
-                "Retrieving sequences for PRIMARY GenBank accessions that:\n"
-                "belong to specific CAZy classes/families AND\n"
-                "do not have a sequence in the db OR the sequence has been updated in NCBI"
-            )
-            (
-                genbank_query_class,
-                genbank_query_family,
-            ) = query_sql_db.get_all_gnbk_acc_from_clss_fams(
-                session,
-                config_dict,
-            )
-
-    else:  # retrieve GenBank accesions of records that don't have a sequence
-        if args.primary:
-            logger.warning(
-                "Retrieving sequences for PRIMARY GenBank accessions that:\n"
-                "belong to specific CAZy classes/families AND do not have a sequence in the db"
-            )
-            (
-                genbank_query_class,
-                genbank_query_family,
-            ) = query_sql_db.get_prim_gnbk_acc_from_clss_fams_no_seq(
-                session,
-                config_dict,
-            )
-
-        else:
-            logger.warning(
-                "Retrieving sequences for PRIMARY GenBank accessions that:\n"
-                "belong to specific CAZy classes/families AND do not have a sequence in the db"
-            )
-            (
-                genbank_query_class,
-                genbank_query_family,
-            ) = query_sql_db.get_all_gnbk_acc_from_clss_fams_no_seq(
-                session,
-                config_dict,
-            )
-
-    return genbank_query_class, genbank_query_family
-
-
-def parse_genbank_query(genbank_query, taxonomy_filters, kingdoms, ec_filters, session):
-    """Parse SQL query result and retrieve GenBank accessions of CAZymes that meet the user cirteria
-
-    :param:
-
-    Return list of GenBank accessions.
-    """
-    if genbank_query is None:
-        return []
-
-    if (taxonomy_filters is None) and (kingdoms is None) and (ec_filters is None):
-        accessions = [item[0] for item in genbank_query]
-        return [x for x in accessions if "NA" != x]
-    
-    genbank_accessions = []
-
-    for item in genbank_query:
-        if item[0] != "NA":  # if GenBank accession not stored as 'NA'
-
-            # check if CAZyme records meets the taxonomy criteria
-            source_organism = item[-2].genus + item[-2].species
-            if any(filter in source_organism for filter in taxonomy_filters):
-                genbank_accessions.append(item[0])
-                continue
-
-            # check if CAZyme records meets the kingdom requirement
-            if item[-1].kingdom in kingdoms:
-                genbank_accessions.append(item[0])
-                continue
-
-    if ec_filters is None:
-        return genbank_accessions
-    
-    # check if the parent CAZymes of the GenBank accessions meet the EC number filter
-    filtered_genbank_accessions = []
-    for i in tqdm(range(len(genbank_accessions), desc="Applying EC number filter")):
-        ec_annotations = query_sql_db.query_sql_db.query_ec_number(session, genbank_accessions[i])
-        if (set(ec_annotations) and set(ec_filters)):
-            filtered_genbank_accessions.append(genbank_accessions[i])
-
-    return filtered_genbank_accessions
diff --git a/scraper/expand/get_pdb_structures.py b/scraper/expand/get_pdb_structures.py
deleted file mode 100644
index 515eb57e..00000000
--- a/scraper/expand/get_pdb_structures.py
+++ /dev/null
@@ -1,311 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-# (c) University of St Andrews 2020-2021
-# (c) University of Strathclyde 2020-2021
-# Author:
-# Emma E. M. Hobbs
-
-# Contact
-# eemh1@st-andrews.ac.uk
-
-# Emma E. M. Hobbs,
-# Biomolecular Sciences Building,
-# University of St Andrews,
-# North Haugh Campus,
-# St Andrews,
-# KY16 9ST
-# Scotland,
-# UK
-#
-# The MIT License
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in all
-# copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-#
-# Bio.PDB reference:
-# Hamelryck, T., Manderick, B. (2003) PDB parser and structure class 
-# implemented in Python. Bioinformatics 19: 2308–2310
-"""Retrieve PDB structures from RSCB PDB and write to disk"""
-
-
-import logging
-import os
-import sys
-import time
-
-import pandas as pd
-
-from datetime import datetime
-from typing import List, Optional
-
-from Bio.PDB import PDBList
-from tqdm import tqdm
-
-from scraper.sql.sql_orm import (
-    Cazyme,
-    CazyFamily,
-    Kingdom,
-    Pdb,
-    Taxonomy,
-    get_db_session,
-)
-from scraper.utilities import config_logger, file_io, parse_configuration
-from scraper.utilities.parsers import build_pdb_structures_parser
-
-
-def main(argv: Optional[List[str]] = None, logger: Optional[logging.Logger] = None):
-    """Set up programme and initate run."""
-    start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
-    start_time = pd.to_datetime(start_time)
-
-    # parse cmd-line arguments
-    if argv is None:
-        parser = build_pdb_structures_parser()
-        args = parser.parse_args()
-    else:
-        parser = build_pdb_structures_parser(argv)
-        args = parser.parse_args()
-
-    # build logger
-    if logger is None:
-        logger = logging.getLogger(__name__)
-        config_logger(args)
-
-    # create session to local database
-    if os.path.isfile(args.database) is False:
-        logger.error(
-            "Could not find local CAZy database. Check path is correct. Terminating programme."
-        )
-        sys.exit(1)
-    session = get_db_session(args)
-
-    # create output directory
-    if args.outdir is None:
-        # save structure files to the cwd
-        outdir = os.getcwd()
-    else:
-        outdir = args.outdir
-    file_io.make_output_directory(outdir, args.force, args.nodelete)
-
-    # check if any classes or families were specified to retrieve the sequences only for them
-    file_io_path = file_io.__file__
-    config_dict, taxonomy_filters, kingdoms = parse_configuration.get_configuration(
-        args,
-    )
-
-    # retrieve protein structures from PDB
-
-    # retrieve sequences for all CAZymes
-    if config_dict is None:
-        get_every_cazymes_structures(outdir, taxonomy_filters, kingdoms, session, args)
-
-    # retrieve sequences for specific CAZy classes and/or families
-    else:
-        get_structures_for_specific_cazymes(
-            outdir,
-            config_dict,
-            taxonomy_filters,
-            kingdoms,
-            session,
-            args,
-        )
-
-    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")  # used in terminating message
-    end_time = pd.to_datetime(start_time)
-    end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
-    end_time = pd.to_datetime(end_time)
-    total_time = end_time - start_time
-
-    logger.info(
-        "Finished dowloading protein structures from PDB."
-        "Terminating program.\n"
-        f"Scrape initated at {start_time}\n"
-        f"Scrape finished at {end_time}\n"
-        f"Total run time: {total_time}"
-    )
-
-    print(
-        "=====================cazy_webscraper-expand-pdb_structures=====================\n"
-        "Finished dowloading protein structures from PDB."
-        "Terminating program.\n"
-        f"Scrape initated at {start_time}\n"
-        f"Scrape finished at {end_time}\n"
-        f"Total run time: {total_time}\n"
-    )
-
-
-def get_every_cazymes_structures(outdir, taxonomy_filters, kingdoms, session, args):
-    """Get PDB accessions in the db, and coordinate downloading the structure from pdb.
-
-    :param outdir: path to output directory
-    :param taxonomy_filters: set of genera, species, and strains to retrieve structures for
-    :param kingdoms: set of taxonomy Kingdoms to retrieve structures for
-    :param session: open SQLite db session
-    :param args: cmd-line argument parser
-
-    Return nothing.
-    """
-    # retrieve PDB accessions
-    pdb_query = session.query(Pdb, Cazyme, Taxonomy, Kingdom).\
-        join(Cazyme.pdbs).\
-        join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-        join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-        all()
-
-    if taxonomy_filters is None:
-        for query_result in pdb_query:
-            pdb_accession = query_result[0].pdb_accession
-            pdb_accession = pdb_accession[:pdb_accession.find("[")]
-            download_pdb_structures(pdb_accession, outdir, args)
-
-    else:
-        for query_result in pdb_query:
-            source_organism = query_result[-2].genus + query_result[-2].species
-            if any(filter in source_organism for filter in taxonomy_filters):
-                pdb_accession = query_result[0].pdb_accession
-                pdb_accession = pdb_accession[:pdb_accession.find("[")]
-                download_pdb_structures(pdb_accession, outdir, args)
-            elif query_result[-1].kingdom in kingdoms:
-                pdb_accession = query_result[0].pdb_accession
-                pdb_accession = pdb_accession[:pdb_accession.find("[")]
-                download_pdb_structures(pdb_accession, outdir, args)
-
-    return
-
-
-def get_structures_for_specific_cazymes(outdir, config_dict, taxonomy_filters, kingdoms, session, args):
-    """Retrieve primary PDB accessions for CAZymes meeting criteria in the config_dict.
-
-    :param outdir: path to output directory
-    :param config_dict: dict, defines CAZy classes and families to retrieve accessions from
-    :param taxonomy_filters: set of genera, species, and strains to retrieve structures for
-    :param kingdoms: set of taxonomy Kingdoms to retrieve structures for
-    :param session: open SQLite db session
-    :param args: cmd-line argument parser
-
-    Return nothing.
-    """
-    # start with the classes
-    if len(config_dict["classes"]) != 0:
-        # create a dictionary to convert full class name to abbreviation
-        cazy_classes = config_dict["classes"]
-
-        for cazy_class in tqdm(cazy_classes, desc="Parsing CAZy classes"):
-            # retrieve class name abbreviation
-            cazy_class = cazy_class[((cazy_class.find("(")) + 1):((cazy_class.find(")")) - 1)]
-
-            # retrieve the CAZymes from the specified class
-            class_subquery = session.query(Cazyme.cazyme_id).\
-                join(CazyFamily, Cazyme.families).\
-                filter(CazyFamily.family.regexp(rf"{cazy_class}\d+")).\
-                subquery()
-
-            # Retrieve PDB accessions for the selected CAZymes
-            pdb_query = session.query(Pdb, Cazyme, Taxonomy, Kingdom).\
-                join(Cazyme.pdbs).\
-                join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                filter(Cazyme.cazyme_id.in_(class_subquery)).all()
-
-            if taxonomy_filters is None:
-                for query_result in pdb_query:
-                    pdb_accession = query_result[0].pdb_accession
-                    pdb_accession = pdb_accession[:pdb_accession.find("[")]
-                    download_pdb_structures(pdb_accession, outdir, args)
-
-            else:
-                for query_result in pdb_query:
-                    source_organism = query_result[-2].genus + query_result[-2].species
-                    if any(filter in source_organism for filter in taxonomy_filters):
-                        pdb_accession = query_result[0].pdb_accession
-                        pdb_accession = pdb_accession[:pdb_accession.find("[")]
-                        download_pdb_structures(pdb_accession, outdir, args)
-                    elif query_result[-1].kingdom in kingdoms:
-                        pdb_accession = query_result[0].pdb_accession
-                        pdb_accession = pdb_accession[:pdb_accession.find("[")]
-                        download_pdb_structures(pdb_accession, outdir, args)
-
-    # retrieve protein structure for specified families
-    for key in config_dict:
-        if key == "classes":
-            continue
-        if config_dict[key] is None:
-            continue
-
-        for family in tqdm(config_dict[key], desc=f"Parsing families in {key}"):
-            # Select the CAZymes under the specified (sub)family
-            if family.find("_") != -1:  # subfamily
-                # Retrieve GenBank accessions catalogued under the subfamily
-                family_subquery = session.query(Cazyme.cazyme_id).\
-                    join(CazyFamily, Cazyme.families).\
-                    filter(CazyFamily.subfamily == family).\
-                    subquery()
-
-            else:  # family
-                # Retrieve GenBank accessions catalogued under the family
-                family_subquery = session.query(Cazyme.cazyme_id).\
-                    join(CazyFamily, Cazyme.families).\
-                    filter(CazyFamily.family == family).\
-                    subquery()
-
-            # Retrieve PDB accessions of the selected CAZymes
-            pdb_query = session.query(Pdb, Cazyme, Taxonomy, Kingdom).\
-                join(Cazyme.pdbs).\
-                join(Cazyme, (Cazyme.taxonomy_id == Taxonomy.taxonomy_id)).\
-                join(Taxonomy, (Taxonomy.kingdom_id == Kingdom.kingdom_id)).\
-                filter(Cazyme.cazyme_id.in_(family_subquery)).all()
-
-            if taxonomy_filters is None:
-                for query_result in pdb_query:
-                    pdb_accession = query_result[0].pdb_accession
-                    pdb_accession = pdb_accession[:pdb_accession.find("[")]
-                    download_pdb_structures(pdb_accession, outdir, args)
-
-            else:
-                for query_result in pdb_query:
-                    source_organism = query_result[-2].genus + query_result[-2].species
-                    if any(filter in source_organism for filter in taxonomy_filters):
-                        pdb_accession = query_result[0].pdb_accession
-                        pdb_accession = pdb_accession[:pdb_accession.find("[")]
-                        download_pdb_structures(pdb_accession, outdir, args)
-                    elif query_result[-1].kingdom in kingdoms:
-                        pdb_accession = query_result[0].pdb_accession
-                        pdb_accession = pdb_accession[:pdb_accession.find("[")]
-                        download_pdb_structures(pdb_accession, outdir, args)
-
-    return
-
-
-def download_pdb_structures(pdb_accession, outdir, args):
-    """Download protein structure from the RSCB PDB database
-
-    :param pdb_accession: str, accession of record in the PDB database
-    :param outdir: path to output directory
-    :param args: cmd-line args parser
-
-    Return nothing.
-    """
-    pdbl = PDBList()
-    pdbl.retrieve_pdb_file(f"{pdb_accession}", file_format=args.pdb, pdir=args.outdir)
-    time.sleep(2)  # to prevent bombarding the system
-
-    return
-
-
-if __name__ == "__main__":
-    main()
diff --git a/scraper/expand/pdb_structures/__init__.py b/scraper/expand/pdb_structures/__init__.py
deleted file mode 100644
index e69de29b..00000000
diff --git a/scraper/sql/sql_interface/add_cazyme_data.py b/scraper/sql/sql_interface/add_cazyme_data.py
deleted file mode 100644
index 5f5be4e0..00000000
--- a/scraper/sql/sql_interface/add_cazyme_data.py
+++ /dev/null
@@ -1,225 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-# (c) University of St Andrews 2020-2021
-# (c) University of Strathclyde 2020-2021
-# (c) James Hutton Institute 2020-2021
-#
-# Author:
-# Emma E. M. Hobbs
-#
-# Contact
-# eemh1@st-andrews.ac.uk
-#
-# Emma E. M. Hobbs,
-# Biomolecular Sciences Building,
-# University of St Andrews,
-# North Haugh Campus,
-# St Andrews,
-# KY16 9ST
-# Scotland,
-# UK
-#
-# The MIT License
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in all
-# copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-"""Add CAZyme data to a local SQLite database"""
-
-
-import logging
-
-from tqdm import tqdm
-
-from scraper.sql.sql_interface import insert_data
-from scraper.sql.sql_orm import Session, CazyFamily, Genbank, Kingdom, Taxonomy
-
-
-def add_kingdoms(cazy_data, connection):
-    """Create a dict of taxa information and add kingdoms to db.
-    
-    :param cazy_data: dict of CAZy data
-    :param connection: open sqlalchemy conenction to an SQLite db engine
-
-    Return taxa_dict, dict of taxonomy data {kingdom: set(organism)}
-    """
-    taxa_dict = {}  # {kingdom: {organism,}}
-    
-    for genbank_accession in tqdm(cazy_data, "Compiling taxa data"):
-        kingdom = list(cazy_data[genbank_accession]['kingdom'])[0]
-        organism = list(cazy_data[genbank_accession]['organism'])[0]
-        
-        try:
-            taxa_dict[kingdom].add(organism)
-        except KeyError:
-            taxa_dict[kingdom] = {organism}
-
-    kingdoms_db_insert_values = [(kingdom,) for kingdom in taxa_dict.keys()]
-
-    insert_data(connection, 'Kingdoms', ['kingdom'], kingdoms_db_insert_values)
-
-    return taxa_dict
-
-
-def add_source_organisms(taxa_data, connection):
-    """Add taxonomy (source organism) data to the local CAZyme database
-    
-    :param taxa_data: dict of taxa data {kingdom: set(organism)}
-    :param connection: open sqlalchemy connection to SQLite engine
-    
-    Return nothing
-    """
-    taxonomy_db_insert_values = []
-    with Session(bind=connection) as session:
-        for kingdom in tqdm(
-            taxa_data,
-            total=len(list(taxa_data.keys())),
-            desc='Adding Tax data to db per Kingdom',
-        ):
-            # query db to get the kingdom db object, retrieve only the kingdom ID number
-            found_kingdom = session.query(Kingdom.kingdom_id).\
-                filter(Kingdom.kingdom==kingdom).\
-                first()[0]
-            
-            for organism in taxa_data[kingdom]:
-                genus = organism.split(" ")[0]
-                species = ' '.join(organism.split(" ")[1:])
-                taxonomy_db_insert_values.append((genus, species, found_kingdom))
-
-    insert_data(connection, 'Taxs', ['genus', 'species', 'kingdom_id'], taxonomy_db_insert_values)
-
-
-def add_cazy_families(cazy_data, connection):
-    """Add CAZy families and subfamilies to local CAZyme database
-    
-    :param cazy_data: dict of data retrieved from CAZy
-    :param connection: open sqlalchemy connection to an SQLite db engine
-    
-    Return nothing"""
-    families_db_insert_values = set()
-
-    for genbank_accession in tqdm(cazy_data, desc='Adding CAZy fams to db'):
-        for cazy_fam in cazy_data[genbank_accession]["families"]:
-            subfamilies = cazy_data[genbank_accession]["families"][cazy_fam]
-            if None not in subfamilies:
-                cazy_data[genbank_accession]["families"][cazy_fam].add(None)
-            
-            for cazy_subfam in subfamilies:
-                families_db_insert_values.add( (cazy_fam, cazy_subfam) )
-
-    insert_data(connection, 'CazyFamilies', ['family', 'subfamily'], list(families_db_insert_values))
-
-
-def load_taxa_fam_data(connection):
-    """Load taxonomy and CAZy fam tables into memory.
-    
-    :param connection: open sqlalchemy connection to an SQLite db engine
-    
-    Return dict representing the Taxs table and a dict of the Families table
-    """
-    with Session(bind=connection) as session:
-        all_families = session.query(CazyFamily).all()
-
-    db_fam_dict = {}
-
-    for fam in all_families:
-        if fam.subfamily is None:
-            subfam = '_'
-        else:
-            subfam = fam.subfamily
-            
-        db_fam_dict[f"{fam.family} {subfam}"] = fam.family_id
-
-    # create dict of tax data from db, valued by the tax obj id number
-    with Session(bind=connection) as session:
-        all_taxa = session.query(Taxonomy).all()
-        
-    db_tax_dict = {}
-    for taxa in all_taxa:
-        if len(taxa.species) == 0:
-            db_tax_dict[f"{taxa.genus}"] = taxa.taxonomy_id
-        else:
-            db_tax_dict[f"{taxa.genus} {taxa.species}"] = taxa.taxonomy_id
-    
-    return db_tax_dict, db_fam_dict
-
-
-def add_genbanks(cazy_data, db_tax_dict, db_fam_dict, connection):
-    """Add GenBank accessions with tax data to the db
-    
-    :param cazy_data: dict of CAZy data
-    :param connection: open sqlalchemy connection to an SQLite db
-    
-    Return set of tuples, of (genbank_acc, fam_id)
-    """
-    # create set of tuples for inserting into the Genbank db table
-    # associates a genbank accession with its source organisms db tax id num
-    gbk_db_insert_values = set()
-
-    # ...and create set of tuples for inserting into the CazyFamilies_Genbank table
-    # associates genbank acc db id nums with CAZy fam db ids
-    gbk_fam_values = set()  # { (gbk_accession, fam_db_id) } 
-
-    for genbank_accession in tqdm(cazy_data, desc='Adding GenBank to db'):
-        organism = list(cazy_data[genbank_accession]['organism'])[0]
-        tax_id = db_tax_dict[organism]
-        
-        gbk_db_insert_values.add( (genbank_accession, tax_id) )
-        
-        
-        for cazy_fam in  cazy_data[genbank_accession]['families']:
-            subfamilies = cazy_data[genbank_accession]['families'][cazy_fam]
-            
-            for cazy_subfam in subfamilies:
-                if cazy_subfam is None:
-                    fam_name = f"{cazy_fam} _"
-                
-                else:
-                    fam_name = f"{cazy_fam} {cazy_subfam}"
-            
-            fam_id = db_fam_dict[fam_name]
-            gbk_fam_values.add( (genbank_accession, fam_id) )
-
-    insert_data(connection, 'Genbanks', ['genbank_accession', 'taxonomy_id'], list(gbk_db_insert_values))
-
-    return gbk_fam_values
-
-
-def add_genbank_fam_relationships(gbk_fam_values, connection):
-    """Add GenBank accession - CAZy family relationships to db
-    
-    :param gbk_fam_values: set of tuples (gbk_acc, fam_id)
-    :param connection: open sqlalchemy connection to an SQLite db engine
-
-    Return nothing
-    """
-    # retrieve genbank id numbers from the local database
-    with Session(bind=connection) as session:
-        all_genbank = session.query(Genbank).all()
-
-    db_gbk_dict = {}  # {genbank_accession: db genbank id number}
-    for gbk in all_genbank:
-        db_gbk_dict[f"{gbk.genbank_accession}"] = gbk.genbank_id
-
-    gbk_fam_db_insert_values = set()  # { (gbk_id(int), fam_id(int), ), }
-
-    for gbk_tuple in tqdm(gbk_fam_values, desc='Adding Protein-Fam relationships'):
-        gbk_id = db_gbk_dict[gbk_tuple[0]]
-        fam_id = gbk_tuple[1]
-        gbk_fam_db_insert_values.add( (gbk_id, fam_id) )
-
-    insert_data(connection, 'Genbanks_CazyFamilies', ['genbank_id', 'family_id'], list(gbk_fam_db_insert_values))
diff --git a/setup.py b/setup.py
index 38e9bfb2..fbff2184 100644
--- a/setup.py
+++ b/setup.py
@@ -74,9 +74,12 @@
     url="https://github.com/HobnobMancer/cazy_webscraper",
     entry_points={
         "console_scripts": [
-            "cazy_webscraper = scraper.cazy_webscraper:main",
-            "cw_get_genbank_sequences = scraper.expand.get_genbank_sequences:main",
-            "cw_get_pdb_structures = scraper.expand.get_pdb_structures:main",
+            "cazy_webscraper = cazy_webscraper.cazy_webscraper:main",
+            "cw_get_genbank_seqs = cazy_webscraper.expand.genbank.get_genbank_sequences:main",
+            "cw_get_uniprot_data = cazy_webscraper.expand.uniprot.get_uniprot_data:main",
+            "cw_extract_sequences = cazy_webscraper.expand.extract.extract_sequences:main",
+            "cw_get_pdb_structures = cazy_webscraper.expand.pdb.get_pdb_structures:main",
+            "cw_cazy_genbank_coverage = cazy_webscraper.genomes.cazy_genbank_coverage:main",
         ]
     },
     install_requires=[
@@ -85,6 +88,7 @@
         "pandas>=1.0.3",
         "pyyaml",
         "requests",
+        "saintBioutils",
         "sqlalchemy>=1.4.20",
         "tqdm",
     ],


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
index 5ec435b7..5a29c229 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -77,9 +77,13 @@ def time_stamp():
 # Define fixtures for testing SQL ORM and interface
 
 
+@pytest.fixture()
+def db_path():
+    return Path("tests/test_inputs/unit_test_database/unit_test_2021-04-27--11-54-58.db")
+
+
 @pytest.fixture(scope="session")
-def engine():
-    db_path = "tests/test_inputs/unit_test_database/unit_test_2021-04-27--11-54-58.db"
+def engine(db_path):
     return create_engine(f"sqlite+pysqlite:///{db_path}", echo=False)
 
 
diff --git a/tests/test_webscraper.py b/tests/test_webscraper.py
index 6a69a634..ed0ea476 100644
--- a/tests/test_webscraper.py
+++ b/tests/test_webscraper.py
@@ -45,15 +45,61 @@
 """
 
 import os
+from cazy_webscraper.utilities.parse_configuration.cazy_class_synonym_dict import cazy_synonym_dict
 import pytest
 import sys
 
 from argparse import Namespace, ArgumentParser
 
-from scraper import cazy_webscraper, crawler, sql, utilities
-from scraper.crawler.cazy_html_pages import get_cazy_pages, parse_local_pages
-from scraper.crawler.parse_cazy_families import scrape_all, scrape_by_kingdom
-from scraper.utilities import file_io, parse_configuration, parsers
+from cazy_webscraper import cazy_scraper, crawler, sql, utilities
+from cazy_webscraper.crawler.cazy_html_pages import get_cazy_pages, parse_local_pages
+from cazy_webscraper.crawler.parse_cazy_families import scrape_all, scrape_by_kingdom
+from cazy_webscraper.sql import sql_interface, sql_orm
+from cazy_webscraper.utilities import file_io, parse_configuration, parsers
+
+
+@pytest.fixture
+def mock_building_parser(*args, **kwargs):
+    parser_args = ArgumentParser(
+        prog="cazy_webscraper.py",
+        usage=None,
+        description="Scrape the CAZy database",
+        conflict_handler="error",
+        add_help=True,
+    )
+    return parser_args
+
+
+@pytest.fixture
+def mock_parser(*args, **kwargs):
+    parser = Namespace(
+        config=None,
+        classes=None,
+        database="fake_database_path",
+        ec=None,
+        force=False,
+        families=None,
+        genera=None,
+        get_pages=True,
+        kingdoms=None,
+        log=None,
+        nodelete=False,
+        output=output_dir,
+        retries=10,
+        scrape_files=None,
+        subfamilies=True,
+        species=None,
+        strains=None,
+        streamline=None,
+        timeout=45,
+        verbose=False,
+    )
+    return parser
+
+
+@pytest.fixture
+def mock_config_logger(*args, **kwargs):
+    return
 
 
 @pytest.fixture
@@ -125,445 +171,435 @@ def config_dict():
 # test main()
 
 
-def test_main_get_pages(output_dir, cazy_dictionary, taxonomic_filter_dict, monkeypatch):
-    """Test function main() when retrieval of CAZy HTML pages is enabled.
-
-    Argv is None, logger is None, args.output is not sys.stdout, args.subfamilies is True.
-    """
-
-    def mock_building_parser(*args, **kwargs):
-        parser_args = ArgumentParser(
-            prog="cazy_webscraper.py",
-            usage=None,
-            description="Scrape the CAZy database",
-            conflict_handler="error",
-            add_help=True,
-        )
-        return parser_args
+def test_main_citation(mock_building_parser, mock_config_logger, monkeypatch):
+    """Test main() with args.citation"""
 
     def mock_parser(*args, **kwargs):
         parser = Namespace(
-            config=None,
+            email="dummy@domain.com",
+            cache_dir=None,
+            cazy_data=None,
+            cazy_synonyms=None,
             classes=None,
-            database="fake_database_path",
-            ec=None,
+            config=None,
+            citation=True,
+            db_output=None,
+            database=None,
+            delete_old_relationships=False,
             force=False,
             families=None,
             genera=None,
-            get_pages=True,
             kingdoms=None,
             log=None,
             nodelete=False,
-            output=output_dir,
+            nodelete_cache=False,
+            nodelete_log=False,
             retries=10,
-            scrape_files=None,
-            subfamilies=True,
+            sql_echo=False,
+            subfamilies=False,
             species=None,
             strains=None,
-            streamline=None,
             timeout=45,
-            verbose=False,
+            validate=False,
+            verbse=False,
+            version=False,
         )
         return parser
 
-    def mock_config_logger(*args, **kwargs):
-        return
-
-    def mock_making_output_dir(*args, **kwargs):
-        return
-
-    def mock_retrieving_configuration(*args, **kwargs):
-        # excluded_classes, config_dict, cazy_dict, taxonomy_filter, kingdoms, ec_filter
-        return None, None, cazy_dictionary, taxonomic_filter_dict, [], []
-
-    def mock_filter_set(*args, **kwargs):
-        return set()
-
-    def mock_get_pages(*args, **kwargs):
-        return
-
     monkeypatch.setattr(parsers, "build_parser", mock_building_parser)
     monkeypatch.setattr(ArgumentParser, "parse_args", mock_parser)
     monkeypatch.setattr(utilities, "config_logger", mock_config_logger)
-    monkeypatch.setattr(file_io, "make_output_directory", mock_making_output_dir)
-    monkeypatch.setattr(parse_configuration, "parse_configuration", mock_retrieving_configuration)
-    monkeypatch.setattr(cazy_webscraper, "get_filter_set", mock_filter_set)
-    monkeypatch.setattr(get_cazy_pages, "get_cazy_pages", mock_get_pages)
-
-    cazy_webscraper.main()
 
+    cazy_scraper.main()
 
-def test_main_invalid_db_path(output_dir, cazy_dictionary, taxonomic_filter_dict, monkeypatch):
-    """Test function main() when an invalid db path is given.
 
-    Argv is None, logger is None, args.output is not sys.stdout, args.subfamilies is True.
-    """
-
-    def mock_building_parser(*args, **kwargs):
-        parser_args = ArgumentParser(
-            prog="cazy_webscraper.py",
-            usage=None,
-            description="Scrape the CAZy database",
-            conflict_handler="error",
-            add_help=True,
-        )
-        return parser_args
+def test_main_version(mock_building_parser, mock_config_logger, monkeypatch):
+    """Test main() with args.version"""
 
     def mock_parser(*args, **kwargs):
         parser = Namespace(
-            config=None,
+            email="dummy@domain.com",
+            cache_dir=None,
+            cazy_data=None,
+            cazy_synonyms=None,
             classes=None,
-            database="fake_database_path",
-            ec=None,
+            config=None,
+            citation=False,
+            db_output=None,
+            database=None,
+            delete_old_relationships=False,
             force=False,
             families=None,
             genera=None,
-            get_pages=False,
             kingdoms=None,
             log=None,
             nodelete=False,
-            output=output_dir,
+            nodelete_cache=False,
+            nodelete_log=False,
             retries=10,
-            scrape_files=None,
-            subfamilies=True,
+            sql_echo=False,
+            subfamilies=False,
             species=None,
             strains=None,
-            streamline=None,
             timeout=45,
-            verbose=False,
+            validate=False,
+            verbse=False,
+            version=True,
         )
         return parser
 
-    def mock_config_logger(*args, **kwargs):
-        return
-
-    def mock_making_output_dir(*args, **kwargs):
-        return
-
-    def mock_retrieving_configuration(*args, **kwargs):
-        # excluded_classes, config_dict, cazy_dict, taxonomy_filter, kingdoms, ec_filter
-        return None, None, cazy_dictionary, taxonomic_filter_dict, [], []
-
-    def mock_filter_set(*args, **kwargs):
-        return set()
-
     monkeypatch.setattr(parsers, "build_parser", mock_building_parser)
     monkeypatch.setattr(ArgumentParser, "parse_args", mock_parser)
     monkeypatch.setattr(utilities, "config_logger", mock_config_logger)
-    monkeypatch.setattr(file_io, "make_output_directory", mock_making_output_dir)
-    monkeypatch.setattr(parse_configuration, "parse_configuration", mock_retrieving_configuration)
-    monkeypatch.setattr(cazy_webscraper, "get_filter_set", mock_filter_set)
-
-    with pytest.raises(SystemExit) as pytest_wrapped_e:
-        cazy_webscraper.main()
-    assert pytest_wrapped_e.type == SystemExit
 
+    cazy_scraper.main()
 
-def test_main_db_raises_error(output_dir, cazy_dictionary, taxonomic_filter_dict, monkeypatch):
-    """Test function main() when aan error is raised when building the local db."""
 
-    def mock_building_parser(*args, **kwargs):
-        parser_args = ArgumentParser(
-            prog="cazy_webscraper.py",
-            usage=None,
-            description="Scrape the CAZy database",
-            conflict_handler="error",
-            add_help=True,
-        )
-        return parser_args
+def test_main_version_argv(mock_building_parser, mock_config_logger, monkeypatch):
+    """Test main() with args.version"""
 
     def mock_parser(*args, **kwargs):
         parser = Namespace(
-            config=None,
+            email="dummy@domain.com",
+            cache_dir=None,
+            cazy_data=None,
+            cazy_synonyms=None,
             classes=None,
+            config=None,
+            citation=False,
+            db_output=None,
             database=None,
-            ec=None,
+            delete_old_relationships=False,
             force=False,
             families=None,
             genera=None,
-            get_pages=False,
             kingdoms=None,
             log=None,
             nodelete=False,
-            output=output_dir,
+            nodelete_cache=False,
+            nodelete_log=False,
             retries=10,
-            scrape_files=None,
-            subfamilies=True,
+            sql_echo=False,
+            subfamilies=False,
             species=None,
             strains=None,
-            streamline=None,
             timeout=45,
-            verbose=False,
+            validate=False,
+            verbse=False,
+            version=True,
         )
         return parser
 
-    def mock_config_logger(*args, **kwargs):
-        return
-
-    def mock_making_output_dir(*args, **kwargs):
-        return
-
-    def mock_retrieving_configuration(*args, **kwargs):
-        # excluded_classes, config_dict, cazy_dict, taxonomy_filter, kingdoms, ec_filter
-        return None, None, cazy_dictionary, taxonomic_filter_dict, [], []
-
-    def mock_filter_set(*args, **kwargs):
-        return set()
+    monkeypatch.setattr(parsers, "build_parser", mock_building_parser)
+    monkeypatch.setattr(ArgumentParser, "parse_args", mock_parser)
+    monkeypatch.setattr(utilities, "config_logger", mock_config_logger)
 
-    def mock_db_build(*args, **kwargs):
-        raise TypeError
+    cazy_scraper.main()
+
+
+def test_main_double_db(mock_building_parser, mock_config_logger, monkeypatch):
+    """Test main() with args.database and args.db_output"""
+
+    argv = [
+        "dummy@domain.com",
+        None,
+        None,
+        None,
+        None,
+        None,
+        False,
+        'database',
+        'database',
+        False,
+        False,
+        None,
+        None,
+        None,
+        None,
+        False,
+        False,
+        False,
+        10,
+        False,
+        False,
+        None,
+        None,
+        45,
+        False,
+        False,
+        True,
+    ]
+    def mock_parser(*args, **kwargs):
+        parser = Namespace(
+            email="dummy@domain.com",
+            cache_dir=None,
+            cazy_data=None,
+            cazy_synonyms=None,
+            classes=None,
+            config=None,
+            citation=False,
+            db_output='database',
+            database='database',
+            delete_old_relationships=False,
+            force=False,
+            families=None,
+            genera=None,
+            kingdoms=None,
+            log=None,
+            nodelete=False,
+            nodelete_cache=False,
+            nodelete_log=False,
+            retries=10,
+            sql_echo=False,
+            subfamilies=False,
+            species=None,
+            strains=None,
+            timeout=45,
+            validate=False,
+            verbse=False,
+            version=True,
+        )
+        return parser
 
     monkeypatch.setattr(parsers, "build_parser", mock_building_parser)
     monkeypatch.setattr(ArgumentParser, "parse_args", mock_parser)
     monkeypatch.setattr(utilities, "config_logger", mock_config_logger)
-    monkeypatch.setattr(file_io, "make_output_directory", mock_making_output_dir)
-    monkeypatch.setattr(parse_configuration, "parse_configuration", mock_retrieving_configuration)
-    monkeypatch.setattr(cazy_webscraper, "get_filter_set", mock_filter_set)
-    monkeypatch.setattr(sql.sql_orm, "build_db", mock_db_build)
 
-    with pytest.raises(SystemExit) as pytest_wrapped_e:
-        cazy_webscraper.main()
-    assert pytest_wrapped_e.type == SystemExit
+    cazy_scraper.main(argv=argv)
 
 
-def test_main_existing_db_scrape_local_pages(
-    output_dir,
-    null_logger,
-    cazy_dictionary,
-    db_path,
-    input_dir,
-    monkeypatch,
-):
-    """Test function main() when passed an existing database scraping data from local HTML pages.
-
-    Argv is not None, logger is not None, args.output is output_dir, args.subfamilies is True,
-    and valid db path is given by db_path.
-    """
-
-    def mock_building_parser(*args, **kwargs):
-        parser_args = ArgumentParser(
-            prog="cazy_webscraper.py",
-            usage=None,
-            description="Scrape the CAZy database",
-            conflict_handler="error",
-            add_help=True,
-        )
-        return parser_args
+def test_main_new_db_exists_force(db_path, mock_building_parser, mock_config_logger, monkeypatch):
+    """Test main() with args.db_output and the database already exists"""
 
     def mock_parser(*args, **kwargs):
         parser = Namespace(
-            config=None,
+            email="dummy@domain.com",
+            cache_dir=None,
+            cazy_data=None,
+            cazy_synonyms=None,
             classes=None,
-            database=db_path,
-            ec=None,
-            force=False,
+            config=None,
+            citation=False,
+            db_output=db_path,  ###
+            database=None,
+            delete_old_relationships=False,
+            force=True,   ###
             families=None,
             genera=None,
-            get_pages=False,
             kingdoms=None,
             log=None,
             nodelete=False,
-            output=output_dir,
+            nodelete_cache=False,
+            nodelete_log=False,
             retries=10,
-            scrape_files=input_dir,
-            subfamilies=True,
+            sql_echo=False,
+            subfamilies=False,
             species=None,
             strains=None,
-            streamline="streamline_args",
             timeout=45,
-            verbose=False,
+            validate=False,
+            verbse=False,
+            version=True,
         )
         return parser
 
-    def mock_config_logger(*args, **kwargs):
-        return
+    monkeypatch.setattr(parsers, "build_parser", mock_building_parser)
+    monkeypatch.setattr(ArgumentParser, "parse_args", mock_parser)
+    monkeypatch.setattr(utilities, "config_logger", mock_config_logger)
 
-    def mock_making_output_dir(*args, **kwargs):
-        return
+    cazy_scraper.main()
 
-    def mock_retrieving_configuration(*args, **kwargs):
-        # excluded_classes, config_dict, cazy_dict, taxonomy_filter, kingdoms, ec_filter
-        return None, None, cazy_dictionary, taxonomic_filter_dict, [], []
 
-    def mock_filter_set(*args, **kwargs):
-        return set()
+def test_main_new_db_exists(db_path, mock_building_parser, mock_config_logger, monkeypatch):
+    """Test main() with args.db_output and the database already exists"""
 
-    def mock_none(*args, **kwargs):
-        return
+    def mock_parser(*args, **kwargs):
+        parser = Namespace(
+            email="dummy@domain.com",
+            cache_dir=None,
+            cazy_data=None,
+            cazy_synonyms=None,
+            classes=None,
+            config=None,
+            citation=False,
+            db_output=db_path,  ###
+            database=None,
+            delete_old_relationships=False,
+            force=False,   ###
+            families=None,
+            genera=None,
+            kingdoms=None,
+            log=None,
+            nodelete=False,
+            nodelete_cache=False,
+            nodelete_log=False,
+            retries=10,
+            sql_echo=False,
+            subfamilies=False,
+            species=None,
+            strains=None,
+            timeout=45,
+            validate=False,
+            verbse=False,
+            version=True,
+        )
+        return parser
 
     monkeypatch.setattr(parsers, "build_parser", mock_building_parser)
     monkeypatch.setattr(ArgumentParser, "parse_args", mock_parser)
     monkeypatch.setattr(utilities, "config_logger", mock_config_logger)
-    monkeypatch.setattr(file_io, "make_output_directory", mock_making_output_dir)
-    monkeypatch.setattr(parse_configuration, "parse_configuration", mock_retrieving_configuration)
-    monkeypatch.setattr(cazy_webscraper, "get_filter_set", mock_filter_set)
-    monkeypatch.setattr(parse_configuration, "create_streamline_scraping_warning", mock_none)
-    monkeypatch.setattr(sql.sql_interface, "log_scrape_in_db", mock_none)
-    monkeypatch.setattr(parse_local_pages, "parse_local_pages", mock_none)
 
-    cazy_webscraper.main(["argv"])
+    cazy_scraper.main()
 
 
-def test_main_existing_db_scrape_direct(
-    output_dir,
-    null_logger,
-    cazy_dictionary,
+def test_main_database(
     db_path,
-    input_dir,
+    mock_building_parser,
+    mock_config_logger,
+    config_dict,
     monkeypatch,
 ):
-    """Test function main() when passed an existing database scraping CAZy directly.
-
-    Argv is not None, logger is not None, args.output is output_dir, args.subfamilies is True,
-    and valid db path is given by db_path.
-    """
-
-    def mock_building_parser(*args, **kwargs):
-        parser_args = ArgumentParser(
-            prog="cazy_webscraper.py",
-            usage=None,
-            description="Scrape the CAZy database",
-            conflict_handler="error",
-            add_help=True,
-        )
-        return parser_args
+    """Test main() with args.db_output and the database already exists"""
 
     def mock_parser(*args, **kwargs):
         parser = Namespace(
-            config=None,
+            email="dummy@domain.com",
+            cache_dir='cache',
+            cazy_data=None,
+            cazy_synonyms=None,
             classes=None,
-            database=db_path,
-            ec=None,
+            config=None,
+            citation=False,
+            db_output=None,
+            database=db_path,  ###
+            delete_old_relationships=False,
             force=False,
             families=None,
             genera=None,
-            get_pages=False,
             kingdoms=None,
-            log=None,
+            log='log_dir',
             nodelete=False,
-            output=output_dir,
+            nodelete_cache=False,
+            nodelete_log=False,
             retries=10,
-            scrape_files=None,
-            subfamilies=True,
+            sql_echo=False,
+            subfamilies=False,
             species=None,
             strains=None,
-            streamline="streamline_args",
             timeout=45,
-            verbose=False,
+            validate=False,
+            verbse=False,
+            version=True,
         )
         return parser
 
-    def mock_config_logger(*args, **kwargs):
-        return
-
-    def mock_making_output_dir(*args, **kwargs):
-        return
-
-    def mock_retrieving_configuration(*args, **kwargs):
-        # excluded_classes, config_dict, cazy_dict, taxonomy_filter, kingdoms, ec_filter
-        return None, None, cazy_dictionary, taxonomic_filter_dict, [], []
+    def mock_parse_config(*args, **kwards):
+        return (
+            None,
+            config_dict,
+            {},
+            {'GH'},
+            {'CE1'},
+            {'Bacteria'},
+            {'genus': 'a genus'},
+            {'a genus', 'a species'}
+        )
 
-    def mock_filter_set(*args, **kwargs):
-        return set()
+    def mock_connect_existing_db(*args, **kwards):
+        return (
+            'connection',
+            'logger_name',
+            'cache_dir',
+        )
 
-    def mock_none(*args, **kwargs):
+    def mock_log_scrape_in_db(*args, **kwargs):
         return
 
     monkeypatch.setattr(parsers, "build_parser", mock_building_parser)
     monkeypatch.setattr(ArgumentParser, "parse_args", mock_parser)
     monkeypatch.setattr(utilities, "config_logger", mock_config_logger)
-    monkeypatch.setattr(file_io, "make_output_directory", mock_making_output_dir)
-    monkeypatch.setattr(parse_configuration, "parse_configuration", mock_retrieving_configuration)
-    monkeypatch.setattr(cazy_webscraper, "get_filter_set", mock_filter_set)
-    monkeypatch.setattr(parse_configuration, "create_streamline_scraping_warning", mock_none)
-    monkeypatch.setattr(sql.sql_interface, "log_scrape_in_db", mock_none)
-    monkeypatch.setattr(cazy_webscraper, "get_cazy_data", mock_none)
-
-    cazy_webscraper.main(["argv"])
-
-
-def test_main_new_database(output_dir, cazy_dictionary, monkeypatch):
-    """Test main() when a new database_file is created"""
-
-    def mock_building_parser(*args, **kwargs):
-        parser_args = ArgumentParser(
-            prog="cazy_webscraper.py",
-            usage=None,
-            description="Scrape the CAZy database",
-            conflict_handler="error",
-            add_help=True,
-        )
-        return parser_args
+    monkeypatch.setattr(parse_configuration, "parse_configuration", mock_parse_config)
+    monkeypatch.setattr(cazy_scraper, "connect_existing_db", mock_connect_existing_db)
+    monkeypatch.setattr(sql.sql_interface, "log_scrape_in_db", mock_log_scrape_in_db)
+
+    cazy_scraper.main()
 
-    output_path = output_dir / "test_webscraper" / "temp_dir_for_db"
-    os.makedirs(output_path, exist_ok=True)
+
+def test_main_db_output(
+    db_path,
+    mock_building_parser,
+    mock_config_logger,
+    config_dict,
+    monkeypatch,
+):
+    """Test main() with args.db_output and the database already exists"""
 
     def mock_parser(*args, **kwargs):
         parser = Namespace(
-            config=None,
+            email="dummy@domain.com",
+            cache_dir=None,
+            cazy_data=None,
+            cazy_synonyms=None,
             classes=None,
+            config=None,
+            citation=False,
+            db_output=db_path,  ###
             database=None,
-            ec=None,
-            force=True,
+            delete_old_relationships=False,
+            force=False,
             families=None,
             genera=None,
-            get_pages=False,
             kingdoms=None,
             log=None,
             nodelete=False,
-            output=output_path,
+            nodelete_cache=False,
+            nodelete_log=False,
             retries=10,
-            scrape_files=None,
-            subfamilies=True,
+            sql_echo=False,
+            subfamilies=False,
             species=None,
             strains=None,
-            streamline="streamline_args",
             timeout=45,
-            verbose=False,
+            validate=False,
+            verbse=False,
+            version=True,
         )
         return parser
 
-    def mock_config_logger(*args, **kwargs):
-        return
-
-    def mock_retrieving_configuration(*args, **kwargs):
-        # excluded_classes, config_dict, cazy_dict, taxonomy_filter, kingdoms, ec_filter
-        return None, None, cazy_dictionary, taxonomic_filter_dict, [], []
+    def mock_parse_config(*args, **kwards):
+        return (
+            None,
+            config_dict,
+            {},
+            {'GH'},
+            {'CE1'},
+            {'Bacteria'},
+            {'genus': 'a genus'},
+            {'a genus', 'a species'}
+        )
 
-    def mock_filter_set(*args, **kwargs):
-        return set()
+    def mock_connect_existing_db(*args, **kwards):
+        return (
+            'connection',
+            'logger_name',
+            'cache_dir',
+        )
 
-    def mock_none(*args, **kwargs):
+    def mock_log_scrape_in_db(*args, **kwargs):
         return
 
     monkeypatch.setattr(parsers, "build_parser", mock_building_parser)
     monkeypatch.setattr(ArgumentParser, "parse_args", mock_parser)
     monkeypatch.setattr(utilities, "config_logger", mock_config_logger)
-    monkeypatch.setattr(parse_configuration, "parse_configuration", mock_retrieving_configuration)
-    monkeypatch.setattr(cazy_webscraper, "get_filter_set", mock_filter_set)
-    monkeypatch.setattr(parse_configuration, "create_streamline_scraping_warning", mock_none)
-    monkeypatch.setattr(sql.sql_interface, "log_scrape_in_db", mock_none)
-    monkeypatch.setattr(cazy_webscraper, "get_cazy_data", mock_none)
-
-    cazy_webscraper.main(["argv"])
-
-    # delete newly build db
-    file_io.make_output_directory(output_dir, True, False)
-
+    monkeypatch.setattr(parse_configuration, "parse_configuration", mock_parse_config)
+    monkeypatch.setattr(cazy_scraper, "connect_to_new_db", mock_connect_existing_db)
+    monkeypatch.setattr(sql.sql_interface, "log_scrape_in_db", mock_log_scrape_in_db)
 
-# test get_filter_set
+    cazy_scraper.main()
 
 
-def test_get_filter_set():
-    """Test get_filter_set."""
-    taxonomy_filter = {"genera": ["Aspergillus", "Trichoderma"], "strains": []}
-    cazy_webscraper.get_filter_set(taxonomy_filter)
 
 
-def test_get_filter_set_none():
-    """Test get_filter_set when no filters are provided."""
-    taxonomy_filter = {}
-    assert cazy_webscraper.get_filter_set(taxonomy_filter) is None
 
 
 # test get_cazy_data()