Package: rcsb.exdb
py-rcsb_exdbRCSB exchange database extraction and loading workflow toolsIntroductionThis module contains a collection of utility classes for extracting data from
the RCSB exchange database and subsequently reloading processed or integrated data.InstallationDownload the library source software from the project repository:gitclone--recurse-submoduleshttps://github.com/rcsb/py-rcsb_exdb.gitOptionally, run test suite (Python versions 2.7 and 3.7) usingsetuptoolsortox:pythonsetup.pytestorsimplyrun

toxInstallation is via the programpip.  To run tests
from the source tree, the package must be installed in editable mode (i.e. -e):pipinstall-rrequirements.txt# OR:   pip install -i https://pypi.anaconda.org/OpenEye/simple OpenEye-toolkitspipinstall-e.Installing in Ubuntu Linux (tested in 18.04)You will need a few packages, beforepip install .can work:sudoaptinstallflexbisonInstalling on macOSTo use and develop this package on macOS requires a number of packages that are not
distributed as part of the base macOS operating system.
The following steps provide one approach to creating the development environment for this
package.  First, install the AppleXCodepackage and associate command-line tools.
This will provide essential compilers and supporting tools.  TheHomeBrewpackage
manager provides further access to a variety of common open source services and tools.
Follow the instructions provided by at theHomeBrewsite to
install this system.   Once HomeBrew is installed, you can further install theMongoDBpackages which
are required to support the ExDB tools.  HomeBrew also provides a variety of options for
managing aPython virtual environments.Command Line InterfacesA convenience CLIexdb_exec_cliis provided for performing update and loading operations.exdb_exec_cli--help

usage:exdb_exec_cli[-h][--data_set_idDATA_SET_ID][--full][--etl_chemref][--etl_tree_node_lists][--config_pathCONFIG_PATH][--config_nameCONFIG_NAME][--db_typeDB_TYPE][--read_back_check][--num_procNUM_PROC][--chunk_sizeCHUNK_SIZE][--document_limitDOCUMENT_LIMIT][--debug][--mock][--cache_pathCACHE_PATH][--rebuild_cache]optionalarguments:-h,--helpshowthishelpmessageandexit--data_set_idDATA_SET_IDDatasetidentifier(default=2019_14forcurrentweek)--fullFreshfullloadinanewtables/collections(Default)--etl_chemrefETLintegratedchemicalreferencedata--etl_tree_node_listsETLtreenodelists--config_pathCONFIG_PATHPathtoconfigurationoptionsfile--config_nameCONFIG_NAMEConfigurationsectionname--db_typeDB_TYPEDatabaseservertype(default=mongo)--read_back_checkPerformreadbackcheckonalldocuments--num_procNUM_PROCNumberofprocessestoexecute(default=2)--chunk_sizeCHUNK_SIZENumberoffilesloadedperprocess--document_limitDOCUMENT_LIMITLoaddocumentlimitfortesting--debugTurnonverboselogging--mockUseMOCKrepositoryconfigurationfortesting--cache_pathCACHE_PATHTopcachepathforexternalandlocalresourcefiles--rebuild_cacheRebuildcachedfilesfromremoteresources
________________________________________________________________________________For example, to construct and load tree nodes list data collections, the following
command may be used:exdb_exec_cli--mock--full--etl_tree_node_lists--rebuild_cache\--cache_path./CACHE\--config_path./rcsb/mock-data/config/dbload-setup-example.yml\--config_namesite_info_configuration>&LOGTREE\