Package: datacube-explorer
Data Cube ExplorerUsage (quick-start)Assuming you already have an Open Data Cube instance, Explorer will use
its existing settings.Install Explorer:pip install datacube-explorerGenerate summaries for all of your products:cubedash-gen --init --allRun Explorer locally:cubedash-runIt will now be viewable onhttp://localhost:8090Developer SetupThese directions are for running from a local folder in development. But it will run from any typical Python WSGI server.Firstly, install the Open Data Cube. Use of aData Cube conda environmentis recommended. You may need to alsoconda install -c conda-forge postgisTest that you can rundatacube system check, and that it's connecting
to the correct datacube instance.DependenciesNow install the explorer dependencies:# These two should come from conda if you're using it, not pypi
conda install fiona shapely

pip install -e .Summary generationInitialise and create product summaries:cubedash-gen --init --all(This can take a long time the first time, depending on your datacube size.)Other available options can be seen by runningcubedash-gen --help.RunAcubedash-runcommand is available to run Explorer locally:$ cubedash-run
    * Running on http://localhost:8080/ (Press CTRL+C to quit)(seecubedash-run --helpfor list of options)But Explorer can be run using any typical Python WSGI server, for examplegunicorn:pip install gunicorn
gunicorn -b '127.0.0.1:8080' -w 4 cubedash:appProducts will begin appearing one-by-one as the summaries are generated in the
background.  If impatient, you can manually navigate to a product using/<product_name. (Eg/ls5_nbar_albers)Code StyleAll code is formatted usingblack, and checked
withpyflakes.They are included when installing the test dependencies:pip install --upgrade --no-deps --extra-index-url https://packages.dea.ga.gov.au/ 'datacube' 'digitalearthau'

pip install -e .[test]Runmake lintto check your changes, andmake formatto format your code
automatically.You may want to configure your editor to run black automatically on file save
(see the Black page for directions), or install the pre-commit hook within Git:Pre-commit setupApre-commitconfig is provided to automatically format
and check your code changes. This allows you to immediately catch and fix
issues before you raise a failing pull request (which run the same checks under
Travis).Install pre-commit from pip, and initialise it in your repo:pip install pre-commit
pre-commit installYour code will now be formatted and validated before each commit. You can also
invoke it manually by runningpre-commit runNote: If you use Conda, install from conda-forge (This isrequiredbecause the pip
version uses virtualenvs which are incompatible with Conda's environments)conda install pre_commitFAQCan I use a different datacube environment?Set ODC's environment variable before running the server:export DATACUBE_ENVIRONMENT=stagingYou can always see which environment/settings will be used by runningdatacube system check.See the ODC documentation for config anddatacube environmentsHow can I set different timezoneDatacube-explorer default timezone is configured to:Australia/Darwin.To configure the instance to a different timezone, the following configuration needs to be applied:os.environmentvariableCUBEDASH_DEFAULT_TIMEZONEapp.configvariableCUBEDASH_DEFAULT_TIMEZONECan I add custom scripts or text to the page (such as analytics)?Create one of the following*.env.htmlfiles:Global include: for<script>and other tags at the bottom of every page.cubedash/templates/include-global.env.htmlFooter text include. For human text such as Copyright statements.echo "Server <strong>staging-1.test</strong>" > cubedash/templates/include-footer.env.html(*.env.htmlis the naming convention used for environment-specific templates: they are ignored by
Git)How can I configure the deployment?refer todeployment instructionsandapp-configHow do I modify the CSS/Javascript?The CSS is compiled fromSass, and the Javascript is compiled fromTypescript.Installnpm, and then install them both:npm install -g sass typescriptYou can now runmake staticto rebuild all the static files, or
individually withmake styleormake js.Alternatively, if usingPyCharm, open a
Sass file and you will be prompted to enable aFile Watcherto
compile automatically.PyCharm will also compile the Typescript automatically by ticking
the "Recompile on changes" option inLanguages & Frameworks -> Typescript.How do I run the integration tests?The integration tests run against a real PostgreSQL database, which is
automatically started and stopped using Docker. This requires Docker to
be available, but no further database setup is required.Install the test dependencies:pip install -e .[test]The run the tests with:pytest integration_testsHow do I add test data for the automated tests?Most of the automated tests for Datacube Explorer require sample data to run. This comprises
definitions of ODCMetadata Types,ProductsandDatasets.These are contained within YAML files in theintegration_tests/datadirectory.Test data is loaded using a pytest fixture calledauto_odc_db, which is activated per
test module, and will automatically populate the database using files referenced in module
global variables. Activate and use it similar to the following example:pytestmark = pytest.mark.usefixtures("auto_odc_db")

METADATA_TYPES = ["metadata/qga_eo.yaml"]
PRODUCTS = ["products/ga_s2_ard.odc-product.yaml"]
DATASETS = ["s2a_ard_granule.yaml.gz"]To add sample datasets required for the test case, create a.yamlfile
with the product name and place all the sample datasets split by---in the yaml.If the sample datasets file is large, compress it withgzip <dataset_file>.yamland reference
that file instead.Roles for production deploymentsTherolesdirectory contains sql files for creating
Postgres roles for Explorer. These are suitable for running each Explorer
task with minimum needed security permissions.Three roles are created:explorer-viewer: A read-only user of datacube and Explorer. Suitable for the web interface and cli (cubedash-view) commands.explorer-generator: Suitable for generating and updating summaries (ie. Runningcubedash-gen)explorer-owner: For creating and updating the schema. (ie. Runningcubedash-gen --init)Note that these roles extend the built-in datacube roleagdc_user. If you
created your datacube without permissions, a stand-alone creator of theagdc_userrole is available as a prerequisite in the samerolesdirectory.Docker for Development and running testsYou need to have Docker and Docker Compose installed on your system.To create your environment, runmake upordocker-compose up.You need an ODC database, so you'll need to refer to theODC docsfor help on indexing, but you can create the database by runningmake initdbordocker-compose exec explorer datacube system init. (This is not enough, you still need to add a product and index datasets.)When you have some ODC data indexed, you can runmake indexto create the Explorer indexes.Once Explorer indexes have been created, you can browse the running application athttp://localhost:5000.You can run tests by first creating a test databasemake create-test-db-dockerand then running tests withmake test-docker.And you can run a single test in Docker using a command like this:docker-compose --file docker-compose.yml run explorer pytest integration_tests/test_dataset_listing.pyDocker-compose for Development and running testsTesting with app.configedit.docker/settings_docker.pyand setup application config. Thendocker-compose -f docker-compose.yml -f docker-compose.override.yml upto bring up explorer docker with database, explorer with settingsSTAC API ExtensionsThe STAC endpoint implements thequery,filter,fields, andsortextensions, all of which are bound to thesearchendpoint as used with POST requests, with fields and sort additionally bound to the features endpoint.Fields contained in the item properties must be prefixed withproperties., exproperties.dea:dataset_maturity.The implementation offieldsdiffers somewhat from the suggested include/exclude semantics in that it does not permit for invalid STAC entities, so theid,type,geometry,bbox,links,assets,properties.datetime, andstac_versionfields will always be included, regardless of user input.The implementation offilteris limited, and currently only supports CQL2 JSON syntax with the following basic CQL2 operators:AND,OR,=,>,>=,<,<=,<>,IS NULL.