Package: eumdac-tmp
EUMDAC - EUMETSAT Data Access ClientEUMDACis theEUMETSATDataAccessClientIt provides simple access to the EUMETSAT data of all satellite missions. As a Python library, it comes with many methods and helpers to use EUMETSATs APIs and services, like Data Store and Data Tailor. It also provides a variety of useful command line utilities for data search, translation and processing.PrerequisitesYou will need a python environment to run the library implementation of this code. EUMDAC requires Python 3.7 or higher. We recommend that you install the latest Anaconda Python distribution for your operating system (https://www.anaconda.com/).Dependenciesrequests,   ver. 2.26.0, License: Apache-2.0 (LICENSE_APACHE_v2.txt), Copyright 2014 Kenneth Reitz,   info:https://anaconda.org/conda-forge/requestsresponses,  ver. 0.16.0, License: Apache-2.0 (LICENSE_APACHE_v2.txt), Copyright 2015 David Cramer,    info:https://anaconda.org/conda-forge/responsessetuptools, ver. 58.0.4, License: MIT (LICENSE_MIT.txt),              Copyright 2020 Jason R. Coombs, info:https://anaconda.org/conda-forge/setuptoolsInstalling EUMDACInstalling with PIPThe EUMDAC Python package is available throughPyPI:pipinstalleumdacInstalling with CondaTo install EUMDAC on the Anaconda Python distribution, please visit theEUMETSAT conda-forge pagefor install instructions.condainstall-ceumetsat-forgeeumdacInstalling from sourceTo install EUMDAC from the development source, clone the repository and install it locally.gitclonehttps://gitlab.eumetsat.int/dso/dso_usr_sup/eumdac.gitcdeumdac
pipinstall.AuthorsCarlos Horn-EUMETSATBen Loveday-EUMETSATNiklas Jordan-EUMETSATPaulo Carmo-EUMETSATRafa de la Hoz-EUMETSATPlease see the AUTHORS.txt file for more information.Developer NotesPlease note that the points in this section are only relevant if you can access the development repository. They do not apply to access via thepublic mirror.ContributingIf you feel like something is missing, should work differently or you find a bug ineumdac, pleaseopen an issue on gitlaband select the
respective template for feature requests, change requests, or bug reports and fill out the sections.Become a developerIf you would like to propose an implementation for an issue, please get in contact with the project maintainer,Rafael de la Hoz Sevilla, to become a developer on the gitlab project.Then create a new branch with a meaningful name relating the branch with the issue, e.g.feature_download_progress_indicator.
If you want to get feedback or discuss design choices of your implementation, while the branch is still under active development, you can already create aDraftmerge request. Any implementation related conversation should take place in corresponding merge requests.ADraftMR will already enable gitlab to run style checks and simplified unit testing on your work. Once, your development reaches a certain degree of maturity, where the implementation actually solves the issue, you can remove theDraftstatus. This enables enhanced code testing and signals the maintainer, that the branch is ready for review. If the reviewers ask for larger changes, you can always change the MR back toDraftfor the implementation.If your MR passes all tests and is accepted by the reviewers, the final decision on when or at all to merge is on the maintainer (product owner). Please be aware, that there could be merge conflicts with other MRs or simply just other solution candidates.TestingYou can avoid failing pipelines by local testing. We are usingflake8,blackandpytest, which you can already use locally.pipinstall.[test]blackflake8
flake8eumdac/tests/setup.py
black-l100eumdac/tests/setup.py
pytest--coveumdac--cov-report=term-missing--full-traceOr you can usetoxandtox-condato run the entire testing pipeline.Mocking in TestsIn Tests, we avoid requests hitting the real Data Services web API, because the test suite should be independent of the actual availability of the web services in order to be fast and reproducible. Since the actual requests are hidden behind the object methods and properties, there are two strategies. We can either mock the low level object doing the requests, or we can mock the responses by the web API.For the first case, it might be useful to know, that all objects are lazy loading, which means that the instance creation will not trigger any request. Relevant request are only done on demand and populate some private attributes. This can be used to mock those objects by populating these private attributes
with mocked values.Example# mocking the tokencredentials=("abc","xyz")token=AccessToken(credentials)token._access_token="mock_token_value"token._expiration=time.time()+1000asserttoken.access_token=="mock_token_value"# mocking a Collectiondatastore=DataStore(token)my_collection=Collection("MY-COLLECTION-ID",datastore)my_collection._properties={"title":"THE COLLECTION TITLE"}assertmy_collection.title=="THE COLLECTION TITLE"For the letter case, we use the libraryresponseswhich allows to register predefined responses for therequestslibrary and prevents unregistered requests to pass through. To be more precise, we have a customunittest.TestCaseimplementation which has the main purpose of recording and replaying responses from the real web services. The idea is to dual-use these tests for integration tests by hitting the real endpoints and for unit tests by replaying the recorded responses. Still there might be some responses, which cannot be reproduced on demand from the real API, e.g. returning a 500 error code by the server when the service is down. For those cases, a test can be excluded when recording and register handcrafted responses.ExampleclassMyTest(DataServiceTestCase):deftest_something(self):# this is intended to hit real endpoints...@unittest.skipIf(INTEGRATION_TESTING,"Reason for sipping, e.g. irreproducible on demand!")deftest_something_else(self):# this should never leave the systemurl="https://www.foo.bar"self.requests_mock.add("GET",url,json={"data":"the data"})response=requests.get(url)assertresponse.json()=={"data":"the data"}Note, that in order to hit real endpoints, the environment variablesCONSUMER_KEYandCONSUMER_SECRETneed to be set in the test environment. For example;CONSUMER_KEY=abcCONSUMER_SECRET=xyztox-eintegrationOr in order to re-record the responses, run# re-record everythingINTEGRATION_TESTING=onCONSUMER_KEY=abcCONSUMER_SECRET=xyzpytest# re-record responses for a specific test file, e.g. token testsINTEGRATION_TESTING=onCONSUMER_KEY=abcCONSUMER_SECRET=xyzpytesttests/test_token.pyMore information about the mock object library you can find in thepython documentation.LicenseThis code is licensed under an MIT license. See file LICENSE.txt for details on the usage and distribution terms. No dependencies are distributed as part of this package.All product names, logos, and brands are property of their respective owners. All company, product and service names used in this website are for identification purposes only